Eyes on Earth Episode 131 – Using AI in Geospatial Work

[00:00] There's always going to be that next tool. When I first started 32 years ago, it was to help start the first NLCD. And so I've seen that progression of methodologies over time. There's always going to be that next best thing. AI is that next best thing now. Five years from now, I'm sure it's going to be AI based, but it may be completely different. And so I think that's the thing that's... [00:23] is always the case in science, always staying on top of the game, looking at what's happening in the field, and always adapting. [00:36] Hello, everyone, and welcome to another episode of Eyes on Earth, a podcast produced at the USGS Aeros Center. Our podcast focuses on our ever-changing planet and on the people here at Eros and across the globe who use remote sensing to monitor and study the health of Earth. [00:53] My name is Tom Adamson. [00:57] Artificial intelligence, also known as AI, is quickly becoming a necessary part of geospatial work at Eros. [01:06] I had a conversation with Eros Director Pete Doucette about AI and its uses here at Eros. I also spoke to others at Eros who are using AI in their work to help clarify what it all means and what its benefits and challenges are. [01:21] Fair warning to listeners, we did have to use some of the jargon surrounding AI, like machine learning, deep learning, and convolutional neural networks.

[01:32] Now, don't be intimidated. It's a new technology and a very big change we're going through. [01:39] PEAT encourages us to be open to the new technology and its great potential to helping us at Eros make the work we do better, more efficient, and more useful. [01:48] And besides, at Eros, as the home of the Landsat Archive, Landsat data's quality, consistency, and continuity are optimal for AI exploitation. So let's get to it. We'll start with Eros Director Pete Doucette. [02:08] Last summer, we did an Eyes on Earth episode with a USGS scientist about mapping the California coastline. He described AI like this. [02:18] to learn by example. [02:20] Is this a fair way to describe AI, or is there more to it than that? [02:25] I think Sean's characterization was a useful way to think about AI. [02:30] in terms of a concise way to put it, I think to better understand the bigger picture of what AI is, because a lot of people are still wondering, what is it really? It's useful to understand the... [02:43] the different phases of its evolution. So the term AI was initially coined back in 1956. At that time, what kind of dominated the AI methodologies, if you will, had more to do with rules-based systems. For example, creating a set of rules that an algorithm would just carry out.

[03:13] probably a couple of decades until we entered into the machine learning, which is kind of what Sean's alluding to here, learn by example. Up until that point, it's really just computers are doing things faster than humans can. Yes, in a way, depending on what the application was. So oftentimes applications were very targeted and specific, not very flexible, but [03:38] But they were effective. Okay. [03:39] If we think about learning, say, to ride a bike, [03:44] Okay. You can learn to ride a bike from [03:47] from a manual, a user's guide on how to ride a bike. And so that's kind of a knowledge-based way of thinking about [03:54] Okay. [04:10] in that you're taking in the information and learning from the information that you take in in real time. And so that's kind of the, I guess, metaphor I would use to describe what AI was in the past, essentially a manual for learning how to do something versus a trial and error way of learning it. [04:31] So what is your background? How did you get interested in AI and promoting its use among geospatial professionals? [04:38] So my first career job was with the Defense Mapping Agency, which is now known as National Geospatial Intelligence Agency, or NGA. The mission of DMA, which hasn't changed, is basically to map the planet.

[04:53] largely from satellite data, that part of the planet that the USGS doesn't map. So the USGS is charged with mapping the United States, and NGA has the rest of the planet. So back in those early days, the mapping process was conducted through digitizing or tracing features from imagery, oftentimes from satellite imagery. [05:23] film-based products, right, to digital products. That was a major transition for the map-making process. And that's roughly when I entered into that scene of map-making, and NGA was transitioning. With digital imaging, of course, that afforded you all kinds of [05:42] computer vision based methods [05:45] to assist in digitizing or tracing out features such as roads, buildings, rivers, these kinds of things. [05:53] And so it was a course that I took at Purdue in 1993, if I recall. [05:59] That was called Engineering Aspects of Remote Sensing. It was taught by Dr. David Landriebe. [06:06] who was actually one of the pioneers on using Landsat data. And at that time, he was one of the forward thinkers on how to extract information from multispectral scanner MSS data back in the day. And he used that data in his course as part of the labs and such. And so that was my kind of introduction to multispectral data analysis through David Landgrove's course.

[06:36] name was Landgrebe, right? You've seen Landsat data. It was a heck of a coincidence there. But that's what first got me kind of intrigued about using computer vision methods to automate the process of digitizing features or characterizing features from multi-spectral data. We're talking a lot about AI right now, but it really seems like there's a lot more buzz about it now than [07:06] through the history of AI. One of them occurred around 2010, which was the use of GPUs, graphical processing units, which was a hardware acceleration. [07:19] that was brought to bear on a method that's based on artificial neural networks. And so they hadn't really demonstrated neural networks had their value or potential up until that moment using GPUs. And the realization was, well, we can just scale out the size of these neural network models, because GPUs allow us that additional compute power needed to train these models. It wasn't really recognized as a great breakthrough moment at the time. [07:49] But it crescendoed up into the mid-20-teens where it really started to take off. So let me get to the more recent time you're asking about, even just five years ago. So when I talk about artificial neural networks, and that today is really what identifies what I'll call modern AI.

[08:09] The foundation represented by modern AI is really the neural network methodology. Okay. And there are other methods that have evolved through the decades that are more conventional. The way I would characterize... [08:22] the real difference with a neural network based [08:25] of learning is that it's more inspired by how humans learn. And so it's not as though it's simulating, you know, a human or mammalian brain. That would be kind of overstating it because neural nets is essentially a very simple mathematical approach to learning in a way that biological neurons in a human brain, you know, [08:49] don't do, it's much more complicated in a human brain. So in that sense, it's inspired, as I said. But the big difference is [09:01] the way we train neural networks is inspired by the way humans learn through that trial and error process I was talking about before. So that's really the big breakthrough. And then just in the last few years, November of 2022, I believe, is when ChatGPT entered upon the cultural discourse of the day. That big breakthrough... [09:25] was led by the concept of the transformer. So that's the T in GPT. And again, transformer is another one of those terms where most people just say, well, you know, what does that mean? Why should I care? And how does it change the way I do business? The transformer is,

[09:44] is what allows... [09:47] the algorithm to make use of context. That was the big breakthrough. It's an absolutely pivotal concept in the world of machine learning and AI, the ability for the learning algorithm to make use of context. If you think about, for example, reading a novel, so any particular page that you [10:11] what's going on in that page in the story. But when you get to the end of the story, at the end of the book... [10:17] what you read in the first chapter may factor into understanding the plot of the story. And so that's context that's well beyond just the immediate surroundings, but the full story. So now you understand what happened, or you're able to factor in what happened in chapter one to understand what happens at the end of the book. And so that's a much more expansive way of thinking about context. [10:47] itself. So one might ask, well, what's being transformed? And so what's being transformed is the input data is being made to be more context aware. That's the transformation process. [11:02] So we've covered some of the AI terms and learned about Pete's background and where he's coming from. I'm going to turn now to Riley Fleckenstein, who works at Eros and is just about to finish his PhD in computer science at Dakota State University in South Dakota and see how he would define AI. And we'll get into something called deep learning. He also has another interesting take on the context that Pete just talked about.

[11:30] Give me, in a quick nutshell, how would you define artificial intelligence? And I know we call it AI. It's kind of the broad field. It encompasses a lot of different subdomains. The two main primary subcomponents of that would be machine learning, which is really essentially using algorithms or statistical models, or computers to help learn from data and learn to either make predictions or detect different patterns without explicitly instructing them or explicitly coding them to do that. [12:00] domain even of machine learning then really is deep learning, which is actually my main passion and main focus. An extension of machine learning where you're using neural networks. So that's really been kind of the big explosion in the whole AI ML boom is deep learning, these neural networks and the different architectures and how they're able to be applied to these perception based problems, which are traditionally extremely difficult to tackle with more traditional methods. [12:30] a certain kind of set of standard models and architectures that are primarily leveraged, and there isn't a lot of flexibility in mutating the algorithm to fit your data or fit your problem. Deep learning, though, is a little bit more like, I would say, more like a box of Legos. And instead of being forced to build a specific race car or something out of the box, it's really a box of Legos that you can construct in any way that you can think of that actually can better be better fit to your problem.

[13:00] a lot more data, but they have been shown to be able to extract more meaningful information from data as data volumes increase and outperform traditional machine learning methods. Okay. Can we talk more about Legos? Sure. This sounds like an interesting way to understand this a little bit better. You can buy a Lego set that does a race car, and it comes with instructions, and you follow the instructions. You go step one, step two, and you put them together, and when you follow the instructions, you end up with a race car. [13:30] You're kind of given, here's a box of Legos, build this race car, because that's what it's supposed to build. Where deep learning, it's like, here's Legos, and you can kind of build whatever you want. So it's kind of an engineer that has more flexibility, right? Can you tell me more about convolutional neural networks? Back to a bit of that Lego analogy, all neural networks are really kind of made up of these little Legos. They're actually referred to as nodes. And you construct these nodes, which are essentially just like elementary tensor operations, they call them, or mathematical operations. [14:00] mathematical operations, and you construct them into layers, and then the layers into modules, and then the modules into architectures. And you can kind of construct or customize what this architecture looks like based off of your data. Well, convolutional neural networks are constructed specifically to deal with images to help extract that spatial information from the image. And essentially, it's a neural network that's constructed, you know, specifically to be

[14:30] in image data, surrounding pixels have relevant information from next to each other, right? Yeah. Spatial kind of orientation. It's not just looking at the single pixel. Right. You know, this is a square. Right. What is it? If you look around it and you see agriculture. Right. I mean, that makes sense to us. But you were able to get the algorithm to kind of figure that out, too. Yeah. So that's what's like the beauty of the CNNs, you know, the UNET, which is not something I developed. That's been developed. Ronenberger, 2015 is the paper, if you're curious. [15:00] At Rondenberger et al., they developed it. But, yeah, this UNET, you know, is a convolutional neural network that can pull that information out, and that's spatial information, you know, right? So we knew that, and so we incorporated that into our algorithm, you know, to help pull that spatial information. [15:16] So we have some terminology taken care of. Machine learning, that's algorithms learning from data to detect patterns in the data. Deep learning, which uses neural networks, and neural networks is machine learning inspired by how humans learn. And then there's transformers, that's the algorithm making use of context. Now let's start talking about applications of AI. [15:45] How is AI going to help science projects at Eros going forward? One of the areas where it will be particularly useful is in the area of data integration. So one way to get more information from remote sensing data

[16:02] is to combine from different sensor types [16:07] we often refer to as multimodal data from [16:10] different sensors, say, from the electro-optical part of the [16:14] of the spectrum, the visible part, there's the thermal part, there's the radial part through synthetic aperture radar, for example. These are very different modes of information, which have been traditionally difficult to integrate. And when we do that, we come up with methods such as HLS, harmonized lens at sentinel data, where we wind up resampling one data to look like the other data set. [16:44] What transformers allow more naturally is to integrate multimodal data in a way... [16:52] that allows us to use those data sets in their native forms, without having to realign and resample things. Kind of like what happens in, for example, a human or a mammalian brain. So we have our senses. [17:07] that our brain integrates. So you're looking at me and you're listening to me. [17:13] So there's a visual source of information and an auditory one. Two very different [17:19] signal types. But the brain puts those together in a very almost seamless way. And we still don't understand quite how the brain does that, but it does it. And so can we take that same concept of integration of very different signal or observations and make sense of it in a more complete way?

[17:42] So that's where I think Transformers, the modern AI concept, has the largest potential with where we're going to take our remote sensing science. And in the future, we're expecting Landsat next, which is going to have even more of those spectral bands. Absolutely. It feels like this will be even more important to be able to understand by the time we get to that. So I think the future way to message the value of Landsat, if you will, is to demonstrate how we can combine it [18:09] with other modes of remote sensing data that make it a more valuable product beyond just itself. That's what AI will do. [18:20] will provide us the ability to do. In theory, it's yet to be demonstrated in a very concrete way, but that's where the future research resides, is the integration aspect of multimodal data using transformers. We should be seeing an increased amount of research, at least I would encourage research in that particular direction. [18:45] Next, we talk about another application of AI at Eros. For that, I'd like to introduce Terry Soule. Hi, I'm Terry Soule. I'm the Science Branch Chief for the Integrated Science and Applications Branch here at Eros. I've been here about 32 years and I'm glad to talk to you today. All right, thank you. [19:03] So I'm going to look at the merger that we did recently of the National Land Cover Database and the Land Change Monitoring Assessment and Projection Project. So two projects that did land cover, both used forms of deep learning, and I would still call that some form of AI, although a little bit more rudimentary. When we looked toward the future, we realized that we needed to become more efficient. We needed to merge those projects, save some money, create a product faster,

[19:33] a product with more accuracy, and that's where AI came in. And so we had a very daunting task a couple of years ago to take these very longstanding projects with a very long history and in two years completely revamp them. And I can't say enough about the team at Aeros, both the contract and the government side and how they did that, and that a completely new methodology was stood up, all AI-based, linking three different AI models. We're faster. We're more efficient. [20:03] projects, we've saved the government and the taxpayer money, and we're creating a superior product. So it's a win all the way around. [20:11] Now we'll go back to Riley for some more detail on AI's use in NLCD. For the last two years, the LCNEXT project, which is really the, we have our legacy projects, the NLCD, the National Land Cover Database, and then the LCMAP, or the Land Change Monitoring Assessment Projections Project. So they essentially merged, it was about two years ago, into our LCNEXT, or Land Cover Next project. The product that came out of that at the end of 2024 was annual NLCD, [20:41] Landsat data to classify all these different kinds of land cover. Yes. My primary job has been working on the classification algorithm for the annual NLCD product. What does classification mean real quick? Yeah. So classification, you know, just for this particular problem, giving each particular pixel a label, right? A land cover label. Oh, okay. Given input data of Landsat

[21:11] agriculture or whatever the land or urban or trees right yeah and there are several of them yeah yeah okay and there's trillions of pixels across the u.s yes so so you do need some kind of some kind of ai to help you do it faster otherwise it would take forever yeah so um to looking looking at the legacy projects a little bit um legacy nlcd did a phenomenal job of generating these [21:41] legend, the same spatial resolution, but there was some of a product generation latency. It usually took them two to three years to generate a new map because they relied pretty heavily on, you know, expert interpreters and scientists. They use some machine learning algorithms, but to support that, they also use a lot of like expert interpretation, right? So they make an extremely high quality product, but it just took more time, right? It took a couple of years. We want to reduce that latency. We want to get the annual product out there and do it quickly and [22:11] So that's kind of where we needed to rely on more advanced deep learning algorithms to try to replicate that high quality and retain that spatial cohesiveness and that temporal consistency and really keep that NLCD legacy feel that we know our customers love and maintain that high level of scientific consistency. And so, yeah, we really needed to leverage some more automated techniques to do it, be able to generate an annual cadence and now working on updating every year.

[22:41] There's the change detection and then there's the classification. And the classification component is we've coined as LCAMs or Land Cover Artificial Mapping System. [22:51] And, yeah, I was the chief designer, you could say, or the lead designer and developer of that algorithm, leveraging deep learning. We actually, you know, using convolutional neural networks, specifically the UNET convolutional neural network, and then also transformers as well for the temporal component. So introducing, you know, some of these advanced spatial and temporal models and ideas into the algorithm to extract that spatiotemporal information from the Landsat time series. [23:21] Thank you. [23:22] If this doesn't sound like a dumb question, why didn't we think of this a few years ago? We didn't have the capability. And there are definitely, from a staffing perspective, but also from a computational and a resource perspective, it's very data hungry, it's very computationally hungry, and those resources continue to grow in terms of our access to them, and we need that. And so to be frank, some of the things that we are doing right now, five years ago we couldn't. [23:52] I mean, we just didn't have the computing resources or the technical skill to be able to pull it off. But that's part of the challenge, too. It's a combination of people, the IT resources, and the will to try something new. It is changing fast. It is changing fast. Are there any other projects besides that one that are going to benefit from that, too?

[24:11] There are multiple projects that are pursuing AI, and they're linking into some of the work that's been done on NLCD. I'm also looking forward to the next generation of AI in the building, and that's something that Neil Pastick is helping us lead. And that, you know, right now, traditionally in the building, it's been project by project in terms of how we generate these products. [24:41] model that is something that could form the core of many projects. And so the thought is that by having this foundation model as the core, we can put different heads on that model and produce land cover, produce evapotranspiration, produce impervious surface and other data sets. And the advantage of that is from a cost perspective. But from a user perspective, too, it also ensures that all of those products that we generate are very consistent with each other. [25:11] And that's a huge advantage compared to the every-man-for-himself paradigm that we have right now for our projects. Okay, it's kind of cutting across different research areas is what it comes down to. Yeah, and it has the potential to less training data, less compute time, and leveraging work across multiple projects. It should help us become more efficient. Terry dropped the name of our fourth guest for this AI episode, Neil Pastic. Thank you. [25:40] Neil will tell us more about the foundation models that Terry mentioned.

[25:44] Will you go ahead and introduce yourself? Give us your background. Yeah, absolutely. Well, first off, Tom, thanks for having me on the show today. My name is Neil Pastic. I'm a research physical scientist with the Eros Data Center. I've been conducting data science research for environmental science applications for the past 15 plus years. So over the past half year or so, I've been developing a suite of geospatial or Earth foundation models. And you can think of these things as large scale deep learning models. [26:14] pre-trained on [26:16] huge corpuses of geospatial information. For example, imagery collected from Landsat, Sentinel-2 and the like. [26:23] And the idea is akin to how large language models operate on text, [26:28] But instead, we're really just working with remote sensing data. [26:32] So we're crawling the entire archive, trying to exploit patterns they're in, [26:36] and using that information to better understand the landscape around us. So it's all about developing and leveraging artificial intelligence, and more specifically, machine learning, deep learning, to help land use managers and scientists on the planet do more. [26:51] What else can you say about foundational modeling? How can that be useful to someone who downstream is going to use the data that you're working on? [27:00] Yeah, so there's two sides to that going there. So if we develop this geospatial foundation model, which is, [27:08] like I said, pre-trained on huge amounts of geospatial information, [27:12] It's basically learning a compressed representation of what that data is. So it's a distilled knowledge hub, per se, that can be mined by others. So it's a compressed representation of the data set, and these compressed weights can then be used for downstream tasks with a little bit of fine tuning and a limited amount of labels.

[27:33] The outputs of these models range from historical and future estimates of surface reflectance, [27:40] So folks can use those outputs. Say, for example, we have this pesky issue of clouds within optical satellite imagery. These foundation models are really good at gap filling the clouds. [27:54] making use of patterns that they see in other parts of the landscape. So we can gap fill in the tree, which can be very pivotal for [28:04] some applications. Another thing would be using those predictions of surface reflectance and turning those into snow abundance maps at a weekly and or finer temporal resolution. [28:15] So not only are the model weights, [28:18] useful, but the outputs from said model can be super useful as well. [28:22] Okay, so what kinds of things can that do? [28:25] What's really exciting about these models is they're designed to be incredibly effective, even in the face of limited label data. [28:33] and we are [28:34] in a position as [28:36] the land change monitoring mecca of the world per se, we don't have a lot of good labeled data describing the landscape. So once we train these foundation models, they're really highly reusable tools that you can then use or repurpose for a bunch of different applications. [28:54] So case in point, we've developed geospatial foundation models, trained on LATSAT-7-2 data, [29:00] And we use those features inherent within the model

[29:03] for mapping burn scars and progression. [29:07] forecasting invasive species cover in the Western United States. [29:11] monitoring snow depth abundance, [29:14] as well as snow water equivalent. And all that information is pivotal, pivotable or pivotable for land use managers and water resource managers. So even if they don't have the technical prowess to actually use the model themselves, the outputs are super important for them. But this also means once we scan these models up, researchers and users don't really have to start from scratch. They can leverage the weights within these models to kind of speed up deployment of solutions. In [29:44] effective tool for gaining insights from geospatial information. [29:52] We're going to end part one of this episode of Eyes on Earth right there. We learned about machine learning, deep learning, neural networks, transformers, and now we can add foundational models to that list. These models are pre-trained on huge amounts of data, in our case geospatial data, and [30:14] for lots of applications and users don't need to hash through all that data from scratch. And we talked about a few science applications of AI. In part two, we'll discuss how AI can help us fly the Landsat satellites and we'll explore the challenges and benefits of using AI. Check out our social media accounts to watch for that and all future episodes. You can also subscribe to us on Apple and YouTube podcasts.

[30:44] This podcast is a product of the U.S. Geological Survey, Department of Interior.