Global Scale Analysis with Google Earth Engine (Cloud Next '19)

My name is Joel Conklin. I'm a product manager for Earth engine Earth engines. The petabyte scale processing platform for geospatial data. Google used by a wide range of Industries swivel developing over the last 10 years, and I'm excited to tell you a little bit about it and show you a few demos illustrate. Some of what I can do. So when we think about Earth engine, we think about two things one is it's a massive data catalog with open data from satellite imagery and many others and the computation platform where you can analyze that data

in a variety of different ways to commit you use cases. We see users from agriculture industry forestry industry oil and gas natural resources in a wide variety of others and we'll go through a use case here in a second. Will do that through the lens of an agricultural use case. So imagine you're working at a company. You have a customer who is a farmer in Arizona. He wants to know how his agricultural fields are doing maybe is farming corn wheat soy, something like that and we're going to go through a use case that shows how that works in North engine.

I mentioned the Earth engine data catalog about 500 data sets of geospatial data 30 petabytes and growing with a wide range of data types from climate and weather data imagery and radar data geophysical socio-economic air quality data many more you can bring your own dated or attention as well. So if you're doing analysis where you like open data and your own data that's easy to do and what that does is it lets you get started faster. So instead of figuring out what day do you need? Where do you find it? How do you download a how do you process it? What you can do with our attention

is just identify that they did in the catalog and start start your work. Your hypothesis. And so I'm going to go to a demo right now will show us a little bit about how that works. so here with Five lines of code there in the middle of the screen. We've done some bolts and it in for analysis the first line identify as a dataset. Second line says, you know, what date do we want to look at here in a third line? We're doing some visualization parameters adding it to the map and telling it what part of the map to to show. I had run.

And you can see the status at this is Lance at 8. It says satellite imagery program run by the US government. It covers the glow of every 16 days. We've asked it for one day and you can see the path of the satellite from over the course that day. So we haven't asked much of your attention yet in terms of computation. But when you do start to do computation the Austin start in the code editor what we call the code editor and here at the top in the middle is where you write your script right above that. There's a run button which I use just

now which gives you the ability to run some calculations in our attention and then you can look at your results on top of the map either here at the bottom or if you put someone on the math you can see the results in the top-right. We also have access to all of your scripts all of your data and documentation about how to use Earth engine in the top left panel there in addition to the code editor environment. We also have programming apis you can use the programming of guys in jupyter notebooks or pull data into your teeth if you are close with the AP as

well. I want to go through a series of additional channel is here where will start to answer that question for the farmer. How are my feels doing this year? The first thing I'll do is show you what it looks like when you pull in 16 days of data. We should give us pretty full coverage of the United States and the whole globe. So that's that's giving a landsat satellite time to to cover the entire globe. As a you know, someone who's working to answer questions for a farmer you might care about a longer time

serious and not so I'll make this a six-month Time series. And you can get run there. And so again with relatively few lines of code. Not that much work to do you're pulling in data from you know, 6 months worth of Lance at hundreds of the data at your fingertips. And Lance catalog itself goes back 30 or 40 years so you can go wait wait for the back in history the knot. So we've also got this median value here, which is basically looking at over six months. You got 12 images for each part of the each part of the

planet is looking at the median value for each of the bands in. Image. It turns out that clouds are highly reflective. So if you want to find clouds change median to Max and once that comes out to see that there are files almost everywhere over the United States at some point during that 6-month. A quick and easy way to mask out clouds, which is the more common thing to do is to change the median to men. And run that and you've got a decent Cloud free Mosaic of the United States or anywhere on the globe again to do

Northampton. Not that much work. In the agricultural space you care. How well is your vegetation doing? How healthy is the vegetation in one common way of doing that is where the normalized difference vegetation index or ndvi the pretty simple calculation with the same a code that we had before to bring that imagery in Define a function at ndvi use existing North engine operators to do that calculation, you know, one band minus another / 1 band plus another Add the results to the image that you're working

on and then hear you math that across all the imagery in your in your collection. Define the pallet attitude the map what part of Matthew on a show and it's not going to show part of Arizona where this and agricultural fields. And hit run and I'll start pulling in ndvi data on top of some imagery data. What the NDP is showing as if you get higher values that goes to darker green and those dark green values to register for lots of Arizona has lots of vegetation. You can also move that transparency Fighters up and down to get a sense of what's behind that imagery. You can toggle delete layers on and

off. So Friday things you can do to start of inspect the data for the few more lines of code. We can do a lot more. And here we are again starting with that same basically that we had in the previous example. Adding a little bit of UI on top of that. And in this case for adding a map and a label on one side and on the other side linking those two maps with a function called Linker and adding a split panel on top of that so it a few more lines of code. Now you're able to go into Earth engine and scroll back and forth and see, you know, ndvi and how that

relates to the imagery underneath the pretty useful tool for analyzing that data An organic farmer, you might want to carry my team care a lot about how that data is changing over time. So in this code, I've added a another panel at the bottom with this this code here. And I added a series chart to that panel. And so when I click on the map that's going to show me the ntvi overtime with you know, that 0 to 1 range for energy. I found it and you can see that generally speaking for this farmer overall in 2018 and DVI was pretty good in July 18th. Looks like there was probably a cloud over

there over the over the farm. So that's why that's showing us a low number. the last time I have here is got to run a comparison to adding another time series chart running comparison between 2017 and 2018. So is that comes up we'll have ndvi for a 2017 on the left hand side and DVI for 2018 on the right hand side and we'll we'll start to see comparisons between the two data sets Battlestar to give us the answer of of how healthy are the fields how much vegetation is on those fields 1 year versus another and so

again clicking on that same roughly. Same spot. And looking at the results we can see that the NDP on 2018 at the bottom here. It was higher for most of the year. That was in 2017. This is all using Data Center public catalogs. You can bring your own data into this as well and using components that are easily available North engine to let you inspect that data in various ways. They're going back into the slides. So we've spent a lot of time a few minutes in the code editor, which is what we do interactive analysis.

We've started to get a simple answer for 1 farmer. If you want to do that at scale you start using Earth engines basket building so that allows you to scale globally so you got thousands of farmers that you're serving around the world let you automate runs and lets you basically take advantage of Earth engines parallel Computing architecture and on the right here, we've got an example of from not the agriculture space, but from work that we did with a European jrc joint Research Center where they wanted to attract surface water over the course of 30 years, you know,

where is their father been changing over 30 years and up in 10 million hours of computation put ten thousand to use against it would allow the conversation to finish in less than two days. So, you know when you really want to run up a job or tension can can help with that. And if you want to build a retention analysis into an app into a web page into a mobile app engine HDI is likely to do that because your users the ability to customize their date customize their the answers that they want and yeah, basically get a lot of the value and power of Earth engine without

writing code editor code themselves. So that's another few other things. You can help you see how your analysis have a greater impact with your analysis help it reach more and more of your users in a way that makes sense. Does my last side if you are interested in our attention to take my pill to help you solve some of your problems faster, you can go to Earth engine. Google.com click sign up and or sentence for you to evaluate and you can start getting the tires start using the code editor the FBI's whatever makes sense to you and take it from there. That's all I have. Thank you very

much for your attention. Appreciate you listening and depth.