Terry Ryan is a Developer Advocate for the Google Cloud Platform team. He has 15 years of experience working with the web - both front end and back. Before working for technology shops like Google and Adobe he worked in roles that combined web development with system administration at the Wharton School of Business. He also wrote Driving Technical Change for Pragmatic Bookshelf, a book that arms technology professionals with the tools to convince reluctant co-workers to adopt new tools and technology.View the profile
About the talk
Coming to a new cloud platform can be a daunting task. Figuring out how the tools we offer track back to the tools you are familiar with, or need to solve your problems can be challenging. This session will explain the entire GCP platform offering, explain what everything does at a high level, and help you to make the right choices about which tools to use.
Hi, my name is Terry Ryan. I am a developer advocate for Google. So let's get started there. There are a ton of DCP products. When I went to go put this talk together. I went to go to our canonical list of all of our products like okay what things should I talk about and so I went and I saw the east And I saw these and I don't know. What did I sign up for you or this talk? I can't get through all of this right there. No other choice ones that we also do a lot with so I can't get through all of this. So what is
what is my goal here with this talk? My talk is to give it to her gcp from the perspective of someone coming to DC for the first time, right? So what are these things were all these hexagons? Are they all mean a secondary goal but not something to get to is explaining all the various things are available Google Cloud by they just I literally cannot do it and a lot of those like there are a lot of individual slides I have here that are actually placeholders for an entire talk again somewhere else. Like when I get into compute serverless vs. VM vs. Container like that the whole talk
on into itself, so I'm going to try to point you all in the direction of where you want to find the things that you want to do and hopefully that will be good for everyone in the room. So one of the resources that I used to build this content is gcp and 4 words or less which is available that very long GitHub address. If you just do a search on a search engine of your choice for gcp in 4 words, you will find this and it's really helpful. If he is very concise definitions of a lot of our products and where they are concise. I hopefully will kind of help explain
them. Let's get started with I think one of the most important parts of gcp which is infrastructure, right? When you move your work over us. If you need to run somewhere, you need to to have a BM. Do you need to have network connectivity? You need basic Computing resources. So infrastructure is a gigantic important part of the story about me just so you kind of see where my biases Are I Am by definition of appdev mostly on the web. So you'll see that I have much more knowledge on the computer side of the house and the technologies that sort of compliment them
as opposed to stay out of big data jobs. So I'm going to skip this slight detour and go right into computing. And we'll talk about the issue. I already brought up svm's vs. Serverless vs. Containers. This is one of the first decisions you have to make where are you going to run the code that you need to run to do what you need to do? So we start out first with compute engine, which is where I think a lot of her customers start off virtual machines discs and network. What do they do? What can they do for us? So we will start compute engine you can get
anywhere from .52 96 processors and actually that the actual actually see in the united new can go all the way up to 160 but you can't customize them same pic with memory a half of just under half a gigabyte of ram to 624 gigs of RAM. You can attach up the 64 terabytes of standard disk or SSD. You can do three terabytes of attached SSD. You can buy gpus RTP use and like to say they spit up intensive seconds. You'll see that in a minute. I'm just going to go right to my demo and then if I could switch over to my machine, that would be great.
There we go. All right, so you'll see here. I have a whole bunch of the ends already, but I'm going to create a new one and Imma have to call it. Delete me to it's like rated one earlier today. and let's see will put it put it in LA right now Los Angeles Data Center and you'll see that I can go ahead and customized I can create with the Anywhere from 96 I can play around this. I'm just going to do a basic one cuz I don't want to have to explain that on my expense report. I can choose a whole bunch of
different images whole bunch of different flavors of Linux core OS Debbie and Sue's whatever you want to use all and we have flavors of Windows as well. I could use standard or persistent SSD persistent disk or standard persistent disk again, I will choose standard and we see that I get an estimate of how much it's going to cost per month. Great. I can open up firewall for it. But one of my favorite features of the the the interface here is down to the bottom. I use this a lot. I can figure this in the lab and then I hit this and I get you'll see the long stick a text with this is
the g-cloud scripting command to build the VM that I can figure it so I can use the tool to dial an exactly what I want and then I don't have to write like out these really super long URLs. That means something in terms of what Scopes the Vienna's able to access. I don't have to remember all of that the Beast the tool do it for me, Go ahead. and create this VM and like I said these we have to spit up intensive seconds, which means what in my hotel room practicing making sure I don't screw up in front of you all takes like 10 seconds in front of
people it takes 40 and I'm not really sure why but my hope is that as I slowly crab walk over to the console. It'll be ready when I get there. All right good. So thank you for not looking directly at the screen and you know Sarah press the button to SSH into the machine that's available through the console all the time. I could just like I don't have to pull down certain want to set up my own terminal if you want to you can I don't want to I just do it. I do all high maintenance duties for these browser windows and you'll see here that I'm connecting and by the time I walk over
here is all ready to go and I can you know, let me pump up the fun a little bit here so you can see that I'm going to apt-get update. I had to start selling software. Remember that I was supposed to do that supposed to do. Bang bang again. There we go. And so now I've got to be and I can start falling around with installing software on all ready to go right to the browser. So I'm going to switch back. to the preso and move on to that was compute engine not want to talk a little bit about a penchant for words or less to manage application platform
manager at platform flavors standard and flexible. I think these run times are off cuz I know PHP Canal g72, so I apologize for that. So standard and flexible. There's a little bit of difference between the two flexible. You can do more different run times and we're different languages but it has opened capabilities, but it is it's not as available as standard which is much more constrained. You have to really adapter code style to write for a pension, but you get incredibly rapid scale out of it and its gills down
to zero. So if you're not using it, you don't get charged for it. If you are using it you can scale up to however much you said you need to handle. I'm relatively quickly and I'm actually going to show a demo of a pension scaling in just one second here. So let me switch over to my machine. And I'm going to call this up. Hopefully let's make sure everything is loading. There we go. Explain what's going on before I fire this off. I have an app engine app to speed up the top you why that's this thing. This is going to
either give it a variable amount of load. It's going to take that load distribute it basically do all the math to divide it amongst the 5 p.m. That I have down there those 5 p.m. To the Apache bench and they're just going to hit those going to send a whole bunch of request against the the app now they are at the bottom. I totally real app at a spot. Com. The reason why that's obscured for now is because I want to show this cold so I tested this last night and I let it all go cold. It's been sitting idle for overnight. It has zero usage right now. I want to show you how fast a pan chicken
scale from not running at all to handling large amounts of traffic. So I'll be here at the top you'll see that I have a request of 10,000. So me to send 10000 request at that. Real service there at the bottom are going to go ahead and send load and you'll see a pigeon spitting up instances of my code to handle my requests and something for me just ignore that happened. I seem to be having issues with this demo today. Maybe I know wait. Yeah, I'm going to get Walmart eye and then say I should have tested this morning
in addition to last night. Alright. Alright. Well then I'm just going to punish it by sending a hundred thousand two hundred thousand on it so I can sit down. Okay, well normally happens you can scale beautiful 8 when the app I wrote to to show this off shows up what you should see and what I will totally go back and make sure is working. The next time I could just talk is that usually it will go and scale. However many operations I needed to take and it will in the beginning the QPS Terry see if the top it's a 3600 will be loud be like 500 because it's just warming
up from cold and then I can eventually get this up to being able to handle QPS is of 8500 QPS which when you do all the math is about 30 million request an hour or what Wikipedia handles. I am sure app engine is fine. I am sure I pushed the change at some point that broke this front end. So with that I'm going to switch away from the demo and be glad that my sacrifice of the demo gods did not get accepted this morning. So what's the word application as a service writer you build an application?
We schedule it for you. But sometimes you don't need a full application. You just need a little bit of code one little bit of code that will do one thing enter Cloud functions event-driven serverless functions. What you can do is have a function a single function respond to an event and that event could be HD HTTP based, right? So you get a URL you can use to hit the your your code or it could be something we called background background is writing a file to a bucket or responding to a message on pubsub languages. We have a restricted nodejs
Python and go but this is another one where I'm just going to go to the console and show directly. So if we can switch back here. today and have a little bit of networking. So I'm going to blame the other one on the network. Just Wireless here functions will automatically populate this stuff for me so I can just call this greetings and you'll see that it has a hello world message down there. I'll spell it correctly that helps. I have a couple different options. I'm
going to make this trigger on HTTP. I'm going to call the function hello world, right? And what school is I can experiment with this I can. Now, I'm alright. Thank you, though. I'm were covering up for my phone. Sorry. Summer go ahead and create this an error occurred fantastic. Maybe I should have taken the hardwire but not too late now. So I already built one of these so I can I wasn't prepared for this. So I will see that I have a function here already to go and I have a a
trigger this I go ahead and write this function. That was the same exact function. I just wrote there and we'll see yes and something with you. All right. So this takes about thirty seconds to deploy and I can have functions that respond to anything that I need but based on just one function the rate we're going to switch back to the life present. I will talk a little bit kubernetes engine, right so we talked about vm's we talked about serverless and now we're going to go with containers. So how many people here have played with what's a peculiar play with
doctor? Goodnight, that's normal with kubernetes from scratch the hard way. Alright, so here's what you burn in his engine does for you. Right? So if you are used to running kubernetes, you may you may recognize, so you go to set up the vanities and you build yourself a cluster of machines, right? You want to make sure you have attached discs to the node machines because they're going to need extra storage the run your containers and whatnot. I'm going to go ahead and starve kubernetes and wire up all the
connections to make sure you have a network that will allow all of them to talk if you want to make sure you have your nose when you make sure you have a masters will make sure they're all talking to one another and then you're going to want to add DNS and service Discovery and then probably someone will say I like how do we know stuff is running so you got to add like logging and monitoring and then you got to make sure the house is all stay up-to-date and how to make sure kubernetes day is up-to-date and just a lot of work. Right? Like it's a lot of work. I made it sound like faster
and simpler than it is. Is it a hold. That's raised the hard way that like brakes people when they try to do it or just saying you could go to kubernetes engine and hit that button. This is Craig lesser, just press it and like it does all of that for you and will we will manage all the updates for you. All you have to do is say update and we'll update. And it's the first better experience than running kubernetes the hard way and so I just want a real quick show what kubernetes engine can do for you. So I'm going to switch the laptop here in the let's give this another try.
So I have a kubernetes engine. Sorry as a communities cluster all ready to go. Until the first thing I'm going to do here is a launch a visualizer. I all I'm going to do all this with make because I don't want to live type in front of anybody. So let's make sure this is all up and running. Visualizer. Okay, we should see nothing hear good. There's nothing we're going to do is we're going to lunch. Install kubernetes a system here, right? So make lunch. We should be able to see this live.
No, make the play Sorry Meg deploy. All right stuff is happening. So if we go back to hear we should slowly see my containers and my services load up. Any moment now? All right, I've got I've got work press I've got a custom PHP front end. I've got a note app and all of this is running on the same Hardware. It's all running into the you would never run Drupal and the seven on the 888-7772 saying box or run a version of no don't you had this set said about allocating resources that you can just keep adding containers on until you fill it up and then when
screen is engine in Loxley skillet for you, but I have the databases all set up in the back end and I have all of the front end is running and working. So that was a system that was running a tall 2 seconds ago. And now I'm starting an IP addresses. I'm starting to get kind of activity. I am not going to risk it. I'm going to say this work and this is great. So I'm going to switch back to presentation mode and then we're going to talk about something that got announced today, which is really exciting Cloud run now Cloud run sort of straddles because if I would get it for name
the four words, it would be serverless but with containers which seems sort of oxymoronic, but basically what you get with cabron is you get what we call Canadians but managed to Native managed you give it any container and we will steal it down to 0 or scaled up to however much you needed to do and we are able to scale that under load. So I'm going to show a quick example of a cloud run app to show to Cloud Run apps. So one of the things I'm going to switch back to my my laptop here. and let me make sure that I have access and let me talk about
a little thing that I was trying to do, which is I have a tab-delimited file. It's not that exciting. It is all the sessions from next there's some extra proprietary data in there. So don't like you no, read it or anything. It's still trust me. It's all the sections from next and I just want to take it displayed on a screen. So I wrote a little Go app that does that. And let's see. We we have that Ingo. Sorry, I'm running so many pull up run here. to do do Yo, and what city is my Go app? And you'll see that I have a URL for it,
right? So let's go click on that and we'll see. Let's 2019 Satchel s great right just a little bit of code. I may go to show it to you. If it's not that exciting. It's just string parsing code, but when I went and looked at this We go down as it eventually someplace it just kind of falls down right like the string parsing and go I just could not get it to do what I needed to do. So I struggled with go for a little while and I remembered I'm old and I used to know a language. That was really fantastic at parsing strengths.
fantastic at Parsons Springs, but like no one would build a server list run time around Pearl right that seems ridiculous, but I was like, but maybe Maybe it would do better than go at parsing a tab-delimited file. So we'll see here. I have the same app you'll see it is Pearl. You may not recognize it as Pearl because I did not write it to be as inscrutable as possible, but it works so I'm going to go and deploy this. code play that again. Alright this time it works to see how long it takes to push a container to Cloud run. So there we are.
Alright, so when go back to Cloud run here. Go back, and we'll see that I have a version of this Pearl. I'm a go ahead and go to this API PL right? Cuz it's ugly and we go look at that girl. Did it perfectly curled in it fantastically. Now what I love about this is what I get to make fun of go, which I love I do love go, but I do like making fun of gophers, but the other thing here is that like we all have this kind of code lying around right that like some know someone 20 years ago in 10
years ago and is still like being maintained still needs to run and we would like to get it off of our machines. I got a Latino at however old Pearl is doubt running on cloud run and solving my problems without having to do a lot of work to get Pearl running on filed run. So that's what I love anything. You bring will just run. And that's really fantastic and free. All right, great. So I am going to go back to my present and go a little bit further. And now we have a question to ask do I choose? Right? I've got containers are going to be M's. And I've got
service which one do I choose and that is like I said in like an entire talk on its own but let me try to give you like kind of a bullet points of this if you got a lot of stuff in PM's nbm's are the gravity that are holding you to being on Prem. If you want to move the cloud will then I would just go to compute engine just do migrations to the DM's and fine. Especially if you have a large number of systems people here in a shop that's more systems than Dev like it totally makes sense to go to VM Zinn and not switch over if you have a lot of developers and you have a lot of Greenfield
projects. It's a lot easier to start Greenfield projects on serverless that it is to started on your own Hardware. So that would kind of lens you towards going service. If you have a good mix of both contain both systems people and developers and you're looking to get the most out of the hardware that give the most out of your spend for for your resources container and container engine is the way I would go buy one of the advantages of computer containerization is one preparing your apps to run on containers pretty much matches what you need to do to run them on server list, except
the whole doctor part, but especially running entertainers, you can choose to run a whole bunch of different wedding doubt. I would say containers is probably the way you want to go for now serverless. I think it's eventually going to be what everyone wants to do. I think it is today. I like what a lot of developers want to do but there's there's things blocking people from going but I think I think so lot of EMS stick with computer engine if you have a lot of developers go with
serverless to be a lot of systems people and a lot of developers you can go with Painters with the added thing of if you can learn containers now or make future transitions for you much easier. Now that's a lot condensed. All right, so I'm going to switch gears Here and Now talk about networking. One of the things that we are proud of us about about Google cloud and our infrastructure is we have our own private Network so that when you are transmitting data between our services it transmits completely on our Network before it ever touches the the public internet and we can route stuff
all around the world. So if you are a user in Australia, and you request a resource in North America, what's going to happen is the request is going to go to a local point of presence in Australia for us and then travel on our Network to North America and then back and what this does is it provides you get chili from a lot of the First Tee nature of internet traffic because our network is little bit more consistent and you get pretty good therapist through that back at Network and I actually want to show a little demo of of of an app that we built the kind of show this off sofa. Switch
back to the laptop. Okay. So what am I to do is I have a picture of a headshot on my desktop that I took it earlier there. I am right where I sent me around the world send connecting and then we're going to be able to see this travel in real time across our Network. That we're not so that transmit that go in from Taiwan to Tokyo to Sydney to Oregon to Iowa to Virginia to London to Frankfurt to Sao Paulo took 4.24 seconds on our Network and what's great about the setup that we have for this is one that
that's fairly good that we were able to send this message across from Staunton all the places each get a VM and each location that took the damage in stamped it and then sent it on to the next spot. What's really great about this is that I didn't have to like set up any networking to make this happen. If you have your VM zelle on are in one of our projects, they can automatically talk to one another so you don't have to like go to different consoles or different parts of the world. You can just have all of your VMS in one place and wire them up to talk to one another just by adding them
to the same project. If you want to play with this, we actually have a full version of this out in the dead Lounge start the death zone and you can actually get selfies that print out and your picture has travel around the world and And you can take that home as a souvenir. So my switch back to the present. I'm going backwards. Then we go. All right. Now some basic things that people want from the network here. They won't load balancers, right? We have load balancers. We divide them into HD pssl, which you would think of this more than L7 load balancer. We also have a
network load balancers, which is what you think of it until 4 load balancer. You can also set up a bouncer's to be internal only so that you can only have traffic you don't have to expose your services to the world to get them to look down. We have firewalls pretty easy to set up you set up a rule open port 3306 you get that rule name MySQL you apply that tag that name to a machine and it'll open up that poured on that machine relatively easy to do in pretty simple to to go through. We also have Cloud virtual Network. So if you want to configure routes to handle
communication between you and your Google Cloud instances and another location, you can do that with Cloud interconnect. If you have enough traffic that you would like to directly Pier with us. It's something that we can talk about. It is obviously something wherever there's a hot a lot of upfront cost but the advantages is once you start that pipeline going you can transmit large enough data for a better a better experience VPN. We do provide EPN but it's gateway gateway only this isn't for hooking up Road Warriors. This is basically used to connect you to another set of some place securely
with your your own tunnel in place. CDN we do have a CDN if I wouldn't think this is not like stay competitive with Akamai. This is like you just want to add some Edge caching to your your work. You can do that you set up for a bouncer still use HTTP headers to determine whether or not stuff cashew can invalidate stuff. But if you need more advanced tools, you probably want to go with one of our partners instead of just the cloud CDN also. Yes, I love this because it it irritates the lawyers whenever I give this talk because it is the only product we have that has
100% SLA and every single time I get the lawyer say there you can't say that sounds like it says it right there on the web page. I'm staying at the only product we have that is 100% at the light. So you can run your DNS off of our Network and our nameservers. Do you have any hair? Which means that if you go with this and we have an outage don't mention me in your lawsuit I think is what was, going to coming from what I just said, right? All right, I'm going to move to yours to storage so storage we faced with divided into structured or unstructured storage
structured are things like data application data user data and unstructured things like files and blobs and and pictures and and other things so we'll start with unstructured. I will start with cloud storage cost overages object storing a serving if you want to have a giant bucket where you put all your files and serve them up to the world. This is the way you would do it. Divide this kind of into three areas public content process data and long-term storage. So we look at the chart public data. We have at the bottom there multi-regional storage. It is 99.95%
of ailable its Jill redundant. So this is for web country this or stuff you want to push to your users as close as possible so that they get the fastest pull-down of the data that you possibly can give it is 2.6 GB 2% 4 * 2.6 cents per gigabyte per month to store it there. Now that's for public facing data. If you have large data jobs that you've got a lot of files or batch data that you want to run a lot of habitation on you don't necessarily need it close to your users. You need to close to the machines that I can do that
work so you can go with regional where the date of will stay in one region, but you it's okay cuz you reset your V ends up there to Prada to turn through them there. And so they're only going to be or like let's say you're just a regional you you have a regional presents and you don't want to necessarily serve the world. You can just set up your data here. Now. It's important to note that just because it's regional doesn't mean it's only available in that region of just means that it will be faster served in that region. Then somewhere else is sold worldwide available. If you wish it
to me, it's cheaper to cents per gigabyte per month. And that's that's Regional now. We have two long-term storage and palpitations one is only called near line, which is 99% of ailable. It's got a 30-day minimum which means you can put it in and delete it in under 30 days, but we're going to charge you as if you left it there the entire time that you actually deleted that's totally fine once a month and unlike the other ones. There is a charge for pulling data out. So if you are if you've envisioned needing the date of multiple times, will you probably want to
do is pull it out leave it someplace instead of pulling out repeatedly? Search for things like backup or maybe long-term longtail multimedia, that's not getting a lot of views now. Just leave it there and have it be stored cheaper. The final one is cold line, which is our cheapest. It's 99 available. Just like home near Neal. I got a 90-day minimum storage again. You can pull it out earlier, but will just charge you and this is for things like Disaster Recovery or compliance the archives right things that you really hope you never have to pull back out cuz if you do you probably
have like a resume open in another window right disaster recovery of compliance archive two things that like are not happy making But if you have to at least it's the cheapest one to store and you will not complain about the five sixth sense for gigabyte to pull it out. Right because you got bigger problems. But the instead of here is to try to push you towards making sure that you're using it in the way we kind of envisage was this is not like this is not pulling stuff in and out very quickly. This is you know, four or Disaster Recovery compliance. All right. Well something other than call
Cloud file store, which I'm not really going to go too much into this is basically just to manage NFS server, right? Do you need to file system? Do you need multiple machines? Look at the same file system. There you go. We have this product if you really if you need in NFS server, this is what you need any part of that argument. I have my preferences you have your preferences. Let's all just agree to be friends and not getting to a nosql is web scale discussion. So we'll start with
nosql. We have two products here Cloud datastore, which is a horizontally scalable documentdb and we have Cloud firestore which is a strong and consistent herbalist document pee pee and you might want to say I don't think Tyson makes that forwards and I agree the other thing you might say as these two things if I look it up like, Toys similar know if they're both documentdb Zorba, you know, sorry horizontally scalable versus Riley consistently basically datastore is a little bit
more capacity a little bit more scale little bit little bit more robust firestore is based off of data store. But with the Firebase API on the front end which means one there's a really great kind of activity to to local clients. So the spk for the to Redington mobile and web is I think really really fantastic and easy to use and it has real-time capabilities. In fact that Network demo that I show her you're seeing in real time at hopping from place to place that was powered by fire store cuz I needed that for the real-time kind of aspect of the experience. Okay, so that we
have Cloud SQL managed MySQL and postgresql. That's what it is. It's like it's just mad at these things. You don't have to worry about maintenance or do you know you can still do all the DBA task you might need to do but you don't really have to do the systems work to keep that machine running. So that's about SQL and we have finally Cloud spanner, which is horizontally scalable relational database, which is basically combined some of the best features of nosql which is the the fact that you can start. It separated a bunch of whole bunch of different
instances and have that drive better performance and better is better scale, but it comes with a cost which is it is not like a it's more of a BMW than a Volkswagen. So if you go to experiment with it, please watch the the the pricing meter cuz it's it's a great tool when you have that level of scale on that level of client that you need to access. Give access to but it is it is a pretty robust tool. So how do I make the decision of which one to use among all of these? Well first it's easy to split up if I've got unstructured stuff. I want to use
one of the other dude wanted to know if I have structured stuff when I use it on the right if I have pictures or documents are other things how I choose ring cloud storage Corp FiOS store. There's a really easy. Do you need to file system and attach it to a VM, then you probably want fire store. If not use cloud storage break it also if you want to share it with the internet cloud storage like one-click to do where is with Todd files or heater depending on the VMS that you run to run web servers web server software in order to survive anything that's on there. So I would you want to share it
pots versus probably the way to go. All right. So let's say use your data or financial tracks or a stack of paper, which is I don't know what I was going for with that but You've got structured data, if you're going with nosql if you were talking directly between your database and web clients, I would I would highly recommend Cloud firestore. If yours is going a lot of server server communication cloud datastore is probably the way you want to go. Do you have a crap ton of users all around the world and like high high needs for availability?
Will then baby cloth banners right call for you, but you really have to test it out and see where does where does traditional SQL start falling down? And when does Scott's Banner starts become worth it to you to get the kind of performance that you can get out of it. So I'm going to switch to security last part of infrastructure. And one of the first things people do is how do I let people in to the resources they want and use cloud I am to do that. It's pretty basic. You've got a list of users you can add permissions and rolls to them and that's how you do it the permissions and
rolls definitely vary from product product and they are done on a product product basis. So different ones will have different abilities to do things. for example, one of the things that you can do is create custom rolls so you can take a whole bunch of permissions and combined them into a new custom roll, but that's only available for certain products in certain combinations so that it might we also have the ability to do Key Management Service. So is you want to host your own Keys you can like basically everything on our terms of data is encrypted by default. The only
question you have is not whether or not it's the Cryptids encrypted. It's whether or not you manage the keys or we do we'll do it by default. We're happy to do it for you. We also know that you know trust but verify your your perfectly entitled to a host your own keys and manager and keys if that's what you want to do. All right. So now that's all infrastructure and that's all the stuff that like you you absolutely have to have to move the cloud some of the things that we can add to the picture because again, we're Google and we have some experience with some of these things not
intelligence and not me but that's what we call it. So sorry Alex, so I'll show off is bikhri bikhri. Is this amazing tool that allows you basically to do Very fast lookups of large huge huge sets of data. And in fact, I'm going to do one now. I'm going to Brave the demo Gods once again. load All right, so I'm going to open this query. The run is very now. Let's see. I'm going to show options. Yeah, that's fine. So what am I running? I have all of Wikipedia's locks loaded up and so I can see the request for everything that's come in and I
can no no, it's not using cash. This is going to take a while just to explain was going to do basically long story short. I can I can in a relatively short. Of time look through over a trillion records in bigquery and come up with the answer of what are the top 100 most searched things on Wikipedia for 2013 and 2014. I do know this is going to take longer than most people's attention spans. I hope I hope that it remain cash but it didn't so long story short. This allows you to search through trillions of Records in just a few minutes
as opposed to maybe spitting up your own data job and run. In you know hours or what not but I am going to better part of valor and switch back to my present. That one more demo to give and that one. I'm happy about so other data tools that we have pubsub pops up with her messaging bus at the publisher subscriber model. You do many to many communication. Basically we can scout the hundreds of millions of operations millions of hundred million messages per second dose of data flow data flow is a managed service for data analysis. Basically, it's Apache beam, but
managed by us. Also managed by us is damp Rock, which is managed spark and Hadoop jobs, but with auto scaling So it's important that cloud pubsub and cloud storage forms of a bus so that you can if you need to talk between any of our services, that's probably going to be one of these two that used to do it with cloud storage bin for unstructured data like this to file or as Bob pubsub will do messages for you. All right, we're going to switch to machine learning in artificial intelligence. And typically we have three kind of ways of using ML
on gcp. The first is to use our model II is to create a model and the third is to Extended go further with models models. We have some of our models that we use the most already exposed in the form of these. Ml apis Vision API natural-language API speech API translation API there a few more the basis of what they allow you to do is you wanted to text faces. We have a vision API that allow you to check faces in your in your apps. I'm sorry in your pictures because it's a very common use gate same thing with logo detection or language detection
translation all of that stuff. Now typically what happens if someone uses this and like this is great and does the same but I really wish you could do a little bit more, you know case and point someone came to me and said they wanted Vision API, but they wanted to train It's so it can detect whether not someone's face was covered because they broke security cam software and they wanted to be able to text you have someone masked coming into a location. I could that's usually bad, right? We all agree that they're not there for fun your fun, I guess
so we also have this thing called automl which takes some of our are apis and allows you to train your own models with base with an arrow constraints and so much and it is a dental near and dear to my heart called a pizza dinosaurs switch over to laptop or we're going to refresh explained. So I was trying to teach myself ML and I'm not from California. I'm from Philadelphia and whenever I'm here I live here now. I really don't enjoy pizza here. And if you're from California, you don't understand why if you're
not from California, you're not in your head you understand. So I wanted something that would detect whether not pizza was authentic like it if it said it was New York style is New York style pizza if it's Chicago-style is Chicago stop. So we're going to see here on my desktop how to pull up a picture. And we go. So this pizza comes from my local pizza place. That is very authentic. Like I think it's very good for this area Pizza. Going to go ahead and upload a picture of it.
When we go and we have it all in my model that's going to go look at it first to determine whether or not it's pizza or not. And then after it figures out that it is in fact Pizza. It should give me a score and it says wow, that is 95% New York style pizza that has good pizza at least according to me. I was able to train this model like without any knowledge. I was able to just take pictures classify them put them in that wasn't that tremendously hard and now I have my own Auto ml model that can determine whether or not a man like pizza if you want to play with this this is also in
the developer sound you can go and were we have at various times that we're not announcing we're bringing an actual pizza and when there's not actual Pizza you can take use it with pictures of pizza which we know where they came from, but you can play with that kind of see how this works. It's also get a project so you can go and download it and like build your own model and play the give you want to you know rates other tools other Regional Other regional foods, you can do that cool. I'm going to switch back to the present with clothes that out. So you want to go further and
like not trained a narrowly constrained model. You probably want to use tensorflow and Cloud. Ml engine is probably the way to do that on our platform. Right? It's scalable. It's managed tensorflow where we will make recommendations on the how to run out of run those those jobs. All right, so we're at the conclusion that I'm close on time, and they're so there are a ton of gcp products. So we work here. We only showed these products right? We didn't show everything and here. We only showed these and here we only truth.
He's right. I didn't get into all of it. But my hope is that now you have a better idea of where the tools are and what kind of problems you consult with them in the computer and another level you can get to the resources that you need so that I'll say thank you very much for coming on your feedback. You probably got the email spamming you to say, please write your sessions. I apologize, but if you would that would be great, and I'm going to say thank you. I hope you enjoy the rest of next and have a good day. I'll Stick Around for questions from at the end of my time.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.