Duration 35:00
16+
Play
Video

AI Adventures: art, science, and tools of machine learning

Yufeng Guo
Developer and Machine Learning Advocate at Google
  • Video
  • Table of contents
  • Video
2018 Google I/O
May 10, 2018, Mountain View, USA
2018 Google I/O
Video
AI Adventures: art, science, and tools of machine learning
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
10.13 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker

Yufeng Guo
Developer and Machine Learning Advocate at Google

Yufeng is a developer advocate for machine learning on Google Cloud Platform, where he is trying to make machine learning more understandable and usable for all. He is the creator of AI Adventures, a YouTube series about the art, science, and tools of machine learning. He enjoys hearing about new and interesting applications of machine learning -- share your use case with him on Twitter.

About the talk

Looking to get more insights from your data, but don't know where to begin? Dive into machine learning and the discovery journey of applying it to your datasets with this session based on the YouTube series "AI Adventures".

Share

Morning, my name is Izzy playing well. And I host a video series on YouTube called AI Adventures where we explore the art science and tools of machine learning. I wanted to bring a little piece of that here today in person. So let's go on an adventure together. For the purposes of this session. Let's start with just a short definition of machine learning and use that to build up kind of a workflow and get into the tools. Well, then use that today to help us

see how we can solve machine learning problems and see how we can use that to apply to your problems outside. And so will say that machine learning is programming with data and will use that data to create a system or model which is trained to do some tasks and we trained this model very importantly using data and so everything comes back to the data. So that's really the first step of machine learning your data it all centers and starts from there. Next there's

typically some kind of preparation needed of that date of the raw data is often not so suitable for use. And so we need to prepare a little before it's ready for your machine learning. Search will need to choose the model. We want to use then train it up and evaluate its performance do some fine-tuning and finally make those predictions do the tasks. So we'll go through the steps in detail over the course of the session. So first, let's see what question we are trying to use Armada to address in the first place.

Full walkthrough kind of a simple example data set and go through the 7 steps using the task to categorize some data about animals. Okay. What kind of animal is this? Based on some input data? Is it a bird? Is it a mammal or a reptile fish or amphibian a bug or an invertebrate one of these 7 types and will be based on a variety of different kind of stats about an animal. Soar model will take in this data this structured data and use that to try to make a prediction one of these 7 types of animals.

Now that we know what kind of situation let's dive into each of these steps in more detail. First up is gathering data just because we know what problem we are trying to solve work doesn't always mean that we have the data needed to solve it and without data your model is going nowhere. So ask yourself this where does my data live in what format and how can I get to it? These are often times the early stumbling blocks as we tried to use data to solve an address our problems and you don't have all the you need to

collect your own data. This is also, use case and you can get creative in solving this problem by building some systems for data collection. Perhaps you make it again one really interesting example of this is Quick Draw. Quick Draw is an online game that lets you draw a specific picture and try some predictable you are driving. So it tells you draw a basketball or draw a boat. And now it's it's kind of funny people like it and as a result, the game has generated over 1 billion Doodles of hand drawn images from around the

world and this data set is open source on GitHub and there's actually an AI Adventures episode that we just put out if you want to learn more about using this data set and playing with that but back to her steps. What's next? Well, we got to prepare that did dinner preparation could be its own separate talk entirely. So we'll just before you do any kind of machine learning can really help you understand your data better in this fundamental because it gives you this intuition about your data

set and can save you a ton of time and headache later on in the process. And so I really encourage folks to clean their data to look at the data and intuition about it helps you identify gaps and you'll learn a lot about what you're working with along the way and maybe you'll find that you actually need more data or that certain piece of data. You collected Warren Ashley so useful after all. I like to play with to do this is called facets will take a little brief detour and talk about that has an open source data visualization tool from Google research if you give a

better get a better sense of your data distribution. And yes, there are a lot of tools that enable you to do very similar things. You can write all commands and call functions. But this one kind of put it all in one place. And because it's a UI, you won't forget the computer particular statistic or look into some corner of the data because it's all computer for you. It just shows up. And so you can just kind of brows it and so here we can see fastest apply to our data set of these animals from the zoo and most of them are kind of 108 assets. Does it breathe doesn't have a tail

that they have feathers things like that. And so that's why we see this kind of funny distribution 021 but a lot of times with numerical distributions will see something far more interesting. Now, let's move on to our third step of machine learning choosing a model. And we'll discuss a little bit about model selection how many we might go about doing this using the least effort possible? And for first let's build an intuition around what is happening when a machine learning model is working well, So here we see how the model is trying to divide up a reason

we were trying to say which side is blue. Which side is orange to match the data which are the dots. So it's not the heart of the grandpa in the world, right? Everything's already kind of decorative forever separate it out. But that's something a little more interesting struggle for a little bit to draw that enclosing circle around the blue. And it's really need to see that these tools are available to you today, which are sophisticated enough to handle all of them for you. So you don't really need to worry about the details of how is going about it. It's just a matter of is it a

cheating the attack but it is fun. And different models can model different kinds of data and different degrees of complexity. So that's what model selection is really about without seeing what model is the right fit for your particular data set and a particular problem. You're trying to solve no concrete or we can do is use tensorflow cheapest, right? So there's lots of talks about tensorflow here at Ohio this year, so I won't rehash those bits, but we'll see an example perhaps of what it might look like to use tensorflow to kind of do model selection 2-3,

right. Code, especially if you haven't had too much hands-on experience with tensorflow. This may be kind of a useful look at just how accessible it can be. So this is an example of a linear classifier similar to the first animation. We saw it's good for a kind of drawing a linear line between two places and we can see that it's pretty straightforward. But what if we had more complex data right this one line call? Well, it kind of stays in one line call. When you try to try it out on a different model going to replace this with a deep neural network and it only takes a few

small sweet. And in this case, we just have to add one additional argument called hidden units. This defines the structure of our Network. In this case. We have three layers with 30 20 and 10 neutrons in them. That's all we needed to do is literally just change the name to replace a text in add one argument. You don't have to replumb your entire system. You don't have to delete everything to start from scratch. That's really nice. And it's similar or other models. You can try a lot of them just involve one slip of the tongue of the method call and

you can busy play around with these different a menu of choices. And so in that Paradigm, it allows you to not have to commit to a model kind of beforehand. You can see how it does and then come back. That's what that brings us to our next step. We've gathered our data. We prepared it we've chosen a model for now at least and now we can do the training training a lot of attention because in some ways that's where the magic happens but in practice is often very straightforward. You just let

it run and you wait. So let's take a look at what happens during the training and then we can see it in when training a model we passing our data and we asked the model right away to try to make a prediction for the prediction will be pretty bad because our model is initialized randomly, but we make this prediction and we take it and compare it to the true answer the correct answer that we know. And we use that information to update the model. And so each time we go through and

update the model with a new piece of data is it works its way through the training set. It allows the models get better and better over time with each training Loop, but in practice when you actually need to coat it up, we take our function we created them and we just called. Train on it. literally one line you just need to provide a function that gives it the training data inputs. And that's it one line. And so once the training complete run this function you go off and you grab a sandwich and you come back

and it's time to do a valuation cuz we got to evaluate our model to see how accurate it is. See if it's doing a good job of Performing its primary task. So we can show some examples of things that we know the correct answers to and the model did not see those questions in Chinese didn't see that did in training. So we'll use this evaluation data that we set aside to measure the performance of our model. Conceptually, it looks very similar to training right? It's still the same shape, but the important difference is that there's no Arrow at the top is

not a closed-loop. The model does not get updated. Once we check its prediction results. And so if we did up-to-date the model, then it would be no different than training and would be essentially cheating. Right? So the evaluation did it it's kind of precious. No way. We don't use it to update the model. Will you hold it aside and we reserve it for evaluating the performance of our model. Encode it's just as simple as everything else. We've seen. We just call. Evaluate 1 evaluate you called. Evaluate and noticed

that we are passing in evaluation did a rhino say he died out and put function. So it's not the training data and if we were using the train. I left that we copy that line down. We just swapped out training for evaluate and we forgot to change the date. Of course now, I'm worried about you eating the performance of our model using the training data. And that would be a big mistake because the model has optimized itself for the training data. And so if you evaluate using that same data, what happens is you misrepresent the True Performance of your model and it would

likely perform poorly on real-world data. Once you deploy it since it's never seen and you never measured how it does with previously unseen data. So this brings us to our next Point, how can we actually improve our model then with ran the training? We did this thing? We're an evaluation. Now. What do we run train some more? How can we tweak it further? We could always swap out for a new model, but what if we decided on this model, but we want to improve it as it's this is where I prefer Amador tuning comes in these

models have parameters of their own but what values should we use and how do we pick them? Much like how you can try out different models. You can also try out lots of different model parameters this process called hyperparameter tuning in some ways Still Remains a active area of research, but that doesn't mean that you can't take advantage of it. Conceptually, we take our training and instead of just training one model will tweet that model and create a couple of different variants until run training on different variants of the same underlying model, but

with different model parameters and see how that affects our accuracy will train and evaluate all of these different variants of the same or similar models and then see which one performs best. That's how it informs our parameter choice. Will you see a lot of that is experimental you got to try them to see what works so you might have to break out your for Boop. And so yeah the task of choosing good hyperparameters is something perhaps you intuited this you could probably turn into a machine learning problem itself optimizing the parameters for the model that you're using for machine

learning. But we'll have to say that for another talk. All right. So we've gathered the data we've prepared it chosen a model and trained it evaluated the data and we tune of hyperparameters at long last. We have reached our last step making those prediction. This is the whole point of training up the model in the first place, right? It wasn't just to get a really good accuracy or to create heat on our server. It was to actually do something useful with it. And so making predictions we take the model and we deployed somewhere or isolated out

and we can show the model some data that it hasn't been trained on. And then see what kind of outfits for dicks. That's what that's our seven steps conceptual seven steps of machine learning. But let's see what it actually looks like in practice for I will turn to the one you use Beyond just tensorflow how you might actually achieve the seven steps. So we're going to go and look some code and in particular. Well, I'll feature a couple of tools and get Hands-On with and I'll point you to some additional resources as

well. First up is a tool called collab if you've ever worked with any sort of notebook environment or a rebel on the web before this is basically a way to run python in the browser for not just run python. It's a whole notebook environment. You can put mark down the code actually executes and because it's hosted essentially on Google Drive you can share with other people. So let's switch over to the demo and I can show you what it looks like to use collab. So here with

you loaded up collab and we can run a various commands. Right all the all the things that we would expect. So, let's see where still connected and in collapse because it's a notebook environment. You have to upload your own files and it connects the internet. So you can also authenticate and pull things down in my case. I have our data set loaded on the machine and so we can download that. And I've got pandas loaded here so we can take a look at our data set your I promise it's a bunch of animals and a bunch of Statistics at the very end. There is a Class Type. And so

kolob really lets you just execute the code on the screen in real time and more over because it's all hosting on Google Drive. I don't have to spin up any machines in the bathroom. You just works seamlessly and you can put comments in and share your work with others and makeup and do collaborative research or just work in general that's really need. And so, let's see. If we have some time, so let's actually walk through a little bit of an order. I've ended up doing that. We took our data and what

I'm doing is shuffling it and then splitting the data so we had a whole data set right in this household datasets happens to just be a hundred and one different animals. So it's probably the smallest dataset. I've probably come across but you still need to do the same types of best practices in this case. I taken it and split it. Into training and evaluation did because if I use all 101 to do training, what am I going to do to evaluate my model, so we'll split it in this case. I chose a fraction of 60% and 40% but that ratio can be adjusted and so we can see we have our

training data all 60 values. And then we have our valuation to tableau. And I do a little bit of pre-processing. So this is the processing your date a bit notably. The last column of the data set is the label. It is the Answer 1 through 7 of what the correct type of animal it is reptile amphibian mammal etcetera. The problem is the labels go from 1 to 7, and I need them to be 0 through 6 in this particular star just shifted by 1 by subtracting one and and that's it. So pretty simple pre-processing. I would expect that

a bigger data said perhaps would do more things. And so we can see that our data. Apparently, I didn't make this run run always run your code and function here is pretty straightforward. It takes the raw data and if we wanted to be Shuffle we can call Shuffle on a dataset and I just let it repeat and batch into chunks as needed. And so this will just feed right into the training and believe there was a talk about this as specifically earlier, and I also do miss that

you can cast recording or roll back to live stream. So we have our data set or our input function and I think I forgot to run the cell again. And this is a little cell I have here to just kind of try out my input function to make sure it's working always good to test things. And so we can see that my input function is indeed returning all the data of each types of the feathers when it has eggs where there's Airborne does it have a backbone things like that. And so each one of these are raised represent a batch of data. And I was also playing around with this check out

the unique values for each column just to make sure like most of them are ones and zeros, but there's a couple here that are little bit strange. For example, there are animals with five legs apparently, so not all data sets are perfect. And tensorflow uses this notion of feature columns to represent. The incoming data models are pretty generic. And so by using feature column that allows you to customize it for your particular data set in our case. I just Loop over all the date all the columns all 17 of them which happened to be numeric and set that as

numeric who's really just a consideration to let people know how many columns and what type are coming in. So we'll run that and if things break is because I forgot to run to sell so yell at me if that happens. And so here's that line of we saw before with the creating a model through here will create a linear classifier and pulling those feature columns and say that there are seven different classes seven different possible values for the animals. And I combined the train at eval is training evaluate calls

into a function just for convenience so that they both run together. I want to run the training on the training data and I want to run the evaluation on evaluation data the wrapping this together helps make sure you don't introduce bugs as you rerun sells to go along and turn and so this is all backed by Google servers and apparently it is taking its sweet time. While its doing that. Well, this one's done. So we got 90% which is like okay, right. There was only a hundred

Rose. We took 60 of them for training and use 44 evaluation. So it's not the most reliable metric. The idea here is that the tooling is there a code is there you should replace this with your own data, which will be much better than this data set and get awesome results. So as promised the Deep neural network is very similar. It's literally the same as before but I replace it with hidden units. I guess there's one notable difference is that I do take the old feature accounts that we had from before and we wrap it in an indicator, So that's just a space to put the

linear networks data into a weighted that the Deep neural network can represent. So it's a little bit out of school for this talk but it's a it's an adjustment just for deep neural networks. Linda networks don't have to deal with So we'll let that guy created the new South. So I guess that's also a good thing to to show off here so we can easily create, you know code blocks in cell block in between and you can have all sorts of great editing kind of abilities here and you can turn it into markdown you execute the cell

and it'll it'll be there. So I forget if I ran this Mabel running again. We can also in like push the model for the right the percentages here are sometimes it's wonky because they are because there's only $40 and evaluation. This is effectively something over 40 is getting it rain. Any other ones are wrong. So in this case, it looks like that 10% probably about 4 or wrong or three or four are incorrect. So let's make some predictions and see what the Mist tensorflow also has a. Predict

function just like it had. Rana. Evaluate and so you can pass in I just took it out of a she didn't slice. A couple of examples to take a look at it so we can see no hear that when we do that you get prediction and a correct answer. So in this case those five that arbitrarily shows happened to work out but what if we wanted to see the exact ones that we got wrong, so this was a little experiment Iran just because I was curious and which ones were the linear and I were getting wrong and which ones were the And we're getting wrong. In this case. There are there

different right? You got the same number of incorrect predictions, they all have for wrong, but they're actually different examples. And so this would be an opportunity to dig in further and play around with that has a GPU support and so you can toggle on a GPU. So if you have big data sets and a big fancy models and you want to access that sort of stuff go and get that GPU. So let's switch back to the slides briefly and take a look at the another fool. Aside from collab. There is another

school that we have that's very similar is also Notebook based and you might have heard of it called. And what kind of oil is most known for its competitions or discussion forums and datasets? It also has a feature called Colonels and Colonels is really just a fancy name for notebook in the Colonel's to look something like this this might start to look familiar from collab except it's blue and different in a couple of subtle waves firstly I want to let's just

switch over to the demo for cattle panels and we'll see what that looks like. How is that? So I took the notebook that we had earlier in her lap and I downloaded the notebook itself as a high python notebook file and datasets. We actually have a zoo animal classification dataset. How convenient that I chose the exact I did it said that already exists on tagging right? And so we can click new Colonel and I'm going to choose notebook intitle kernels not only run in Python, but you can also choose to run them in our for those of you who

prefer car. What I can do is I can actually upload a notebook. Well first I need to download that notebook. So let's do that. So from collab, we will download. This is what I get for assuming a lot download The Notebook. And then once it don't matter we can upload that notebook back into cattle panels. And so boom we have that same notebook now in kernels. There's one small sweet because we had to upload a file from the local drive for collab looking to get rid of that. And the only other tweak is that the data lives in one directly up call dimple. So if I run the first cell

and then we'll work our way down will see that the day that lives in import and it's the same kind of thing. And since we already saw the rest of the notebook, it's literally the same thing. What is interesting though. Is that in the Colonel's we can Let's give it a name. And we can commit that notebook. And what kind of condoms does is it will run your notebook in a new Fresh environment separate from the session that you're in this will generate a notebook with all of the outfits that we've been seen in a ton of nice view

only format, which is really useful for sharing because what if you want to share your notebook as well as how it was executed to other if you wanted reproducible notebooks and so that's kind of what we see here. So we we're a notebook is done and We can do that snapshot it so it's kind of like a good Hub model of like you can commit versions of your notebook and so by default. Your notebooks are private. And you we can see here that things ran the outputs are shown and I didn't have to go through and run these one by one right and if I ran these cells out of order manually

earlier, you're my results running them top to bottom like this will help catch those kind of bugs that would have helped would make it hard to reproduce my performance down the road when I come back to this next week or next month. And if you share your notebooks you can then Fork them. You can force them. You can also get others before their notebooks as well as add collaborators to your notebooks. You can add users to join you on the same notebook rather than pork in them. So there's a lot of great collaboration models across and depending on your particular

use case and I guess it would be good to also just briefly mentioned that like collab we can Also enable GPS intitle kernels as well. So don't let that be a deciding factor for you. So that's kind of kernels. Let's switch back over to the slides and talk about just a few other little kind of tools and tips and tricks before we kind of wrap things up. And if we could switch back into slides awesome, so let's say you don't want no books. Let's say you tried notebooks and they just weren't for you

or you have a bigger work. You need to run a long-running job is going to run 4 hours or you have a really big data set and it doesn't fit and I will kernels or it doesn't fit on your local directory. So you're not going to upload it to drive manually. Then you can use something like Cloud machine learning engine where you can kick off a job to run a long-running job that maybe runs across a set of distributed machines all of them potentially with GPS attached and once you're done you might want to serve your model, right you want to make those prediction, but perhaps you

want to do it at scale you building an app you train a model and you want to create a rest endpoint that can serve predictions to the world and sew machine learning engine also includes a auto-scaling prediction service. You can literally just take your tensorflow model give it a name. And didn't literally point to the file and give it a name there. There aren't any other steps because you can't do anything else in terms of creating a model. Which is really neat. And if you were working with things like scikit-learn or a tri booth and you want to serve those as well,

we'll take those two. And so then you don't have to deal with the OPP's aspect of machine learning. You can just play with your data to your model train and deploy with ease. And I think most not most but many of you are also perhaps aware of the machine learning API that are also available. These are pre-trained apis for doing various tasks their little more canned right? You don't provide your own data, but it does mean that everything just works out of the box. You need something you need a picture to be turned into a description you

need audio to be turned into text it just works and so that's kind of nice but the limitations of course are that you can't customize it for your specific use case. That is still something real. You have to just wait a little bit longer four would be one notable exception of vision. So the automl vision isn't available right now in Alpha and so you can apply for that and it says the train and customize your own Vision API. The animations that we saw earlier that I showed with the orange

and blue dots came from the tensorflow playground. So if you want to play around with neural network in your browser and just like toggle things on and off and just mess around head over to the playground. Tensorflow. Org and you can do that right there. And don't worry. You can't break it just to browser. You can only break your own machine. And so what's next the code that we just saw is I made it public that don't tangle. So that's a good thing to eat. If you want to learn more about tensorflow head on over to

tensorflow. Org and thirdly there's a machine learning crash course that Google released recently. And so if you want to learn really dive into the contest machine learning go further than what we talk about here today head over there. There's basically a whole curriculum of videos and kind of assignments that you can do and really build up your machine learning knowledge. And finally if you're interested in doing machine learning in the cloud you can head over to the clouds Dome the white tent next to the Google Assistant Dome and we're heading over to Cloud. Google.com / machine

learning. Ml. As I mentioned at the beginning I host a video series by the same name as the session Adventures. We explore some interesting nugget in each episode of about machine learning and try to do some Hands-On and do some interviews with interesting spokes. So hopefully we'll check it out and subscribe. So I want to thank you for joining me in this session, and we really know I really appreciate your feedback on session information. So please head on over to the schedule and you can log in and get some sleep. Thanks a lot.

Cackle comments for the website

Buy this talk

Access to the talk “AI Adventures: art, science, and tools of machine learning”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “2018 Google I/O”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Software development”?

You might be interested in videos from this event

September 28, 2018
Moscow
16
159
app store, apps, development, google play, mobile, soft

Similar talks

Andrew Gasparovic
Lead Developer at Google
+ 2 speakers
Jeremiah Harmsen
Lead of Brain Applied Zurich at Google
+ 2 speakers
James Pine
Software Engineer at Google
+ 2 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Josh Gordon
Developer Advocate at Google
+ 1 speaker
Laurence Moroney
Staff Developer Advocate at Google
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Daphne Luong
Director of Engineering at Google
+ 3 speakers
John Platt
Software Engineer at Google
+ 3 speakers
Rajen Sheth
Senior Director of Product Management at Google
+ 3 speakers
Fernanda Viégas
Research Scientist at Google
+ 3 speakers
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “AI Adventures: art, science, and tools of machine learning”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
558 conferences
22059 speakers
8190 hours of content