Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
Predicting the Unpredictable Cruise’s Continuous Learning Machine
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
212
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

Driving in San Francisco is hard. Teaching an AV to drive the city's chaotic streets is even harder. In fact, an AV driving the streets of SF experiences a challenging situation 46x more often than in suburban areas. One of the roots of this challenge is prediction: Will that pedestrian cross the road in front of us? Is that car about to cut us off? Will the car in front of us run the yellow light or hit the brakes?

The difficulty of prediction can be deceivable. One can write a single rule, i.e. cars follow their current lanes, and be correct most of the time. However, the real challenge is to predict the long tail, which are the events that the AV sees very infrequently (and are often the most dangerous): U-turns or K-turns, bicycles swerving into traffic, or pedestrians jaywalking unexpectedly.

At Cruise, we have built a Continuous Learning Machine in order to predict the unpredictable. This session will go inside the inner workings of Cruise’s Continuous Learning Machine and explore how generalizing across the long tail is the foundation of solving all machine learning problems. Tianshi will cover how self-supervised learning addresses dataset imbalance challenges by automatically labeling data and sampling error situations, as well as the testing and metrics pipelines needed to ensure new models exceed performance and generalize well to the nearly infinite variety of scenarios-- all without requiring human intervention!

About speaker

Tianshi Gao
Principal AI Scientist at Cruise

Tianshi Gao, Ph.D. is the Principal AI Scientist at Cruise. He is a machine learning expert and a published author with over a decade of experience. Before joining Cruise, Tianshi was the lead machine learning (ML) scientist and engineer at Facebook. He led and scaled a team of 50+ ML engineers, scientists, and managers to increase ad matching and conversion rate through ads ranking and personalization. His team developed fundamental ML and AI technologies spanning sparse neural network architectures, distributed training algorithms, optimization, counterfactual evaluation and learning, and more. Tianshi’s team was one of the teams responsible for Facebook’s 40% ad revenue growth over the last seven years. Tianshi is also a prolific author, having published nine papers in five years. Three of his papers were selected to be presented at NeurIPS and ICCV, an honor that less than 7% of submitted papers received. He has been published as first author in every top computer vision and machine learning conference, such as CVPR, ICCV, ECCV, ICML, and NeurIPS. Tianshi has a Master’s and Ph.D in Electrical Engineering from Stanford University. His advisor and co-advisor at Stanford were Prof. Daphne Koller and Prof. Andrew Ng, the co-founders of Coursera. Tianshi earned his Bachelor’s in Electrical Engineering from Tsinghua University, where he graduated with honors.

View the profile
Share

So next week, Cruise predicting the unpredictable with present, continuous learning machine. / 2. Yep. Thanks to Allah. So, how do I do one of my name is Chauncey at Cruise? At the day, I'd like to share with you about how we predict the unpredictable with cruise companies during recession. So here's the all I know is going to cover today. So first of all, I'm going to give a quick intro to cruise and then we're going to Okay. Alright. I think I'm I'm hearing some Alaska. Me to wrap up.

So should I eat today? I'm going to take a quick intro to the crude and then we're going to keep driving to the production problem and it's Challenge and some of the key inside and then we're going to actually take a close. Look at cruise continue running person and how it is applied to solve the prediction problem. Who we are is the San francisco-based driving company building the word. Most of all self-driving, technology, be the first company to go driving in a major u.s. City. What's happening? The biggest technology challenge in our

generation in order to provide transportation option that he stay for more efficient and more accessible for everyone. The other word for working to be strapped, the status quote of Transportation today in particular. There are actually. Forty thousand people died in car accident. In the us alone every year, and 94% of these crashes are caused by human arrows. Our goal here is to remove the arrow Pokemon driver from the equation. Second delay, the current exhibition is not going at all in the

US make up the largest share of Green House in Newton. You decent, I'll pick our computer is actually will waste a week of their lives in traffic year. 116 of those 290 hours were spent in traffic that's longer than the average vacation. Toronto to be fixed trying to do is be Cruise origin. The world's first production ready all electric vehicle. It is perfect for the Butte all with no training wheels and Park Meadows, but yet I was busy as a party in mind in order to provide a

location is also fully electric and power two by 100% renewable energy. I think the rest of our Fleet which means it can be both conduction and greenhouse gases. About to do, this is not easy. So here is what it's like to drive in the downtown area of disco. What do I see here? Is the first person view from the car? In order for us to navigate such a thing. We first need to receive and understand what and where are the objects? And then we need to actually predict what they're going

to do. Next. Schedule can done plan. What we're going to do, safely. So this actually typing to me, the prediction problem. We were just briefly talked about. So what is producing again? How does people example to understand it? So what we are seeing here is a bird eye view of the word disease at intersection with four-way. Stop sign. Autumn, it's white car. Here is the coolest car and this is the car that won most interesting in this particular example. In this case, the Keuka wants to drive straight. But when

is to go is in fact, the function of one on coming car is going to do next. If this oncoming car wants to go straight, then it has no interaction with the coolest car, the both can go and continuously in Powell. However, is it, is it uncommon? Car wants to turn left then because it arrived at this intersection earlier. We should be healed and wait for it to clear exactly what we're trying to do a store in the, in the, in the bottom Graf, rink her shoes. The shoes are estimation of the possibility of this oncoming car going

straight and this yellow curve shows, the probability of this oncoming car turning up over time. So, at this particular moment, nothing that even the orientation of his car, both, going straight and turning left on. Both are likely But in the few moments, we can see that the car starting to turn. And you can see that our prediction about his car starts to favor of turning left. And as a result, we cannot wait for it to go for the price of the product is really just about understanding other than the user and then protect their

future motion. American drive safely. Auto V. This also look out another interesting cuz I'm cold. The other increasing sample corresponding to the talk to another interest intersection. There is plenty more complex. So we want you to understand all the things I'm kind of driving pattern as a busy intersection point in this graph indicate a position of the car. We are preserved as a particular point of time. So if we accumulate accumulate over time, you can speed that the cars tend to follow the Lynn.

And this black region indicating a specific accident at the intersection. What was interesting is that when chatting about driving trajectories that start from somewhere? I think the section under an exiting from the specific name, and here are some observations about the pattern. The first of all, and you can see Dorothy of the driving the directory, basically driving street from this land, and an exit from the same name and number of trajectories from a different name.

And I think something changed sometime in the middle and then acted in. Luminati wealth was being a small number of trajectories that correspond to a lump left arm. That's what most interesting are these to use introductory. These are in fact in in Tampa is the most challenging prediction problem. I think we're going to make a prediction, right? Kind of reward. It's looking at another example, this corresponds to actually a Meat Block. As you can see here, the almost

all the driving trajectory Street in following the lane. Even though it's extremely rare, but they're do eagles, eat a true U-turn trajectory in the middle block. And then these are actually the check read that when you could predict safely in order to be better than human. So now we understand the prediction challenge before, before we dive into the ocean at the year that the prophet problem. I like the first highlight a couple guys that are important to make the

burning bush in. The front seat. The prediction of driving is in fact, a self supervised learning cock reason for this, is that the future is always revealed itself over time do ink historical. They know eventually what actually happened and in retrospect. I will, can you open the basement door entry tomorrow? The second is that Active Learning. Super Bowl. Party is actually free and can be done till all this is because for every instant is the prediction and the label of readily available, you can easily break that could compare these two and then

find a interesting number for the mall of green poop. So, the next minute I would like to highlight the data. I only ask what's nice about driving in San Francisco for cruise. Is that our cars get exposed to changing cenarios with more often than this, a thumbs-up in particular? You can see, from this table where we are comparing. Basically, the number of Maneuvers per miles driven between two location. Is another not. The other is Phoenix. As you can see, focus on pole, the frequency that we actually see a construction navigation between San Francisco

and Phoenix is that you actually have almost 40 40 at more than 15 is actually a lot of that we can but we can't we can be a model to predict these maneuvers. Okay, so let's put all these animals together to be with a person Street. So at every single moment of time, this mother is actually going to make a prediction for every object, the car. And I'm going to wait a little bit and see what actually going to happen and then we can then compare what we predict and what what actually happened to find mistakes. In this

case, when we find trajectories that I'll do we use her. No, no. After finding, these are all the old stuff and then we can then augment our existence without wasted arrows are assembled. And then we are going to fix this mistake again and hoping that this new version of model at least can do better. Got to the previous model made an overall achieving higher accuracy on the new mistakes over time. Hopefully, this mall is going to focus more and more on those really, really challenging and then killed in

the fighting those chewing on those psycho are we talked to Matt intervention automatically, and then hopefully tomorrow I can continue to improve especially for those long time. But does it work and make your screen bigger? Can you just like expand your screen a little bit? Your presentation. Is it better to small? Much better. So, I maximize it. So. Olympics. okay, I'm going to maximize the Have you working out? Yes, it works a lot better now. Okay, great. So. Yeah, let's look at whether this works or you can see that

this white car is the object of interest rate. Is that there is there is a parking garage and this car actually be the calling from Tulane the part to this the parking garage. This is not a very frequent kind of maneuver Old Sugar Creek. Nasty was not time enough on the right hand side on the prediction on time for this car and be. Basically be the future position that model estimate. After doing what intuition of a b o n and folded and you can see that it can now or

correctly. Understand that all there is out. There is a garage here and we need to basically predict that is car is going to going to term and cutting into the to a parking garage. We also talking about the you turn it on post earlier in the day. They think of reward prediction for for it and they can be here. This car is the current view of the same car that is basically our prediction how to predict if you interest me very early on. So if you got this fine.

The car is slightly to the right which is a big no. This car is actually going to going to do the way you return. Here's another you don't put which is why we call you turn in real-world or been driving. You actually, even though it's pretty rare. You will actually see people just turning around in frontal in front of in the middle of the road where you can feed their like the car is going to reverse. And then and move forward. In this case. Our model actually did a very good job to really understand is pretty rare in a conflict. When you burp.

Do you think somebody like some kind of highlight of this bed is the prediction for for a car that goes around a song obstacle? So this is actually a good thing to be kind of easy for human but it's actually a hard prediction hack for mercenary model because it means to understand this thing very well. In this case, as you can see that, our model actually did a pretty good job to predict that the car is actually going to going to use a different then to to go forward. The last Ali is actually right now, powering

the two million miles driven in San Francisco. Sunday seared are corresponding to a corresponding to a 75-minute of. Driving, and you can basically find a video full video on cruise YouTube channel. yep, so that's pretty much it would be happy to thank you, Tiana. She that was great, very cool, lies. And sorry we weren't able to. I should have told you earlier to increase the size of your wife's, but great. Okay. So, looks like we have a couple questions

from you to have a lot of problems their problem. You can't solve. Yeah, like I mentioned earlier the mercenary model. You can we ask only you can see her house cuz those are the kind of problems. I think requires a different approach to ball. Okay. Next question is from she gently at do you have a human in the car? And it's at and drive all over the San Francisco while the eponymous car makes predictions? Yeah, the most significantly we actually have to kind of what we call a vehicle operators, presenting the car

they are actually going to basically they actually let the car to drive and if there is a dangerous situation that they think the car is not going to take over so that it can be state drive manual. That's great. Thank you. We've got another question By Yuji Oda. Thank you. What? Insight into human. Judgment could improve the trajectory prediction at a certain location? I'm not exactly sure. So, what do you think? You mean like? Also, if you think about how human make predictions of, what kind of information or like information, can you stream prediction,

is he? Is, what is ketamine? Kind of something featured in your me. Only the signals is one part where two men can share a lot of material supposed to say we can maneuver. So I understand that because there is a static and then the car and it takes the car with me to go to buy. But understanding Hall, human reason about that when we display the object of wrongs of the target object, and then maybe, maybe they say that they are too mean to assume that we can leverage tomorrow.

Which ml methods are being employed. Yeah, I'm freaking account talked about the the details here. I can stay like it is powered by Journey. Deep learning. Can you give any more details there? What kind of deep learning next question is? What are the ballpark size of the data being handled every second or every minute. Oh, yeah, so that's a great question. So driving on cars. Actually it operates at the meaning like, for every every second or they're going to be kind of about the word.

Imagine. Like, I'll take the wrong to you per person app saw them in time. So that's basically the ballpark. OK, Google skip. The next question from Dad, and we already answered a question. He had. Let's go to Vinos question. Very cool work with continuous, training of model. Sometimes you can be a problem in CLM, what helps maintain the stability of the models eg? It does not go into an unstable cycle. Yeah, that that's a decent, very insightful question for, we know details that needs to be taken care of between sure that it is stable and a run.

Call Barbara, ignore that scientists use the risk of the model. S A P not stable or maybe even come because I'm kind of bad. But the wise phone number of the day. Maybe you don't want to always only thing on, like, only the mistakes from every Teresa. Otherwise in the mall is overly 15 to a specific area distribution until maybe you wanted to consider this a perfect. Some reason. I like the older distribution. like a Samsung calculator tomorrow and also the learning rate of things like that. So the only needs to be taken care of soon on

instability. How do you avoid instability in the model? I mean, it's similar to the one. You just answered learning one predictions, on having it to subdivide the prediction into multiple use cases if you can answer this. Quickly. I'm not exactly sure. I fully understand this question. I want me to come and maybe in real-world applications. The production is not going to be a single point estimation. There are Insurance opportunities in this world, and in order to drive safely you put me. One fact, in fact, why didn't you hear is? That

will be great. Thanks. How much can I see? That was really great talk? Thank you everybody for all your question.

Cackle comments for the website

Buy this talk

Access to the talk “Predicting the Unpredictable Cruise’s Continuous Learning Machine”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
106
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Meghana Ravikumar
Machine Learning Engineer at SigOpt
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Nitin Sharma
Senior Research Scientist at PayPal Risk Sciences
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Agustín Mautone
Machine Learning Engineer at Tryolabs
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “Predicting the Unpredictable Cruise’s Continuous Learning Machine”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content