Combining Machine Learning and Control for Reactive Robots - Aude Billard | AAAI-20 Invited Talk

This is this money first planet. We talk by Professor Audi pillow is a full Professor on head of Lhasa laboratory for the School of Engineering at a sweet 16 of Technology law Sankey pfl. She was a faculty member as University of Southern California prior to join epfl in 2003. She has a bachelor's and a master's degree in physics from epfl and a PhD in artificial intelligence from the University of Edinburgh her research using machine learning and everybody with a particular

emphasis on learning from Sprouts, stay there and Performing fast and rabastan retrieve old and works and supplication robotics. Human robot human computer interaction and computational Neuroscience This research Ricci best paper award from several venues amount which IEEE transaction and Robotics ISS. Diana Ross a professor at least the operative Montara bring the interest of time and some mechanical short and with the group pleasure as well, Good morning, everyone and thank you very much for the very kind of interjection. It's a pleasure to be here. I know I'm in a robotic. So I'm going to

show you a lot of videos from robot what you think of this morning. I will try as much as possible to give you a little bit of a hint as to what is a difficult problem in robotics. And why would think that machine on his he is a solution but I also want to emphasize that machine solution for every robotics problem. And so all the waters between my lab is the combination between traditional control and machine learning and we also use an off-the-shelf machine learning techniques. So I went to Starling that's a

teaser. I always like to start Waze is this problem? Let's try to catch an object real fast within a few minutes seconds in here. We were talking about how the second the flight and the object is complex to catch. Hernia, Ridge apartment with a video. It's alright. But if they don't know, all right, so the object that we have to to catch a flying very fast, but also prove that there are not bored. So that means that they have at least only two axis of symmetry. Is it having a symmetry that means that you have to move and catch it along the along the main axis

of the of the object when you catched a rocket flying this is even more difficult because you have to catch it at a certain point which is to handle and of course I'll go will emphasize leader in the major difficulty is that when these objects slide if I was very lonely now Dynamics, so you have to sort of model design a mix and you have to be able to moderate and retrieve this information to run time. But before I go there I lost you for a little bit of fishing. So we're going to look at with a very very busy problem of Christ in robotics. It seems a fairly simple problem.

You don't think picking up an object. And then please give me somewhere but as a matter of fact is still a very very difficult problem. And if you look at the way that human rights for blacks, which is here on a little list of them the ways in which the fingers are placed on the object are not around the stable across a stupid Ross will be something other than globes your tracks here at the bottom. But you know why this person is handling this object is ways to present too late to come out and to prevent liquid to come out the other bras like those which are crazy unstable, especially this

one on the bottom, but which are used for politeness when you hand something over to somebody else and then this is the most magic part is when you use Finger the to text Arielle to hand to hold different objects. I will not show you any video on this but it is something that we walk on news crew use a hold of her fingers in different ways to to to grasp objects is what is still escaping robotics to some extent so we we try to tackle this problem by saying, okay. Let's go back to a problem. We know who to solve.

We just have a simulated robot hero is a cylinder in a cylinder is a fairly easy shade to wrap a present and then we will now compute traditional control. We will find the minimum of torque for three fingers placed on different part of this object stuff that just able to cancel the depravity of kinematics in either with That leaves you with a very non-convex optimization and you'll see a damn it actually this is a strength and his friends that I will use and reuse

over and over again this month. It will give you many different feasible solution. And in fact, it turned out that we don't care so much about the team 980 the same way as what you seen for the video about is feasibility. So we ended up with more than 1,000 crossed and then to make sense of his Frost. We applied clustering very simple clustering where we looked for the Clusters of a simple came in looking for the light fuse estimate mbid equatorial to find the optimal number of customers. We found about 14 grass types. That

means that include cross the river similar. Can I give you a couple of example here is one of those drawers you see on Top This is the robots and on bottom human. We did not put this knowledge in Rutherford. This is what is discovered right simply something or if he even his face. But not the problem is of course, we canceled our fun time. If we don't do that. You don't solve each time jobs musicians like you came to this room until don't like me you look at the map and then you know, you use thought about it and you

did your past tanning, but you've done that once or twice and then by now if you've been to his home sweet Farms than you know your way and you're not optimizing again free from from a nap. So we thought well, that's where I can help us all of this and then we would build a probability map which represents a distribution of food of people posture on the cylinder. So this is a 14 dimensional space is 48 dimensional and an imposition orientation are on over 60 minutes and then we can populate. And I represent he replies which is given by PDF.

It's the most of the redundancy in the system number of possible that are similar and this is something that we would expect when we catch object in Flight is redundancy gives you some certainty that you would catch it. Even if you have to move a little bit your finger around here check. So we have these manifold of feasible and stable craft in clothes for we even have any clothes from read the battery which you can compute and which is the variance of

prostatitis. So what happens when your outside is visible space what can happen typically wonder what is not yet on the object on it sitting there with her initial configuration lines of Guided by black dude, and we go to the closest in joint space the closest posture that is feasible and do evil for the robot on the road. I'm going to go off and you see a stranger and we can Extenze do you see if I could see images that we find can actually be used for many different

objects with just a tiny amount of closing a little tighter the finger on this with a given cross. Your Alder examples of typically this very nice cross where you hold the glass upside down you can you see if we can realize that we hear this will last just painted for vision and then here is a cell phone and we can also hold or type of what objects. Right there ready to the Luna more complex. If I put these objects into Hanover boss. Yes, it is but it still doesn't know to regress p23 the tiny turtle Vision as we see here

for somebody puts it into finger and slightly move the object of interest from this year. So in order to teach robots now how to be rubbish. It's not worth one thing that I would have brought but she's doing both visible and is the closest given this current current configuration, but once it's on the object's position rapidly, So we going to get in use the name from the most recent we couldn't have done that from automatically forward you the form of simulation. It's very very hard to transfer

information about forces contact. So we wants to live and here is an example so we can guide fingers of the robot moving the object in the hand moving a different configuration showing different orientation a showing the feasible type of account screens CBC World poster in Bose technical information and position information you requested that kind of formation of the fingertip the position of the finger as we do that by bike riding the robot and then we can shoot something a little more complex which is useful for control control his finger at the exact levels of

the room. That means that we would control now for stiffness impedance at the objective and we will have how far to finger estimate the Central Central Flats without having it just by the position of the finger at an implicit and bring to keep the finger along the center of mass which determines facts does Crimes of the of the pressure and is bringing you would have a spring of spring in this from the Sun than if you have it on the other way around here

and then What can we do with this? Will it run time? What we can do is simply to get expectation over the conditional of this data to determine what is the price difference between Asian or what is a right posture? So here is what happens for instance. If somebody was the object as a robot space which is determined by this distribution would be acceptable, but this is not acceptable acceptable position. Okay, the next step of course is to be able to adapt to different amount of wait. Don't forget that we don't have the weight of the object. We have it

implicitly with this notion of center of mass which we computed from the position of 3. Increasing the width like we're doing here which is the natural thing that will happen. If you're not a robot manipulate objects, you need to adapt to this and here and drop it stops moving a little bit of information about we don't don't how much we feel that mean no more that we count as we have a glass of coffee or glass of water in our hands. We just need many different examples. And we add this to the distribution and we get some video distribution that

still has the ocean and the information the pressure but this time the database is larger. And so we get a more informed representation of the data and condition of this what you see here for instance as we can condition when we increase the amount of waste and naturally if we are a little nasty stuff to pull your breath away from from the Great Wall of the finger only one of them will actually stiffen up the one where it's the most Essentials going to

three. Andy's last video just wants to emphasize once world. That's what is this distribution of feasible space. We also still retain what we just go along. This is what is compliance with something for division and resist or change the behavior Steven. Only when it snows What are all things that are Beyond just increasing the stiffness and here is an example what you don't see it because it's Above. The Rest of body here feeding slowly is object with some beans. They're playing. I don't see it. Say it

again. Yes, girl, some beans if you can see that at the bottom which are being filled by someone putting some beans inside and at some point it would be too heavy and even though the robot has done in order to regain disability and then replace it you received it is not the question is where to place your finger and you have to move it Superfast joking about some to know we did not have any information about the objects. We did not have any explicit information. We couldn't fail to ship from the position of a senior, but we did not have to be able to

move the finger at one time only objection if we need to have the shape of the object. So we can do it a different way. Either of weed. Affect rate if you can recognize it so we can get flowers and a mother is from a phone call from a camera and we need to have a puzzle 3D position because of finger horror 3-D or we can do it by scanning. It will use the back probability of an indeed slide and are covered with the cousins so I can use to detect to normal. So is this

weekend and Loan. She only with cushion process regression but was determined the normal to the surface such that we could put the finger tip on the normal now, we have the shape of the object and if there is a small for to Vision as we seen before then what we can do. Is concrete a new contact points along this and see if you don't look at explosions moving do it taking the bridge inside only slow manifold Constitution gives you the money. The posture of the

object and you can slide. That's what you're seeing here. And then we can slightly slider the object to Stone post and you have to quickly jump what you can do with switch on Fast and if you have a complete representation of the object undefeatable posture. Do you have to understand that this combines both the physical space of my fingertip as well as in Modern English your face and he thinks so poorly here with me to please again. Hopefully with no. She is supposed to

shake the dollar as much is a benchmark Now using robotics to send you a pic to really see for how long can you hold the object in the hand and you see on Boss video here on the left hand side door slides on the object. They have to jump because it's is too fast. I would like to listen to buy size up here on the left hand side. We had trained. It's only on the first bottle a simple bottle this one, but then the weather tonight. I don't know why the video was actually think that's much

too. Nice of a problem with the projector. The refreshing him well control and here is an example where we can detect from EMG. This is so cool classification with different type of bottom here. You see on collaboration down with some with office on University where we applied at an extra two cities. How to prove that we could use this control system weed and walk down some more. This is this is so dumb. Decoding using different fingernail to detect the position of the singer

on the bottle as you can with what happens if you only use for carrying the bottle is that you don't have any information feedback feedback to the person who's using mg about the title information. And again, this will happen. Scripture and it will drop it and you will not be able to return from it. So when we combine voice detection bcmg. We would actually closed and stabilize your checked as we move it then we have a very very stable control which is moving this way. It's very very sad that it Shakes as much. Do you have any suggestions on what I get

out of them and start again? Fluoxetine I love the screen. Scream 4 party stores in our discussion I can take it now. Kevin upstairs but there All right, we're going ahead of ourselves. Alright, haven't seen any of that. Okay, that's in so much better. And I know you have a second decision, which is on as the person carries the object and put it back on the on the table. So we couldn't we could prove that. They were really enormously improved. But you have a son. We have many different

aspects that are Prisoners the ability to to reset belies your objectivity and quickly do when I search to find the closest OnPoint. So I like to go back to this which was easier to start with his how can weed and grass things that off line. So to do that, the first thing that we need of course is to be able to predict where the object is flying and the difficulty is a mention in Tradition. Is that the object flies with the initial translational and rotational velocity,

so you may have handled when it comes to you and Yuri wants to cross money to handle or not, but I'll spot. So in order to do that, we think he flew a rocket and we tracked them we tried them was the very first camera out there and we put marker. This is so slow down remix YouTube spin on the Rockets today will have a very annoying. What works as well as superfecta regressions, but we chose finally because the problem is is that we needed to predict ahead without having any data

and it was Grace that you see here or the Blue Line. It couldn't resist very far from you did a very very rapidly and give complete the nonsense to the very last place in stop moving in the vertical Direction. So we ended up choosing you always and a good single. So again is that is closed and so we could and then we could No, we need to grab the object. I've told you about how we can grab objects and represent. So what we did was to represents the the way that we should

put the finger on the object by having human something from this and this created a sample distribution that you see here is licensed to be the same as what we seen is a simpler distribution because basically it's a cylinder 2, which you can place your finger and it is sort of a region which is physical that means that any of this region is doable. Robot. Now we have the projector Realty of the of the racket which is flying in space. We know the workspace of a robot. And so we have to determine West broad beans and as you can see we seem to work space of the robots for

sometime sometimes means usually 10 minutes ago, but it feels right away at the beginning before it's gone at the very end for women have such a good idea. So maybe in the middle of something else and something else is what is doable for the robot in a joint space, which is what we call dangerous kinematics. So far this the most likely one and the most likely one is the one the densest and if it's the densest position, that means that this is the one where I can watch it or change like

in my position, which is very necessary because I Plies I would have to add that to position or orientation. And so I better be in a region where I have many different rooms not such a good idea so we can buy distribution of the workspace of the robot with the project planning. But the reality is supposed to be different if flies so fast, even if you have very fast camera and we have these forward planning, which is fairly good what happens in reality is a measurement and then we did sell mine. In fact, it's not flying where I thought it would be fine by something somewhere else. So the

question is, how do I that I just have computed the flight to the complete the way I feel pretty. What is the most likely and now I have to redo that at one time I can do this, but the problem is that I have to computer the best position so How do I compute snow an infinite number of paths which would lead me to this in clothes from this is why I chose not to repeat it. Here is a lot of machines on him being used for during control. So what we do is that we teach robot we show the robot from like you're different

character in space to grab a rocket. And what we do is if we build Piccoli dynamical system, it's a fruit salad or equation which resembles like this. It's a field of motion, which is known in Floresville. And it is a combination between machine learning techniques as well as controlled from Fort Worth. Every time. I have many different trajectory Schroeder put something which is the frame of reference on the object such that it moves with the object all the time and the

effects on this flu. So what happens is that when there is a rerun few months that goes to some other places then I simply move both the flu and the center so the original system in which I can we compute the pass inside the passage given to me in close for so don't know planning so we are again. So when you're off course many different videos showing the walking lights. So you can appreciate that as we stroll the robot has to stop moving right away. This one I particularly like the one with the

How to spell this video I think it's real problem with cream, but if I can catch a bottle of water and flying you can find all of this video on online if you're interested because you will see that it will catch the object when it goes on the side has to go down and there are lots of adaptation to be done. Of course when you're free. I don't think that you've expended to with a Twist to be able to text from Sing. That's Fool's it says phone. It's much easier the Dynamics of the simple. But still there is a problem. Is that it for

Turn the thing that you catch it at the top but some of the four tiny tiny little changed at the neck and and you say why all these two different tractors tractors because you're at theater in trunk space is very different. These are two different grip and so in a few minutes. Can you have to decide this wasn't for another thing that we have developed that was published of need. We tried that we could actually no man. O matic partitioning across different song

wake up system for each of those petition to grow at different attractions. And this could be alone in conjunction with a different resting point on an object to object as you can see this Dynamics, which is embedded in the in the in the brain of a robot is actually moving with a frame of reference of job site and letting the robots which aquatic ecosystem. Sasha you a live video here to catch a bubble its last 40.2 second and the robot has to choose between catching it an echo catching it at the at the top and so it probably depends on your intuition at the beginning

and his phone so we have to take into account stats of the ocean. Remove that one step router to have two robotic Alum Crossing in conjunction an object. So same thing we can have it ready to read for the object and we can have the to walk with the robot which we can compute and then we can determine resting points on this subject and computes. What is the most likely point where they were about to robot could last year and this is what we use them for a recessed to catch a flying out here with a difficult subject

because the subject is non-rigid. So you have to keep tracking and keep updating inside the Dynamics of the moving object. What is interesting also is that the two Robata resynchronized and their synchronized but they don't follow the same path. And at one time they're just completing a different past tense of the system to be able to reach the different point in synchrony. The order before we found that I presented the beginning with controlling p1642. We were conferring at 4:30 fingertip. Then we move to be able to

control for dinosaur or something that moves and I'm pretty Sons on the object. I'll post what we did here was to extend so they soon discovered that I can think of to your house to figure out which is very similar to what I put in it. And so they have a natural in business support division, which is never runs today and how the rock come back and then the Align so yeah. Consult you to the position orientation of the object, but they also stabilize for the phone so we should change

orientation for instance or if you try standing back to the forces. So. That's a very interesting application one of them which was presented at 3 for the eye and just last year with this one. It was to be able to guide in fact about using it as such a to leave the other hand free. That's what we're seeing you. So the two robots are now Koppel still control for the force that they're flying here on the object to the whole Stevie an object. And even though the person is doing it for the force and the food interface that you see below there. Is there

only to help move orientation of position of this couple system. So it's one way of choosing these cross the stable draw statistician of the of the robot creep in conjunction with the direction which which is given by the human We have stuff to do on developing these for a population for surgery, which I showed him briefly, but I was President for a AAA. So there is so nice to everybody, I'm also have to synchronize and stabilize and Hold Us defeat of an object's it brings us also to another problem of control and I don't want to take a

look when she's complaining for stuffed in there for me. But here you have just the first video. This is pretty walking Progress Drive to fit into a physical 441 and we me make a little bit what will happen when you do surgery inside of a womb. the person has his two arms there inside to do we set a ring is the ring Huron on a plastic elastic and then one of the robots in peace Finally, I guess I'm running has for real-time interesting is always to scale up to bring things to more and more complex little so it's one thing to balance with finger holes in the hands. He's

wanting to balance with 2 hours at 6 on the table, but something's terribly difficult thing to balance things when you actually working. So we started doing the same thing holding safely objects. Why do we hunt financing a robot? And here is about to kill Ali complex one which II Chozen pupils, which is that the robots was he here has to go down down down to almost Unstoppable Isaac self pick up the object and then bring it up that we created in simulation and we also robots to be able to do. So as soon as it sees that the object has moved

her weight and it comes closer to the object and try to grab it goes down again and then off to work right again, if a situation just when is about to close Have to go back. So what is it this way to the left is a soda from there. It was fixed between cruel to hold on it. So I'll show you life next few months just to show again. That's his notion of grasping and stabilizing can also be applied now in conjunction with stabilizing the full body. We see here. Robot is holding a box. And by the way, this is not very

heavy but it's heavy for you robots in in comparison. And so I'm not in the Ninth Configuration without a robot is already stable before the robot to destabilise itself. And we also apply something to Visionworks. The boxes is being held as you can see here. Photo more we can we can just test also the ability of the robots to to to adapt to different configuration of the Box itself as a box is being moved just before I dropped it and I think it's about to go on interesting here as you can see is that when the box was too

far down the road, but even if it doesn't line search, he will lead us to an invisible posture ourselves because it will be outside. It's on its gravity and so it will not be able to reach that. So we moved ones that are we included the walking and navigation Jeep what you've seen before. And then the way to come by and visit with ovulation, we have the assistant that you've seen restaurant in simulation before that the robots can a dad. That's what if we move the box

and go to the boxes and close a hand the robots here can handle carrying a box together with someone who's only older than $70 generate. Its release. I did come close to dropping the boxes or sometime. So we just don't stop here and I was just making a quick summary or all of their work that are presented here is available on our webpage Elm. And I want to thank of course all the people who walk behind. They're all the people who walked a lot of the catching up jet in Flight Cena and especially for the mother of the coordination of the all the running and manipulation draw

business was done by meow and and hand and an old is people well behind the control of the humidity in France with the humans with Issa. Thank you for your attention. I will go to come? Sorry, if you could go to go to the Mike email Pathways that be great. So why am I waiting for people walking up there? I asked you to have a question. So I'm done about this but I have seen a lot of work. I'm using deep reinforcement learning particular learning country policy

into end using visual images and the radical to a controller and my impression from a Wocket. This is not a typical approach you would what would you contrast this poach? I meant to highlight your philosophical thinking about general direction of the alarm. Alright, thank you for the question. So it's true that song 1/4 of the way we try to learn from spot later and later. So in order to do that, it's it's really difficult you to do that because you will need to do simulation and a lot of a

required to actually have control over the falls in the friction. So we need to review a baby. That's one thing that I have just as much as possible to have clothes for men and completely understandable model after of the Dynamics and what is has been wrong so that it's fairly easily interpretable in a way back by construction. So that's that's maybe the major difference but we have used the reinforcement learning in in in some cases. We did not see for the type of person that we have the needs to to go to two very complex and and Steve model because

often in fact, we already know what is the control system that we need and what Missing is a dynamic surrounded by people. I wanted to ask how you would handle an even more difficult problem where you want to catch like a fragile object, like maybe an egg that you don't want to break. Yes, it's a very good question. I don't have the supplies here. But the one thing that sweetie blocked because I got a problem with that when we catch the object. We actually stopping so the interaction forces chief of Throne so but I do not feel that it's here because it's so I told you that we've

developed was to to actually move with the Dynamics of the object for sometime. We have probably said a couple of years ago and they were something because it applies also to to space so you put yourself if she wants to know that I object used to remove with the diameter of the object. You make it first Contact and then you slowly you'll see the object and the objects that you really reduce the forces at one time. You try to be able to control for the first when we're catching, but the problem is that we just don't have the time so few minutes ago. That sounds even if the hand is super fast

and there was no time to actually change all the infant of compliance chances are also that we would not have the time to read. So yeah, just astrology to to fly to move with your braces is fine and then you know absorbed in the body and the motion of the body creates out of compliance which diminishes the contact force. Thank you, very interesting and impressive book one question. I had was that the usual examples of bottle and a tennis racket. So is this some learning that you can transfer from for example, but you're learning about the bottle to the tennis racket or

are they like in Oakland or to be trained right from scratch? If you mind reminding us closer to the mic and Reed radio from your pressure? If it's winter, but we could not hear very well. So it's a question how to transfer and then you try to say you don't do any training for the tennis racket. How much does it still work in lightning? What are the parts? That will not work? So they're all things that do not transfer SOL SOL for infants the running of the Dynamics of flying objects. We stole that, you know, the importance of the bottle of water which is how steals will fly completely

different from a bottle of water which is completely empty and so on so we could not find some directly you can whoop strap a little bit but it's not so sorry. Unfortunately. So that's that's a weakness for the other stuff and going on a cylinder has too many different objects. And we also learn a different bottle and smaller bottles. So there is some but not all of it. I think Indians indeed one needs to see more more object. It's something that I find is a missing point in the war that we have currently

is really to be able Turn off read the complete knowledge of a database and then rebuild from one to the other that is really something I would love to have. It is billeted lifelong learning and going to ready many years ago and you don't have it. It's not not so easy. So I thank you for the question. I think it's a little bit of generalization. We have something on his vision in sampa. I still believe that's one can internalize for everything but how to Dan build really A large group of knowledge and then pull from this no lecture at one time for different type of Steel for

different computation. That is something that is still a little bit missing. Don't forget to change the behavior for the new environment only has changed a lot of disturbances. Curry shoot hoops infection robotics that actually just felt so much the robot in bump into them first what they have in the hands put more liquid glass Etc it since it's a filly Colt train set up OK as moving and so I think this is a change in the environment condition to which we are fairly robust, but

not completely. I mean very very different changes what we have done because of the stability and the vehicle frame of reference, but just so I don't know what's what you have in mind in time of change of environment. You can discuss it offline. Thank you one question for a quick question answer hopefully. Tasks and problems like I mean many animals and humans, of course, they have solved this right they can do that very quickly and catch things. I just want to hold Russellville. Is it that biological systems work like this

actually have funny videos where you throw a bottle or I mean you can train yourself. So Superior people walking in circles and different each other and they know who to expect if the nice to come up, but if I just now take something and I'll throw it at you through a very little chance that you would catch it. But the G, which is Young Living Prodigy. You may you grab it with whatever you can do know, it's not what you and can do what you want to do after love. What's all them and I spoken

silly little about that on purpose because I've given the other talk about steinecker system in machine learning. So often that I found it to ship the topic but the ending song in this other approach him in everything that we do is a dynamical system control system stability. So I cannot say that this is the way that the CNS control the central nervous system controls humans, but certainly an approach to explain human behavior phone on phone. So it is close to the way that human control of

body of animals. In fact, it makes sense because it's a lady free us from from time, which is what we're doing on a regular basis is that we don't use it. So we just use relative position relative movement from one person to the other question. I'm sorry for the essay that there is no images explanation of how the human body works, but you can see the inspiration from from it. It's amazing to think out dogs. For example, you can catch something and it don't have a 3D tracking system with infrared cameras

and all this stuff. So that's pretty amazing. This has been agreed to talk and discussion. Let's thank the speaker again. fantastic