AAAI-20 AI History Panel: Advancing AI by Playing Games

Three of The Five Percenters was so slides during the wouldn't have remarks like in the beginning and then close the laptop and I'm just have discussion. Hi. We need us. I thought I did. But we are going to start the computer up to you. So we can go out there. subway ride today I've always. restaurant I just took out both of my bullet points to make it feel like a presentation. I was trying to find it. I was I was hoping it wasn't when I looked around and didn't see equipment.

is a red. Scream because I'm going to introduce everybody first. I think I'll introduce each person before like I would like to know what time is it? I think I will begin everybody ready. Do I leave everybody? Can you hear me? Is Mike on? No, yes. Yes. Thank you. Okay. Hello. Welcome. Welcome to this year's AAA. I history panel. advancing AI by playing games as it happens. It's an appropriate time and place to hold this panel as it is the 50th anniversary

of the 25th national meeting of the ACM where the very first computer chess tournament was held So not surprisingly that tournament was held in 1974 have surprisingly it was held here in New York City and more surprisingly still it was held on the third floor of the New York Hilton Midtown. In 1970 the New York Times reported on that event, and they said quote the world's first major chess tournament played by computers proved that Masters have little to fear from machines. And that not

even a million-dollar electronic Marvel is blunder proof the computers in their shortcoming shortcomings. Excuse me. We're almost human. So we are here today to discuss our achievements as a field in the area game playing since 1970. I by the way, I'm Amy Greenwald from Brown University. I am very fortunate to be joined by such an esteemed panel. So from left to right we have Marie Campbell from IBM. Garry Kasparov Kasparov help me out Jets Grandmaster Michael

bowling from the University of Alberta David silver from deepmind and Kitano song from Sony Corporation. So today's panel operate as follows first, the panelists will present introductory remarks in which they will tell you their stories about mostly their crowning achievements in one case, perhaps disappointment his computers playing game and after they present I will ask the question of the panelist the series of questions mostly pertaining to the relationship between success and playing games and

achieving our disciplines Collective goal of building an AI. But before we begin sorry, and then we will open the floor up to questions from the audience. But before they begin I do want to acknowledge several other esteemed researchers who are not on our panel today, but who have also made worthy contributions in this face. The first is Jerry to zarrow who is best known for developing TDM in a neural network that learn to play backgammon at human world championship level in 1992.

TV Peter gammons showed for the first time the great power of combining reinforcement learning with general-purpose nonlinear function approximation and stochastic search indeed the deepmind team who developed alphago expanded upon methods that were first used in TD Gammon. Second. I'd like to acknowledge Jonathan Shaffer. Professor at the University of Alberta and the person who told me the story about the first chess tournament. My colleague Michael lippman has called Jonathan the prince of

human-computer competitions. He's best known for his Checkers program to know which one the human world championship in 1994 and 95. The program was retired from competitions in 1997 when it was clear. It was superhuman but shaper continued running Checkers calculations in the background until 2007 when he was able to announce that chick Checkers a game with a search face of 5 * 10 to the 20th had been solved. I'm finally 12 my stands home and Nolan Brown in 2017 their program Lee broadest feet the top humans at head up. No limit, Texas Hold'em

a form of Poker and then in 2019 their program pluribus beat the top humans in multiplayer poker. So if not, Of note pluribus was the very first super human AI milestone in any game with more than 2 players. So we'll begin. Marie Campbell immediately to my love is a distinguished research staff member at the IBM TJ Watson Research Center and a manager in IBM research organization. He received his bachelor's and master's degrees in computer science from the University of Alberta

and his PhD in computer science from Carnegie Mellon University. He was a member of the IBM team that developed which was the first computer to defeat the human world chess champion in a match. He has received numerous Awards including the Alan Newell metal for research excellence and the fredkin prize is an ACM distinguished scientist and a fellow of the association for the advancement of artificial intelligence. I'd like to discuss the history of James in the eye by giving for perspectives on how it's progressed mostly focusing on computer chest.

Show me start with this graph. The blue bars are in an indication of how popular games have been as a vehicle for a I restore you can see that over the decades. The number of Publications that are referenced. Both a on games have increased dramatically topping 70,000 in 2010. And also even more interesting is that the Orange Line shows the percentage of all AI Publications that refer to games and that stayed apart from a flurry of interest at the very beginning of AI. Stay pretty concert in the mid-to-high teens percentage-wise over the

decades just papers to refer to games actually use game system is better a substantial fraction of the new deal. Do IBM also has a long history of using a game for a i the very first working program chess program was developed by Alex Bernstein back in 1957. And in those days the challenge wasn't to get it programmed to play. Well, it was together to play Chess at all. The computers were very limited in those days. One of the very first demonstration to machine learning was done by

Arthur Samuel in in the game of checkers and he develop mini foundational forms of a rushing running back in those very early papers any already mentioned Jerry to sorrow. I think the thing to note here is did this work was done in 1992, which was maybe 10 or 15 or 20 years before one might have expected that it would have happened. The next picture is is myself a 23 year younger version of myself sitting across from our other panelists. And this was the

of course the the deep blue match in 1997. And of course the Watson system was able to tackle a completely different class of games that involve language and defeated the top human players at the game of Jeopardy back in 2011. So this is a a graph of Chess ratings over time and there is there's one big story in this and that is the green line. That's how computers have progressed over the decades. The blue line is the rating of the top player in the world

at the time and the red line is the rating of the average of the top 10 players in the world is the one big story but there's a bunch of interesting little stories in the picture as well. First of all at the the far right of the green line in recent years. Some of that increase is due to the work of David and other zero and interesting ly you also see in the blue line. Round 2018-19 World chess champion current world chess champion Magnus Carlsen has been using the output of Alphas gyro and I'm looking at its games and studying them and learning

from those games and he's able to in a recent article said he's used that to increase his advantage over his his rivals on the blue line. And that is that there's a big gap between about 1985-86 or so and all the way through 2005. There's a substantial gap between the top human player and the average of the top 10, of course Grandmaster Ted Burrows. Was that blue line for almost dead entire time? And I would expect. And I will let you know in a particularly large gap from the field in the late 80s early 90s and maybe a question for a for him later

will be Kenny attribute some of that Gap to the use of Technology will will ask him that later. Okay. So now I have just a few quotes mostly referring to the computer chess that help us understand the question why have games been interesting for artificial intelligence? First quotes from Claude Shannon who wrote a very famous paper 1950 games will act as a wedge in attacking other problems of similar nature and greater significance of this is back in the pioneering era another interesting quote from that time.

We didn't 10 years a digital computer will be the world chess champion. Unless the rules bar it from competition. This was 1958. So wrong for two reasons one is that you got the time frame completely wrong and the approach the imagined it would be used to eventually when was none completely unlike what they thought this was time in an hour annual founders of AI Turing Award winners Simon with a Nobel Prize winner. So they were completely misunderstanding. the state-of-the-art in those days I won't read this entire quote, but if it gets gets quite interesting that

researchers in those days were really thinking that chest was a great problem for stuttering studying how people think about games and using the same techniques planning reasoning perceptual abilities and so on it turns out That this is a quote from Jonathan Shaffer. Who and you mentioned and and John's Play It's perhaps unfortunate that came along so early in computer chess history what they're saying there is that Alpha Beta which is a relatively simple algorithm doesn't require a lot of the techniques that Alan Newell imagines would be needed Hawkins worked incredibly well in

chest and that was what the basis of the kinds of programs that until alphazero. That was what caused it to reach human level human Grandmaster level and then superhuman level. And finally, I'll mention a couple of quotes. I won't read them in their entirety, but you can see there is a new era of optimism. There's been several fantastic pieces of work in the past three or four years that are focused on using games the various sorts, including chess to make progress and and the

optimism is that these techniques will lead to Major breakthroughs in other areas of a I will be providing material for some of the debate. We're about to house. Thank you. Thank you. Marie. Next we will hear from Grandmaster Kasparov. Kasparov came to fame at the age of 22 as the youngest World chess champion in history and 1985 retaining his top ranking for 20 years his matches against the IBM supercomputer deep blue in 1996 and 7th. We're key to Bringing both Ai and chest into the mainstream.

He created Advanced chest in 1998, which is a form of human machine collaboration in which humans playing a game of chess are assisted by programs. Te-de cascara is chair of the human rights foundation and on the Executive Board of the foundation for responsible robotics. His latest book is deep thinking where machine intelligence ends and human creativity begins. I hate being treated as a victim for many years the blue matches that I want. The first one was a curse or blessing

20 years ago. I would probably say course now, I believe it was a blessing because it was inevitable and I was very happy. I'm a lucky man to be part of this great experiment. But since you know why I cannot compete with the system panel on just in computer knowledge so I can probably tell you the human side of this of the story about mistakes humans made from the greatest of the mentioned them from very early days as fascinated investigative intelligence like Alfred Binet IQ. Find out how the Braves of chest when he

would actually unlock the secrets of human intelligence. I always say that it's it's very flattering. But let me tell you it was out here setting my steam college for mechanics chess players that definitely my case is nothing else that the applicant for playing chess, but it's also not surprising that the chest attracted such Geniuses as Alan Turing North wheeler and Claude Shannon and many others they searched as as as a great opportunity to up to make machine stinks like humans. So

for them, which was not a question if but when machine prevails in the game of chess, that's the dawn of a new Europe didn't know the truth. Baseball batting machine to win to beat oxy with pliers. It will have to play like humans do diddly. Brute Force could be successful because you go computing power in their hands and they knew that test was mathematically and Iran sous-vide complicated on water call Shannon describe the type B machines the machines that will sum of a mule a human cognition was a big mistake

because definitely be blue was enough. It was a classical type a machine as intelligent as your alarm clock very busy $1 a piece. Because what we didn't understand show or I'm just too late. Is that for machines 232 to surpass humans. He doesn't have to fold the game. All we have to ask is to make a few mistakes. That's if someone would talk about driverless cars or about you know, where Google translate so we are complaining profession. He doesn't exist or we'll have to ask East for machines making fewer mistakes

and humans, even the best of us a push to make mistakes and you need a machine that way a little benefit on this mistake was the blue stronger and Garry Kasparov back in 1997. I don't think so. I totally disagree with this rating. I think that's the only way to compare objectively when you look at the computers now and you know when people ask me about today, And that Speedy the diagram shows today the difference between Magnus Carlsen and stock fish that you can buy online and install your laptop the

difference between all these machines that you can just use on your laptop and current world champion is about the same as a golden Ferrari. To know any interesting to see the phases of you know machine learning and it is an ant and charging human at first it just you know, it's it's a it's a joke, you know, this is not even machines can compete but that week and then the third one which is the way I own on the history timeline. He's he's just one. It is a short one. It's a window competition or machines

are strong enough to compete humans are still strong enough to resist. But again, it passes out very quickly and then machines are much better forever after going back to this to this to this mattress. And then we we can look at these games analysing or analyzing them with our guests just asked today. I'm free chess app On your mobile device now is stronger than a blue. Okay, that's so that tells you ever think about and Ward law and tells you these games. It's not about

machines being perfect and understanding how to play It's All About Lucian Circle level 100 human including the best of us will be to make the mistakes that machine will use that's what's happening in 99 to 7 again. Could I play another match and probably know I stink. I think that's a few more years. I was in a blue-eyed play to all the matches with other machines like the new deal with the priest boat ties. But what I knew from 1997 what's next for my game for my for myself? I recognize that it would be just a Race Against Time.

3 years 4 years 5 years, but are you just it was over there now? It's because people say why I was overreacting when I was pretty blue the match to computer. Esther but speaking about machines defeated humans. We already had these unpleasant experience chest sore chest engines like breeds and others are showing 92-93 choose them lost. Someone many more then I played rabbit bird 5 minutes 25 minutes chest and Oso y Los Dinos imagine 1994 to Days Inn in Rapid chest. Then I came back to this computer. It's his job and

we leave and impression. That's the only reason why we were losing to these illegal business because the game was Queen 5 minutes 25 minutes to see if we had more times then of course, it will be different because more time for us more time for the machine is getting better. Eventually it will it will go to the same equation so I can understand it. The movie not only for us but for machine as well, so that's why now 22 years late, but I could probably resist better. I could play brother by the magic, you know, when

these games by having your chest out today. So and this is the moves that was celebrated in 1997 as the product of a genius human genius and silicone Gino's in 30 seconds. Fish or any other problem will will tell you so big so fast that is a couple of games that will just you know, so that I don't like in 97, I would believe that it was its they will tell me when 30 seconds that it's Gary. How you wish to conclusions after 1997 that is it's not about us against machines and she's so that was my idea

Advanced chess play against other human + machine for future with shorts to understand how well so we we found some interesting things you notice that explains. What is the best ways of hummus in collaboration? And that's why when people say so this is whatever is that the general public is a skin. Maybe you will be comforted by some of these great minds on the panel, but I think that is wild the general public thinks today is Windows 10. I feel it's early days.

From where did Whitney's new booty in the direction, but it's just this time so it's so I think it's important for us to stop, you know feeling that so that's why I tell people I think it's 6:00 this but it's rather call it intelligence than artificial intelligence for two reasons why it sounds friendly terminate shows on Netflix and Killer Robots and other bad things. So not to lose your sleep to oversee of technological progress. It's a better reflection machine collaboration.

So because it's why gays Adobe men against mention to you, why do games or nothing at all just in a demonstration of machines and that's what I believe in every closed system machine will eventually Prevail. So the question is that where the machine she never could do, you know could move information one closed system. That's getting the stuff for me to discuss. But right now it's a let's agree that every game even such complicated games has Texas HoldEm Poker. It's still a closed system. It

always knows them better if it makes you a mistake and sleeping mouth, Michigan Lottery. So why then why don't we just you know use our adventures we can dream and it's of the past hour. I've been a stronger or faster Oriole just for instance like telescope. Is it outside in a way to make this intelligent machines make us smarter. So I believe that it's okay. It's a promotion potential promotion so we can different upgrade on software. So that's what's important is like was a telescope if you put them

in the ground will not take machine learning and new computers and technology has blessing. Thank you. Thank you. Next we have Michael bowling. Mike is a professor at the University of Alberta a fellow of the Alberta machine intelligence Institute and a senior scientist said deepmind Michael leaves the computer poker research Group, which develops counterfactual regret minimization and deepstack arguably the two most significant algorithmic advances in poker AI. Michael also orchestrated the

development of the arcade learning environment, which is a ubiquitous sweet of Atari games for benchmarking reinforcement learning algorithms. And Michael's research has appeared in exhibits at to Smithsonian Museum several of our panelists have had art exhibits or exhibits in museums. Anyway, so I wanted to mention that Thank you. It is daunting to go after Mister Kasparov. But I guess I don't have a choice in the matter. We've been talking a lot about Chess and often. We associate chess we think back of chest being right at the beginning of

computer science in the minds of people like Alan Turing and Claude Shannon, but there were actually another game in the minds of some of the foundational fathers of computer science and in particular, I'm thinking of John Von Neumann and the game that bugged him was poker and so in the 20s John Von Neumann started developing, what would become the modern Game Theory as we know now and in that Seminole book that was published in 1944, there's an entire chapter on poker and bluffing and what I think was trying to do was trying to really come up with a

mathematics of intelligence the mathematics of rational decision-making the mathematics of how we go about making decisions and that's what he was fighting to get all mathematic self. I think maybe it's a little bit hyperbolic to say that his shift in later in his career to looking at computers could hardly be said to be about taking that mathematics and saying if I'm going to make it real I'm going to compute it more than just on paper at the size. We need to make the kinds of rational decisions. We need to make I'm going to need Computing devices to do it because more than we can do on

paper and I really think that there's some Colonel of this this this game of poker bugging him going all the way to we need computers and building the models with first computers. Modern maybe more modern reboot of what we think of AI and poker right now really started in the late nineties though. Mostly chest took over the world is being the game that we were going to measure a I-4 and it wasn't really until the 90s pretty much after deep blue. There was actually a graduate student DARS Billings who walked in the Jonathan Shaffers office

mentioned Jonathan already and and basically started try to convince Jonathan but the game after chest was poker and so it's a why was that what is special about poker soap, so hopefully there's the obvious thing if you know anything about the game, which is I don't have all the information. I need to make my decision. I don't have the other players card, but that's going to be critical to making the right decision and that's very different than what you might see in a game of poker where all the information is on the board in front of you but it's another thing to think about when you

realize that you don't have all the information cuz you don't know the phone it's cards you have to remember they don't have all the information. They don't know your current and when you reason you have two equally reason about that as well too, and this is what makes me want to One of the reasons the game is very challenging notably poker is a game that does not just have a win loss or Draw out come and actually has a sort of continuous outcome of how many chips are actually changing. Can I add some complexity of the game as well? It's also a game that has chance involved in it, which makes

it very hard to evaluate programs the difference between a professional and amateur player. If you really wanted to specifically separate those things might take thirty thousand hands of game three thousand planes of the game to figure out who's actually ahead. So that adds another challenge in addition. If you want to actually try to please play much like we would want for chest that is a program that can't be beat then we have another problem that is if we're going to actually measure that it is harder when you have programs that are trying not to lose chips not to guarantee that

they're they're winning that this idea of just playing them head-to-head doesn't always give you the answer is easily as it does in the end games of perfect information like chest and go Social starting from that I think really momentum started a few years later with the annual computer poker competition. So it's almost hands home at at Carnegie Mellon University and and my group at the University of Alberta started a essentially competition that had lived a AAA high for over 10 years this competition involve dozens of many dozens of entries being submitted every year involve

the workshop re-shared ideas, and I actually can't think of a better instance where you can watch a community building up around a particular scientific question and making massive progress in a short. Of time. Is it so what happened is the result of that massive progress in 2008. We had the Milestone that Polaris was the first program to beat professional players at a variant of poker at all because the game heads up limit. This is the first time we had seen a computer beat humans add an imperfect information games that they played in 2015 in a similar pattern Maybe dejana. Schaefer

my group who developed Polaris also went on to develop cepheus which attempts to solve the game of heads up limit. Do you want to go further? We want to have a program that could be guaranteed not to lose no matter who it play it again now to do that. There was some approximation involves a what we could actually guarantee was it was so close to perfect that human in its entire lifetime of playing against it couldn't figure out that it wasn't perfect. And that was the bar in 2016/2017. We have the release of both Labrada Soviet mention out of Tomas and homes group with known Brown

and we have to teach that coming out of my group that we're both programs. There were able to be professional players at heads-up no-limit, which was the game that has now many many more actions in front of it compared to the game of heads up limit. But you also again one of the challenges is being able to interpret what do the other players actions means when they make them. And in 2019 we heard about pluribus that took similar ideas from those and went on to actually be human players at the multiplayer game. So what's actually maybe a little bit interesting is when I go

to those Milestones there was actually for science papers out of those Milestones coming out of the combination of dr. Sam Holmes Group in my group, which is a little bit more than go I think brush it was exciting about that been pointing to a feel like But I can swing at that and say what's really exciting isn't really those milestones at. All. Right, it's not it's does it really matter for us that we have computer programs that play poker better than human prayers and answered that it doesn't matter unless

we learn something scientifically from that progress in this is leading toward something else, but it was really exciting about what happened over this a 15 year. From the beginning of the computer code for poker competition and now is what science is actually happened. So the counterfactual regret minimization algorithm was developed specifically with poker in mind and this is an algorithm that they able to solve imperfect information games that are thousands of ten thousand times larger than the best methods that came before it and not now this isn't anything about Moore's Law.

This is accounting for Moore's Law. I was able to do that. We also have the idea of decomposition. It was originally thought that it for imperfect information games. You had no way of breaking the game up into smaller sub games the way you would search from a particular position and analyze the position chess. There was just no way to do that in poker with me now actually know how to decompose imperfect information games. Particular poker not the basis for things like you and I also have very introduction techniques if we're going to actually validate these results and get sadistically

significant inclusions. We're going to have to do it with very little data. We're not going to be able to run thirty thousand hands as we've actually developed methods that can do so in ten times fewer hands and a totally unbiased way. And in fact, these ideas are generally they're using policy gradient methods in reinforcement learning. And so some of this came out of what happened over the last 15 years and poker, so I can't think of a more ideal setting to say this is what a game you can do. It can crystallize a bunch of researchers to care about a set of problems and then make rapid

progress over short. Of time in poker is a perfect example of that. Thank you. We'll need the projector back on. The next we're going to hear from David silver a principal research scientist a deep mind and a professor at University College, London. He could let the project that combined deep learning and reinforcement learning to play Atari games directly from pixels. And he also led the alphago project culminating in the first program to defeat a top professional player in the game of Go and the alpha zero project which learned to defeat the world's strongest

chess shogi and go programs. Most recently he co-led the alpha star project which led to the world's first Grandmaster level Starcraft player. His work has been recognized by the Marvin Minsky award the Mensa Foundation prize and a Royal Academy of engineering silver metal. And as far as I know, there's no gold medal. I think that's the top country. Thanks, baby. I must say it's a real pleasure to be here. I thought it'd be fun to talk a little bit about the history of alpha go and find you to

tell them about the story in a way that maybe people haven't had in the past like to begin with I think it helpful to take a self back to understand the context in which Alpha go in the question of God was situated. So we already have but the 90s wear on some samsa a golden era newest exception in in games. So many of the The Matrix Games such as chess Checkers backgammon a fellow of the royal cracks in the sense that are people able to build AI systems that could defeat the wild champions in all of those programs respectively. There was one game which

remains stubbornly immune to the kind of approaches. What should be used to know about the game of Gaara and it turned out that the domestic set technique to which is being used in those other games since he didn't stay up to the challenge of God. I could have a difficulty about be just a normal human difficulty understanding. What position is it even possible to understand what's going on. When you see this mess a black and white stones me to tell you how to evaluate that position many people to really feel that maybe going back to the any questions. Like I'm

Simon and you'll but maybe go really had this stick that if we could make progress in understanding what was going on in position already cracking this game that would really represent a set forth by I the creation of which was that 1.4 million-dollar prize. I thought it was put forward at the challenge for every year up to the S2000 when expired I know competitions where the best computer guy program the winner of the phone about stopping and the price will be given to the Fast Park times to feed a human professional Chi flat.

No force me to go program for that time when quite off of The Challenge start a fact even up the year 2018 actually put forward a nine-year-old child to play against the strongest guide program and the enormous 9 Stein handicap to play against that child and still lost. In fact this position here is an interesting position sign up there cuz it actually was played against the strongest program from that era buy a computer expert Marci Miller. I keep played against that program. What is 29

* in handicap undefeated? At least if I let someone playing to play against the human chest that would make you one poem. I got an appointment with all the pieces. so that was kind of context in which in which the challenge of God was situated. Was there a way to make progress and understands in a way which would mean something to all of II? I thought that led to two revolutions which athlete made significant steps in the abilities of computer guy prank programs. The first was the multicolored Revolution don't 2006

Monte Carlo tree search and the fundamental Innovation, which was drawing on hold right is actually dating back to Northwestern computer guys really were that even back to you in a way I can in Backgammon what this idea that you could evaluate a position by the outcome of random games that you can play many many random games out the end and average them the back if you an idea of what you going to win or lose an opposition. I'm not inside let you go to

small size of go up to human Master Level since we were trying to make progress here. But somehow it plateaued a plateau. Level after 19 by 19 go program for stealing able to beat human. How much is great progress professional level up affect. I'm still far far away. The second revolution. I think it's Familia too, many of the people in this room. Which of the deep learning revolution revolution. I mean decades but in some sense in one moment where where things came together was the kind of image neck my back in 2012 and up myself and others

as well if it's possible for a machine to understand images in a way which it can look at those images and take that very complicated and I more questions try not to stand something. Why not? Why not? Why is there something fundamental about the nature of God which means that we can't evaluate it is something which is special to the domain of human knowledge or is this really something could be tracked both machines to Say that question led to the birth of alpha car. So I'll think I really started as a

scientific investigation into the question of whether a deep cleaning system could really understand the nature of the guard position that was started with myself an intern Chris Madison trying to kind of private question and one of the initial successes what was that? We were able to to build a system which was able to actually predict. What is human might do in a position in a way that let us to think that the ask me that question valuation Watts. Now now I'm something could be addressed. So the second revolution is some sense was being able to

evaluate Sky positions Note 3 random Behavior about through my eye which Israeli leninghausii understands that position. Challenge that led to the staff about the car which was like, oh good project started with a bat and the bat was amongst the team members whether or not a neural network without any sense whatsoever could defeat a showdown level human player. I thought that stimulated I will get back to you a lot of research and we finally had someone in the company who joined us Showdown level fire and challenged the first batch of

Alpha Chi to a match and he became the first victim of alphago and as a consequence had to dress up in the passion of traditional type last 24 hour. now meaning for word from that I once we saw this result felt it was really just a matter of time that we if we had the ability to evaluate positions. I'm in a way we could actually defeat level class without any set from top of that. It felt like an inevitability. At some point we would be able to scale

out. There's method to the point that we would be able to beat talk to you in class. And so if it was to be in baseball right height to be a house that would have to go and do it cost to pull together a team to do this witch lead. I'm in a few months time to us defeating the European world champion found play. I need not to that point. Let me talk to challenge the well Champion. We had to try an estimate and think to myself based on how how I progress seems to be accelerating. Should I make an estimate at what point we would be able to

challenge the world champion and have a chance of defeating them. I'm 5 months later that we we're in. So I'm competing against the 18 time world champion. Lisa doll based purely on looking at the gradient of a curve of all right evaluations. I'm trying to predict whether that would be enough about to go and beat the amazing world champion player and I felt confident until arriving in the room and discovering that the entire world. Press was that 100 million people watching and suddenly started to wonder if maybe maybe this after you could turn out other

than how we imagined United pressure was a man supposed to be even more. So I want only $2 but he was really a tree well champion and there's something really special turn off The Hobbit 3 while Champion with us, but there is something amazing in their ability to actually competing for a high-profile high-stakes one would like this to try everything. I visit all tried everything he had this amazing ability to push alphago in every possible direction of probe for weakness is kinds of situations. We

expecting in in the second gave me seven became known as the moment when computer has started to show something creative. I mentioned an idea, so if we could just get some sound the sound okay. Well, you can just leave from the facial expressions. What what he saying that what he saying there is I thought this was a mistake like this is a very professional, 27 I'm looking at it is absolutely convinced that this move that I think I played with a mistake that he

thought that that somehow something is going wrong that made you was looking at the wrong move and make it humid in correctly attempted what it was playing because it was Unthinkable to human play as it broke every convention that human guard class knew about in in a nap late according to you and yet it turned out the key moments in the game that went on throughout the night win the game but change the way that the human go fly us think about the game of Go finished and we ended up winning the match full games to one piece it all go flat. I'm in

the audience and and he was crying and I thought all this is bad weave. I'm really upset someone and I thought I should go to talk to him and Text Amy said he was crying because he never expected to see something. So beautiful as the move to Africa played in these matches and I just I guess I have a hope now and Anna. Let me type is Hope which we can have that same feeling elsewhere and II that actually, you know, one day we can be creating that kind of beauty and domain for all of us can appreciate that. Just put in an

hour to spare if you go past and understand what's going on then. So I should also say that alphago wasn't Fina 100% success in that they also had issues and one of those issues are something we cool delusions inability to understand domestically sometimes for No, 10 20 50 moves in a row. It was just kind of Miss understand what was going on in position little bit longer. I wish I could if I could happen that made me want to the end of the journey there was still mostly could be done to the next chapter announces are ready for me was about going back

to First principles and asking if it's something which all systems don't understand despite our best abilities to kind of put a hold of human knowledge into them fire supervised learning and then build on top of that with with me for that and I said, that's what was missing. Principles and taking out all of those extra pieces we put in there with you and thinking that we were helping by taking one of the principles. We allowed the system to you discover. It's our knowledge. I correct rights are not about Ashley corrected all of these delusions and let too much much

stronger chiropractor in addition. Yes. It was also something we could then because it was General a general pests. I'll be able to apply to other demands. So he played against you know, what is a fast times on a computer program ever chose to sacrifice. A lot of material of the alphas are giving out coupons and the roofer Bishop very very far behind Dakota. And yes, it's completely stifle. Black as night moves with music at least friend likes in some of the systems are based

on on more direct set ideas. I just finally wants to close with you know, where we went off to this which was more recently the last chapter in some sense musiro has just recently released. The idea was to say ultimately we have to go back to these questions that they found his way are asking us in setting us which was sold games in a way which has meaning for the real well, One characteristic of the real world is it it's a really complicated place and we don't really know they

have access to the rules by which the well. Rhymes and if I do a microscopic or macroscopic and then achieve their goals within that has the model for itself in such a way that that model contains all the knowledge that it requires to be able to plan ahead and sold them for whatever else I was able to know any of the same kind of stupid human performance in chest and sodium go clean it out. There are also something which we could end up lying and Mike bowling at Mansion Del Villar vitari enough to get time to Central HS

take the day off and that's the main. I just wanted to finish because I did mention number of major papers by pointing out cuz I think it to go off to him. I have to advance his hair. That's how it turns out that I'm too out the pipe major papers were published by ourselves on alpha0 which used it to. I think I cracked some other problems. I think this is like a puppet on a midnight run on using officer to crack retro synthetic chemical Pathways Quantum Computing problem speaking of all the states of the often in

Germany. Thank you. And finally we will hear from your alaskey Gitano who is president and CEO of Sony Computer Science Laboratories corporate executive of Sony Corporation ahead of Sony Ai and professor at Okinawa Institute of Science and Technology Graduate University. He is also a founding president of the robocup Federation past presidents of international joint conference on artificial intelligence is Sky and past member of the AI and Robotics Council of the world economic Forum. He received the computers and thought

award in 1993 and was an invited artists in the Venice biennale and right here at MoMA. World events so far has been talkin about what they have achieved and the challenge in action, which is 20 + years ago. We thought about what is the best way to push the state of the oven Ai and Robotics? We thought about a guy many possible topics. Can be challenged and brainstorming about it. And we decide. The soccer game game of soccer age that probably best way to push. In the reason why we decide to do that is if you look at the

problem that we see in the 20 to 30 years in prison or probably even the 50s by pricing. We should it have the very concrete ball and Jacks even that go to provide immediate and sore throat on the Middletown feedback food industry and Technology as well. So that's why we Define the initial Target to be by 2050 Bill Totino. Young in a game of soccer and the definitions are the moon and the reality is the fire chief in the way to achieve this. We should have more technology more discoveries and

that will feed into the society. And then I said like a sucker was decided after through the Constitution and the what will be the important area of the AI and Robotics to be used in the future so I can buy an Audi 90 we envision. What is the most important Ariana and Robotics 30 years from there and probably like a 2024 which is about time now and also the game between 50 and I we come up with you and service for about an elderly care. I don't have the area that we thought when I first

issue in creating a robot and AI system to be able to achieve the high performance in the area is incomplete information and annoyed. And there's no obvious correct answer to complete the information about there's no noise and address and no it's not taking games. So is the RoboCop represent very different kind of a challenge from the water having a party test and go has been pursuing and so bloated AT&T Reward task in the old to be the real peaky a discipline

that need to be integrated and where and how we can apriso get you no idea of having a robot play. The soccer game is held in Nagoya as a part of each guy conference in 1997. pause after deep blue caplet which advocacy is confusable, which is quite a small field and you see the video which is embarrassing and everybody's not moving at all because we going to have a real fox soccer World Cup game and CNN BBC NH gave everyone, you know, she's trying to broadcast internationally and then they

asked me Mystic Tan, oh when the game going to start. I said I got we have 5 minutes into the guest. So this is really interesting that I say is like a okay. I'll be your weakness in something very important. You should be cold. This is how we have a range of weeks of a competition competition driven make it work and uncertain environment in work. If you're very difficult State. We have a rock shop and discuss what you have done and then we got the poster paper and even if we disclose the source. In a mixed media, you know the Dirty Mac Show

usurpers the winner of the previous year championship. So that means we make sure the progress of sharing with a competition that makes it very interesting now after 20 years and this is the final game of The middle-sized League last year against the German team and then a Chinese team. You can see it. I can barely real time and dynamical, you know, passing around and formations and then the older and older what's the game started? You cannot touch a little bit unless you have to leave early movie or just like a DJ and everything goes now you can see that nobody said

intercepting and then I'm going to pack a bag and put it back again. So if you know, there's a continuous learning strategy and so we could change. And that's what's going on right now with goes to that guy. We have a few minutes with the range of these are not with us. We have to have like a sophisticated to the bottocks component in it. And that's very difficult. If a size that you know, when you played so the robot completions, it looks like a show but it has a little science behind this and every time you may have decided

So after as a mission after the completion we get together and everything we published and call a jiffy. He's the computer is out from the scientific contribution as well. And also as you can see some of the examples of hate the slime with the most widely used Slam in Los Angeles is out from the population in a which is like Amazon the bikes at the time he was reading the Cornell big red small size leak in their baggage fee and then, you know, we apply to small Do the Lucas problem and it not going to

distribute to the G6 a warehouse management system. So I think it's very important to emphasize if you look at the 1990 game with imagines, you know, this is what happens and half the people in your position alive and that's very important if we are getting into the local cup Virginia and receive many faculties decide to pursue technology and science. Because you happy at the same time. You see like a 90 or 2001 we have like a very first Global Capital risk you a simulation and the body

that was found in Seattle. Denali in the August and this is the first time like, you know, the rescue robot teams, bring the robot outside the lab and make it work and the bugging on site. S911 this is a foolish from the one of the risky for this fance professor Obi mouth is rough. ER in other time. We versus in Florida, but Cora ponds immediately after the 9/11 attack and the blows halibut recipe rub inside the base camp in the Midtown and it going to the Ground Zero keep the search-and-rescue operations for 3 weeks. And this has been that we have like a 200

hours of footage and and they are rescue professionals and make a notation. What did she be looking into and distribute to the older rescue people and that is one of the major contributions hydrocap rescue a maiden for the barely plowed with this operations. So what you're looking at, is she a tornado in between Carolina and Solitude and you know, this key operational risk have the new the boys will have to find out if they are Survivor out there on all them people cannot getting so deployed.

Don't just so you know Ground Zero. This is the screenshot. This is like a boat from the Fukushima. Number one be after 3 o Dai East Panda getting muddled approval into the ER to be upgradable in the radioactive environment. Do stop harassing what's going on in out near the reactor and did not deploy into the you know, the Fukushima number one on your side help Gregory into the containment operations. And this is also very very courageous actions that made

We are having every year we have like a national or Global competition. Also Regional and national competition. We see like a 400 teams are coming to the RoboCop like a 3000 researchers and the South-East became to maximum events in European. Open Asian open for National open in Japan open General open. So this is a global movement add one more thing as my personal you trying to push like a new challenge. I'm trying to Before Time you challenged for the AI to make a major scientific discovery.

Is Verizon is a minute and Addie's Estrada's cannabis packaging and in the restricting Bynum because you think we know you know games are in the bolded restricted and contrasting. It's a dynamic game and you know the time cycle of the decisions and perception is Missi second WhatsApp music and an app to damage and it's very important to physical the inverted game. What is a missing order is no real time and images and day is New Year's a long time Horizon deep deep deep thinking and strategic resource allocation an open-ended

problem. That is the area between the missing. So I tried to find out if there's any change in go and probably scientific discovery. Probably one of the most important topic. Ai and the boss can contribute in a suicide in the future is going to be a formidable, but I think that would be very interesting go. Thank you very much. Thank you all. That took twice as much time as I budgeted for. But we will still proceed at least with my question. First of all, many of us are aware that we feel that way. I've

goals are a moving Target right first playing chess at The Grandmaster level was said to embody intelligence and then when this goal is achieved Some claimed it was without any intelligence just required massive search capabilities. but even when chess was conquered go still seems Out Of Reach so that became the new Target but once that was achieved that was no longer a I either so what is a i and how will we know when we achieve it? Will we achieve it if we meet robocup RoboCop's goal of building a team of

humanoid robots that can defeat the World Cup champions. or once again will we move the goalposts, but I'm from Mary would you like to take that one? I could start here. So one big problem of course is terminology. There's no universally accepted definition of intelligence or artificial intelligence. So it makes it easy to argue both sides of the moving the goalposts statement. I think we're now becoming realized that generalization is a key part of of what intelligence is being able to take what you've done in one task learn from it and apply it to another Tab and

in that sense. Deep blue which generalize has within the game. It can solve tackle any position within a particular game is less intelligent than Alpha zero because alphazero can transfer to probably you could Define just about any two player zero-sum perfect information game and it would it would do a credible job at playing it. So we have to be really careful when we talked about these kind of things defining what we need. And if we mean generalization,

then it is possible to have very limited generalization. And I think we're moving in a positive on positive slope here towards more generalization in our in our programs. Devon Jones, very slippery ground because we are human characteristics to something that is compounded you one so we can look at the bottom and then you know that your intelligence is very cautious get it free to stay off of 0 was created. The Creator was a human Turtle. I don't need you can apply to the machine for symbolism creativity means you're looking for

something new septic the chance that you could be wrong and you can fail but she doesn't do it. It looks creative to us and I have to accept things that I just said that he process created from machine simply falling the Panthers Adidas blue pit sacrifice to be true because comparing this position with 60 million other games to play in believe it was the better our Machine music but somehow is a beer or maybe both of us because we created machine thinks is the best book. And when does Universal reset it scheduled as a shin to Ken machines

do things based on similarity optimization would profess effectively human-generated data operates within the human created framework, but it it it it it was able to generate an incomparable any positions that he preached against When she was do something so we both creative is because we wanted waiting. So is there at least for now and always be called? You know, it's just I think it's great definition of semantics is a 97 to 6:18 in his book computer power and some reason the great iPad papaya here. She talked about Difference between Pacific and choosing

machines can't decide if you can choose because at the end of the day you go to the very bottom and made his decision because I was told human made made a choice because I want to text so I think we should debated but we should not do you know what you'll get stuck because what I believe, you know, if it will let us know. On a slightly different note. I just I wonder if it's just a property of a rapidly progressing field goal post have to move if the

progress is fast enough the moment that we achieve something in a blink of an eye. It's commonplace and it's something that everyone can do and you can another thing which took a supercomputer that cannot be done if you'll help us to me and maybe that's okay. I know continue to progressing they'll continue to move and will that be a moment where we have a problem is that no limit to intelligence. Trying to find out what they continue to move. The goal is to make it two more times for the much more difficult as nature.

So according to beat mines Mission it is to quote solve intelligence and then to use intelligence to solve everything else. So my first question for the panelists is our deep mines gold ambitious enough. But seriously IBM deepmind Universe Elder too many others have invested Sony many others have invested tremendous resources into mastering game playing is this a path to solve the intelligence? Yes. I think I think there's a there's a there's a common answer that that many researchers in games would give which which which which which is you have

closed systems and closed systems are great for allowing you to advance and measure your progress in and so that's a lot of what we're looking for in good domains to help us move. Our science forward is being able to find a nice clothes system where we can really be able to Benchmark progress be able to really crystallized. What is the core issue that our systems are struggling with and then focus on that and be able to advance science that way so I think that's I think that's half of I think it's half of what's going on with why this is a pathway. I think there's another half of what's

going on and it has more to do with humans than with science. And I think it has to do with what games mean for us, and maybe I'm getting somewhat speculative here, but One of the classes I teach is in back on game design. So I have thought quite a bit about games long outside of like the actual design of them and where they come from and in this field, they're sort of this the story that people will supposed to designers will say about the role games play that in fact games are largely about

humans are sort of in something's hardwired to want to play games and it's actually comes from the source of our intelligence maybe even act like the primate stages that early on that you needed a hunt for food and you had a couple of options. It's very dangerous to go hunting so you can hunt every day and then you would engage in this very dangerous activity and that wasn't good for your health and your promotion of your jeans say another option is you could hunt only when you're hungry and then you would not be skilled in this would not be good for the promotion of your jeans. And then

another option was you could practice and what's going to motivate you to practice would be that you would invent some rules to a game that would engage you in this practice. Start throwing rocks at a tree to see if I can improve my aim and we're hardwired to do this. I think that that fact that we are maybe a little bit hardwired to play and the games are about adding structure to play is in fact, I actually really important to the development of intelligence and that maybe it's not just about we pose games to our agents, but our agents are posing games to themselves. So I

think I got depends. What kind of game out of probably gave me to a bowl game is about to come over there. And that actually unit was the specific role of the pop in the real world may be able to define a friend that way at the same time to be describing something else. Like I would expend didn't go well and found out how this is actually applied for something completely different way, but I got you know who I'm trying to find out. There's a new goal out from there some

sort of unexpected things like, you know, what kind of game and should I evolve to have a mouth? Google what am I single and I think that would be very similar to do. Very complex problem what we are facing in the world to think about music CD to solve a problem. I think it's much closer to that. You know how the game can be used as a pistol builds for the real world problem and many of the real problem is very difficult to evaluate if if you are doing okay or not, you know, if you know, this is very difficult to evaluate in a certain state of the move and it's

not obvious and how can you ask about it. How can you prove that to achieve what you want what you discover expected path, which result in a very big Reno turn out to be a real challenge. Interesting games at volt to that way that kind of game I think like a game can be interesting, you know of a modo for the role that we have a problem. Just coming from the deep my admission scientific question of Our Generation and so is it not entirely appropriate that we have companies devoted to try to crack that question and would we not be disappointed if they would not

companies trying to do that and not just companies communities academics. It feels like it needs all the attention and focus it can get in using games as a pop to get that. I see it is an expedient pathway whilst it's helping us so long that go and whilst games are able to provide meaningful and useful challenges that help us to make progress towards that goal. We should use them because they have very nice properties that constrained. We have play benchmarks Clarence and easy to work with If it turns out

that I'll call ends up needing something Beyond games and we stopped to require properties Beyond games. I wouldn't hesitate to two step in that direction. James besides the Practical benefits that did games have that they are easy to use. There's practically an infinite variety of games each stressing one particular Dimension or maybe more than one of of what we consider to be part of intelligence. And so it's easy to go out and find a game that is beyond the state-of-the-art and figure out what

it would take but not too far beyond the state-of-the-art and figure out what it would take to actually make serious progress. The last year, Sam's home and his AAA. I invited talk he claims and I'm paraphrasing game theory is all you need to solve games data are not necessary. So is this true? And if so with this preclude the possibility that sell them games is a pest to building intelligence. Just one quick comment. There are games. We're particularly Cooperative games. We're

modeling your partner or collaborator. RR is a very important part of the game. You can and doesn't emerge naturally from the structure of the game that requires if data a prize experience. but I think I think I don't have any contacts for that quote, but I will have to respond to it in its context list net which is to say that so maybe in the context of Poker. This is a try but you're not good at all. And then there is a rigorous definition of salt and we rarely use it in that freezing.

I think $0.01 you might be able to say do we need to be at a c high-level play to be able to play it ourselves and that's the form of data that you have that seems like the answer is no we don't need that. And in that case I would have totally agree with it. If is a sense in which may be what you mean by that is we don't actually we Actually be practicing the game we should be taking and we should have to take actions and use our own say self play but I think self play such a fundamental idea that it would be obese and seemingly too embarrassed to abandon this what is really really

powerful methodology for being able to understand both games through being able to play yourself. But then that same idea should be able to and in fact new zero shows that we can use that same idea of just being able to not know the rules be able to take data as the right experience and be able to learn to play Atari games which where we don't have the rules of the game. So it's unclear where Game Theory would give you the answer. In 2013 deep deep mind famously announced that hasn't made a pioneering Breakthrough by training dqn. Aziz Network to play Atari games at

super human level using only the real pixels on the screen as inputs. And otherwise, we had Advanced Beyond AI game playing programs that excelled only in their reasoning abilities to AI Incorporated both perception and reasoning is it possible to achieve intelligence with reasoning alone without some if not, all of the other core components of AI representation perception reasoning planning learning communication actuation Etc. And related does this mean that there will be no AI in anything less than fully embodied agents.

since you mentioned was recently so I want just you tell a joke, so it's the That's happened. This is so real real story December 2018. I was asked by Demis hassabis just to write about alphazero and I just as if it was a few of reasoning. To that. I've been receiving numerous invitation for all sorts of insect Congress has received yesterday for Congress of microbiota in Harris. Wild Wings and Things I'd like to think that the goal is maybe to try and connect reasoning to perception

that ultimately any intelligence has to operate weather in this real world On All or anything like that. Well in some dream of observations coming in and actions going out ultimately going to be necessary to connect. Stream of what it's seeing and an acting to own ability to reason about what's going on in that. Well, I'm sorry that process of going from from that's coming in and actually going out to see something which enables it to understand and and and and reason about it may take many.

So you make some dqn which is kind of model free varies directly addressing the problem of how to how to reason about the Weld and whatever the mural that happens to be doing. Is its reasoning process? It's nice not explicit such mechanism that happening alphago, maybe even more remarkable than whatever it did eventually at the fact that you own that by itself with no such a tool was able to reach the standards of very very strong human class who devoted years and years to studying and maybe this is the canonical the main way we previously what I said. Well you have to set you have to

have explicitly things to do anything that is so so maybe the way in which we connect reason to perception may not look like how we used to think it may not take a form of mixed message search for it man. I think we should be open-minded but I think it's clear that the goal is to develop have a greater understanding of a Wells witch comes only from this Century my to stream of interactions and to comprehend. Well through some process that captures all of what we mean by reasoning even if it doesn't do in the way we If you have

a reason to Knox ratios in the real world, look Ghostly have the physical interaction so busy which is our products and of the votive convention out like a prescription to finding everything you planning and inactivation for that reason that the feedback as well and is all kind of control will be made to properly Controlled Products systems were complications. I hear it's like I think like I'll be you know, very important to look into the embodiment contractions and then multiple layers of

interactions. Let you know big kind of symbolic. You actually does something you actually think it's rather than go into the higher. You know, that that's how things operate saying about your system. If you find the robot operate in the real world, you know, you have to have a different kind of important aspect that can be done in a bit. Did my last question of my good we can do something else? I was just thinking of the word a little bit of embodiment. And then sometimes that word

is used to say it needs to be in our world getting our perception taking actions in this, you know, this space here em, and and I think when we talked before intelligence is about generality and so I think there's some degree one way to think about this is that you know, is it going to be that intelligence only makes sense. If it's in that will it should be General to other embodiments as well to you know, it it it doesn't always have to go every direction but I like we would expect that there would be some intelligence in a system that could be embodied say only playing Atari games. Now,

is that going to be the end-all if we do that have we got it in our world to know not necessarily but I think I think about him it is important, but it doesn't mean that I have to think about it this world. So last point I think I wanted to come back to mister Kasparov mention of the bottom line. It relates to a conversation. We were having a lunch which is whether or not you know, once the machine is able to even if it's stored likes it. Let's think about Chinowth can Checkers being solved maybe it has a table and it can look up

the move to making every possible position somewhere so we can achieve its goals. So is this intelligence is this enough? We've achieved the bottom line. So what more is there? Bottom line in chest doesn't mean it's a perfect move. It's just gettin it's a move of a high-quality. So bottom line ever gave who's winning the game this it's going to snow Perfection. So I accept that definition winning the game. You don't making fewer errors than humans. Maybe this is Seville while you forget I don't I

don't think so because Genesis we mentioned what war generalization many times have humans can learn from mistakes against just to use this experience so far. I'm not sure that it's it's it's doable baby. I'm just wrong but it's get I don't know whether this is StarCraft problem can easily transfer Knowledge from one map to another without extra extra credit. So don't have nothing magical to humans. I think it's just there's so many philosophical questions that would never answer key the history of perfume science. So that's why he just now trying to

just trying to do find someone else to answer questions for us. It's like we talked about so you don't like what you see there. Just you know, just or the beer or just you try to fix something with you. So I agree there are many definitions of intelligence, but that shouldn't stop us from trying to get one and let me try which is to say community has is the ability to achieve goals. And so in that sense if something play very well it at the check is or a chest. We should say it's achieved its goal. The one that has been sacked and what differentiates

things which were able to generalize the data that there's some goals which require generalization in order to achieve X-Type example, if you take no human in order to achieve their goals of whatever they are that beset by say evolution of survival and so forth that in order to achieve that all humans have to be able to eat very generally kinds of different problems along the way how to manipulate objects how to move around in the wild how to communicate how to influence other people to do some things. These are all things which are required in order to To achieve their goals as effective

as possible. So the level of intelligence depends on what you're asking the what is the goal that you're setting the system and if you'll testing it a challenge and gold and environment then there's all kinds of level of intelligence that can be achieved. If you if you're asking for Golden environment, which kind of such rights and you completely crack check is in itself. Well, there is light faster intelligence that can be achieved enough to me. So I think that the the nature of intelligence depends on the nature of what we're asking it to do and it's it's it's something which we

if we satisfied with with something achieving its goal weld. Soviet we can also put something hot or Oculus entity that is wild humans. We are setting goals and achieving them intelligence how to find problem must have some ultimate goal that there is the ultimate goal of a problem. Otherwise, it's kind of like shifting sand with no definition. Now in order to achieve ultimate goal the system itself all kinds of stuff that might need that in order to achieve its ultimate call. It might have to set

yourself what kind of motivations and proxies of things that it wants to achieve it on the way to do it to do that ultimate Pakistan and I think if we don't have the ultimate Pappas, I'm not sure that we can pin down intelligence in any way that will be meaningful for us to study and understand it. 70 * 1 what is a my battery says well, and the ultimate goal is a survival and reproduction River invite us strategy. I was a vitamin C the survival and reproduction and so like how you know, it depends on the

You know, so the problem is actually down to the survival. I feel so like what we want is the general principles of intelligence. So so so coming off of Dave's comment. I feel like what we want to do is is for any goal that we might set for the agent and which were setting it but made me the agent is adopting a now for any of those goals. We would like a procedure that that they could go about trying to achieve those goals in an ideal. He has a broader range of Demands as

possible and I think that comes back to this generality piece and I think that is the direction. I think that we see even AI game playing and it's very narrow way starting to move to where it becomes a lot more about can we really show that these principles that we're finding are more General than just their ability to play any one game but can demonstrate that they can achieve goals across a variety of things in terms of solving certain kind of siding. Exactly know the concert is a different you see I have different definitions interview

and they have a good defense strategy from what we see as the interest has no in the context of the most of the movies that will come fix problem section 4 years old. But that's one peach. They may be a future organism, which I'm in the same context is able to achieve much better by letting a lot of Haitian revolution. And I think that would be you know what we are looking at. I think it's important to distinguish between the the system that executes on a goal and the system that builds the system that executes on a goal and in the case of alpha zero.

The learning part achieves a build the executing system that actually is trained to play chess and I think that that distinction will be useful in. I think with that we have to stop I'm afraid. This is a lot of fun. We're keeping everyone from their dinner. However, so I would like to thank our panelists. Making the trip for being out there. Thank you very much. They've obviously made tremendous contributions to our field and two games in particular, and I think they've convinced me that making progress in

games is making progress in AI, so hopefully they have convinced all of you as well. Thank you. You will walk around to find information quite as One Max.