About the talk
AI is already within our lives and it is serving in many fields; from deciding to give you bank credit to driving your car, from deciding your medical process to securing your house. On the other hand, science and technology accept that AI is unexplainable.
Assist. Prof. Dr. Beril Sirmacek | Department of Computer Science and Informatics Jönköping University
JOIN over 30.000 AI Enthusiasts Receiving the Rise of AI News with news, updates and special offers before everyone else.
Rise of AI Summit 2020 | Berlin
My expertise is computer vision en AI (artificial intelligence). I have been using my mathematical and software skills for earth sciences, especially for processing satellite images and other big data. I am eager to contribute to applications which can automatize data processing to identify land use and impact on natural elements as well as on biodiversityView the profile
Because we know spoke a lot about how we could build a PC fan based on a i and how we can do a i trading. And yesterday we already touched the topic around regulation. What is really? What is a I would be doing. Do we really understand what's happening there? And this box this black box of a I can we actually unbundle it and I'm very happy that sir Professor doctor Barrel. I have to read the stories for my check. I hope I pronounced it. Well from the YouTube on University in Sweden is here with us as she has done tremendous work in all different field, Industrial Field, but also
machine on the ice and she will help us shed some light on this topic. I hope you're ready and well,. Let's stop. How do you spell? C'Mon, you pronounce my birthday pics? Thank you for this and good afternoon. Everyone. Great pleasure for me to meet you at this virtual events in Rise of a, i n twins when I hope it's really went again next year and to meet each other. And then I'm a scientist working on computer vision and a is, I'm very nice introduce. Thank you very much for. This is my academic career, but besides I'm just looking robots, Uncle,
Tim spot industrious. Well in both academic work and industrial work, it's extremely important to know what we're doing with our AI or machine learning models and to explain whether we can trust on the Stations of this mobile or not, when they're using. Are there in scientific work or in industrial word. Therefore we have to give up transport to a i and another words. We have to know how we can build up a i Trust. This is going to be my topic today and I would like to have a short discussion with you in my
short time to address these topics on our she will need. Why do you steep or trust in? All areas of our life and our daily needs but are also industrial business Needs Trust. And how can we do look, trustworthy AI or is it possible at all to do you look trustworthy Ai and what kind of mathematical tools do we have to see whether we can trust on our mobile or not? And finally what's coming? Next. I would like to have a discussion with you a little bit brainstorming, including my ideas and Imagination. What could be waiting for us in the field of trust work today? I
We do not only have physical needs like food and drinks water. And we also need each other. We also have other needs to survive and Abraham Maslow in 1943. He's written a paper called a theory of human motivation and he suggested are there about no muscles and in this pyramid, he described human needs for our survival. The largest layer in the body responds to do layers, which we need more for survival. And then their importance are reduced when you grow up
earlier and they are getting smaller. And if you see here after food and water are humans, basic survival needs our safety security. Which we can also summarized as trust, assume that a baby who is born, who can find food and water but not in an environment that he or she can trust and feel safe. Baby will die. And the adults aren't much different than a small baby. But we are just in the derby speaker by this but our human knees. Are just the same like a baby
beyond our physical needs. We need to trust in all other areas of Our Lives. Not only close relationships, are businesses, color plates, and working together under the same roof. And I assume that you are a developer and you want to sell a motor or before he's on a cloth and make money with it. So other people can use and pay you monthly with subscription fee, for instance, is mobile gives a result. Is it safe to use? Is it just once he can other people really rely on its decision?
What it says, and do they really get what they pay for? And they're supposed to work. It's actually in ethical rules for these kind of questions, opens up as the skills of trust work today. I already there. I mean, they're already used it. So I got star for 1. Good news and bad news. Good news. Is that guy is already in all areas of Our Lives. And the bad news is that guy is already in all areas of our life. Assume that you apply for a credit card and it's rejected. This comes from a result of a i r
court and you want to know who's going to answer that question. And who's going to explain how this motor is, given the decision is breathing adequately reject or not assume that you are in the doctor office on dr. Analyze. Your CT scans blocks measurements with an AI algorithm, which is already on. Doctor says, on the way to a Cheerio is still have a surgery. Can you read it first? You will ask me why,. This decision is taken. Somebody needs to explain us how these
mortals are working in order to trust you. Say you made the trip application, you seen some other speakers in the beginning of the day. I heard a job applications are these Are also going through a our mobiles because there are so many applications to his kin and Civic a good candidate and then decision is taken. And they say, you are not accepted to the position. You want to know why? The question today is County really trust. If she is making decision for us, and I is really regulation always of our life.
It's some examples of some of these models are affecting our life and then I will show you some tools designer developers or the business owners or people who wants to know how I can use to know. Is there a way I could be transported to your nuts? I start with his story. This is LaTanya Sweeney. She's a nice lady is you see if she hurt and exciting news. From her colleagues, her call it fate of our research, your name on search engines. You only get some advertisement. That start with Anya is arrested. The truth about Sonia is lasagna is trustworthy.
They really want to know what's in her story. Bringing. The lasagna was a data scientist. She was very curious about what's going on with it. And she wasn't told me a scientist, but she was also a Harvard University professor in this area. So, she decided to make it a good analysis on how this engines are working. And she found out that her name is close by, in a black minority group, which are also in the park associated with some events, which ended up with And they are
for these engines are bringing up results like Isla tires arrested or not, but not for the other kulluk. So tell me how much a month or two before summer and to use, is it ethical to use, you know, the issue happens as a police officer and they use pictures of the people to decide whether they've got some risks to do for a crime or not. And his AI, Moto say, high risk for Bernard is a real story and low-risk for Dillons. And then this both of them go and get along
as three days later ended up in police station again, but Bernards head no issues later. So this decision will do is working on. Is it looking at long clear tattoos or is, it is rather too keen on some patients expression. It's has to explain us but it is looking at what is his folks. To rely on the results ATM. Maybe you heard this examples of people added some little noise on a pictures, a picolas cage and Edinburg is ended up classifying it as something else.
I'd like, another animal noise is even I cannot distinguish noise, but this changes the decision to stop driving car and suddenly this answer became little bit dirty, and it's cannot give the right decision. Like anyone more better, working example and Mike's, working example, but still needs explanation,. Did you vote them out of discrimination, of wolf and Husky and AT&T, they sold the motor was working with 99.9% accuracy. Surprised. Wolf and husky look like each other's phone much.
This isn't very high it. Seriously, why is looking very nice today? So, they also, even if the result is very good, still had a question whether we can trust or not. So high performance of algorithm doesn't mean that we can hide the trust. We still need explanation of why we can't trust on this. High performance, that has been many studies of explainable. What is here in this more often in Lost 2-3 years, maybe before that. It was mostly clothes interpretable, artificial
intelligence algorithms and it's been searched for almost 10 years at the moment ideas. We need some mathematical to say we've got some training examples. Can be trained or Elmo talks continue data came from a real be more months and your mother which is trained for training day circuits. New example gives a decision and what is trying to do to answer why this decision is taken. Don't be quote, The Next Step. The next step is to explain this one depending on the audience
who is asking the question of why? If it's the people who is asking why maybe we need to explain it. More accurately about Slayers what's going on in this is very Pious issue. Maybe you need to try to restrain it this way. But if it's an audience, who is the end user somebody waiting for a bank card? Maybe we have to explain the decision-making process different. They are there by How to find the neural network or making more examples from home, similar examples in the training set, looks like they're for the decision is taken.
So explainable a are all go to choosing process is the river depending on who's asking the question of why, who is the audience to whom? We need to explain this and buy me want to explain why we will we have developed a mathematical tools to decide whether we can trust or not. Well, maybe we can trust by looking at this. We try to get ideas about training day to set about a motor itself and about the input Barbie looking at in this in football and they are
making our decision in order to explain our mortal little bit, little bit to open up the dataset. I'll just give an example of My story. I bring my dog to the vet and then it says, I have to make this vaccine to your dog. And then I asked why and don't she says, for instance, because every other dog has this vexing, so I have an idea, whether I can trust on this astronaut, she explains me different day. I got this, this blast report and dance just shows me that this has to be cured with his message. Maybe if it's explain different things. I can see how
it is explained to us. Also brings us to the point of trust, whether we can trust on explanation or not Angeles, Millworks in mathematical think. I mean xai metal start explaining training, dates, are the most of its of Gargamel looking at input. Explaining this mathematically but there's no quantitative video to discuss trust. So the answer that the West has been given might be this trustworthy for me, but maybe more trustworthy people other person and there's no quantitative issue about a trust is the end-user audience must discuss. We are
opening the Black Box. Little bit. We are bringing little bit transparency. Why are metals Opening up with transparency is possible by using some interpretability approaches in the mathematical mathematics, and choosing the approach is the safety dance on the audience. Who's asking the question, who is the answers that we had to pick one metals of explainability depending on the audience and depending on how much transparency is needed? Is it does need to be high level, or does it need to be
better details? Maybe it said you were bored, and we need to explain very, very detailed. And what problems we are focusing on what's mathematical methods could be used in this development environment, forces are possible to be used in this environment and better explain about the are used during the training process or after maybe Moto is pretty, ain't you got it, but you have to explain it afterwards. So defensive. Which of these problems are occurrence. We have to pick and all curtains
and Albertsons are also classified. Is weather day explained the entire model or not. And whether they can stitch, all the neural network layers and two components and explain or not. And whether they're focusing on the DL Gorton is trained or how the algorithm is functioning in real life or not. I listed some mathematical methods which are frequently use, what is impossible to explain all of them right now, if you're interested, if you're a developer, please speak to splice and you searched and protection. I will just give you some examples. One of the most use
example is shop, or shapely features of switch. List the feature important that we can decide whether they all go swimming when its decision-making looking at the right thing or not or what it is. Looking at the Titanic. 326 classification example, where are these listed under the passenger died or not? Stampy looked at the features of the passengers and this particular importance in the decision-making, whether he or she died or not and don't be solved, it mostly female survives and the mail start. So we discovered this old is Kevin number ticket price
age of the person oldest features are looks but which one which features the most important in that? It's the gender, whether it's female or not. So this explains that the decision is made, even if it's very high time. I smelled a toasted and this is like, Hypixel level 2 images to explain, which pixels are contributing to the decision-making. For instance. If the dog is classified as a dog, which fee is pixels are contributions to this decision, making process of decision-making process. Tim Peaks. A-level is looks
to the Husky and wolf example, that they looked in the beginning of the presentation, they sold that, which was working with 99.9% performance was looking at the background, and if it sees snow is supposed to close to my needs as a wolf. If it sees no snow. It was closed, find the animal as husky. So that's that's how it was working with very high performance because all the both examples were coming with snow but it wasn't looking at the animal at 12 and its decision-making. So High Performance Tools doesn't mean that's the
AI motor is working well behaved. Xai missiles to know whether we can trust or not. So I can tell of excel, metals can tell her as a lot about whether we can trust or not. Whether there is a bias or ethical issues in this all going to or not, how to fix it. Because if we know how the decision is made by it's looking as we can. Also fix it later if it's wet customers to squat, do do. But hopefully, what is coming next? Bees are just my idea. What? I'm just imagining that these xai process will be
fine. But someday it's going to be like the Ai mobile view, explain to use it real son, why? The decision is taken and user will give feedback. I don't trust either of you give this decision. Maybe you didn't look at the writing and then maybe which trains itself if this way or looks at other teachers and an improved. Results, maybe you've been use this vision and explains again. I'm assuming that is Luke still be real time. In the future. May be getting feedback by verbally and improving the model real time and
what we've seem to wrap it up, simple motor doesn't mean that it's low performance or more complex. Model doesn't mean that it's high performance, and higher accuracy, doesn't mean that we can trust more. We still need to explain why this high accuracy is taken and Beyond comes with both advantages and disadvantages. It might also end up with some security leaks. Of course, if they explain something with meat, doesn't need to be explained at this point. So this one for my presentation and thank you very much. I hope you reach me
on websites on Twitter, on my new sweater YouTube and very nice to meet you sir. Thank you. Thanks a lot better than eating out sometime. I can see your research and I guess we will have some questions from the audience, but maybe let me stop by at first question. What do you think will be the main drive us in the future for understanding order? The Dimitri understand the eye? Is it the regulation that you see or is a greedy more understanding? The outcome of tomorrow's better training? The models better may be creating better products out of AI models
problem. We have training algorithms with a dataset can be bound to sell the product to Somewhere Else Bar. It says totally different data sets that we even do not imagine and they expect the algorithm to work done. If it's not working. We need to explain why what is Val's looking at. Why? It was working good in this place and not that this is a self-driving. Cards. Which is developed in Germany with all examples taken in Germany, all houses and streets learns for Germany and you sold the car to another place, just in the Netherlands. I mean, a bird country.
I tell you that it will not work with the same accuracy because our buildings are all red color and how old different kind of cycling roads and it will not see the same dataset. So it's important to know by developers, making decision, how we are making this. What is this looking at? And is it trustworthy or not? I want something from Tesla that they roll out. I think the mothers in several areas at several times because of exactly what you just said. And our friend Fabian has a question might be in the room. It seems
like it wants to start with an expandability. Test anyone's understand what technical expertise, does he need for that. Maybe you should be a student of you bounce on the audience, be safe. If the audience is a person who wants to know how the Al Gore it is making decision. We can use Metals, which I just seem to find the decision-making process instead of shoving the country players of neural network. It can be shown by a simple decision tree. I looked at this than I looked at this than I looked at this and this
decision is taken just two seem to find the model we can use such methods or just like this featuring portal which features were important in this decision. Making process, for instance. You applied for a job. And you're rejected, maybe somebody can use this feature important test and say that your Publications were not enough there for your rejected. Maybe the future importance, in this case could be helpful. But if you're a Jew, Can you go to different story? You have to maybe explain in deeper level levels? Maybe you have to really see if it's
finding a wolf if it's real looking at the animal, if it's really looking at this cure, not the background than how can I rearrange my training set so it can learn that even if there's no snow on the background. So it still recognizes the animal. Maybe you have to do some tricks in your training process. So depending on the audience the metals and then explain in case could be different. But the purpose is to communicate with anybody. If it's someone who doesn't know when a programming, we still need to be able to communicate with xai to. This was
what we were looking at and you're making this decision. You might call you tomorrow. Be careful. There's a question from a large navel. Also. I also from Fabian but a question to that M. How can regulation be technically implemented? More like a tooth or a g fpr self responsibility or like an expection standards. How do we make a question? And I believe it's still open questions for all Europe. At the moment. We've got some local Sears in the team's national teams. And also, please pull up
racing, Netherlands. And Germany are also trying to decide what are the ethical rules rules rules and how to describe even the word trust. How do you describe trust? It's too much personal. And these are not mathematically, describes videos. So we have to try and still trying to Define this. Like we discussed for counting Automation and we had an AI that was much better than the human being but the mistakes for different. So you're better than a human being, but your mistakes are
we at? And I guess the same can happen with autonomous driving. So, Actually the cosmos better but then it runs over a child. So obviously that's something we don't want. So how do you do find that? Well, when you, when you look more at the, at the training off of model for explainable AI, is it, is it possible from your point of view? Because, that's something my status really careful to say. Hey, the models not working with that results with the liquid products. We're missing this specific type of data. We're missing the specific type of, I don't know teach engineering
approach and then we can make it happen to get predictable models. Say a story started. Military in in, in America. So DARPA project in in United States has started except because they trying to close by camouflage tanks of its efficient algorithm and down. They ended up with him to close by come flies text with 99.9 performance. Again, it's almost working perfectly. And then they asked the question. How did cambered is? Well, I mean even just you and I cannot see this, flush tank. That way. I can see it.
This is call aixa. I story starts with in the beginning, Sunday Phone, Doctors in the MAOI. They Phone Doctors in the training dataset. All the come flush, tank examples were taken in clouds of steam and all the rest of the images were taken in Bryson, and they are most of us. Whether there's cloth or not, so I suppose most unrelated to come snatch tank again. And this is just like the wolf and husky story. But this is how the XIII development has started. And I need a cigarette. We don't trust on performance numbers.
It's my say 99.944 months, but we also need something else. Maybe he needs list for foremost. But we need some explanation. Then we can decide whether we can trust on it right now or not. If they're self-driving. Car, is still enough. Now, I'm driving on giving the decisions myself. But if this moment it's getting dark and I don't trust my decisions. Alot. Maybe we can try a little bit more carefully and share their attention with the with the car. But if you've got
some videos about how much trust car has on itself at this moment, maybe we feel little bit more relaxed to change the radio and the little bit to become more relaxed. In this point. I felt like self-driving cars. Would be amazing so I can just relax and don't to look at the traffic. Thanks a lot. That is for your time. Thanks for all your insides. And that maybe maybe I posted you all your website in the chat and maybe some people can have some questions and reach out to you. Thank you very much. Good.
Buy this talk
Buy this video
Our other topics
With ConferenceCast.tv, you get access to our library of the world's best conference talks.