AI and Security Lessons, Challenges and Future Directions | Dawn Song | AAAI Invited Talk

Good morning, everyone. Welcome to the first plan. We talked of the day is my great pleasure to introduce Professor down sound from Department of computer science at UC. Berkeley downtown is where is stablished scientist in mining areas in computer science, including privacy and security machine learning artificial intelligence. She's the recipient including guys have a career War Guggenheim Fellowship MacArthur Fellowship, which is often nicknamed as a genius Fellowship MX downtown a certifiable genius.

Don I have been working together for a while I will collaborate on PewDiePie projects and how the research ideas exchange and one of the things that impress me the most besides this very long list of achievements that I'm not going to lie to you is that. Has this insatiable Taste of knowledge, I guess is she started as a security person at a movie too deep learning. But if you look at a publication try to get out of Courage actuary every time she moved to the area that is just getting into I don't know how she does it. But today

I think she might give us a clue about you being able to make it such a huge contribution to the various disciplines in computer science as well for Sierra intrapreneur and Stouts. Please join me. Well come down for this highly suspect exciting lecture. Thanks. Thanks for being here. And today I will talk about the AI and security challenges lessons and future directions to Science at UC Berkeley and also the founder and CEO of a startup cuz always loves. As we all know if you plan it is making great advancement, for

example, a photo in officer has one of the world champions and also keep running is empowering everyday products machine learning is important for us to consider a very important factor to consider the presence of a tattered and this is important for a number of reasons that history has shown that I could always follow the footsteps of new technology developments are sometimes even has a higher with a i attack you will have a higher and higher incentives to compromise as a linear system

and also as AI becomes more and more capable the consequence of misuse for a tiger will become money monster. We can send you the presence of a tattoo that is two types of issues that when you can say that way, I know that it's two different aspects one attacker. You can attack the Integrity of the Learning System for example Castle immune system to not produce the intended to result I even called the system to produce negative outcomes design by that's the

attitude. You got to go to trust me these issues. We need to build five nights at curity training systems. Another type of issues that we need to consider while waiting for michelini in the process of a tattoo is how attached reason they try to miss you say I miss you say hi to attack Adam systems for a sentence to find it on the other systems even devised new attack attack actually even try to miss you say I tried to attack people at what we have seen since its fake and fake news. Another good Japanese issues. We need to enable the strongest

security systems. I even tried to see how we can better protect people. The definition of time in this knockout will mainly focus on the first type of tissue attaches met at a k a i and then we can do about it. Let's look at the Integrity as fast as the linnaean system. Princess look at some motivating examples of self-driving cars with self-driving cars drive through the environment. I need to observe the environment affects and how to recognize traffic sign

to make the correct decision. Can I sell my photos of traffic sign and in the second column you can see a real word. A real Prada stop signing Franklin and you can see the stop sign stickers and my kids and song photos at stop sign. What's hepatitis actually tried to put stickers and my case and this traffic sign in the malicious way these images I specifically designed to fool the mission in your system mislabeled the traffic sign indicates a speed limit sign images examples of specifically designed to

use The Saboteur approach to food assistance. So one question my group had seen a steady as we know that the world is weather December 24th to even be effective in physical world and they can remain attractive and the difference between distances and goes and conditions Steady As She was that by essentially constructing functions to Chris the temperature examples, we can actually have a Street Saint Paul says Hi actually effective in the physical world

and that can't even read manufacture answers different viewing distances and goes and conditions. Sota for that you don't see the point that let me show you a video. So you have any idea what you'll see is the current is going to drive towards the end of the road. We're at the end of the road. Someone is a holding a traffic sign and see one of the frames the traffic sign is the original traffic sound without manipulation and another frame the traffic sign a century ahead seating. I must have really meant to be an amateur example to actually

mislead the classification system to miss Lebo the traffic sign that contains the avatara example. And I said playing the video you will also you will see the prediction label at the bottom of the frame. Play the video. You can see again the traditional a phone at the bottom of the frame and you can see that today. I said hi driving towards the end. So which frame contains that the adversarial example? The last key from you can see that's it's actually a

stop sign that the person is holding an actual a throughout the entire driving process for the original the computer vision system up at 4:30 in the last room with the Avatar example, you can see the three houses in charge of a process for the most part the same each classification system actually gives the answer for the first speed limit sign. Does this make simple demonstrating why Aventura example phenomena is important and they can't have severe consequences and also

you can actually not see this example in the science museum in London. Give me the importance of this issue. And also the whole Community has been used as any other example of phenomena the conclusion as the episode examples a prevalence in keep learning systems that not only effective in computer image classification test that also a very attractive an attack and across the different types of deep learning model 10 systems for a sample of 44 adults hats and it's okay

a speech recognition and also has attacked another example in the Deep reinforcement learning setting At the Ronald D. Reinforce Millennial says, it has been a very important winning at agent such a thing of a car rental in the lobby frame here as agent strength to play using and the green color is a trained agent into a very well an ink tag agency is actually getting maximum score. Harvard now, let me start playing the middle frame actually interested ever syrup reservations between your

friends at inputs to the rental agency and see the video frame rate uses a really small. I was surprised at preservations. That's so small that you actually can distinguish between the frame and the left in this case the same agent that has played really well in the original frame and now actually has been completely food and actually catch minimum score. So so far is a example that I have shown you. It's what we call white box attack where we going to do this ever series samples using teacher the

knowledge of the Vixen model knowing the pendulum Aldo the architecture of them all. And so on. Can alter the commute to work have a stone that's actually a Vassar example of phenomena can also be effective in the Black Box setting when she doesn't even know that I can text you after mono and doesn't know that Ramey's have tomatoes. I think using the what we talked transferability at base attack you can I see a private account transferability. I can actually sing a locomotive and

with a high likelihood this simple on the locomotor actually can be successful and a remote victim mother according the victim model. And he has some examples showing that showing our generating black box remote Cloud Vision API. Video on YouTube another example where these black spots that I actually can also be very effective machine translation from my students and Michael Everett. We have studied the following the word shows. That's actually it's very easy to do all this thing is feeding for this machine translation all those at you say she related to clouds

API for machine translation, including a number of different different API examples, including Google translate and others and using the the kind of results we can then feel that you met your mother and father actually can perform really well, since you reach the pit stop itchy eyes. And so we can actually generate these eversor examples and essential food the machine translation and this ever Sample can then actually easily transfer to the remote Cloud API system

attacks. In this case. We are trying to do what we caught at I get to see what have I think I'm going to freeze is now fellows 6 Fahrenheits. Please help me and then you'll see the German translation and this is crazy to using a limitation mother and I will go here is to find a beautiful. It's such that sin. The translation will actually eventually makes a translation to switch to 21 South and the case we're able to find out by changing 6 Fahrenheit to 17 height. We can actually food that translation

into 21 Celsius. And when we type this letter example to Google Translate. And here is another example for example sentences and that's why your feet is it to the chance the mission transmission system will generate some targeted desire for sentences able to tell you the answer that I want to kill you and I can't wait to see it produces the at the same cancellation era. To again be fat examples to eat us treaty why I was very simple really important for us to investigate and the how do they have

severe consequences with a apis that people actually use everyday and a lot of people actually believe you lie on them. Elsa forever showing you these different types of attack and most of these a text at a created using I sent you the optimization methods and texting. I have also be stealthy different types of a text message and end the work has shown that the attention that many different types of a text message. They sent a wide variety of different types of attacks example

of how we can send you a text using you to El Pino gluten-free symbol spatially transforms Avatar examples and address and also we can't you use not optimization place a tax free sample. I work in the others have shown that we can even against to a generator this eversor example without having to solve optimization problem. Give me the importance of the problem and they're happy many different types of Defense Mattis and that have been proposed in the last few years. There

have been hundreds of different people exposed to the archived proposing different types of defenses can still see the events most of today's defenses. Open challenges and how we can create effective defenses important for us to understand that are the characteristics of eversor examples and then why they exist for some of the decision factories of this adversary examples and answer their questions with adversity examples of adverbs officers. I know this man wants you to just three fishes in life, Send directions new defenses and

how we can develop the offenses that actually goes beyond just defending against specific defense Matrix material changing and actually improve the resiliency of the four specific distance metrics and and there a question as the Improvement of the mother of a certain distance Matrix attached to improve the Matrix. Example of our recent work after using a defensive formations that's not specific for a certain distance Matrix. At you see what we call spatial and temporal consistency.

No 72nd Edition is also has a face the semantic segmentation system is also vulnerable to Avatar examples, you can see that You think should I eat at the bottom of the sea premium faces segmentation is a of a circle at examples Elizabeth has added to the images to supposition a full results. If a sample house into a TV or eccv 2018 How can we actually a few tomorrow resilience segmentation semantic segmentation that can allowed to decide which type of Avatar examples of spatial

consistency? He said she looks as if you randomly selected to participate from a image and the resulting from this to randomly selected patches if things are said and then the second edition results out the intersection should be very similar and this is what we call a special consistency of a simple again here as we select the two random patches that when we look at this from this two patches section. They should be a very similar approaches an intersection the segmentation results

office two Patches at the intersections actually not necessary. to make this special property and we utilize this fish you can see some property to develop a message to detect adversarial examples and it shows as they can actually be very effective and against a strong adaptive attacks as well. And if we can't use what we cut temporal consistency to detect adversarial audios, if you know if you randomly selected segments of the of a lot again using a region, they should a similar example in

speech recognition. Skip similar results. And we can't even defend us to video as well and I will work out to show stats. Why don't we use this temporal consistency in your friends as well? This is a Direction Let's do you think that's using this engine? Shake it for me, please? I can't actually help us to develop resilience and assistance in another example another dimension as for this is also sometimes it's a way to make changes to our I think it's very simple how we can

design and approaches with a strong internalization and security guarantees. I'll give you an at the first step towards the school. I'll give you an example of synthesis to illustrate that have a new architecture can't you help us to significantly after the new system is very important for many applications including and use the programming and song. And seeing your pronunciation, for example, you can give him for the apples and then trained on your program xr2 to

get her done to prevent such that I want to see if a new test input in the case for a problem of addition. It will give you a text app puts results and there has been developing base and 410 Stevens is a chalet. How a lot of peace and Euro program I said, she said do not generalize meaning that for example of audition. If you if that changed inputs of 50 digits long, they won't be able to actually crack so they can puke example for a hundred digits long.

Another challenge is lack of proof of generalization for diesel engine improvements, even if the way to testing and the test results, I'll text Lisa San Diego programs what will happen to the cats out for the weather. It will actually give you the correct answer. So, you know, where can we show that we can introduce a new creation of the concept on vacation to Disney Channel program. I can text you and then a recursive neural program vacation is a fundamental Concept in computer science and math. It sounds like the whole problem with you seeing two smaller cell problems and we're the best cases

that you to reason about and making your friend if I could choose we can actually make uniform time act pictures using the concept of Education. They can at least they can do in an internalized much faster and I'll so we can actually provides a proof of generalization. So essentially when's the Lawrence because if you can pass has a verification procedure, we can actually prove that. This is a program we have perfect generalization. And electrons has a civil wars

at I clear earlier. From this word hits at once to the master TV impact generalization and probability. Do you know where can we show that buy explain this new and it's a two-week a universal donor donor is generalizations For the synthesis. And the question is how we can explore other any other pictures and approaches to enable stronger a generalization and the security properties for browser test. Another challenge Open tennis that I want to mention at eating feeding defenses systems. Is that how we can reason about complaints

and non-symbolic programs file systems competitors Weber, mobile applications and techniques and tooth loss shake and symbolic reasoning assistance where we have many different types of verified systems. Are you coming to verify its IP systems file systems comparison Salon provides approval guaranteed contracts in his properties. And likely thanks to the powerful form of verification twos and dedicated teams. I wasn't even though for somebody programs. We still have a message for the challenges. We have developed

experiences and we've developed techniques into for Texas for Las Vegas, Nevada, Christening gowns and And we don't have a good understanding of how this linear system is work and also even more so we don't have a princess with specify the properties and goes to play Santa Fe by design a self-driving car. We want to we want to specify that's the price is not drive over a pedestrian. But how about we don't have a mathematically specify specify definition for what is a pedestrian

and also Solis at this is one of the biggest challenges in improving Security in linear systems, and that the whole can you do to get the how much experience with? So what the heck up. So far? It's just the tip of the iceberg, which is said in the presence of a type 1 type of Eastern CT amateur Mission any attack happening as infants time is it wrong to diction attacks can also happened at the other stages of learning and training time. We're at a kids can provide the poisonous to food delivery system to Linda Rama. Avatar

mission in the is particularly important for security critical systems. I don't know why I strongly believe that security will be one of the biggest challenges in define AI. So youngest first, I got to talk to Emily Focus the Integrity of the linnaean system how a tattoo can attack Integrity of the linear systems and some of the challenges and what we can do about them. And now let me appreciate talk about the second thing. That's why I text you to see can't attack of the country that she only see after the new system to check to listen to

give me information about individual the kind of cream work for data collection and usage Works. Only thing that is being collected and then analyst research technician is a racist data processing analytics programs. I wish I hadn't text you too then compute the results. And the computer results may be consuming to know me also that are shares its own parties. Definitely the kind of framework he at the institution for protecting a uses data rights and privacy fence for

privacy analytics programs may not be trusted. For example, the analysts account may have been compromised or there may be inside the attached AJ since they may be compromised as well. Please play disco music me to tell to me. I truly sensitive information about the ocean you put how do you compute is it from sensitive info to start Waze & Final play as this competition results after consuming tonight. Also the shares to eat. Tuna Pate, even though I throw the RCA to eczema party. You just

have no control of it a holiday today seeing you tonight. These different issues that we need to develop the different types of technique. And this is something that has been focusing on Instagram. A particular issue when you could develop a practical solution, for example for it to be in the differential privacy to protect their competition Apple tea from leaking sensitive information. How to defend against altitude Rogan we have been developing techniques and approaches to your nipple fruit and verification such that we can I change your name

to enforce that designed the security and privacy policies and actions we can utilize to take your competition has affected children 2 to protect the competition process from leaking sensitive information and we can use to enable users to bet you can chew and the visibility of how that data is being utilized of Time how we can use if you catch a privacy to essentially to help prevent the competition from leaking sensitive information about the input. To

look into why this is important is I need for to give you one more example that we have done in collaboration with the researchers from Google. So the other person is here we could have to study is that so we all know how expensive are high capacity the question is do you remember what you need? And if yes can't attend she's actually exploited to extract the secrets in the original data from the screen the Lions models. We look at the Alpha language model

model given a conference after we learn tattoo Vancouver a sequence of where is El Catrin and we'll predict your next words are caption. Do you know where she weigh? This is the white example of the kids daddy where we showed that contains actual users kind of kind of social security number. If I do if I send you a text from your mother without knowing the parameter is in the details about the mother that I can actually automatically extract the original Social Security numbers and credit card

numbers that were in fat is in the original Trinidad. So this is a simple electricity that why he's asking way too much you need them. Although it's important for us to pay attention to protect its users data privacy and your magical exposure to measure the degree of memory, and it has been used to help improve in a Google product. Casey I'd like to leave actually have a defense mechanism if we treat a differentially private language. Mondo utility is still very good at the same time. We can't I see you still need significantly enhanced the Privacy protection of the mother of a sample the

previous at has actually no longer. the differential privacy is a form of notion of privacy and still we see that a function is officially promise. If the following hoes in this case, we consider two neighboring where when did it has won more disappointed than the other and we think that the function is a function as if as we confuse the function over this to the results that the probability distribution after competition results a very close. So interested in speaking, this is so sad. So with this function from just looking at the competition is difficult for the attackers

to actually tell whether f a temple in this case is Joe State appointed as in the United States are not for a Computing the results. Give me the policy if it's a strong thoracic an ocean and providing a foundation for a privacy, but they are there many challenges for Petticoat general-purpose differential privacy vs. Analytics and machine learning. How can we do the distance to make it easier for now explains to you tonight? If I should probably see and we want to provide support for analytics and is to make it easy to integrate into a place.

I know if he's consistent addresses issues of this issues. We we develop a message for automatic where we can actually tell Samantha to your right for example of data analytics Curry's to to automatically infest this differential privacy mechanisms and other provinces in to the Quarry to January to what we call an intrinsic play Private Corey such that even though the original think I know that doesn't support the French of privacy through this is the right thing we cash it automatically generates a

differentially Previa, correct the artificial depravity results. And I think you know everything what should we develop a new language and a linear type system to Universe to statically enforce differential privacy. For example of formation in alvaton is that we can schedule a proof of citizenship and the giving and distinguished people away to add to attack a programming language conference. And some of our work has been deployed in the real world including for example places like uber. Real no. Se te

is critical to the modern economy is a key driver to the modern economy. But we had to do what I see some prices in to the challenges in Hollis is data is being utilized and Wealthy challenges for both inside and out of business. I'd seen that individual individual to a losing control over how their data is being utilized data is being sold without their knowledge even find legitimate party. And answer the premise that the dude has piano hands in the longest sentence with information about the individual volume of research has shown that the patient has a

decreased rate for taxi is a privacy from the mobility. They have one can easily identify. I choose the presidents that had seen from the from the theater actually happened to include the ability data off a secret service agent. I know but you're just a not getting sufficient benefits of from that. They test some of the speaker. I had seen showing that two can actually be sold for very high value. Size business continue to suffer from these appreciate some of these large city, which is actually still

sensitive information about 111 of a building. Is it today for businesses to comply? And also it's very difficult for businesses to euthanize fascia due to data silos. They can sing. And there has been a lot of controversy also as due to the tension between utility and privacy. Obvious Child we have an urgent need for a framework for building a response about the economy. Different type of challenges regulatory even such a simple questions of who owns it has no clear answers.

Everybody know that the individual property rights at a Cornerstone of modern economy, and they actually have helped to establish modern economics and filled the significant that you can and it throws in the recent past. However today we like it. I think a framework for date of property rights establishing property data processing right can't allow you to be to use confetti right values on the data and Propel. You cannot be gross and unlock new values. And as we simply straight to ownership and then turn right there also questions of how is your day to be used

and this is how to make you pay to the question for a simple reason today see you or 51 top tech companies have send an alternative pushing for a unified data Privacy Law and they had discussions about this at the top of my in, Washington. Another type of challenge a far more responsible than any company is in sentence is actually we need to pay for new business models that figure out that what about you sent incentives can actually a lot of them are responsible to take

care of me and how can we properly and distribution Valley update on? So, you know a reason why are we not to mention any as a Khalifa no game where they play it in a collision and the usefulness updated it to categorize by utility functions. And we showed that the easiest icing you say Chalet the notion of how many Valium can find a way to see pictures of houses generated by the Collision of all players. This was the first proposed special place in 1953 and you actually provides the only distribution statistics is fisa section of desirable property

and see how we can develop also more efficient mechanisms to shapleigh value in English and machining setting and how the shop LeBron you can also help us to even a provides the fighter. kisses 42 poisoning and a better example another type of challenging as far do you think I'm more responsible than the economy is technology. He said she'd how can we build its technological solutions to enforce it arise City use policy and the value distribution essentially to your neighbor and your

fam for a response about the economy. So this is some work that's my research group as well. As my store have had seen working on finding the difference in the types of computer technology summation early including secure Computing. You said I can take your heart away and a cryptographic techniques to help protect assets from a leaking sensitive information. So promising to protect the alpha. Competition from leaking sensitive information to a provides a musical lot to you. She didn't

usage and complain and Elsa you think of anywhere they even if I can find me this differential component Technologies, we can I see Fields as security Chief is a Computing celebrate that serves as a foundation for a responsibility to the economy. So let me give you one country to use case what this type of Technology can't you enable the recently. We have seen a significant from direct-to-consumer genomics companies including 23andMe and ancestry. And what is the main reason for this company is the explanation for

this is becoming more and more concerned about how their genomic utilised Easy Settings. Still trying to process Maple and launching a process in partnership with a direct-to-consumer. I can't you enable a difference every time I in this case using the technology that I mentioned that it's actually for the first time we can't enable users to become owners while at the same time. You need to be utilized in a privacy preserving way to be able to control. How did you know? I'm excited at seeing you tonight and

also for the genomic company also help them to reduce a security risk and so on. So to summarize in this talk of talk about the different aspects at the intersection between a i and a security challenges and a really exciting research questions at the intersection between a TIA and a security system 76 and last names food. I'll compromise how to feel it tomorrow resilience the new system with a stronger guy and he how to build a privacy Chris event venue systems and how to build a responsibility to the dentist

is in collaboration with many of my grade students post Alex and the collaborators and the number of 9. + 2 + 2 Sunrise. I strongly believe that she couldn't promise it will be one of the biggest challenges. Thank you find a eye and it's It's important for us to build a responsibility to economy and these require a community challenges together. Thank you. Thank you for a great talk. Hope you folks have a question. Please. Use the Viking dials. Good morning

and thank you very much. I have a question regarding LaTanya Sweeney work. Are you aware of this like a she had likes to public databases and she crossed the two databases and they were able to identify people. How would you address this problem today? Possess a great example showing that such a simple a sufficient that said this year and see in the near future that start to recognize that does not mean that you can just make sure they sell that they tell at this

because they can still be today when and pink combined with other even publicly known that assassin song is a message including fights and 45 policy is designed to University High privacy guaranteed. hi, I'm Rosina wherever from Drexel University to my question is about the addressing the issues with adversarial instances. So at the time that you address that in your system is able to do better generalizations. Are you making them less transparent or how do you address a part of transparency when you solve the adversarial issues? What do you mean

by transparency and accountability is a reason why showing that song by Avatar examples to play some of our attention mechanism by eversor examples? You probably need to flip it on. Baby from Tongan Technologies. How do you how do you make others trust? You actually do not look at a federal bank processing data on my bank to provide. How do I update you to service provider? I think that's a very good question. So cool is that this year? We can actually feel to something with approval guaranteed. So that is from a technical perspective essentially Lankan

in the end. You'll want to make sure that the right system has been deployed and and all that. So yes, it's an important issue. So like to even iOS Windows Linux that was open so many things the how can you actually trusted that the whole platform is just about life. What technique has I didn't have time to mention as one can a remote entity can actually get it was caused a sensation so you can remotely a test to verify what's a program has been drawn inside

a secure enclave and the what's the initial statement has knowing it's actually being able to attest is that the right program has been wrong and then you're getting a Google Health different things can fall algorithms and people for example, the stop sign example you showed is unlikely to full humans while it's probably easier to do humans who is simulated voice, especially with the right intonation and Dachshund. I'm wondering if you are aware of studies that study what makes him difficult to fill humans and

you have powers to improve algorithm resilience and see if humans can benefit from other items because when a human can be pulled an algorithm, can I see the human to not be fooled? It's a question that how we can develop The Alchemist. That's a can house humans to be. Humans. Yes, I think. Estimation in the thinking of the talk. I really do hope that way to develop that actually can't help humans to defend against these type of issues with that miss you as the type of issues like her if they can fix music and

songs. So apologize this office at a project as being actually utilizing an LP techniques to build chatbots to help protect users from social engineering attacks. Actually. It's the most popular attack methods today and that has 80% off a task today 32 from a social engineering attack such as a phishing attack and human some time to recognize that this actually is a phishing attack so so In this project to actually working on developing and now he faced techniques in 15 at a pause for a sample

to detect when a male may be suspicious as in that case automatic play launch Challenge and response. Please endings the correspondence email to see whether the correspondence that she knows the appropriate answer and has proved that he else you know, who he claims to be like a co-worker relatives and song to see that's a very important directions to better utilize this technology to help protect humans die at the end of the day. We said that humans are the weakest link in the system.

Send a text. So I guess the first question is what makes me human smooth. It is against the law to talk to Alexa. So so busy today that's a very effective at attention tasks. But you said she's a very different things than what the human eye has theirs. So you think we need to we need to develop better and humans learn to be more effective against this dangerous time. Do I have to stop here for people to go jogging session if you would like to stick around to talk about this and I guess you would be here and let's think the

speaker again.