Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
Outwitting Deep Learning Models
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
100
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

Deep learning has captured the hearts and minds of ML engineers and researchers. However, is it always the best tool for the job? In this talk, we describe two NLP applications deployed at Facebook where deep learning classifiers were outperformed by simpler, more lightweight techniques. For detecting online content related to COVID, we show that data-driven regular expressions outperform deep learning in both precision and recall. For detecting clickbaity link titles in 20+ languages, we show that a carefully designed regularized Logistic Regression outperforms the Kaggle-winning BiGRU model by AUROC and pointwise precision/recall. We discuss characterstics of practixal problems that facilitate more pedestrian ML methods.

About speaker

Igor Markov
Research Scientist at Facebook

Igor Markov is an IEEE Fellow and an ACM Distinguished Scientist. He is currently a research scientist at Facebook working on AI infrastructure. He previously worked at Google Search and was a Professor at the University of Michigan, where twelve Ph.D. degrees were completed under his guidance.

View the profile
Share

Regal discussed several application where it is possible to do better than deep learning would simply, I'll get it Sam's, or at least improve deep learning blade. On some Blended learning models with us. And we're out there. I'll start by acknowledging. I was significant scientific and Engineering play with it. Even toward one of your days is from Facebook. Another one, another one. But aside from the time, was you planning a course with matters is the success in application.

And the success has been staggering in its breadth of topics. And Elsa in the quality of results. On the left is an example of a picture is interpreted by a deep Learning System is fines. I'm in a different parts of the picture that can be labeled and it creates a sentence that explains what's in the picture. So you have a combination of nature of language Generation, kieron image analysis and putting things together. Another example, here ulcer from Facebook real-time translation from French

into English, example of the right. So is the Lord and used to interpret the emotional development of fingers of the user interface in Oculus, and at the bottom, you see any alteration of a speech processing. So it's hard to argue against deep learning. But if you look at what it takes to implement, these Solutions are very, very difficult and resources under the trailer wallpaper from Google kind of adjusting. The machine learning is a high-interest credit card of technical debt,

and this is really why How many jobs exist for deep learning Engineers because there's a huge amounts of infrastructure needed to do all of this and this infrastructure changes. So the code it was written 2015-16. It really doesn't work today for many reasons, including your grades and saw and mice model. Huge amount of salt on top of this is difficult. As it turns out that application, only provide a good solution for the recent paper from September. Apply is deep learning to a well-defined problem of predicting future, considerations in The Game of

Life and The Game of Life. If you haven't heard, it's an old invention of John Conway. You have a grid And destroyed every cell, is it either alive or dead? And it start of changes by deterministic rules in terms of whether neighboring cells are dead or alive. All the neighbors are alive. They sell dies of basically over with Elation and all the neighboring cells are dead than to sell dies of boredom, because they're doing all their life doll. Throne of the bending of the rules, develop long-term

behavior of a sophistication of the problem. Here in a researcher has deep learning to solve his predict the future of a consideration given deterministic rules. There is complete information. Here is Candace dold by in a very simple. I'll get a chance with very little memory. Use It can also be sold by a convolutional neural network and they did celibate with me. Listen to. So basically, they found considerations were saying and solution exists. Now, the question is, can be trained on a large numbers label examples, and you can

generate as many as you want. These starts with random. Initial result was the training is good way, of course, you can do Lucky but on the average and high probability. It doesn't predict the future considerations for you. And the only thing that helps his upscale in the network, but then your kind of brute force in the problem. So, I know we have an example. She contrived maybe not very practical. We're deep learning is just not a good thing to you and I'll be going to look for it. That's it. But before we get there, let's review what

it takes to deploy is accepting application in production. The application is to be competition with fish and it needs to be robust in many ways. But stop providing data wise. Simplicity is very important, both for the implementation and for the dependencies, and of course, also for the integration with other Jesus typically in on proved and incremental notified, so it needs to be understandable and working with a solution. Even if it's been a reasonable. It may be undesirable because fragile which

sometimes they could be simpler and also compute power can be a problem. First of all, do you really have high-quality Training Day? Do you need continuous label and the reality changes because I'm production things. Can she stay a lot of attention to principles in Houston, ranking principles must be defensible. I need you can't explain what you're doing is doing. It's really difficult to get through, you know, policy approval and other question is bias, has it snowed in a black boxes that you don't understand exactly what your sister was doing. This could be

a problem. I'll give you examples were, you know, I was asked to work out a solution were. So particular example of how late is correctly and you have to be able to also apply your model, do different context with bigotry, post processing, that made easy or hard depending upon. If you're comparing people often tried and they didn't work and so he just going to say OK Google find. Let's try something else. How do you make a conclusive comparison to have a negative result with this takes time?

And it's not, it's hard to justify his reminders went, but he have a couple of examples here, which which allows makes that compares the application and balance one class much smaller than the others are. The first application is a classification or a text that is related to call the 19th and the 2nd and the comparison of the simpler solution, either spend alone, or as an add-on to when someone with people are many examples here is simplified the systems are a little bit more complicated that are in production, but, you know,

they should do it presented. Perfect. Example of a covid-19 textPlus application. We have deployed. Pacification is 64 languages from America Vietnamese, and it was mostly down by, you know, several people do the for this is applied, all the cross Facebook URL, Facebook, search for his Instagram hashtags account. Ban Vision Group named Boston, Common there, any Integrity applications were. There has been all sorts of misinformation in terms of David like what cures covid-19 first. Application was one of my people do a search and it started in February. We

showed them the car information center of the storm on the right, and this is a fairly restrictive application, right? You being wrong. Is, is it's really painful here by Precision is important. Highway Cole less important in other applications. Optional, you see a hashtag? India, flights, Do you want to detect that? But you know, what is difficult because you is, you need to buy the garbage different towards their spelling, various. We haven't seen before with k with two r's

and so on. Sometimes people use Euros instead of always double Coronel is, is there an addict are treatments that are the ninja and it was really on here much,. I just woke up location. This is some used to be integrated with people buy front page. Because again, disqualification is used all over Facebook. You need to perform real resource usage, Facebook. Mansion from the movie Source results. Resource consumption. In the spring people do in at Weiser more

resources because they were in line. Mower. Are there any full of matches? Did, you know, you could detect a Corona Extra? There's a city in the Philippines, coronavirus there, what's going on in San Diego on 66 languages. If you wanted to train a conventional machine, learning system units by quality data, he will not get this engine or medium in Mongolian. If it's not in time and not just loved it. Don't mention things change and you have to update your system to reflect changes.

Also AutoCheck, precision. And recall of the classification is a very challenging. I'm on Central changes in April, and in April, there were protests against lockdown until all the products that we're going on with work, you know, in the news. But it turned out that some of the systems learned that whatever Pro to stop them. They related to call it and then two months later or after the one month later, there were protests against police brutality and some of these systems, you know, fortunately that

didn't affect production, what they labeled it. Honestly is garbage late. So another issue is a different applications, require different position to check out Clinton jokes, be labeled as baggage related not for search and look for the comet information center, but perhaps for other purposes and one from column a and a borderline case, because this Existed before, of course, knowledge of the different name and also expected to coronavirus. Russia means coronavirus British. What is the word? That means don't get the virus. Fortunately. It's also related to,. Does it

turns out? But it could have been worse. The real deal breaker for a look at the dates. These are two titles from the birds and they describe on what happened after a particular viral video showed up in May. It was scolded. Plandemix. It's basically misinformation video by uniform or doctrine and this video went viral on social networks know, I'm for the second installment of this video. Some third-party research her learned about this one day in advance and you notified the community. On some particular Monday. I got a call that you need to upgrade our system to

account for, you know, this new video. There were several very distinctive keywords there but we did that and also other social networks. And then the same in a person at the verb. No. Included. That social success. Others can write with labels with training. You kind of do listen to 3 hours almost impossible. What about the details? The solution to scalable solution that is fairly old school. It's based on extensive use of regular expressions with big data and the infrastructure here uses Centex,

to equal 500 HP. Would you took some digging to to Define? What's the regular Expressions? By the way, if you haven't heard of them, their templates that in its capture large on in collections of different text variant, Retro Temple. Is any kind of terrestrial is in number, for sitting characters to do test matches everything. And if you can find, of course, but it's because regular expressions are in a one line, they may grow. And they're hard to understand the car tomorrow. One of the first things I figured out

it was on the right, how to create a multi-line regular expression has its own way of supporting multiple and regular expression. It turns out there is a clever trick with negative Lucas had, which I was not explain now, but it uses the sentence with question marks and explanation, mark, where you can create,, and the line ends as comments are not really used, so you can create a multi-line regular Expressions. This okay. Deployed in a very short pieces of configuration to all servers worldwide and make it available to all applications in pretty much trouble light.

Vietnam and their changes with SQL queries on Facebook search. This was a very early every powerful inside. This is important because it requires that people enter their full of keywords. And they also repeat. So you can see basically the most popular search queries of a kind and you see examples here. So the list show the results of a change in the regular expression were you, add some new mattress discount on used with some misspellings different countries and so on and on the right wing boots possessions to be

don't match Corona beer anymore. You don't match koronadal, which is a city in the Philippines. And if you don't match Corona beer and coordination and put another voice like that. So I know how did we discover a cure? This is mrs. Right. In the example, here, we started with just two key words,, North Carolina virus, which are used in many languages and he went through French text via everything that has covid-19 virus. And we see what else are you sort this words by, you know, how often did appear? Okay. So here are you needed to

discover words that are related to the topic. For example, in this case of college. It's a very gentle word. Do they call it a week? Keyword on, Sony read. All seven of stars of the week award, you going to do it by itself, but you can use it with other bills to find and call David garbage with underscores covid-19, and also find the spelling. Okay? And all the question, if they find us, you would you would very quickly how do you put them together into regular expression? I would really love to have the

full automatic deletion of Fortune. For that. Doesn't seem possible. So, we did as much of the nation as possible and as much human and I was thinking an intervention is necessary, you know, several features of regular expression to capture. You do spelling, very special character glasses. They could you see or k80? Muse Ward boundaries for shorty Awards, music 0 width, -2 excludes example, Corona beer, so we can match coronavirus not hold my beer. And when you sent the size

register and on the right is, she is a social distancing appeared early in the development of it says work. It was a good social. There was the word dispensing, but they're meteors when they appear in the face. That's what that's a strong face. So we can bind with the words end in a record for spelling words into bipartite rackets plazas were in the middle. You see in a 02? A t, i n o quantifier for any character and the? Means optional. It sounds like it's a necessary because you can sense evil characters. But what this

really means it means lazy Mansion instead of the default eager matching, which matches, as if you can answer this possible, not as many speed and then produces more inexplicable and also more match. I'll talk about it. Football match is over later, but just trying to express them to come up there and Sylvie distill them by the following piece of code, you know, about the what it does. It splits the records into chunks in keywords under my eyes. You see a list of those drunken keyboard in blanks. All copies of this trunk, rookie word,

indirect minutes of all the recall drop for the Red X and you sort the chunks in here, word by in a whole valuable. They were by this percentage drop. And so, you know, today was important because it's using a time. But then you see the second one was virus and infection. These are really useful if you don't want to do anything. But some keywords for example, epidemic doesn't seem very useful because it's whenever it's used. Also, something else has used. It appears to be close to the end of you and shrinking, these are agates. As we also can approve the match in

time, which is important. Dictate a scale in the timings here and work, you know, the timings for your warning system, you could see. This is much faster than it would be planning, can give you this also for the optimization V optimized regular expression, for Clear, example of Corona match, my, receipt possible, as an optional much, but it's when you look at them because, you know, two pages of text messages correctly because of what might otherwise you have to beat the entire. And it's also supports Black Box process,

some matching pictures work-from-home may be discounted, right? And you can have another layer potentially using some of the snow machine learning tools. You can basically, you, know. In this case, in a vaping epidemic was a smashing be intercepted. But of course, we can also verify the Rex. OK, Google data can be, it's not necessary but decided you're not rained on and help finding nothing cute. If you do have the key words, that are two general. And also for

unlevel order us, an example, where Virginia and trained in Russian was run in August and the phrases that is fine, but are actually related to protest against the Bureau of fraudulent election in August in Billerica. And so again, this is not correct, but we can interpret these things and we can decide whether our records just need to get one. Know what keyword Discovery. Let me see, baby. Repeat. What I said earlier that by looking at the matches, we can discover more keyboards. You can do this at level of

full document. You can also do this at the level of my Corona, and then virus. And there's something in between. There's something between us that I like the thought of being useful. And when you write his things and showed you a person to person is a comparison, to the end result of the sample 300% English Envy labeled than using these included positives of by Jaiden and boy bragging rights for the coverage was pretty good and we prevent male relation of exposed and you see that accuracy is

reasonable time because the prevalence is low tide for Boston, balance problem. I want you to read this to you, one is higher, Precision, Precision is 98 ninety-nine percent. And this is English. But there's over two most out of languages here to as much higher recall. And it's available for fewer languages. But no, more than ten of these are, you know, I used languages. They see that the record says, improve in both precision and recall, upon the email glossifier, which is pretty interesting, right? Given that, this is a much simpler technology.

It's much faster and much more likely, and I had some examples here of false positives for the dinner glossifier. Basically, just a lot of temporal over. What did you hit something to the data? And then where is regular expression encoder, understanding of language? No, I'm Devolution where she didn't perform any relation, 4626 languages, like it's already know Pages text message to Pete and this is more than others to be right now. Bye, bye. And repeat it, and basically build a short safe. Right? Here's an example of an

Arabic for most languages in 03 to 15 per cent per cent of my supposed to be checked. They were not caught by the Saints record. But this is ridiculous, especially confirm the correctness of mansions in most of the kids. And this automation is very impactful for the 3:15 % you can use automatic translations. That's so the many languages. But English that they performed earlier validated this methadone. Just summarize, never thought the classification for my taxes,

very efficient. In terms of performance is quick to revise the software. Self-contained, very internal problems with other systems for bust into changes and data. It's an old word for misspelling, continue individual, keywords and phrases and it's captures expansion Lane key word combinations, which would be difficult. Opponent system uses embeddings precomputed forwards from the Wikipedia. You already lost. You have to use Emergen-C words. We use expressions with negative look around, which is a really nasty featuring regular expression. He provide

explanations for every match and there's a white box and black box cost in Virginia. And unfortunately, you know, many of these things are not available as possible, but it takes time. Different synonyms repeat, you could definitely code them. Interact. Cats know, when their transition to the scepter the second ago. So this is a little trickier. This is click-bait falsification. And there are two types of clickbait on that. You just think of his Facebook exaggeration in the clickbait

in general is a title which would cause some information, exaggerated some information that makes you want to place and then you were disappointed because it's just not there. It's not what it looks, like. Are detailed guidelines and the guidelines to use a more nuanced than those in Chicago contest. We also support 20 + languages and we do have large set the labeled it. So this is kind of a more favorable contracts or deep Learning System. And the one beef have for this comparison is a black cat, that sure which is similar to eat. I kind of

winter, it's improved on. My waist is uses additional teachers. Were the Geico winner was featured less. You just use the word embeddings. Can I soak? You know, this is a simple model single model that uses multi-lingual inviting. Where is the build separate models? Not regular expression for different language, keyboards here because the same key word made participate in a click bait, a title or not. It's really the style of the title. Not the concert. Okay, and also, there's a lot of

subtlety there different scores of clickbait than us. So, just pick yes, or no. Most efficient is not good at. Okay. Here's the problem analysis, that work order in. It's not that important for clickbait because it's hard to find two examples of titles. That essentially have the same words but in different order, but different good bass. OK, Google helpful, but if so, We can use a bag of reward and Grimes action doesn't require understanding, you know, the semantics of the word so we can fit it in beddings are you know, a questionable necessity and also to shallow problem in

the sense that the equipment was the title of this year working with are typically less than 20 words for title and the median of subtle long-range word dependence. This is what deep learning is good for if it's someone could text her but you know what the problem is. I don't know what is important is capture level features and some of the deep learning model that you seen it difficult to combine. The word level features were important because there is a word form in multiple languages and their languages with optional spaces that shows the Burmese and

San Filippo. And also when you deal with one seed, lost languages, that's as important the available and they're limited. You don't have 7 for, you know, Burmese and some other language. They want the generic solution and they both alive with lots of fire that uses Warden, character level and grams is used as a very simple to f. I d affect the recession has recently shown actually surprisingly good for Southwest vacation. Because it can give you a very high dimensional. Usually 2 to 300 Dimensions. If i d, f,

d, 1 million or more David, get away with a hundred to one. And we'll see you tomorrow for teachers for teacher. 35 minutes over. Okay, I see Paul already. So, basically the comparison, I'll just show you the comparison and you know, this delightfully system, improves upon the buy group and this happens in a lot of different languages, but the best results are for on stumbling where you combine Universe, simple way the Dubai crew and the logistic regression

based on this is so I didn't tell him about the teachers that will develop, but that's one of the reasons why it works so well. So we can write a different, maybe a hammer, but not everything is a nail and life with pacifiers for problems, that are simple enough to give you fought the hell up in cycle relatively easy, chewning much, lighter computation and explicable results. And that's it. Thank you. Thank you for your core. Your presentation and I'll turn it over to Steve.

Cackle comments for the website

Buy this talk

Access to the talk “Outwitting Deep Learning Models”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
106
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Jon Krohn
Chief Data Scientist at untapt
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Vanessa Klotzman
PhD Student at UC Irvine
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Patrick Hagerty
Chief Data Scientist at Arena
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “Outwitting Deep Learning Models”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content