Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
Still using one hot encoding? Here’s some ‘HELP’!
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
111
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

Old habits die hard. At the heart of supervised classification in machine learning lies one such ‘old habit’ of One Hot Encoding (OHE) target class labels. In spite of a substantial body of literature that demonstrates the shortcomings of using this simple technique, it continues to be considered the default way to represent categorical labels in a standard machine learning pipeline. In this talk, we embark on an exploration, trying to understand how to and what happens when we replace the one-hot-encoded label vectors with dense lower-dimensional continuous valued smooth label vectors. In cases where we lack any semantic insights into the label space, we intuitively try to keep the dense label vectors as far apart as possible, which maps to solving the equiangular Grassmannian line packing problem. We demonstrate the efficacy of this new technique across many classification problems that arise in Computer vision, Natural Language Processing, Time series analysis and Speech processing. We conclude by sifting through the open sourced python implementation of this idea and carrying out a quick live demo that showcases how easy it is to use in a machine learner’s pipeline.

About speaker

Vinay Prabhu
Chief Scientist at UnifyID

Vinay Prabhu is currently on a mission to model human kinematics using motion sensors on smartphones paving the way for numerous breakthroughs in areas such as passive password-free authentication, geriatric care, neuro-degenerative disease modeling, fitness and augmented reality. He is currently the Chief Scientist at UnifyID Inc and has over 30 peer reviewed publications in areas spanning Physical layer wireless communications, Estimation theory, Information Theory, Network Sciences and Machine Learning. His recent research projects include Deep Connectomics networks, Grassmannian initialization, SAT: Synthetic-seed-Augment-Transfer framework and the Kannada-MNIST dataset. He holds a PhD from Carnegie Mellon University and an MS from Indian Institute of Technology-Madras. In his spare time, he works on his cricketing skills and generative art projects, some of which have made it to the playa at Black Rock City.

View the profile
Share

Hello everyone and welcome back. We have been a problem Chief scientist from unify ID and he is ready to share all this amazing information. Go ahead. Awesome,, Thank you so much for having me hear this my second demo Khan's. I enjoyed the previous Edition as well as for the organizers for pulling this basic and down street, to work on, trying to kind of options Beyond one heart, and coding. And the job title is held on the same problems as it is used in a default. Wait, let me Begin by

session of cattle years. Should you not be able to take out anything? Useful are important for my dog, you'll have the spotlight with you and those who have autonomous muscles in the ears. So, let's get down to the dock of my station. The dog is split into five sections. And then I will be so we can do a quick survey of what people want to kind of demonstrate the else and shortcomings of Walmart in pouring a few reviews in your classification. And visit this mystical realm of brass minion packing an Optimus face back in to try and understand

what I heard from this. Rather make sure an interesting mathematical field and how can also contribute back to this field. Last see we'll talk about how to perform incense if your training with these known one other important factors and then I'm kind of give a brief tour of the tools that we have to return for you to use and how are you thinking of you back to this community? So the marketplace of ideas and trying to understand? Why is it that suboptimal techniques and bad ideas

have been survive the test of time and You find this kind of thing about, you know, preferring the certainty of mystery to the mercy of uncertainty there, different flavors of the, like, the Perfection is, the enemy of something that works or unknown. Angel is better than an unknown devil, which is why, you know, what ideas? Like Siri have still God died, the national debt. And once the idea that I'd like to motivate, but then I supervise classification as using one hot n flooring problems. So what does it

have the option to have a turtle snail and a butterfly? And then you have the ordinal word with that is originality between the categories. So, example, would be a man happy and awesome. So these agents of the level of happiness, that can be, Angola 0.5 a month and then you have the Word of Life by Noriko present, or absent. And so, so, so, Basically means that he's like a snail and a butterfly number to Round. 100 would be represented by the court by DR. Horton, all platforms. Here.

You see relaxation of me using it by using the situ Clan. Pre-processing model that has won more than quarter in a pimp lamented, you have somebody fermentation in all different depending Frameworks. How many large numbers? So it may be worth paper 16th at 4 every lead. They have the number of the cardinality of a set of categories. So they have figured this out of these folks have, you know, basically have made the transition from the world of 1 hour and

go to option to the world of, you know, smooth dance, better representation and this time and we have kind of been based on actual representation of words or sentences for the word where I can send us to wait favors. And if you like the lay of the land board in a few people, have done this, a beautiful survey paper out. Recently by deepmind. Please, take a look at that. The folks who have a grapple with a very large cardinalities. Have kind of mood away from the world of options to load. I'm still in pain. So what about the

output side of things? What happens when the the number of output classes and explodes on the production side machine learning tutorial, San Jose is that you have and then you train your new glasses and Additional interest in the last in the Peace of Mind Slayer and then, but 7th at 10 classes. I didn't like you'll get to see this increase in the output parameters. Every time you set up interviews a new class, right? Or a new situation. But if you have a very large number of classes, the first goal of the heavy lifting doing, this interesting kind of twisting

and turning of the representation Space, Unicorn us different flavors of money for the future. You will see that you are looking into the new real real estate picture in the last layer. Where is just kind of throwing hyperplanes and boundaries between the On a different presentations that are made available in the previous 18,000 different categories in the last layer, time doing the, non interesting of the dumb things. And then, I'm going to repost, you see that most of the times, I see acquired during the

last day of the least interesting things are going to have to send a picture. If you have gone through the worst of and focus on training, only the first few layers. Transmission. Do you have a question. So this kind of frustration over the last hundred kilometers residing in the most dumper than you'll ever get a copy of, my sleep has been attacked on two fronts. The 2015, paper off that introduced into the computer from using these puritanical, one that has benefits both in terms of satellites training, as well as Give me a better

accuracy and the 2015 paper on North Station Tanner Fox on the available in the soft option of the modern from a student model train and but they basically have like soft noises of my values across space. Similarly 2016 paper by Francois salad information Beyond endurance. Recent, you know, you have this very nice paper titled angular bushel hardness by the group from a Catholic and a video of their kind of motivating like looking Beyond using Pandora to cross entropy asses.

Go to Lost at one or two. Use in a supervised. Classification framework. Using brass minion packing and the motivation goes something like this. So I managed to get away from these in time. It snows, password code Adventures to go into space and encoding. So the question is, who's going to give you the importance in the first place? Like, MSP people who had a huge Treasure Trove. Text you later. If I give you a new classification problem, I'm busy trying to pacify different types of nuts and bolts on a production

Pipeline. And the question is, how do you account for these things without domain knowledge? So, let's say that you don't have any looks like. So you busy have no romantic Notions about what the categories are and what you have to assume that given that you don't know anything about the label vectors. The cool thing about 144 that late. Victor has to be paid by secret distance between any two glasses and even a specific and bring Diamond chain, you want them to be as far apart as possible. So the efficient as possible.

And then using this one hundred and headed back to make a should not require a lot of training and using autoencoders and stuff like that. They should be off the shelf and you should have like a bunch of that. You don't have to spend any more time in this. Leave finding better than brains. So what you're trying to do it from the 6 time and trouble had to go to the victors into Lord. And I know for a fact that this visitor of snow will try to understand how it is that they can

a nurse all of these, this massacre books at satisfy all of these damn trains. And knowingly, or unknowingly this kind of forced into the larger Gambit of problems, called the groceries and do this, when they're kind of talking oranges. Do you have like a talk about like the ubiquity of a hexagonal screws packing of spheres in the nature around us nature? Does Optimum speed stacking. I'll be through it in the world of Applied Mathematics, the brief tour and Tails. The fact that you have the problem of the problem, specification

that was done by Craftsmen and sending brass minutes, please starting a special purpose map. That is the space that you're trying to invest in a larger mother space is basically on Dimension one. And a complex. The main campus emergency was heavily involved. Sanitation's, so you have this Treasure Trove of optimal cord books that you can shop for grass when it comes to eat. Still a lot of work to be done and you have to use the spirit of the optimal cord book construction techniques or have to pay Bank on the shoulders of giants who worked

in the domain of a zebra, grass, Terry and using the inspiration. And you know, you can also kind of see that you know, if it has this kind of know which is in the form of like it's going to be. So if you want to keep these lines as far apart as possible, depending on how demanding yard in terms of like the angle of Separation, then you have to kind of going to a higher dimensional space. So this is kind of seeing in the dark mad at you, kind of try to increase the space in between

the lines. When you have to go into higher and higher Dimension spaces, and then, if you say that, you don't want, I want my angles to be or talk to each other. You know, the one that stays so given this into real-world problems in the inside. If you are not willing to relax this demand a little bit and you say that, you know, you don't have to be exactly equal distant from each other in the angler to mean, I'm okay with nearly the angular. Then you can kind of go to match your domains. I saw the Lord Amazon spaces and we have, you know, and you can just take

this in a code poo73 computer and just drag and drop it in your pipeline. And you will see that as you can, I'm going to lower and don't want to go down to 5. And when I mentioned, then you'll see the increasing concentration, that happens, when all of them are almost out of the Fall In The Sweet Spot between 86.5 degrees. 90 degrees of separation between each other and ask you kind of make your if you're more and more demanding in terms of it very well. Then you have to

see this happening. And this beautiful phenomena is recently being studied by covering minds and Applied Mathematics that has been so pleased to pass this. Will they work? It's a very interesting line of work that is from people who are red wires for TV programming and Country to the domain of mathematics to attend at the dock, to Legacy and Tails using one out and coding schemes. And they sleep in your life was when he lost and using our my sport in French, but now, we're going to Bay City using is abusing

and Recreation Frameworks by using a glass or similar loss. The question is, how do we do infants? So we have this work and this time in your life, what kind of speaks out a dense and bring what does not. Like, you know, you can kind of have this off, my flight interpretations disease. Then what you do is you go back and you find the nearest neighbors are the biggest book in the angular to me. And then the infant's now becomes nearest neighbor search. Total

vision problems where your classes are. Basically this one have this dance and bearings and then put in friends. You're basically doing our men over the angular space of the question is, isn't this an expensive thing that you have this extra gadgetry? That's required for infants before they could just do our minds that we have to do our men in the angular to me and science. Lee. This is where you have kind of meeting of two powerful ideas. So one was basically face, which is like a faster-than-light is a child

and bring your business with the competition muscle part of the city that is actually implemented in the library with in the Rapids. Are you have magic that happens? So for example, you can kind of if you have the estimate intense and brains for all of the nation at the $50 glasses. You can do the influence on like a burrito bowl, commercial DP, like sitting right there. So I was on the entirety of the six seconds. So if you were to use something like an office and fermentation, that is the Katy Tree

on scikit-learn, I'm using fire and damage to, and brings it takes me about 25 minutes for the entire incident to happen. Braswell. School. Ml, you get a hundred and three times. So, this is rather spectacular against the holes Boogeyman off. Like a busy having to do this cumbersome. Your nearest neighbor search because you have the meaning of a fast. Play on gpus. So if you basically have a preacher in your life work, that's already been trained on more important matters. If you want to come by and

you can just focus on the last, you know, where are you? You busy, you know that aggression problem and you'd be able to use. And then you can't use this fast, nearest neighbor, search. And you'll be able to deploy it by making like, you know, profound games and Times of like the country without making, even a little bit of surgery in terms of having to do any sort of in a lottery ticket, search kind of proposed this fancy thing doesn't even work. So we kind of have done some studies in the domain of computer vision, the standards

and the endless. Yes, it works off the shelf. We tried it. Yes, it works, fine CDs, classification on the island. And, yes, it's true of by just using off-the-shelf architectures and just be paying the last leader. We also got a slight Improvement since this classification. So I can see the proof is in the pudding. Yes, we agree. And we have demonstrated through rigorous and example, that might happen to me. That, yes, it works seamlessly well, and if you're interested in trying to understand as

to, okay, you have seemed to have this new. We are. Basically using a dense and buildings that are available off-the-shelf and you have ESP computer code books. How do I use this? And, you know, how do I can't bring that intuition about this to mean? And how do I pump your back to this section panties with that? So I can't do a way to take a pass to what we proposed. So what is the official letter? What happened with in supervised learning to use 100 coated labels for your classes and a growing sense of resentment,

and you have this? Motivation towards a golden Community. To go to word. Angular to me when you can. He's refusing this off. My son is returned, not really property values and it's very hard and your interviews. And then the interpretations can be had in terms of like the ocean between the prototypes of the glasses being modern BBC proposed a solution by looking into the huge amount of work available within optimal speed. I think you're going to do you want to keep the Glass Pro twice as far apart as possible and you want to be doing this off the shelf and people have daughters and their kind of

standing on the shoulders of giants by looking into things like to pick up your record book and then we basically so how can use them across different to me and I should you be interested in trying to understand the lay of lines. How to import Discord books from this treasure troves than an orderly available that I've been precomputed by applied mathematicians blink baby kind of give you a demonstration. So as to how to go through these theoretical code books and how to download them and use them in your pipe

lines and American demonstration to all that you need to do is basically type in the number of classes and then you just called one function with his LP and his coach. And then you pass this number of classes. And then you can also give this a patient screen to start the outcomes. And as you can see, it's a super-fast instantaneously will get this code books. You literally have to spend the 6 number of classes Boston into this function and out comes the code book that you can now use instead of one heart and coded labels in your pipeline. Just be Play Ciara.

I mean, Can you go to cross entropy loss with an i and you faint? If you have or don't have anything from scratch? This in a business, all the regression problem for the last year can be done efficiently. Using Nvidia Rapids Hospital. And if you're interested in this kind of looking at Moss Ematic options available to sign of curated this. So, you know, this is semantics available within the little space. You can use it rain in Billings from LP live in the form of beans. Are

you can use if you have more knowledge about your little space so you can kind of different models of now with not 100 so that you can kind of get more, you can get influences in the angular to be similarly. Like I said, if you're proposing a new working on a new architecture, you busy have to clean it up. So you can kind of, you do use a Peach Tree, Inn in Billings and three Train chords, and you can breathe in, only the last layer of your new network, to be to get a moto version. So

here you can see that, you know, you had like 2048 damage nose piece off, my inspectors and then this would have been $1,000 Mansion a local player and you're going to New York, kind of fitting it down to 500%,, go that has to be in broccoli, kind of validated. And if we have time, I can kind of go through some testing done. That's busy say that I'm going to Lorde Amazon space. It's not a remote on a fall and I can see if they have these ups and downs. And that's very interesting in itself. So that's a lot of work to be done in here. So this is an awful contributions

to the community that I didn't stop. Then we want to build a bridge between Both Worlds and the Sabina action breaks. Do you have the word of grass? Begin biking Sandy's optimal packings, second? No, use a GPU stand of Hanford, better cold books in the spirit of The Optimist regions where there are no available and the Honda still going on within the domain of computer. Gently in a supervised classification that you are trying to train your liquids, but especially with large number of classes, you are looking for, you know, a non one hardcore adoptions. Do you want to find some place

your last year when most of the arrangements? So this is kind of, you know, any games here can be identified in to your new leopard out later. I'm by the weather will be in use this new level 10 base design or even hunting algorithms to pass them up with the stipulated that in this league. Have generate more accurate are the optimal codes, especially in regions, where there are absolutely no Altima code books available. He was all great. Please, wrap

up the If you have any further questions, feel free to DM me and this is the applied mathematicians as well as the machine learning of practitioners. If you are busy, using a bun out and current better, especially in the large label space, this is an excellent technique. You don't have to kind of train anything from scratch. These are pretty available code books that I can download from 1 to 3. If you are interested in contributing to the domain of Applied Mathematics, when you want to hang for better code books, no one's actually kind of used deep use two

of them. How many do we have? The bike or scooter available for you to use? Please do go to the suppositories. If you have any questions, just send a message and thank you so very much for your time. If you have any questions, I'm all ears your fantastic vinay. At this point. We have Galena standing by. So anyone with questions, please enter them in your stage chat and they will be monitoring that chat feature and you'll be answering your questions live on that chat feature. So we thank you. It was fantastic vinay.

Cackle comments for the website

Buy this talk

Access to the talk “Still using one hot encoding? Here’s some ‘HELP’!”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
104
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Jon Krohn
Chief Data Scientist at untapt
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Meghana Ravikumar
Machine Learning Engineer at SigOpt
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Cristine Marsh
Data Scientist at Affirm
+ 1 speaker
Isaac Joseph
Software Engineer at Affirm
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “Still using one hot encoding? Here’s some ‘HELP’!”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content