Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
Adversarial Fairness in the Labor Market
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
73
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

Unconscious bias in the hiring process has natural analogs in machine learning with biased training data and latent correlations. Policies introduced by The Equal Employment Opportunity Commission mitigate some of the biases but lag behind advancements in data analytics and machine learning. Adoption of machine learning algorithms in human resources can have great success in predicting key performance indicators (KPI) but has been slow based on the risk of introducing new bias into the process.

We present the notion of Adversarial Fairness to mitigate bias in the hiring process. Adapting generative adversarial networks, Adversarial Fairness uses two competing neural networks: the “generator” to predict a KPI and the “discriminator” to extract correlations to EEOC protected classes. The algorithm trains the networks to a point of Nash equilibrium: the “generator” is optimal subject to the constraint that there are no remaining latent correlations available to the “discriminator”—hence the “discriminator” is degenerately optimized.

We demonstrate Adversarial Fairness in a scaled production environment.

About speaker

Patrick Hagerty
Chief Data Scientist at Arena

Chief Data Scientist, Arena; Chief Data Scientist, Adaptive Management; Director of Research, In-Q-Tel; Applied Research Mathematician, U.S. Government; Ph.D. Mathematics, University of Michigan

View the profile
Share

Play, we talked about adversarial fairness in the labor market. I hope everyone can hear me not getting any feedback from the the backstage,. So I might check in a couple minutes to make sure that there is a that at the size are going through and that everything's an audio is working. My name is Patrick Haggerty. Is that Arena and can all talk about Avatar appearance in the labor market? So just a little bit of background on me. I've been in machine learning for over a decade. I spent over

over a 10-year US. Government doing difference between Applied Mathematics in different fields in Last 5 Years that I was there. I was really focused on on machine learning backs. And after that I went to Ink, you tell and worked a lot with the labs there and and worked particularly on such imagery using some of the emerging neural network structures and type of a generative adversarial networks to the, the Labour market. So, let's tell you today. If bought some of that, the really interesting impacts that we're having

with the with Hot Shot tools. The detailed abstract summary is the we're going to use adversarial networks to a battle or combat, unconscious bias and sometimes conscious bias in the hiring process, the little bit different than the general. The generative adversarial networks because there is not only a decision, whether it's real or not. There's actually another cost function that we try to offer my so that's a little bit of a, a different different strategy, but we'll talk about, not only

the, some of the details of what we do. But we're also talked about some of the ways we communicate that the power to to the clients and hear, the clients are not answering machine experts. The real impact of this. I think differentiates it from a lot about the other other machine learning aspects that I've had, is that this is a skill production environment. There's lots of things that I've done in, in, in the lab and experimenter or, or Affection, level machine learning machine. Learning effort was

about that. The first thing I want to stress, is that there's inefficiency in the labor market. This might not come as a surprise, but, but the hiring process, if you're looking for a job, we're trying to hurry. So it can be frustrating than a lot of our employers realize that that they're bad at hiring. There's lots of reasons why that's a difficult process. Sometimes there's too few applicants, the order in which you see applicants is is not the ideal order and sometimes you just have a fixed amount of times and

you might have only ten time to go through 10 good resumes in a row and after that you don't even look at what's up. What's at the bottom of a pile? That's why don't you just look at what's on the top of the pile. Last-in first-out type of phenomena to make labor market, inefficiency and vacations are required that aren't Directly related to Performance make everyone has seen that the number of years experience required for a certain job role. And what we want to do is we want to identify which of these requirements really are correlated to success in a in a position with which our sort may

be irrelevant requirements that might prohibit you from from hiring a really good person. The unconscious unconscious bias is the heart of this talk and and how to address that how to use machine learning to to attack on one of the most up, most of the problems in the hiring process that could have a large impact on on on, on some of the oppressed people are oppressed up parts of our society. So I think it's great. That way. We can use these tools to such as generative, adversarial networks to to actually make an impact in a positive direction when it

comes to the labor. One of the things that we learn from a lot of employers, is that they failed to Analyse historical data, whether it's why someone was, was was terminated, or why? So what made what made someone a good Roi for hire and in one of the one of those misconceptions that we have to have to have to explain to her class and there's really no such thing as a good applicant and applicant is one part of a multi set aside a problem. There's that the applicant there's the job. There's no employer

may be which specific departments are employed combined in order to have a successful tenure as successful experience and job role. So you might have an an applicant that fries and one position or one client at 1 client, but does not drive in another client and and we we we want to start disabuse them. Stop looking for the what, what's or called, good applicants, but trying to find good fits and that's that's where we use them over machine learning techniques. So the typical strategy is that we have a bunch of data.

We have a Target metric and we apply the standard Machinery, a little more detail of what makes this. This this problem probably a little bit different than other problems data collection is is much more difficult in the study. Look at that later from the applicant. We get data about the job, lots of third-party and open source data about about the region where the job is and the backs of facility may be, where they're deployed, another department of the deployed. But we also get a very large amount of client

data. In terms of performance, their 10-year, their engagement. Are these are all data that we try and we can use to build a remodel, so it's a performance-based. So we have the, the Entertainer we have the label data, which is The golden data, she just had the input. Your let me know what you can do. You can do. That's that's that's a key component of that scale. And that's something that that really helps strengthen our models. One of the typical techniques. We

have a lot of different dimension be used to mention the production techniques to, to reduce the number of features that go to her monologue. Part of the models. We use neural network part of the rainforest and part of the model. We use a logistic regression. And really that the thing anyone take away is that that forms. Well, there's no need to make a more complex model where there's a risk of overtraining. So great thing about logistic regression. It can be really powerful and it can be NBI. If you have enough data, it can

be very well. But if you have a neural network, you might have a risk of overfitting that and that's something that we have to be really conscious of especially if are where are giving feedback on applicants. So, when we do we use the the data available, of course, they amputated me. What do you do with the output? And one of the, one of the one of the challenges is that our customer is is I'm maybe not a data scientist. Could be a c-level person or a hiring manager at a at a facility and

they have no idea what area under the curve is. There's other issues were area curves. A for a 6in, auger them might not be appropriate because some of our employers that we have, might have over over a thousand, different job rules. And so do you create a And Ariana Kerr for each particular position, each particular match. And if you don't know, how do you, how do you find the outliers? How do you identify the outlier? So even the typical data science metrics, get a little confused, a lot of a lot of different role than you can. You can form a

stratified approach here is that we focus on the Gap, the return on investment in the impacts to come from there. So, if we can identify applicants that are not stay longer and there's a huge cost of hiring that has an economic impact. So, what we do is sort translate our our, our machine learning predictions to to actually return on investment and that's much easier for our clients. And our, our employers to To understand. But the one thing I really want to stress is at the prediction of human

events. And this is a really hard problem. I speak Haitian much more difficult than a lot of high performance machine learning techniques similar to a little bit of information that helps you make an informed decision. It can have a huge impact and that's that's what we see. Right. So, I think the key thing to take away, when it comes to the bias in machine learning is how is it measured historically, and how are we going to change how this is measured? So, seriously, this is measured by by a

fight, disparate impact nurses. This rule called the 80% rule, the positive rate of a particular category, compared to the positive rate of the best-performing category category. And then this prediction label satisfies, the 80% rule the current standard in order to measure disparate impact on on a minority class whether it's a race or a religion or sex or national origin. And we can, we can do better than that song that says we're we're we're using machine learning techniques to improve this. I'm putting this or that the scale

that weighs 2 to measure buys medication in a I may be early 2012 or so. Maybe that might be related to a protected class as described in that the equal employment opportunity commission you remove and you don't and puts in the model and then maybe satisfactory, but what we found was that there's Layton correlations that exists. What's the current status quo was? Well, it's as long as you satisfied that the forfeiture or the 80% rule, good job. Mister some statistical significance at need to be satisfied as well, but

still not still can lead to increasing. So we want to address this in a way that is actually takes advantage of the the power of of AI machine learning techniques that are a relatively recent. So the adversarial fairness as a Critic to try to identify bias in your ear predictions and you use an adversarial networks to make sure that you defeat the most powerful critic. So that's, that's the that's the outline and we'll go into a little more detail. Ideas that we have

generally we have a are going to remove lichen correlations in a relationship protected data in the input model and we create a game like network, but the difference is that we have to prediction of measures. We have one is, how old is it due on the actual prediction in terms of success in the, in the, in the labor market? And then after they're hiring? And the second one is the adversary has a typical of gas in. This is a secondary requirement as we have lots of the

EOC in performance data without that. You can't, you can't perform this task. So that's one of the things that makes this possible and the metric that we used to determine how well are are, all good. Works is the AZ of the atmosphere on that works over. This is something that we can quantify. We can quantify. How much a fine, a finer, a degrees in the the previous disparity role in the 80% rule? There are there pyrrole in from that? We, we can have. We can quantify how much, how much Slaton correlations are left based on our

our classifier and that are based on an adversarial Network as well. Some of the some of our implementations that we we've done using real data is that basically this adversarial network will will take to probability distributions over the inputs that might relate to two different classes. So here on the right hand side. We might have started off with a unawareness. What we moved out the correlations that were were, were there. And in what we what we're left with is, is it's over. But a difference that you can

see, maybe a little bit of difference in might satisfy the 80% rule, but after doing our apisero training are indistinguishable and that's, that's that's great. So when we, when we compare the Democratic Party in the patrol, we can see the impact impact is before, or by his medication technically, or we are within the, the good range. But we certainly made it more fair and our our strategies as we get more data. As we as we get more more clients and more information. We can create a stronger medication models. But where are you improving?

Improving? What the, what was, what was the status quo? And I think we're doing it using some of the more advanced machine learning techniques that are available. Throw the key takeaway. I think I'm right at the time but the key takeaway is that the metrics are hard to choose the appropriate metric for a client and the data sources. When you have some of these techniques that remove bias, you ain't open yourself up to more more data sources in order to create the rate. The model, for example, continue time as one that might have

some late in correlation to protective glasses, but could be extremely useful. In determining how long someone's days and fix our position. We can quantify by us are using the AC and this isn't production at Arena or really excited. It's been applied tomorrow more than 500,000 applicants this year. And this is something that relation removal was considered impossible prior and now we're using some of the most advanced Really excited at Arena and I think I'm I'm at time so apologize and get any feedback from the

you're doing. Good. Can you hear me? Patrick Patrick, what a fantastic presentation at the current time. We don't have any questions. So there may be later. So what we're going to do time is fantastic information. You will just share with all the attendees. So since we have no questions at this time, we're going to invite everyone because we do have a break from 3. 23205 to go into track two of the ML and sign. There's a 3:15 to 3:45 as well as tracks 3 ml in business.

So again, we invite you to move over to the track for, from 3:15, to 3:45, ML and Sciences. In businesses, that you'll see the tracks. You have an opportunity to choose will be back to the main stage with another fantastic speaker at 3:15 in Italy. We will see everybody back here to the main stage at 3:45. Thank you Kim, Patrick, and if you can exit the stage Patrick. If there's any questions, Patrick will answer them offline. And again, thank you Patrick for a fantastic presentation. You can exit the stage now.

Cackle comments for the website

Buy this talk

Access to the talk “Adversarial Fairness in the Labor Market”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
106
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Dan Gifford
Senior Data Scientist at Getty Images
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Igor Markov
Research Scientist at Facebook
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Cristine Marsh
Data Scientist at Affirm
+ 1 speaker
Isaac Joseph
Software Engineer at Affirm
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “Adversarial Fairness in the Labor Market”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content