I am a senior platform engineer at Ibotta, a cash back for shopping mobile app. Before getting into the tech world, I worked in public K-12 education for 11 years. I transitioned from education into technology by way of the Turing School of Software Design, a Denver based code school with a Ruby-centric curriculum.View the profile
About the talk
White-boarding is not nice. An unpaid take home project is not nice. We decided to apply the Ruby community motto "Matz is nice and so we are nice," to our technical interview process. Come learn what changes we made, how we enlisted support of other rubyists and non-rubyists alike, and how you can too.
I am a senior platform engineer at Ibotta, a cash back for shopping mobile app. Before getting into the tech world, I worked in public K-12 education for 11 years. I transitioned from education into technology by way of the Turing School of Software Design, a Denver based code school with a Ruby-centric curriculum.
What hello? My friends were coming up on the midday ish, mid to afternoon, ish time, at least Central and United North America, South America time zones, but for the rest of you who are on all over the world time zones, thanks for kicking and staying with us for another live session. I just wanted to show you all so excuse the entire, but I put in my 5K today for the 5K challenge. The virtual Rubicon 520, if you haven't seen it checked, it's there. I encourage you to get going. If you haven't, it's not a 5k. You can also do 30 minutes. Anyway, enough of that.
Let's just Dive Right In. So our next speaker is Jesse, so I want to welcome him to the stage Jesse, Jesse, Jesse. Hello. Hello blue. Alright, okay. So Jess is going to kick it off. Helping end kind of close out. The final sequence of events for day to Jesse the stage is all yours. Thank you. Thank you. So, hello Rudy, I wanted to just start by giving it a huge. Thank you to the Rubicon planning team. So Abby and Marty were like the primary contact people for me, but I
know that there's a ton of folks proposals and put a big thank you to them and a big, thank you to our sponsors. And my biggest thinks of all the folks you were doing and right now. I really, I really appreciate it and thank you so much. And this conference is really, it's just such a treat right now in the midst of some very uncertain times in the United States across the globe. And I'm just so grateful to be here with all of you. So my name is Jesse spivack. Are you see him pronouns? I'm a Staff engineer at
Ibotta, cash back for shopping app and I bought a house a great engineering culture. We get to use some of the most cutting-edge tools in the business. The work is super interesting both because the domain is pretty complicated. And also the scale is pretty big. And most importantly, the team is filled with great people who have like really high expectations for themselves and so yeah, I definitely highly recommend it if you're looking for your next thing. So personally I took a non-traditional path to get into software. I worked for 11
years in the K-12 education, space and so I came to software by way of a coding school in Denver call, the tree school for supper and design and if you, if you have friends or family who were thinking about a second career and we've got this great channel in stock right now. For all the second career folks who are at Ruby, you should definitely, definitely time to tutor and have them check that out. I think it's an amazing program and it's been really good for me. So let's get started. So today I want to First talked about how interviewing is broken
for the job Seeker. Next, I want to talk about how interviewing is broken for the interviewer. I will use storage for my own experience interviewing hundreds of candidates and also being interviewed myself a few times and I want to try to paint a picture of the current state of interviewing. And then I'll share how I worked with my team and I bought a to reimagine our interview process and drawn lessons that that really I've learned in this community. And so finally, I'll share some strategies that you can use to reshape hiring at your own company's whether you're the CTO
or Junior developer just starting out. So They're sort of like been kind of a history of people who talk about interviews in this community. And I just want to shout out a major inspiration for me, which is Eric Weinstein. Real scum, 2019, talk interview them where there where they are. It completely jolted me out of my own complacent acceptance of the current state of the interviewing, and I hope that this talk will have a similar effect on some of you out there. So let's get going. I want to start out by a talking about how interviewing
you experience from almost four years ago. Because I think it is pretty illustrative of a typical interviews at that time and to some extent today as well. So perhaps this will remind you of your last interview process or maybe you closely resembles the process at your current company. So the first step was a phone call with human resources and the purpose of this phone call. I presume I was to make sure that I didn't sound too much like a crazy person and I got my salary expectations were in line with what the
company is going to offer. So right out of the gate, this process is very broken and we're going to go on a small change. So just bear with me, Corporate America, systematically share, salary information by way of Consulting group's that collect data in a way that the workers of America do not. So, nearly every HR team is paid for access to the data. And even if, even if I do not, I have access to that data. They have had far more salary conversations than typical developer and so therefore the interview is always at a
disadvantage at this point of the interview, the asymmetrical, understanding of developer compensation. What's a kind of pause and consider? How to say symmetry can be felt more acutely, buy some groups than others. So, if you're a white dude like me, you were statistically making 5.4% more than your female colleagues in the same developer rolls with the same level of experience. According to Glassdoor is progress on the gender, pay Gap, 2019 hired. Inks wage inequality report shows that similar gaps When comparing the workforce on racial and sexual identities.
So, John has to do with a white male. Colleague is paid $0.91 for every dollar, white male is paid and mail, lgbtq colleagues, and make 96 cents on the dollar and it's even worse for female, lgbtq colleagues and two cents on the dollar in. This is obviously not exhaustive exhausted, but clearly, The takeaway here is that salary. Transparency is very important if we want to end racial discrimination and we need to pressure on employers to stop, trying to save money off the
Ripple effects of systemic racism. Sexism. And we should stop writing job descriptions. Like I'm currently, we should stop writing job. Descriptions us a competitive pay interaction, listing compensation numbers. Do returns my own hiring process. After my phone screen and I had an interview with the engineering manager, whose team I was applying to join and the purpose of his conversation was first to give the manager chance to gauge my potential value to the team and second to get me excited about
about working through the gauntlet that's put me through. So after the interview, I was given a take-home project to build a set of apis that could perform various crud operations on Words and their anagrams. A lot of things said about take home projects in this community and D H, H R Block poster. He said, I've heard many Fair complains that companies are asking candidates to complete massive projects that may take 20 30, 40 hours of work, which is all unpaid and which might be difficult for the candidates. Existing job in life. Don't do that. So for me, I
spent 16 hours in form and seeing all the bonus and clients writing unit in integration tests and putting together a comprehensive readme file. I remember finishing the project and sing to my wife. This is the best project I've ever done a few days. After submitting the project I was invited to do a remote project and I was just so excited to hear like how great the 15 hours of work. I put into my anagram baby. I was, and I was sure that the two days of my wife single-handedly, caring
for our one year old twins. Will I coded in our bedroom was not going to be for not? So during the feedback session, I walk to the code with two Engineers that I bought it. And as someone who recently had no professional coding experience at the time. I actually assumed that these Engineers jobs were just to give feedback on code. That was written that I bought it. And let me tell you, they gave me quite a bit of feedback. So they said things like I think you're meeting calculation is not right. Can you explain why all why your wife calling flat in there? Why did you
choose to have to optimize that query at which I thought I did? All these are just like a few of the highlights in the conversation. It was brutal. I came out of the bedroom, like dripping sweat. Totally dejected. I told my wife that I was not going to get the job, and it was brutal. So let's summarize all the way as it does take them project days is completely broken. So first Ibotta is not an anagram of business, it's a cash back for shopping in retrospect knowing that the past like knowing what the past three and a half years at work has been like for me and I
bought it. I can see there's basically no overlap between the code that I wrote To Sir Bananagrams and the code that I write to Gwar Shoppers for their purchases. The next, the project took me 16 hours for which I was paid $0. I was desperate to get my foot in the door so I did it. And many other folks would not or could not use just try to get in their home life constraints. They're probably the hardest part of the whole thing was just a child care. This would have been completely undoable. Finally, after the feedback session, I was left with
the impression that I was not smart enough to work it Ibotta. But to my surprise, I was Call Jennifer half-day in-person interview. I enter a room with three Engineers waiting for me, and I was asked a few questions about my background and then I was pointed hat to a whiteboard on the wall where I had to write a function that takes in an array where each element is either an integer or a similar Ray and help with the sum of all the Senators. This problem you can solve by writing a recursive function. And after I went through that, I was asked to model the game of
chess on the Whiteboard. After that, I was given the option to ask the interviewer. So few questions about Ibotta. And then a totally new set of Engineers came in and repeated that process. Exactly. So this time I had to write a function to reduce overlapping ranges on a number line. And this question has a trick, which of course I did not know. And that's the tricky is that if you start out by sorting these list, I need a list of ranges in order of either their minimum or maximum point. The algorithm for combined, in the overlapping ranges becomes way simpler
through the problems with this process, right? The first of all right, out of the gate. I was asked to write a recursive function in front of an audience and I can tell you that is quite large. And the only thing I've ever seen in as many hundreds or maybe, even thousands of files, I actually removed a year ago because it was, as much as I do. 3rd, the ranger juice story, trick is really just a trick. And if you read, like, cracking the coding interview or very close to do computer, science course work. You like, we know this trick, I'm not confident at all
that, there's a causal relationship between knowing these kinds of tricks and being a productive member of the team, finally, and I haven't yet, but the only non white male. I spoke to throughout my entire interview with Kayla from HR So it's like gentless generalize my experience a bit. First. I had a time-consuming on pay project second. I was made to feel that I had done poorly on this project and would therefore not be a successful employee. Third, I was asked to perform at a whiteboard on promise that were not representative of the work that I
was interviewing to do. A. Most importantly, I had an easy people make all sorts of assumptions about me based on how I look and typically in and speak and typically, especially in the software functions work in my favor. If I was not a white dude, statistically I'd be doing all of this for the opportunity to earn less money to boot. So in summary in, this is really what I want to run home with the process is just not nice. So next, I want to kind of turn to how this process is broken from the interview with respect. I think a lot of time
looking at the chat, I think a lot of us agree. The interview process is broken for the person looking for the job but I'm going to nap present my case of why this isn't terrible process for the interviewer. Also, over the last four years, I've LED media conservatively like, 200 technical interviews that go through interviews from the interview is to collect data to make a guess at the potential value that the candidate is going to bring to the team. So ineffective process, if this was a good process,
right? We we get the interview with the interview with a lot of data that makes estimating a potential value very, very accurate for Gary e z. Joann Ibotta, we use a 4-point scale of a strong. No, no, yes and strong yet. So I want to show you like roughly how that plays out in my history of interviewing so I given only one or two Only one or two Strong Nose over those two hundred interviews. And now in those cases, the applicant didn't get the job. So I can't
really like use the experience of working alongside that person to to to to to turn in like the accuracy of that prediction. So I've also given fewer than ten strong yeses out of those two hundred interviews and in those cases, the applicants did go on a job but only some of them accepted the offer. So now out of the strongest is that I have had the experience of working with which is like some number between one and ten. I can say that I was definitely wrong one time. So one string, yes was wrong. Which tells me that like roughly 10 to 20% of my
strong. Yes, predictions are not accurate. So what's left now is the vast majority of interviews that I've done where I haven't felt strongly confident in the data. That that I've collected and hedge by only saying either. Yes or no as opposed to strong yesterday from you. So here's where the data quality of this interview process can really lead to some mistakes. So, in some cases, I did go on to work with folks. I said no to and was pleasantly surprised right by the value of the first present, either. Amazing. I can't imagine, like, not working with them. In other
cases, I did go on to work with those. I said yes to answer had the opposite experience, so it could be that, like, I have terrible judgment, and that someone better at reading people asking questions and listening, I will have fewer misses on the interview process. I just subscribed. But I actually think I have like average to slightly above-average judgment and and so therefore I think the problem really is Want to date equality. The process does not provide the interviewer with sufficient quality data, to empower the interviewer to make a decision with a high degree of confidence.
So to see why does might be the case was just look back at that process one more time. We've asked our candidate to complete, a take-home project with very little in common, with the actual job. Then after we have survived our Brew, after they've survived their brutal feedback on their work that we did not compensate them for we make them answer questions on a whiteboard. These are questions that we already know the answer to and are very dissimilar from the work. We will ask them to do when they join our team. So again this process just isn't nice.
So we have a saying in the community and is actually our unofficial motto. Maybe some of, you know, it is the creator of his wonderful programming language should be the opening keynote. Yukihiro Matsumoto or mass, for sure. Right? Mass is nice and so we are nice. So overtime. I'm guessing you can get the full story from someone else. At this conference has heard they acted in the community a little bit longer than I have. But the saying got shortened down into meanest one so mass is nice. And so we are nice. What if
the weather? We interviewed was nice right after all matches nice and so we are nice. So answer the meanest Swan interview, So, how to dive into the components of a Venus one interview? But I want to have first point out that the meanest one interview is. More of a, is, it is a mindset, right? It's not a strict process to hear to the idea that and it's a big idea, right? If we are nice to each other everyone benefits and so I'll just drive our interview. But I'm not arguing that everybody go out and adopt this exact format. As I described our food
processor, I'll try to point out the parts that I think evidence army knife on approach and also describes Mary's Improvement areas where we can be nicer. So arminius one interview is much shorter than the process. I went through. So, all other things being equal, this alone is a win for the candidate and for the team even if our data is just an unreliable, at least you're spending less time collecting it. We still have the phone screen in the manager interview project completely after the manager gives Schedule, a 90-minute technical interview with
engineers and this interview has four parts. So import wine, we introduced ourselves. We give an overview of the interview and let the candidate introduce themselves to the gold here, is really just to put the candidate is nice. We are nice. Some candidates are relieved. When we go to the structured interview, like, knowing what they're going to do, like, takes a lot of stress off of them. And so, we are nice, right summer. Combine, our personal wires. When they realize
you have a ton of experience, that will be great for a team and I'm never just saying that like the entire interview is meant to be framed as we're going to be working together. And let's imagine, this is your first day on the team, worst of the interview. So important, you of our of our technical interview, it to read some code that we've been. We were in, we call this our code composition module. Imagine this is your first day on our team and this is the domain of a code. We want you to work on. So we explained that, as you look at this code, we want you to tell. If I
got a high-level, what it does, what you like about it and what you dislike about it? We do now, we don't actually give them code for Marco base after all, we want them to want to work with us. And the really like the production code is just way too complicated. There's just too much context sentence like a 20 or 30 minute conversation. So, we pick one interesting part of our system, we put together a few miles and some tests. And we send these files to the candidate and ask them to open them up and whatever text editor ID, they feel comfortable using it again. Mass is nice and so we
are nice, right? Often times we interview folks who have not written Ruby before. So when they open up our Ibotta app and see a bunch of Darby files, they get nervous so we explain that. Not only can I use Google as if this was really there, first day on the job. But they're welcome back. I sent him questions and they have about the code. So, to further drive home this point, we might say something like you like most of Oliver's that I bought. I when they start do not have much Ruby experience, but one day our butt on day one, you have to pick up stories that require you to
start writing some Ruby. So we're deliberately using words that pursuing the candidate will be successful. When you join our team, you will start writing Ruby. This language put the candidate and allows them to really demonstrate their strength Another piece of language that we typically throw in is that the exercise is not meant to be a bug hunt developers. Like we were like conditioned to assume that we're about to be tricked or the Visigoths in the interview. So we have to explicitly say that this code is bug free and do not look for bugs like otherwise, it's the only way to
shake them from trying to catch them. So why do we ask for candidates to read our code? So we asked them to re-record because a lot of what they'll be doing, especially when they first get started is just as reading or code. What is the code? Walkthrough actually tell us about our candidates? Well, I personally like to let me just describe what the strongest candidates are able to exert. Look like Tusk connections between the lines of code or implementation detail and the big picture of our business and
end-user experience a question. They may not know why some of our variables have an at symbol in front of them. They may be surprised to see methods that end with a? But most the most compelling Canada is being genuinely excited about working with us. There are the people who is easy to imagine pairing with, on day one on their first story. 8z communicate their thought process. And basically, like if you walk through code with someone for 20 minutes,
I think you will actually have a pretty good idea if you would benefit from having them on your team. So we also agreed to section on a curve for candidates with less professional coding experience. We focus mostly on that first question. What is the code for? What does it do? Recent graduates of like, computer of computer science, programs of Cody schools. Don't usually have a ton of experience reading other people's code. So it's a circus is already kind of a big ask for candidates with more experience. We want to be able to make that. So
after about 20 or 30 minutes, we stop a conversation and we ask her to stop sharing the screen which is a bit of like a psychological break. Again, mass is nice that we are nice. I will try to get some positive feedback at this point. I'm pointing out the strength of the Canada exhibited throughout the conversation and whenever just making this up, people can do pretty well. These types of conversations and it allows them to show off their strengths. So then we move into the third portion of our interview module
and we asked the candidates are right or what was the code to be asked to write requires them to take him some Jason and do something to it. And I'll put some Jason. It's really similar to a lot of the work that we do on a daily basis. We start by showing a sample example, output from what we want them to write so that they have to go online and then we moved back. Example, piece of source data and we walk through the entire data structure together and explain in detail how to go from input to Output. We are literally giving them the answer to the question ever asked.
Again, we have to remind a candidate that there's no trick and there's no gacha, right? We say this is not a speed run. The answer is not interesting to us, we just gave it to you process. They take to put this in the practice is with interesting to us and we encourage our team and so they can just have a question. Just like they went on the first day of the job. So what is completing? Our coding challenge position module, we get to see how they might code varmint, we get to see whether or
not they used test-driven development. What language is most comfortable with and have a problem, the best can make a plan and usually even write out their plan and comments or pseudocode then they begin implementing their plan but they check themselves line-by-line. The folks you tend to not do as well either, don't plan it all or just write a bunch of code. All I wants which makes debugging way more challenging. We're also not shy to ask questions and point candidates in the right direction. To start going down the wrong path and this type of interaction. Give the data on
the Canada, Scarborough about 30 minutes of washing the candidate. We can pretty And also, if they take a systematic approach to their development and then also potentially like how collaborative and receptive to be back. They are. So after any Nicole composition module, we leave in ten minutes for the candidates of questions. Immediately after the interview mutation, technical proficiency, and these things are directly accessed by the exercises. We just went through. So Kendra to do on this technical interview, have passed. The only type of
girl hurdle in her interview process we might them to join us for one more interview with two parts. The first part of the culture at interview assessed by developers based on the discussion of the candidates work experience on me. And then there's also a leadership interview performed by the hiring manager. So to summarize the old interview was not nice. The new interview is trying to be nice. We want to collect data that allows us to make decisions about the potential, a candidate as quickly as possible. We do so by shortening the overall process and aligning each component with
candidate write a lot of. This is why I feel. So we tend to be a bit more hand-holding with more senior candidates. But of course, There's likely a bit of unconscious bias. It's right here and we can be better in that regard by watching each other by practicing together and formalize our process if it were still very new. Especially when working with Canada's from underrepresented groups, So, I hope I now I have helped convince you that that there's an effort to see to this mean if one interview and I like to close up a talk with some tips and pointers
on how to enlist support the first most companies have been remote since March and the transition from in person to remote work, help accelerate our transition to an interview to reboot our process. So next, get HR on board and the easiest way to do that is just too short in the overall process of the process because they got an offer with another company or just don't want to persevere through a 20, step process. Third get Engineers involved in the first, I quickly. Sure, I was
doing with engineering managers individual contributors and these folks gave me awesome feedback and in many cases just started like raining p. R s Hold the work. We've been doing this meanest, one interview for a few months now, and so far, the results have been quite positive and I can't stay that way. I can say, I can let me know they're on the right track. So first Engineers are also a jar, is very excited about this process. So two key stakeholder groups are into it and then finally, the candidate seem to actually be enjoying the process. And so
I want to lick close by sharing a piece of feedback that we got from its candidate. I don't read this email that we got because I'm very proud of it. So I also wanted to provide some positive first. The interview on Thursday, I understand you're currently in the process of transitioning from a take-home assignment to the live parents dial interview performed on Thursday. So I just wanted to say that. Asylum interview, Potentially the best form of sacral interview. I've seen yet, I sometimes get frustrated with contrived leetcode style interviews but I really thought both pieces of the
interview were simultaneously. Very practical in a great test of a candidate's knowledge and capability of picking up new Concepts in languages. I hope that you feel the same and that other candidates feel. Similarly, the industry move in terms of technical interviewing technique. And so we are nice. Thank you very much for me, this is a lot of fun. I will be in Black to take questions immediately after this cough. Anybody has questions, but thank you so much. I'll see you all at the next one.
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.