alt Hacker NewsAn engineer's perspective on hiring
Comments
Everyone calls them Interviews but they are not really interviews. They are exams.
Oral exams, live coding exams, system architecture exams, take-home exams, behavioral examinations, code review exams, extended essay writing exams, case study exams, sample work trials.
You can't be a real professional if you have to take exams in every job change.
In serious professions, people take exams early in their careers for being certified. Sometimes they take additional exams to renew their certificates. And that's all.
They don't take exams from random people in random companies that know nothing about evaluating knowledge. They take official, standardized exams prepared by professional testers/educators.
Engineering jobs can't be standardized. Engineering and required skills and knowledge is too broad for that.
An interview is not an exam. It's a meeting. The interviewer asks questions to learn about the candidate. The interviewer may ask some questions to learn about the company and the position. That's all. That's the universal definition of a job interview. All the other things are additional tests and exams.
Do they need to do those exams for better selection? Probably not. Their "hiring process"es are not backed by any science. Then why are they doing that? They have to filter somehow. If there are 1 to 100s ratio of candidates for each position, they need to filter hard. Exams are the standard method for ranking and filtering.
But we are professional engineers, not students.
I don't mean to be harsh, but as an engineer you owe it to yourself to try and get better at interviewing. Does it suck? Yeah absolutely. Is it an annoying process? Definitely. But even if you have an easy and stable job things can change quickly at any company. You're only closing doors on yourself.
If you get nervous, the main thing you can do is more interviews. My personal anxiety peaks right before the start time, luckily my bathroom is next door to my office! But after doing dozens of interviews I settle in once it gets rolling.
If you hate leetcode, well just get good at it. Yeah it is kinda dumb but it is straightforward to practice. And there is a lot more to a leetcode interview than knowing tricks - you need to communicate well.
Take homes? Yeah they are time consuming. If you really need a job do them, otherwise pass on the company!
Overall as a candidate you really need to try and go one level up on selling yourself - not just why you are a great candidate (which you are of course!), but why you would succeed at this role in particular.
As much as I dislike leetcode style interviews, if I fail one of those, I learn what I can and move on.
Failing a take-home is an entirely different thing. It's a huge loss in time and mental energy.
I've only done 3 of those in my career and only because the projects sounded interesting. 1 of those 3 resulted in a job offer which I can now confidently say in hindsight was the worst job in my career (...so far!).
I'm now leaning towards just filtering out companies that do take-homes because it signals to me that they don't care about their candidate's time and how a company treats its candidates is usually a good indicator of how they treat their employees.
> interviews should: be applicable. reflect the actual job duties
No, it doesn't matter that much what task the candidate is doing in the interview. It matters what the interviewer is looking at. I'm sure plenty of interviewers don't understand this, and I think this is often missed when people debate about Leetcode interviews - including in this article.
> most interview questions have very little to do with day-to-day responsibilities
You're missing what the interview questions are. Yes, one part of the interview is "here is a puzzle, can you solve it?", but many of the other questions should be things like "explain why this doesn't work", "why did you start with this approach?" and "are you sure that is the best name for that function?"
Leetcode interviews are perfectly "applicable", as long as the interviewer is using them as a convenient frame to see how you think, communicate, and write/read/edit code and isn't trying purely to assess your skill at quickly solving leetcode puzzles.
> cannot distinguish a senior programmer from a marketer using chatGPT
This is empirically false, because companies haven't suddenly lost all their hiring signal since ChatGPT came out. But if a company has this problem, they suck at interviewing.
(I admit the style of interviewing I'm describing has its own problems, and in particular doesn't address what I think is the biggest issue: the fact you're partly testing people's ability to (appear to) perform under acute pressure.)
Our process is fairly simple:
1. 60 minutes interview with the CTO.
2. At home coding challenge. Candidates can pick one from 2 coding challenges. But we try to keep them engaging and fun, but still complex in details. Sometimes people don’t want to invest this time. That’s acceptable, but in this case you have to show os some of your work from the past, so we can discuss this.
3. Interview with 2 engineers from the team. We are doing coding challenges as a base for this interview. It’s just a way to get people talking with each other on technology and how they work.
4. Make an offer or say no to the candidate. Everyone involved in this process from our side has a veto right. So if one person says no - it’s a No.
5. Send contract to the candidate, if they accept the offer. This is the first time in the process where HR is involved. Everything else before was done by the Dev Team.
I think this is the most important part, show respect by taking care of the process by yourself and communicating with the candidate.
> companies often over-index on crystallized knowledge over fluid intelligence.
another way to say this: focus on aptitude. in my hiring funnel, this is a core tenet. you need to be able to capture polyglots and systems thinkers. its still pretty hard to design a process that balances this all very well. combine that with an absolute glut of applicants and you have a very challenging problem.
The OP thinks that candidates spending a lot of time on applications is OK, as long as the company shows respect by spending a lot of time themselves. I think this is mistaken - I care about how much time I have to spend, and am a lot less concerned about how much time the company takes.
There's a trade-off: if a company spends more time / requires more effort on an interview process, they can get a better signal on the candidate's abilities, but then they'll lose out on candidates who are unwilling / unable to commit this time. This might just be a hard trade-off in recruiting.
Not sure if they're still doing it, but GitLab does the code review interview, and I too really liked it.
Before the interview, you clone the repo and get the app running on your machine.
For the first half of the interview, you review a pull request in real time. There's a mix of obvious and non-obvious callouts. And the second half, you actually implement your suggestions.
Honestly the code review portion alone is a great indicator of a dev's experience and soft skills.
The situation is not going to improve as long as business stakeholders and engineering managers (some closer to MBAs than actual engineers) think of software engineers as construction workers. They think we are fungible, they don't understand the craft of programming etc, and have very short term mindsets. Took me a while but then I realised that I needed to interview my prospective employers as much as they were interviewing me, and to just ignore those not worth working for.
It's crazy, I've been trying to hire an architect to build a house for me, but I don't trust them (they're all liars) so I asked them to draw up blueprints in front of me and describe everything they are doing in great detail. I need to understand what they are doing. This building is important and I don't want a bad hire. They should be able to do this in 10-15 minutes. If they knew what they were doing, this request wouldn't be an issue.
Then I was trying to get someone to do my taxes. So I've been asking every applicant to do my taxes from last year so I can see if they really know what they are doing. I mean sure, they've done taxes for years, but these are my taxes. I've even tried giving them math puzzles around asset depreciation, but people just keep hanging up the phone.
And then, I wanted to add a wing added to my house and I've been trying to get these entitled contractors to come build a shed in my backyard so I can see if they really know how to use nail guns. I've heard people are just big'ol lairs lately, and I need to see how they work. I mean, sure these guys have built many houses in the past, but we have high standards here and only hire A players.
I've only been getting horrible candidates! No one is worth hiring! There is a huge shortage of qualified people.
If only there was a way to fix this.
One of my favorite interviews (Mixrank) was where the CTO/founder and I picked a random problem on Project Euler and we both coded on it in parallel. I was in the driver's seat as far as design and general approach was concerned but we were each coding an independent solution. We weren't even using the same languages.
Once we both had the correct answer, we started optimizing to try and eclipse each others' runtime. This was actually a fantastic test to display deeper knowledge beyond just regurgitating a solution, showing benefits and drawbacks of different languages and patterns, and seeing how we could work while agreeing or disagreeing on a technical subject.
I was really too junior for the role they wanted and I didn't get the job, but it was a fantastic experience.
My best job (for the last decade) for a software engineering job was a 1 hour technical interview followed by a 1 hour interview with the director.
It helped that it was 90 days contract to hire.
At my first employer circa 2005 we had a simple 1-2 hour interview and a 90 day probationary period. Respected people's time, gave everyone a fair chance to prove themselves. I don't know why it's not more common in the industry.
Part of what lead to it I think is we hired largely straight out of college and doing a 9-hour interview with someone with little experience is a waste of time.
It worked great. In my five years there we only had a couple people not make it past the probationary period.
The primary problem with hiring is that developers are a single status with not performance benchmark. The solution is to segment need by capability.
Let’s face the reality that most developers will never be able to write original software and just put text to screen using a tool or framework. Don’t call these people engineers. These people are the assembly line of software. Measure them according to desired patterns. They are copy/paste but smarter than data entry and understand some of the restrictions in place. Expectations are low and compatibility and replacement are the key business values.
Next are the people who test software, the QA. We expect more from these people and then work them harder for less money at a lower level of reputation.
Next are the people who evaluate software. These people are closer to engineers. These people include accessibility, security, and performance experts. These people are more like a combination of QA and senior developers. Evaluate these people on these criteria: written essay, technical knowledge, force them to measure things in real time and see how they perform.
Next are the people who actually write software applications. Let’s call these people solution delivery. These people are similar to junior architects and actually build things. These people should be evaluated only on the basis of organizational capabilities above that of the engineers that measure things.
Finally are the software owners. These people resemble a combination of project management and junior architects. They must have the experience to know how to build original software, like the junior architects, but also a planning vision to push though demands from competing stakeholders. There is busy savvy to this comes from a solid engineering planning vision plus superior communication skills most lesser software people never honed. Think of these people as senior principals with real authority. Evaluate these people on their delivery experience, using numbers, and reputation.
I agree with the article. Discussing previously created code and do code reviews live covers a lot of ground. Whereas live coding is just meaningless for the stated reasons.
But a 9-hours interview process seems just too much... I think you will only ever know a candidates true fit once you start actually working together and 2-3 short sessions with someone is enough to get that go/no go feeling.
You can't hire without taking risks.
Where I live, you usually have a 3-months probation time in which both sides can quit within a 7 days period... so the risk is manageable.
Just feel a candidates fit and then go for it... and adjust when necessary. Don't overthink it.
If I can get a person talking about tradeoffs - be it speaking to a past project, or a hypothetical I give them - then I think I can tell who the serious developers are fairly quickly.
The fundamental problem with interviewing is that you're not preparing yourself to understand a business' problem and needs, but you're preparing to have to impress your potential colleagues, a non-trivial number of which are already jaded.
Big tech can afford this because they have a constant stream of candidates.
> live coding does not select for generalists
I have a rage memory now of a "live coding" event that makes me want to leave this profession and never come back.
Here's my suggestion for the technical interview:
1. Applicant signs NDA and agrees to not use AI 2. A small bug or feature request is chosen from the backlog 3. The staff engineer pair programs with the applicant until it is fixed/implemented
This does a few things:
a) The interview is suddenly "productive" for the employer rather than a cost sink b) No one will apply that doesn't actually know the stack/framework/language c) The interviewer gets a real world example of what its like to work with the applicant d) Leet coding and puzzle solving is eliminated in favor of real world coding e) A secondary skill set that doesn't actually contribute to the work is eliminated (go pound sand autists)
> this also loses you taste
I won't forget the in-person interview round where I coded a frontend visualization for a data graph (tracking global shipping), then fielded a post-work general interview round from the whole company (~10 ppl) about specifics and "choices" made during a rush to finish. I ended up not going due to comp, but they were acquired months later. Life is funny.
I really loved the uno reverse card interview the author recommended. "Let's debug this piece of broken code"
This can spark so many interesting discussions, from syntax, architecture, cs, product etc
It is literally the job that engineers do 99% of their time yet we don't interview on this.
Early in my career, most of the people I interviewed were not very good.
Eventually, enough time passed that the talent pool grew considerably and most people are baseline competent.
Consequently, I now find that respect and time efficiency matter a lot more.
The best interview process I’ve ever done is sitting down with the team for 3 days, crashing on the founders couch for the weekend and shipping code
The worst ones were the leetcode interviews I couldn’t pass
So glad I changed industries after my first s/w job out of uni wanted 2 hrs unpaid overtime per day and the boss ran the office like he was lord of the manner, swearing and blowing up whenever it suited him. If I had the skills these people want for the price they're willing to pay I'd just start my own business and reap the rewards myself. Most people aren't worth working for. Perhaps the pool of employers is far larger in the US which makes it easier to shop around for a good one.
> select for applicants who will be good employees for years to come
Why would you do this given the expected average tenure is probably like 18 months to 2 years?
I disagree on the "taste" aspect: I don't think it's valuable to take that into account in the slightest.
A lot of the time, technical interviews and take-home exercises turn into 100 iterations of "Do you prefer vanilla ice cream or chocolate?" or "Do you prefer vi or emacs?"
A good employee will figure out the subjective tastes, biases, and cultural quirks of a workplace, and fit in. A senior engineer will have done it perhaps half a dozen times in their career. -- And yet, companies insist on putting interviewees in a situation of having to basically mind-read those subjective tastes and biases to try to tell interviewers what they want to hear, because they'll reject a candidate on the false premise that a vi guy will never fit into an emacs shop.
"How do you feel about unit tests?" I really don't have any feelings about them whatsoever. Just tell me, if you want me to write them or not. Instead you're asking me to mind-read and trigger some hidden trauma in you. I just don't know whether the trauma is that codebase that required breaking production every single day, because there was no way to test it. Or whether the trauma is that one guy that you hired who never got any work done because he obsessed over refactoring the codebase for optimal testability instead.
Well structured temp to perm arrangements are usually win-win?
I've been using a variation on the "Code Review" example for years, and really like it. I give the candidate some fairly straightforward sample code[1] that interacts with a Python library[2] along with a couple user stories that they are expected to implement. I tell them I'm both their product manager and pair programming buddy, and that they should ask me any questions they might have, use ChatGPT, consult Stack Overflow...whatever they do to solve the problem at hand because I'm just interested in seeing how they think.
No system is perfect, but I have a 100% success rate with the engineers that have been hired using this model, and several years of data to back it up.
[1] It actually has a few subtle bugs in it that I'm always curious to see if they'll catch.
[2] The job in question requires working on a Python web app, and so I assume some basic Python knowledge.
I don't want to give away my secrets, because this has actually worked really well for me and I'm afraid that I'll lose my edge as an employer - however I have a very small neck of the woods and HN seems very US-centric so I think I'll be ok.
What has worked for me, honestly, is being directly involved with my hiring pipeline and having conversations.
It seems like common sense, but there's a lot of reasons not to do this and people will make good arguments to prevent it. "What about bias", "your time is more important" etc;
However, bias is an unfortunate consequence of selecting for value fit anyway and I can't think of a more important task than selecting the members that will be the future of the company.
I've had some positions that were open for a weekend where I got 400 applicants, and yes, it was daunting to go through and give each of them an honest shot, but you know: I had to do it. What's the alternative? I might miss a fantastic candidate because someone didn't understand what I actually need.
Evaluating programmers and "devops" people is just insanely hard, technologies are mostly fungible. If you can write one C-like language you can learn the others in about a month, but what can't be taught is what your values are, if you think in a systemic way, if you're easy to work with and respect others.
So, my solution is to have a conversation, challenge what they know, see how they react when challenged, see how they react when they reach the end of their knowledge and see what they're most proud of and if they get excited by it.
No gotchas, no esoteric internal handshake, just: are you defensive? Are you curious? Are you passionate? Can you communicate effectively and are you intelligent.
If you hit those, you can do anything.
"How do you even know who to interview?"
This is a hard question, for me there's not a lot of candidates that are physically located in my region, so those go through as long as they have something on the CV that looks relevant. For others it's a combination of: would it be easy for them to move, have they worked remote (and can do it in a region where I have a tax entity) and how strong of a fit to the role is the CV, lots of experience in games would be what I expect since I work in video games - but if you're going for a backend programming role then: what have you built and what do you list as your responsibility to achieve it?
With this mindset I managed to build a high performing, high trust team that executed very well on (literally) impossible demands. If the ownership of the company was better that team would have easily been world class.
We also exceeded dunbars second (clan) number with the size of the team, so it wasn't intrinsic to small teams (80+).
Honestly, I'd say the best setup would be something like:
1. Live coding/code review/take home test 2. Technical discussion about said test/project (and the company's tech as a whole) 3. General team discussion/culture fit interview
That's because at the end of the day, you really want to filter out the folks that can't do the job as quickly as possible. If someone needs AI to code, can barely use a computer or writes poorly thought out, inefficient code straight out of the last century, it doesn't really matter how charismatic they are or how well they can discuss where they want to be in 5 years.
Way too many companies do things the other way around, and waste far too much time because of it.
Oh, and definitely make sure your questions are specific to the tech you use/field you work in. I still remember having a tough time using React at a certain company because the interview process didn't require any modern JavaScript/SPA knowledge and I'd managed to pass without ever having worked on that sort of web app before.
Similarly, I also remember seeing a WordPress dev agency using Google style leetcode interviews to filter out applicants. Made me wonder how many people they actually managed to hire that way, given that the pay being offered was pretty low, the work wasn't particularly appealing to the people who'd pass those interviews and the requirements to pass the interview in general didn't match up with the job at all.
job ads used to say "thrives under pressure" as a kind of euphemism
now they just use the interview gauntlet to test your will
its like those japanese game shows where they have 10 rounds of humiliation to see how far you can make it
> most interview questions have very little to do with day-to-day responsibilities. all good software engineers are generalist and live coding does not select for generalists
If I had a dollar for every time I heard this (flawed) argument, I’d be rich and would no longer have to sell ads on my Hacker News comments. I’m going to get hate for this unpopular opinion but here we go.
So often, “But Leetcode isn’t like REAL programming” is the siren song of the programmer who probably overestimates their coding skills and experience.
Yes, I hate to say it - live coding is actually one of the best signals you can get on a candidate’s seniority and ability to program a computer (and more importantly, their core computer science skills). A good interviewer is trained to know how to probe your CS knowledge during this, and will watch how you structure code, break down problems, debug, and think about testing. They will even ask you to make changes to see how coachable you are and what you might be like to work with. It’s not about inverting a binary tree while sharing your screen, it’s about showing me how you solve a problem, then translate what’s in your head to code.
Take home exercises provide little to no signal, and screen out people who have families (who wouldn’t bother with a 4 hour take home exercise after work). I don’t want to see how you Google, I want to see how you think.
These candidates always want some version of, “But trust me, bro! Hire me: I’m a senior engineer, I don’t remember how to Leetcode! I’m good, I promise!” But what they won’t admit to themselves is that a good senior engineer is able to do all the things a junior can do PLUS all the things a senior can do.
It’s not perfect, but I won’t hire anyone that can’t pass a live coding round.
This comment brought to you by Poppi. Poppi: it’s soda for people who are silly enough to believe soda can be healthy.
He mentioned that Oxide has an interview process that takes 20 hours realistically between coming up with work samples, answering 8 questions before hand and 9 hours of in person interviews.
I was curious about the pay.
https://www.glassdoor.com/Salary/Oxide-Computer-Salaries-E54...
For their “senior” engineers it’s in the range of offers for I’ve seen for new grads at most of the tech companies and around the range of generic enterprise CRUD developers.
No offense to “enterprise developers”. I spent 25 years as one. But why would I jump through hoops for a job that pays about the same as I could hypothetically get based on interviewing for a few hours and asking generic behavioral questions and maybe some techno trivia about whatever language the company uses.
I find it fascinating that companies want rockstar ninja developers but then offer meh compensation for the positions.
Just talk to people. Seriously. If you're "an expert", then >80% of the time you can figure out if someone is an expert in the same topic just by talking to them. See how they think and problem solve. That's the skills they'll be using on the job anyways.
Here's the traditional engineering interview script:
Ask the candidate how they'd solve a problem you either recently solved or are currently working on. Where does their mind go? What questions do they ask? Do they make the same mistakes you made? Are they different? Do those complement one another? Are they excited?
Honestly, one of the best things to do is get people to talk about their passions. For a lot of engineers their passion is related to their work. You WANT this as a business. They'll work hard because they're having fun doing the work. They'll learn more on the side and get a lot of expertise and ideas you wouldn't have on your own. If you make the job interesting and fun, they'll stay despite offers of higher pay too! But all of that depends on managers. A good manager participates and their most important job is preventing engineers from spending too much time in rabbit holes. You need to go down some but some unravel forever and your engineer will get stuck in an infinite loop.