How to Interview Engineers
Defmacro, 2020-10-12
There are just three things to determine about a candidate: talent, judgement and personality. But just talent (aka the horsepower of a racing car) is useless if not applied properly and consistently.
Talent
Talent is a combination of speed, working memory, taste, familiarity with the available tools, understanding of how things work, and the ability to program. Like IQ for engineers.
In engineering the best specialist is 10x better than the very good specialist, so obviously the challenge is identifying these specialists.
At the interview give a development problem and observe the solution process.
The most talented candidates think first (not too long) then start typing, with the bottleneck being their typing speed. They will also have the best “taste”, i.e. formatting, algorithm, etc.
The slightly less talented candidates will do the same but 1.5-2x longer.
The rest (say, if they don’t produce the result in 15 mins) should be politely rejected. Their bottleneck is their thinking throughput, and that’s a no-hire.
The interviewer doesn’t have to be as good as the person being interviewed, as they’re not looking to clone themselves.
Modern interviewing lowered the bar due to the need to hire hordes of engineers, but if hiring individual engineers – go the non-scalable way and ask for algorithms.
System design and architecture questions are useful for candidates who think slower than they type; bright engineers never have problems in this area.
Judgement
There’s a difference between a tinkerer and an engineer: the former creates complexity and the latter uses complexity, discovering and satisfying constraints.
People who’re in the job just for the money, or like creating complexities out of hobby shouldn’t be hired.
It boils down to programming strategy – one either possesses it or should at least understand when to start asking questions. The worst is having poor strategic intuition but believing otherwise to the detriment of the team and the company.
The judgement section of the interview must be differential (i.e. adjusted to a specific candidate). So have a candidate write a memo on a serious decision (suggest them a topic) to the company explaining why this decision needs to be made. This will show how the person thinks, what constraints they see and how this can be overcome. A piece of writing that can’t be argued with is gets a pass score, otherwise – no hire. The main goal is finding what the person bases their judgement on.
Personality
Checking references and asking about one’s performance and teamwork helps, but only a bit. There’s nothing much one can really find out during the interview, as everyone’s on their good behaviour.
The goal is to somehow find the signals revealing that the person is lazy or hard working, how they treat other people [MK: what’s the engineer equivalent of observing a person talk to a waiter in a restaurant?] and whether they overanalyze their solutions.
So this one is probably better solved by the corporate culture, which will squeeze the wrong fit out other than any analysis of psychological traits.
Theatrics (that’s where fun starts)
Make a 6-hour interview (with a bunch of developers giving increasingly harder tasks) so that the person gets invested in the idea of working for the company.
These increasingly harder tasks should end up in the person NOT being able to solve the last one; then the person is more likely to accept a job offer.
Also, engineers need to know the measure of the person (what the person can and can’t do) to calibrate their expectations of a new team member.
And don’t offer the job on the spot – let the person think about how well they did during the interview, then give them a call next day to propose.
I'm a bit concerned about such advice as "make 6-hour interview".
It's the open market out there and hiring is a two-way process. If a company willing to hire a great engineer, it should show some real value to potential candidates, otherwise, engineers won't be willing to spend 6 hours which usually costs quite a lot in the case of good specialists.
Also, please don't make managers assess engineers. Engineering is a form of art and managers thinking only with numbers and treating engineers as less talented only by how much it took for them to execute a test assessment, won't be able to hire good engineers. Only an artist can judge another artist. Make engineers assess the engineering skills of potential candidates and managers to assess their soft skills, then you'll be able to get an objective decision.