How to Fix Technical Interviews

​Recently published research suggests that one of the primary reasons behind the difficulty in hiring computer science, engineering and IT talent is the way candidates are interviewed and assessed.  

The technical interview asks job candidates to write code and solve algorithmic puzzles on whiteboards while explaining their process in front of proctors. Job seekers hate it. They say it is needlessly difficult and time-consuming, often not relevant to the job, and even discriminatory.

"Having to invert a binary tree on a whiteboard, solve a riddled question or hand write a computer algorithm has little or no bearing on the actual work that comes with the role," said Tom Winter, tech recruitment advisor and co-founder of DevSkiller, a tech sourcing, screening and skill mapping platform based in Warsaw, Poland. "While algorithms and data structures can be useful, their predictive validities are very limited."

In addition to being frustrating, whiteboard technical interviews may be unnecessarily inducing so much stress that candidates' performance is significantly hindered. While tests should allow for an unbiased assessment of problem-solving ability, "we found that technical interviews may simply be a procedure for identifying candidates who best handle stress," said Christopher Parnin, assistant professor of computer science at North Carolina State University in Raleigh and author of the study Does Stress Impact Technical Interview Performance?

Companies that rely on these stressful tests are eliminating potentially qualified candidates to their own detriment, he said.

Parnin and his team designed an experiment in which 48 computer science students were given a problem to solve. About half of the students went through the typical interview format, solving the problem out loud with a proctor present. The other half took the test in private. The differences in test results between the two groups were striking.

"The impact on performance was drastic," Parnin said. "Nearly twice as many participants failed to solve the problem correctly when simply being watched. Participants reported feeling very nervous, rushed, stressed and unable to concentrate as a result of being watched. In contrast, participants in the private setting reported feeling at ease, having time to understand the problem and reflect on their solution."

Two-thirds of the participants taking the private test solved the problem, whereas only about one-third of those who were being watched did so. Notably, for an industry trying to attract more women, "we also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," Parnin said.

Parnin added that employers could consider providing private test settings or dropping problem-solving tests altogether and instead asking candidates to explain how they would perform job-related tasks.

Testing on Real-World Work

Winter advocates a different approach, believing that problem-solving tests are critical to hiring the best tech talent, but the test problems need to be something new hires would encounter in their day-to-day work. The problem with many abstract or hypothetical whiteboard tests and algorithm puzzles is that they offer little insight into a candidate's relevant job skills, he said.

He explained that talented programmers use a variety of resources to write code and solve common algorithmic issues: "Searching Stack Overflow and using prebuilt solutions is a part of a programmer's job. … If you take it all away from them in an interview and ask them to code on a whiteboard, you'll not let them show you their actual skills that you are looking for."

Winter said most programmers he's talked with appreciate when a potential employer is trying to create a real-world testing experience.

"Ask the candidate to resolve a certain problem that you worked on in your company. Don't make it too big—you will lose a lot of programmers if you give them a project that will take them several days or even more than a few hours. When you get back the results, invite the candidate to an interview and go over their submitted solution to learn more about the approach they took. This way you'll be able to not only see how the candidate tackles the problems, but also understand their way of thinking."

The key is to provide the candidate with a test that makes sense and matters for the position they are being recruited for, Winter said.