The art of writing a technical interview question

Posted on

Writing a tech interview question is not something most people study in school. It can be a bit overwhelming, especially if you haven’t had the opportunity to take interview training or follow your peers as they conduct interviews. Maybe you’ve just been hired and already find yourself conducting interviews, trying not to let the candidate know how nervous they are. you are.

We’ve developed this checklist to answer some questions you might have about writing a technical interview question. Although it is not intended to be an exhaustive list, it should help you ensure that your questions do not suffer from some of the common problems that waste applicants’ and interviewers’ time or get the wrong signal. interview.

✅ Does it assess the concrete skills you are recruiting for?

Take a step back from the details of your question and start by asking what skills you want to assess with your interview question. The more complete and specific you are here, the better. What might be a good interview question for, say, a junior React developer wouldn’t be relevant for a senior DevOps engineer.

Also consider that your question isn’t the only question the candidate will be asked during their interview: it’s part of a series of tasks that together should give you multiple perspectives on the candidate’s skills. So, to assess what your interview question will assess, it is important to also know what your colleagues will ask you.

✅ Is it clear and accessible, regardless of someone’s cultural background?

Put yourself in the shoes of someone who has a culture completely different from yours, someone with whom you have very little in common. Would they understand your question? It is natural to assume that the candidate knows the rules of certain sports or games, or knows specific places and events. But you don’t want to find out that those assumptions were wrong when you’re in the middle of an interview.

It’s also best to stay away from interview questions that might introduce negative associations. For example, medical records can be a good analogy for the technical problem you want the candidate to solve. But you may never know if a candidate’s personal circumstances, such as having recently been through a difficult medical situation, might come to mind and distract them from their best performance.

✅ How waterproof is it?

Let’s face it: if you work for a company that conducts a lot of interviews, chances are that after you ask your question a certain number of times, it will be leaked on a platform like LeetCode. However, there are a few steps you can take:

  • Write an open and progressive question. An open-ended, progressive question with layers is harder to leak because there isn’t just one right answer. While candidates can certainly still prepare to solve a more open-ended problem, they will also have a hard time anticipating how the conversation will unfold and the many different directions and explanations you might give in response to their approach.
  • Create variations. If you develop many variations of your question that are similar but not identical in what they ask, it’s like having redundant servers: when one fails (leaks), you’ll have the others on which ones to fall back on.
  • “Watermark” your question. When your question leaks, you want to be able to find out quickly and remove the question, minimizing damage to your talk signal. But since many technical interview questions are similar, it’s a good idea to sneak in an idea that will stand out. Imagine being able to search for something like “pink toothpaste problem” – you’d be able to tell if you’ve been leaked with great confidence!

✅ Have you tested it with your teammates?

Calibration is essential for interview questions. Otherwise, you could end up with a coding task that is either too difficult to complete in the time allotted, or too easy, leaving you with a bad signal.

It’s good practice to test your interview questions with a group of your teammates, especially those who have the type of experience you’re hiring for. They will give you an idea of ​​how difficult it would be for a qualified candidate to solve the question, the types of questions candidates will ask, and any clues that might be needed. Most likely, your teammates will also point out areas where you can improve the task.

✅ Do you have a scoring grid?

Deciding how to grade your interview question can sometimes be as difficult as writing it. To introduce objectivity and reduce bias, you can introduce test cases that should each count towards the candidate’s final score. You may want to weight these tests differently so that more complex tests contribute more to a candidate’s final grade. You can also decide that some of the tests should be hidden, while others should be shown to the candidate as they complete the task.

A better way to write interview questions

Writing effective technical interview questions takes a lot of time and effort, but good questions are key to ensuring your interviews are predictive and fair. At CodeSignal, we take writing interview questions so seriously that we have a whole team of assessment design engineers and IO Psychologists dedicated to the task. We develop skill assessment frameworks that assess coding skills in the field while avoiding bias and question leakage, so your engineering team can spend less time maintaining interview questions. To learn more, request a free demo here.

Leave a Reply

Your email address will not be published.