Take home tests vs. Live coding exercises
In such a competitive talent market, there's never been a more important time for businesses to ensure their interview process is as attractive as possible. This is particularly true in the Software Engineering space, where the technical stage of interview processes has become a hot topic amongst candidates and hiring managers alike. Whilst testing candidates' technical skills seems a crucial way to determine their fit for a role, for many candidates, being required to carry out a particular type of tech test may encourage them to drop out of interviews altogether.
If your interview process is suffering from candidate abandonment or a slow time to fill, here are 4 things to consider to help choose the right testing method for your business.
⏰ Are tests impacting your time to hire? ⏰
It goes without question that having a smooth and timely interview process is integral to keeping top talent engaged. Given this, many candidates prefer shorter testing methods, such as pair programming, which can be booked and completed in a matter of days, keeping the process moving at pace. With pairing tasks often lasting 1-1.5 hours, they can also be completed during the working day, meaning that candidates won’t have to find time during their weekends or evenings- a big plus for those who prefer to switch off or spend time with family.
When it comes to take-home tests, if a candidate has numerous processes on the go, there’s also the risk that they have multiple take-home tests to complete. This often becomes a bottleneck as candidates may struggle to find spare time to complete these tasks, resulting in other processes moving alot faster and increasing the chance of drop-outs.
There is also the common complaint that, whilst companies may say a test takes a certain amount of time (i.e. 2-3 hours), this is heavily dependent on candidates’ familiarity with the challenge (i.e. building out new APIs). Thus, certain candidates may end up spending double the amount of time on such tasks, particularly when there is pressure to produce clean code and write tests, as being able to talk through their approach with an interviewer is not possible. There is also a consideration that some candidates may also seek external help during this time to complete therefore skewing the intended outcome.
To sum: If you are currently asking candidates to complete a take home test, it's worth looking at your return times and evaluating if they could be hindering your speed of process or even causing drop outs. Switching to a live pair programming task could help to speed up time to hire and keep the top candidates in multiple processes moving through your process at pace.
💛 Is your method of testing inclusive? 💛
Another question to ask is whether only offering one standardised format of testing is inclusive for all candidates and minimises bias?
Take-home tests are often considered non-inclusive for working parents, those with caring responsibilities, or people with busy lifestyles who may not have 3-4 hours of time to spare on a task. Therefore, only offering this method could lead to a less diverse talent pipeline due to certain candidates having to drop out due the time commitment that take home tasks require.
On the other hand, pair programming often adds a pressure component, which is off-putting to candidates with anxiety, who feel that they cannot perform to the best of their ability as strict time constraints means they cannot approach tasks as they naturally would at work. In such a pressurised environment, some candidates also become easily flustered, leading to them over- engineering simple problems. As a result, some may prefer to drop out of the interview process altogether, rather than putting themselves through the discomfort.
One way to ensure that you are providing equality of opportunity to your candidates, is giving them the choice of which type of test they would prefer, allowing them to pick the method to best demonstrate their technical abilities and work best for their personal circumstances. This is likely to significantly reduce the number of candidate dropouts due to disengaging with a particular testing method. However, one drawback of this is that offering numerous methods of testing can be difficult to scale and create a level playing field. If you are interviewing large quantities of candidates, it will be difficult to establish a way to accurately benchmark these candidates against one another, potentially making it a less inclusive method.
To sum: If offering multiple methods of testing is not right for your organisation, work towards designing a consistent process that looks after the candidate but also ensures that you are able to measure your key criteria. Your process should be set up to eliminate candidates who aren’t good enough, not those that don’t have the time or desire to do it.
🗣 Does your method allow you to evaluate how well a candidate communicates & collaborates? 🗣
Before determining your method of testing, it’s integral to establish exactly what you’re trying to evaluate from such tests. Are they purely to measure technical ability? Or, do you also want an insight into a candidate’s communication style, character and thought process?
A significant drawback of take home tests is that it gives you a limited insight into how candidates approach problems and work with others. Whilst this is a straightforward method to benchmark candidates’ technical abilities, it gives little insight into how well they might work with the team and if they will fit in culturally. Consider pre-booking feedback sessions with your candidates for after they have
One great benefit of pair programming is that candidates meet with other members of the team, allowing you to see how they may work and communicate with others. Likewise, it affords candidates the opportunity to get a feel for the environment they could be working in, building a rapport with the team and buying into the opportunity more. A great interactive experience could see your opportunity fly up the candidate rankings.
To sum: Whilst pair programming can be a great way of assessing a candidate’s cultural fit as well as technical skills, it is worth remembering that nerves can play a part in live testing methods, so you might not always be getting the absolute best out of the candidate. Try to consider how you can make the candidate feel most at ease throughout the process so they can perform at their best.
👏 Are you providing a positive experience for your candidate & increasing buy-in? 👏
Ultimately, in this market you need to ensure candidates are bought into you as a business to avoid dropouts. If a candidate has a positive first-stage interview and feels invested in the company’s mission and values, they are far more likely to complete a technical task, even if they are less in favour of a certain method of testing.
We have seen high levels of candidate engagement with clients who utilise the first stage interview to essentially ‘pitch’ their business and develop a relationship with the candidate. A successful pitch will convey a strong sense of who the company is, what you’re trying to do and how the candidate will play a part in that mission. Likewise, demonstrating your company culture (progression frameworks, team offsites, celebrations and events) is a great way to increase a candidate’s buy-in and your chances of keeping your candidate engaged enough to complete your interview process.
Likewise, if you are asking candidates to complete a live programming task, a great way to reduce a candidate's nerves and provide a more positive experience is to give them some tips for how to prepare. Explain your expectations around their technical skills and how you want them to communicate with the interviewers. This is a great opportunity to create a more natural setting to demonstrate their skills outside the pressures of an interview. Also let them know if they have the ability to google anything during the task.
If you are opting for a take home test, a good tech challenge should encourage the candidate to feel as though they are writing code to solve the kind of problem that they’ll be facing day to day. There needs to be detailed upfront criteria to encourage the candidate to fully understand expectations. Pre-book a 30 minute test feedback session between the assessors and the candidate (pass or fail) so the candidate feels encouraged that they will have an opportunity to discuss their design decisions and also receive live feedback with a member of your team. This is a top tip to improve overall candidate experience even when being eliminated from the application process.
To sum: Providing a positive experience throughout the various interview stages will significantly increase your chances of attracting and retaining top talent throughout the process in a crowded market.
To Conclude & Alternate Methods of Testing?
To conclude, with the current hiring market as busy as it is, there are numerous factors for consideration when designing your technical interview processes to make it more attractive to candidates.
In addition, although pair programming and take-home tests are the most prevalent methods of testing, many of our clients have designed new means of testing all together that better meet the needs of candidates and provide them with the best experience possible, whilst still evaluating both technical ability and soft skills. These include; Portfolio Discussions (where a candidate shares and talks through a piece of code that they have worked on recently, allowing them to feel more comfortable with the code they are discussing whilst also going into technical depth) and Code Reviews (where candidates are given a ‘bad’ piece of code and are asked to highlight how it can be improved, a great way to assess candidates’ critical thinking and attitude towards problem solving). This method can be more beneficial for Leads, Principals, Staff Engineers, where this will be a larger part of their day-to-day role.
Whatever you decide, ensure you discuss your decision making process with those in your team that will be required to invest their time in assessing the candidates at this technical stage. If you are a small team, consider how interview fatigue might affect morale so it is important that everyone understand the time commitment required to create a great candidate experience whether they pass or fail!
Burns Sheehan are working with several organisations that are leading the way in hiring and are committed to providing the best service to both candidates and clients in the current market. If you’ve experienced candidate drop outs or bottlenecks caused by technical interviews & are looking for tailored advice on adapting your recruitment process, feel free to reach out to me here.
