Engineering Hiring: Technical Evaluation and Making Metrics of Values
Interviewing engineers is a complex task that requires special skills. A new industry has popped up catering to this need — Karat runs interviews for medium to large sized companies, with some customization. Because the engineers and the core of your company, you should be thoughtful in who you hire and why. As the graphic above describes, the cost of hiring a bad employee is huge, whereas the cost of not hiring a good employee is not that high. Always err on the side of caution when making full time hires. Here, I’m going to give some thoughts on how best to conduct your process.
First, determine what you value. There are a few key areas to examine here. Values alignment, communication, ambition, geographic proximity, working style, seniority, willingness to work for equity, knowledge of certain languages, technical proficiency, product knowledge, industry experience, company loyalty. Determine which are important, and create metrics for them. How would you quantify values alignment? Should they have volunteered at an org in a similar space? What makes them ambitious? Promotions in quick succession in a previous company? Decide how it will be quantified and stay objective with each candidate.
Determine the questions to be asked of every candidate — the process is always evolving, but try to stay consistent, especially in hiring the first team, so you have a single experience to judge each engineer on. Rank them on a scale from 1–7 and take qualitative notes. There are many studies on categorization and bucketing in the human brain that state seven is about the maximum number of distinct categories we can meaningfully discern. Think of the 1–10 rating system, is there a difference between a 3 and a 4? In dating, the least attractive people are a 6, we just don’t use the other scores. Limiting to 1–7 forces you to make meaningful choices and bucket accurately.
Work with an engineering advisor to ask a technical question. This question should be a toy problem that is solvable in 45–60 minutes. But it should be suggestive to what will be worked on at the company. The best way I’ve found to do this in writing these questions for startups I worked for, is to create a minimum valueless product. A shell of the problem your startup is solving, with some interesting backend or frontend tasks that you can solve with in memory data, but that’s simple enough to wrap your head around in an interview. Oftentimes, it makes sense to write some scaffolding code to orient the candidate, and have them write functions that extend the features. When evaluating technical proficiency (the main thing you want to be filtering on), break this down into a few sub categories. Code speed, code accuracy, code efficiency, code clarity, clarity of thought, language knowledge, ability to search for answers, communication of technical concepts, test writing.
This is a big subject that requires a deliberate approach. There should be some actionable steps above, and reach out if you want some guidance from someone who’s conducted hundreds of interviews for Google, Karat, and revamped interview hiring for two successful fintech companies in NYC.
