The candidate who explained their architecture decisions fluently, answered every question without hesitation and got a unanimous yes from the panel is now four months in, and nothing is shipping. Nobody on the panel was wrong about what they saw. They were measuring the wrong thing.
Interview performance and engineering performance are different skills, and a loop built on conversation rewards the first while telling you almost nothing about the second.
This guide covers the mechanics of the decision itself: how to calibrate the seniority you actually need, how to design a loop that produces comparable evidence, how to score it without letting the last conversation dominate, and what to check after the offer. It does not cover where to source candidates, which is a separate problem handled elsewhere on this site.
Key takeaways
- Structure beats rapport. In the largest recent review of selection methods, structured interviews emerged as the strongest predictor of job performance, ahead of cognitive ability testing.
- Calibrate the level before writing the requirement. Most bad hires are level mismatches rather than skill gaps, and the mismatch is decided before anyone is interviewed.
- Score during, not after. Written scores recorded immediately after each stage prevent the final conversation from overwriting everything that came before it.
- Take-home exercises have lost most of their signal. Assistants complete standard ones cleanly, so the weight has shifted back to live discussion and code review.
- Reference checks disqualify well and select poorly. Use them to catch disasters, not to pick winners.
Which role do you need before hiring software developers?
Write the answer to one question before anything else: what will be true in six months that is not true now, if this hire succeeds? A requirement that cannot answer it is a headcount request rather than a role.
Three failure patterns show up repeatedly, and all three are decided before the first interview. Hiring a senior engineer for work that is mostly well-specified implementation wastes money and bores the hire into leaving. Hiring a mid-level engineer for work that needs architectural judgment produces an expensive rewrite eighteen months later. Hiring a generalist when the gap is one specific capability means the gap stays open.
How do you calibrate seniority?
Levels are not about years. Calibrate by what the person owns without supervision, which is the only definition that survives contact with an actual team.
| Level | Owns without supervision | Needs from you | Signal to test for |
|---|---|---|---|
| Junior | A well-specified task | Task breakdown, review, direction | Learning speed, receptiveness to review |
| Mid | A feature end to end | Priorities and acceptance criteria | Independent problem decomposition |
| Senior | A component or subsystem, including its tradeoffs | Business context and constraints | Judgment under ambiguity, ability to say no |
| Staff and above | A technical direction across teams | Strategic intent | Influence without authority, systems thinking |
The distinction that matters most commercially is between mid and senior. A mid-level engineer executes a defined solution well. A senior engineer tells you the solution is wrong before you build it.
That difference has a price attached in both directions. Paying senior rates for well-specified implementation wastes budget and bores the hire into leaving within a year. Paying mid-level rates for work that needed architectural judgment produces a rewrite eighteen months later that costs more than the salary difference ever saved.
Tip: write down the level and the evidence that would confirm it before you meet anyone. Deciding seniority after meeting a likeable candidate is how scope gets rewritten around a person.
Why do structured interviews beat conversations?
Because unstructured conversations measure interviewer comfort. When every candidate is asked different questions and evaluated against a different mental bar, comparison becomes impossible and the most articulate candidate wins regardless of capability.
Research on selection methods has consistently found structured interviews, meaning the same questions, the same dimensions and the same scoring rubric applied to everyone, near the top of predictive validity. In Sackett and colleagues' review of personnel selection assessments, structured interviews emerged as the strongest predictor of job performance, ahead of cognitive ability tests.
Structure is cheap to add. Fix the questions per role before the first interview, define what a strong and weak answer looks like for each, and require every interviewer to submit a written score before hearing anyone else's opinion.
What does a good developer interview loop look like?
Four stages, roughly four hours of candidate time, each producing evidence the others cannot.
| Stage | Duration | What it tests | What it cannot test |
|---|---|---|---|
| Screening conversation | 30 min | Communication, motivation, basic fit | Technical depth |
| Live technical discussion | 60 min | Reasoning, depth, ambiguity handling | Sustained code quality |
| Code review exercise | 45 min | Judgment, standards, communication of critique | Ability to build from scratch |
| Practical work sample | 90 min, paid if longer | Applied capability on realistic work | Long-term collaboration |
The code review stage is the most underused and the most informative. Give the candidate a small piece of imperfect, working code and ask what they would change and why. Reviewing well requires judgment that is far harder to fake than producing a correct answer, and it maps directly onto what senior engineers spend their days doing.
Keep the loop to four hours. Longer loops do not improve accuracy and they lose good candidates to faster-moving employers.
What should you test at each seniority level?
Testing the same things at every level is why loops fail to differentiate.
- Junior: how they respond to correction, whether they ask when stuck, whether they can read code they did not write.
- Mid: how they decompose an unfamiliar problem, whether their solution handles the unhappy path, whether they test what they build.
- Senior: what tradeoffs they name unprompted, how they handle a requirement that is wrong, whether they can explain a technical decision to a non-technical stakeholder.
- Staff and above: how they would sequence a migration, how they build agreement across teams, what they would choose not to do.
The senior-level question worth asking in every loop: describe something you built that you would build differently now. Candidates with real production experience answer specifically and without defensiveness. Candidates without it describe a success.
How has AI changed developer interviewing?
The take-home assignment is the main casualty. A standard four-hour exercise is now completed cleanly by an assistant, so a strong submission mostly proves tool access. Grading them the way you did three years ago measures very little.
That is not an argument for banning AI in interviews, since almost every working developer uses assistants and candidates who do not are frequently slower for no benefit. The differentiator has moved to judgment about output: what they rejected, what they rewrote, and whether they can explain submitted code line by line.
Three adjustments follow. Shift weight from asynchronous exercises to live discussion. Add the code review stage, since critiquing requires understanding that cannot be generated. And if you keep a work sample, run it live or follow it with a walkthrough where the candidate defends specific decisions.
Ask directly rather than policing. "Which assistants do you use, and where do you not trust them" tells you more about seniority than any algorithm question, because the answer requires having been burned.
What are the interview anti-patterns?
Most of these persist because they feel rigorous rather than because they work, and the two hardest to give up are the ones your own team was hired through.
- Algorithm puzzles unrelated to the work. They test preparation for interviews, not capability.
- Unpaid take-homes over two hours. You lose the strongest candidates, who have other options.
- Consensus meetings before written scores. The most senior voice in the room becomes everyone's opinion.
- Interviewing without a rubric. Each interviewer applies a private bar and calls it judgment.
- Trivia about syntax or APIs. Everyone looks these up, including your current team.
- Stretching the loop for a maybe. A fifth interview rarely resolves genuine doubt and usually manufactures a yes.
How do you run reference checks that work?
References are weak at selection and useful at disqualification, so treat them accordingly and keep them short.
Ask for references you choose from a list of past employers rather than accepting two nominated names, since nominated references are selected for enthusiasm.
Then ask three questions: what did they own, would you hire them again for the same role, and what would you make sure they had support with. The third produces the only candid answer of the three, because it lets the referee be helpful rather than negative.
Treat hesitation as data. A referee who is warm about someone's character and vague about their work has told you something.
What happens after the offer?
The hire is not finished at acceptance, and the first month determines whether your evaluation was right.
- Week one: access provisioned, a named buddy assigned, and one small real task merged.
- Day 30: contributing to normal work with review, and the codebase walkthrough completed.
- Day 60: owning something at the level you hired for, with reduced review overhead.
- Day 90: the evidence to judge whether the level calibration was correct.
If someone is not owning work at their hired level by day 60, the problem is usually onboarding rather than the person, and it is fixable at that point. Waiting until month six turns a fixable ramp problem into a performance conversation.
All of this assumes someone owns the cadence, the review standards, and the ramp itself, which is the job an engineering manager does and the reason ramp failures cluster in teams without one.
Who runs the loop if you have no technical leadership?
This is the question most hiring guides skip. If nobody internally can evaluate a senior engineer, no process compensates for it, and hiring one anyway means paying senior rates for work you cannot assess.
Two workable options exist. Borrow senior technical judgment for the evaluation itself, which is one of the more common uses of fractional CTO services, or hire the leadership layer before the individual contributors so there is someone to set the bar.
Get a shortlist worth interviewing
Four hours of interview time only works if the three people in front of you are worth interviewing. That filtering is the part we do: technical screening, delivery evidence and communication assessment happen before you see anyone, so the loop you just read runs on a shortlist instead of a slush pile.
Your rubric, your questions, your decision. Ours is the step before that.
