What the Research Says
A meta-analysis of over 85 years of hiring research found that unstructured interviews have a predictive validity of 0.20 - meaning they explain roughly 4% of variance in job performance. Structured interviews, where every candidate is asked the same questions in the same order and scored against defined criteria, reach a predictive validity of 0.51.
That is a meaningful difference. Structured interviews are more than twice as good at predicting whether a candidate will actually succeed in the role.
Why Unstructured Interviews Fail
In a typical unstructured interview, each interviewer:
- Decides in the moment which questions to ask
- Allows the conversation to go wherever it leads
- Evaluates based on overall impression rather than specific criteria
- Has no consistent way to compare one candidate to another
The result is that hiring decisions are heavily influenced by factors unrelated to job performance: how similar the candidate is to the interviewer, how articulate they are, how warm their handshake was. This is what researchers call affinity bias and the halo effect.
Components of a Structured Interview
1. Job analysis first. Before writing a single question, define what success looks like in the role. What competencies actually predict performance? For a customer success manager, this might be problem decomposition, written communication, and managing escalations.
2. Consistent questions. Every candidate is asked the same questions in the same sequence. This makes comparisons valid. You cannot compare candidates if each person is asked a different set of questions.
3. Anchored rating scales. Each answer is evaluated against pre-defined criteria, not general impressions. An anchored scale for "communication clarity" might define a 1 (answer is unclear and requires follow-up), 3 (answer conveys the point with some ambiguity), and 5 (answer is organized, specific, and backed by evidence).
4. Independent scoring. Each interviewer scores candidates independently before discussing with the team. Group discussion before scoring leads to anchoring bias, where the first opinion dominates.
Implementation Challenges
Most teams know structured interviews are better but do not adopt them. Common barriers:
- "It feels robotic." It does not have to. Structured does not mean cold. You can still build rapport, ask follow-up probes, and engage naturally within a consistent framework.
- "We do not have time to design questions." The upfront work takes 2 to 3 hours per role type. For roles you hire frequently, this is a one-time investment that pays dividends across every future hire.
- "Our best hires came from gut feel." This is survivorship bias. You do not see the hires you missed because they were screened out by arbitrary judgment.
Getting Started
- Pick one role type you hire at least 3 times per year.
- Define 4 to 6 core competencies for that role with the hiring manager.
- Write 2 behavioral questions per competency.
- Build a simple scoring rubric with 3 anchor points per question.
- Run the next 3 hires through this process and compare outcomes with historical hires.
The evidence is compelling. Structured interviews take more upfront work, but they produce better hires with less bias and lower turnover.