Something has shifted in how companies write job requirements, and it is easy to miss if you only skim the bullet points. Roles that used to list tools now describe decisions. Instead of "proficient in SQL," the line reads "can decide which metrics actually answer the business question."
That change is not cosmetic. It reflects a real problem hiring teams are running into: when a candidate can produce a competent first draft of almost anything in a few minutes, the traditional test of whether they can do the work stops separating people.
The task stopped being the bottleneck
For most of the last two decades, hiring tests were built around production. Write this function. Build this model. Draft this campaign brief. The assumption was that producing good output was hard and rare, so measuring output was a reasonable proxy for capability.
That assumption is weakening in a lot of knowledge roles. Drafting is faster, boilerplate is cheaper, and the floor of "acceptable output" has risen for nearly everyone with access to modern tools.
What has not gotten cheaper is knowing which output to produce, catching the moment when a plausible answer is quietly wrong, and deciding what to do when the requirements conflict. Those are judgment problems, and they are stubbornly human.
What judgment actually means on a scorecard
"Judgment" is a vague word, which makes it dangerous to put on an interview rubric. Vague criteria are where bias creeps in, because interviewers fill the gap with gut feeling.
Teams doing this well break judgment into observable behaviors. A few that show up repeatedly on well-built scorecards:
Problem framing. Does the candidate restate the problem before solving it? Do they ask what success looks like, or do they jump straight to a solution?
Constraint awareness. Do they notice the budget, the deadline, the team size, the regulatory limit? Strong candidates surface constraints unprompted.
Error detection. Given a result that looks reasonable but contains a flaw, do they catch it? This one separates people faster than almost any other prompt.
Tradeoff articulation. Can they explain what they gave up and why? A candidate who presents a choice with no downsides has usually not thought it through.
Escalation instinct. Do they know when to stop and ask? Knowing the edge of your own competence is a skill, not a weakness.
Each of those can be scored on evidence rather than impression, which is what makes them usable.
Interview formats are changing to match
The most visible change is the decline of the pure production exercise. Take-home assignments that ask a candidate to build something from scratch are being replaced with formats that assume the artifact already exists.
Review exercises are becoming common. The candidate is handed a completed piece of work, a pull request, a financial model, a go-to-market plan, and asked what they would change and why. This tests critique, which is much harder to fake than production.
Debugging a flawed deliverable is another version of the same idea. Give someone an analysis with a subtle sampling error or a plan with an unstated dependency, and see whether they find it.
Some teams have started letting candidates use AI tools openly during technical exercises, then focusing the evaluation on what the candidate did with the output. Did they verify it? Did they notice what it missed? Did they know enough to reject the suggestion that looked confident but was wrong?
That last format is still uncomfortable for a lot of hiring managers, and reasonably so. It requires interviewers who can evaluate reasoning in real time rather than diffing an answer against a key.
The interviewer skill gap nobody planned for
Here is the part that gets underestimated. Testing for judgment is harder on the interviewer than testing for output.
Scoring a coding exercise against a rubric can be done by someone with moderate familiarity. Scoring whether a candidate reasoned well about an ambiguous tradeoff requires an interviewer with strong judgment themselves, plus the discipline to score behavior instead of agreement.
The failure mode is predictable: interviewers reward candidates whose conclusions match their own, and penalize different but equally defensible reasoning. That is not a judgment assessment, it is a similarity test, and it narrows a candidate pool fast.
Structure is the fix, not a new question list. Defined criteria, independent written scores before any group discussion, and a debrief that asks for evidence rather than impressions. Those practices mattered before; they matter more now that what is being measured is fuzzier.
What this means if you are job hunting
The preparation that used to work, memorizing question patterns and rehearsing polished answers, is losing value relative to something less comfortable: being able to explain your reasoning out loud, including the parts where you were unsure.
A few things worth practicing:
Talk through your thinking before you land on an answer. Silence followed by a perfect response now reads as less credible than a visible reasoning process.
Name your tradeoffs. When you describe a past project, say what you deliberately did not do and why. Interviewers are listening for that.
Be specific about how you verify things. If you use AI tools in your work, say so plainly, and be ready to describe how you check the output. Vagueness there is a red flag; a clear verification habit is a strong signal.
Say "I do not know, here is how I would find out" when it is true. Confidently wrong is now the most expensive failure mode a candidate can demonstrate.
The honest caveat
None of this is settled. Judgment-based evaluation is harder to standardize, harder to defend when a rejected candidate asks why, and more vulnerable to interviewer inconsistency than a scored technical exercise.
Some companies will overcorrect, drop structured assessment entirely, and end up hiring on vibes while telling themselves they are measuring judgment. That is a worse outcome than the take-home they replaced.
The teams that get this right will be the ones that treat judgment as something to be measured rigorously, with defined criteria and real evidence, rather than as permission to stop measuring at all.