What AI Interview Prep Does to Structured Interviews

Structured interviews were one of the few things in hiring that research clearly supported. You ask every candidate the same questions, score the answers against the same behavioral rubric, and compare people on equal terms. For decades, studies found that this predicts job performance considerably better than an unstructured chat, by some estimates about twice as well, and it cuts down on interviewer bias too.

But the method rested on an assumption nobody bothered to state, because it seemed obviously true: candidates arrive with roughly similar amounts of preparation. If everyone has prepared about the same amount, then differences in their answers reflect differences in experience and judgment, which is what you're trying to measure.

For most of the history of structured interviewing, there was a practical limit on how much anyone could prepare. You could read a few guides on behavioral interviews, write out some STAR stories, and do a mock interview or two with a friend. That helped, but only so far. A strong candidate still stood out, because practice couldn't fully stand in for real experience, and a calibrated rubric could usually tell a well-coached average candidate from a genuinely good one.

AI interview prep tools have removed that limit. A candidate can now run as many mock behavioral interviews as they like, have each answer scored against the same things your rubric looks for (STAR completeness, specificity, how clearly they describe their impact) and get corrections after every answer. Give a motivated but average candidate a couple of weeks of that, and their answers can become hard to tell apart from those of someone who really does think and communicate at a higher level.

So the interviews aren't getting worse. The candidates are getting better at being interviewed. That would be fine if being interviewed were the job.

Economists have a name for this. Goodhart's law says that when a measure becomes a target, it stops being a good measure. A behavioral interview was meant to be a neutral instrument, a way of observing something that existed independently of the interview. Once a large share of candidates are training specifically against that instrument, with a tool giving them instant and detailed feedback, what it measures shifts. It starts measuring how well someone can produce the expected answers under known conditions. That used to track job performance reasonably well, because the limit on preparation kept it honest. Without the limit, I'd expect the connection to weaken.

If that's right, you'd expect a particular pattern to show up in outcome data. Some candidates would score in the top quartile on your rubrics, advance through every round on merit, accept the offer, and then underperform at 60 or 90 days. They wouldn't have lied about anything. The person in the interview would simply have been a rehearsed version of them, and the person who shows up for work is the unrehearsed one.

Where do you look instead? What behavioral interviews used to capture, how someone thinks when they don't have a polished answer ready, hasn't gone away. It just shows up in different places. As far as I can tell, AI prep tools haven't learned to reproduce how a person actually reasons and talks in the moment. Someone who thinks clearly, asks good questions without being prompted, and copes with a real surprise in conversation still sounds different from someone with 40 hours of interview reps. You just have to reach people before they've switched into interview mode.

That makes first contact more informative than most recruiters assume. A call that comes before a candidate knows they're being formally evaluated gets you closer to how they normally think and talk. Few teams designed their process around this, but it's worth paying attention to how candidates come across in the moments they didn't prepare for.

It's a large part of why Asendia works the way it does. Look at it from the candidate's end. A few hours after applying, before any formal interview invitation and before they've started rehearsing, their phone rings. The call is structured enough to check them against what the role requires, but open enough that they end up describing their experience in their own words, without the STAR scaffolding, and asking whatever they actually want to know about the job. It's exactly the kind of moment they didn't plan for. What the recruiter gets back is a qualification summary built from what the candidate said, written into the ATS.

Something similar is happening to written applications earlier in the funnel, which I wrote about in the post on AI-generated applications flooding your ATS.

None of this means structured interviews should go. The research behind them was real, and reducing interviewer bias still matters. What has changed is what a high rubric score tells you. If you keep reading it the way you did five years ago, I suspect you'll keep hiring excellent interviewers and wondering why the 90-day reviews don't match the scorecards.

Ready to transform your hiring strategy? Schedule a Demo with our founders today!