After AI Interview Prep, the Follow-Up Question Is What's Left

What is a question like "tell me about a time you handled a difficult stakeholder" supposed to measure?

Not really the story. The question was a way of getting someone to reach into their past without much warning and seeing what came out: which episode they picked, how they framed it, whether they blamed anyone, how they sounded while telling it. "Walk me through a project that went wrong" and the old greatest-weakness question work the same way. To the extent they worked at all, it was because the candidate had to answer more or less on the spot.

That assumption has quietly stopped holding. Candidates now run their likely interview questions through an AI tool before a first call or a panel. One estimate puts it at about seven in ten active candidates for professional roles. Whatever the exact number, you should assume the person on your next screen has done it.

Consider what that does to the answer you hear. Suppose a candidate spends an evening preparing with a chatbot. Their stakeholder story comes out in tidy STAR format because the AI tidied it. The numbers in it are well chosen because the AI helped choose them. Their weakness is vulnerable in exactly the right amount, enough to sound self-aware and not enough to worry anyone. None of this is dishonest. But what you're listening to is a final draft, and a final draft mostly tells you how good the editing was.

So a team that relies on behavioral questions as its main screen ends up partly grading the candidate's prep. That would matter less if these questions had been very predictive to begin with. They weren't. One commonly cited figure puts the predictive validity of structured behavioral interviews at around 0.35, which is real but modest. A fair amount of what interviewers took from those conversations was noise before anyone prepped with AI, and rehearsal pushes it further that way. Meanwhile teams spend several rounds and a debrief collecting better and better versions of the same prepared material. The effort is real, but what it produces is thinner than it looks.

Where did the useful part of the interview go? I think it moved one step later, to the follow-up.

A candidate can rehearse the likely versions of a question, because a question on a list only has so many likely versions. What they can't rehearse is your reaction to what they just said. If they mention that the project "recovered in the end," you can ask what they changed and who pushed back. If a number sounds too clean, you can ask how it was measured. Their prep didn't anticipate those questions because they depend on an answer that didn't exist a minute ago. The second and third exchange on a topic is where you find out how someone reasons, and how they talk when they actually have to think.

Most interviews don't work this way, though, even ones that feel probing. An interviewer with a rubric hears an answer, writes a note, and moves to the next competency. That's exactly the format AI prep is good at. An interviewer who listens and follows the thread learns something different. Most teams believe they're doing the second thing while mostly doing the first, partly because following up well is tiring, and hard to keep doing across a day of back-to-back calls.

This is a big part of why we built Asendia the way we did. The alternative to an interviewer working down a rubric is one who follows the thread every time, including on the last call of a long day, and that's easier to get from software than from a tired person. Asendia screens candidates in live spoken conversations, and when an answer contains an inconsistency or a claim with nothing behind it, that's where the conversation goes next, which is the kind of question rehearsal doesn't cover. Afterward the ATS gets a verbatim transcript of how the person actually spoke, along with qualification notes, key quotes and a ranked shortlist. A hiring manager can see how someone handled being pushed one level deeper before giving them an hour of their own time.

The same thing is happening earlier in the funnel, where applications themselves are increasingly written with AI. There's a separate post on why AI-generated applications break text-based screening. Put the two together and almost everything a candidate hands you in advance has been through an editor.

I don't think any of this makes AI prep bad. Candidates arriving better prepared is fine, probably good. What's changed is that the spontaneous part of a behavioral question used to carry the information, and for most professional roles it no longer does. Writing new questions won't help much, since new questions get prepped too. What helps is a conversation where the next question depends on the last answer. If you want to see where your own process stands, pull a handful of recent screening notes and count how often the interviewer asked a second question about the same answer.

Ready to transform your hiring strategy? Schedule a Demo with our founders today!