Where Bias Gets Back Into AI Hiring

Suppose you run a careful audit of the AI tool your team uses to screen applicants. You check whether its recommendations show disparate impact across protected groups, and they don't. The model passes. Is your hiring now fair?

Not necessarily, and the reason is simple enough that it's odd how little attention it gets. The model doesn't hire anyone. It makes a recommendation, and then a person decides whether to follow it.

Most of the worry about AI bias in hiring over the past two years has been about the model. The standard argument goes like this. A model trained on historical hiring data will learn historical preferences. If a decade of successful hires skewed toward certain universities or certain backgrounds, the model will learn to weight whatever those people had in common. That's a real concern, and regulators, researchers, and compliance teams have been right to look at it. But it's only one of the places where bias can get in.

What happens at the override

Researchers who study how people work alongside AI have described a pattern called selective override. People tend to accept AI recommendations that agree with what they already thought and reject the ones that don't. Apply that to screening. When the tool puts forward someone who fits the recruiter's picture of a good hire, the recommendation goes through without much thought. When it puts forward someone who doesn't fit that picture, the recruiter overrules it more often.

If that's what's happening, the AI is reflecting recruiter intuition back rather than checking it. The one difference is that there's now a record that seems to say a neutral process made the call. That may be worse than the old way, since at least then nobody claimed the process was neutral.

Compliance reviews of AI hiring tools usually audit the model's outputs. Far fewer look at the gap between what the AI recommended and what people finally decided, broken down by candidate profile. That gap is where I'd look first.

A made-up example shows why. Suppose the tool recommended advancing 30% of candidates from non-traditional educational backgrounds, and recruiters followed that recommendation for 9% of them. The model would pass every fairness test you ran on it, and you would still have a clear bias pattern in who moved forward. The tool did its job and the workflow around it didn't.

What you'd want is a log of overrides: who was passed over despite a positive AI screen, who was advanced despite a neutral one, and whether those decisions line up with any candidate characteristics. This is easy to build. It's hard to want, because it measures the judgment of the same people who would have to ask for it, which is probably why so few companies have one.

The problem that came before the AI

There's an older source of unfairness that the AI debate tends to skip. Resume screening rewards people who are good at writing resumes. Career changers, people whose skills came from outside formal credentials, and people who have never heard of keyword optimization all score worse in an ATS than candidates with the same abilities and a better-formatted PDF. You don't need a model trained on biased data for that to happen. The format does it on its own.

A conversation has much less of this problem. It's hard to be filtered out for formatting when you are explaining out loud what you did and how you did it.

That was one of the reasons we built Asendia around a phone call rather than a resume. It calls each applicant within hours of applying and runs a structured screening conversation, with follow-ups based on what they actually say. What the recruiter gets afterward is a written record in which every candidate was asked the same questions in the same order, so the career changer and the person with the target-school degree can be compared on their answers rather than their formatting. Each question and answer is logged, and the notes and verbatim excerpts are saved to the ATS, where anyone reviewing a decision later can read them. That's more than you can say for a resume score whose weighting neither the candidate nor the recruiter can see. There's a separate post on agentic recruiting if you're curious where the line sits between AI that acts on a pipeline and AI that only helps you search.

A reviewable conversation doesn't fix override bias by itself, though. Someone still has to read the log.

So if you want to know whether your hiring is fair, asking whether the model is fair is only the start. The harder question is whether you could reconstruct, for any candidate, what the AI said about them, what a person then decided, and why. Most companies can't yet, and that seems like the right place to begin.

Ready to transform your hiring strategy? Schedule a Demo with our founders today!