Your ATS Stores Resumes. It Doesn’t Tell You If a Candidate Can Actually Code.
ATS and screening are two different jobs. Most companies are paying for the first and assuming they got the second.
A hiring manager opens a shortlist his recruiter just sent over.
Five candidates. All “in the system” for the last two weeks. All routed properly through the ATS — the right tags, the right pipeline stage, the right neat little status icons next to their names. The dashboard, by every measurable corporate standard, looks happy.
He jumps on the first screen. Twelve minutes in, he realizes the resume and the human do not match. The candidate is fluent on the headline of their last project and immediately lost two levels in. He politely finishes the call, closes the tab, and stares at the four remaining names on his shortlist with a sinking suspicion.
The ATS did its job. The ATS just never had this job.
There’s a quiet assumption baked into most hiring stacks — that the applicant tracking system is doing the work of screening. It isn’t. An ATS is filing software. It catalogues, routes, and reports. It does not validate.
When a recruiter exports a shortlist from the ATS and sends it to the hiring manager, the candidates on that list have been stored well. They have not been evaluated. The first real evaluation happens in the technical interview, which is round three. By the time the gap shows up, an engineer has already lost an hour to it.
This whole post is about that gap.
Hiring and recruitment are two jobs. The ATS only does one.
Worth saying out loud, because at most companies nobody actually has.
Recruitment is the part of the work that finds, sorts, and moves candidates through the pipeline. Inbound applications, sourced profiles, scheduling, stage tracking, communications, reporting. That is the recruiter’s craft, and that is what an ATS is built to support.
Hiring is the part of the work that decides whether the candidate can actually do the role. Capability. Depth. The judgment calls about whether this specific person is going to make the team better — or quietly drag a sprint into the dirt for the next six months.
An ATS is a recruitment tool. It manages flow. A screening assessment is a hiring tool. It produces evidence. The two get conflated because they share the same vendors, the same dashboards, and the same vague marketing pages. And that confusion is exactly where most of the funnel pain in a typical company lives. Leadership sees a clean pipeline view and assumes the column labeled “Shortlist” means validated. It means filed. Those are very different words.
The screening test in the interview is the wrong place for it.
A 45-minute coding test wedged inside an on-site interview is what happens when nothing earlier in the funnel validates capability. It is a patch on a structural problem. A fix at the wrong stage of the pipe.
The cost of placing screening at round three gets paid in compounding interest. The recruiter has invested two weeks of coordination. The hiring manager has lost an hour they did not have. Two engineers have come off sprint work to sit on the panel. The candidate has rearranged their week around your loop. And only then does anyone discover that the resume and the person were never the same document.
By that point, everyone is too tired and too invested to act on it cleanly. The interview becomes a polite continuation of a conversation that already ended. The “thanks, we went in another direction” email gets drafted that evening. The dashboard, importantly, still looks fine.
A real screening technology runs the test before the interview, not during it. It treats the assignment as evidence, not theatre.
What a technical assessment test in recruiting should actually do.
The technical evaluation meaning of a real assessment is simple, and it has nothing to do with scores.
Get SkillBrew.AI’s stories in your inbox
Join Medium for free to get updates from this writer.
A real assessment captures how a candidate solves a problem in their actual domain — not just whether they reach an answer. What did they try first? Where did they pause? What did they do when their first approach failed? Did they even notice it had failed? Did they ask a clarifying question — or did they invent the missing context and hope?
That is the data a hiring manager has been trying to extract during a panel interview for years. It does not need a panel. It needs an assessment that is realistic, scoped, and observed properly.
A good one outputs something a hiring manager can read in three minutes and act on. Not a score. Not a percentile. Not a percentage of test cases passed. A trace of thinking — the artifact a senior engineer can look at and immediately recognize as “yes, this person reasons the way we need them to” or “this candidate is going to need more handholding than the role allows.”
That artifact is what the panel interview was always trying — and largely failing — to produce.
AI in the recruitment process, used honestly.
Here is the part of the conversation where the pitch usually starts getting embarrassing, so I’m going to try to keep it boring.
AI is doing the work this layer needs because the work is, frankly, reading. Reading code submissions. Reading written reasoning. Reading work samples carefully, at a depth and at a speed that humans cannot sustain across two hundred applicants per role per month.
That is it. That is the whole capability claim. Not “AI picks the best candidate.” Not “AI eliminates bias.” None of the other oversold sentences from any of the other vendors with a chatbot on their landing page.
None of this replaces the hiring manager. None of it produces a hire decision. It puts an evidence pack on the hiring manager’s desk before round three — a structured artifact showing how each candidate actually approached a relevant problem, with the recruiter’s shortlist already mapped against it.
The honest framing: AI doesn’t decide who gets hired. It closes the gap between resume claim and demonstrated capability before that gap costs a hiring panel a full day.
Speed of hiring is an advantage of — what, exactly?
A faster talent acquisition process that still ends in a wrong hire isn’t actually fast. It just feels fast on the dashboard.
Speed is an advantage only when the funnel is also validating. Otherwise, fast hiring just means fast attrition four months later — when the new hire is gently moved off the project, then off the team, then off the company, and the role re-opens with a slightly more anxious JD attached to it. Net velocity: negative. Dashboards: unbothered.
The companies that hire well aren’t the ones with the lowest days-to-offer. They’re the ones whose days-to-offer are spent on the right candidates. The two metrics look identical on a slide. They have almost nothing to do with each other.
The footnote
The ATS will keep filing. That part works. Nobody sensible is suggesting you rip out the system that keeps your pipeline organized; that would be a strange recommendation.
The validation layer the ATS never actually claimed to provide — that is the gap SkillBrew.ai quietly fills. Before round three. On the way to the hiring manager’s desk. As evidence, not theatre.
That’s the footnote. The argument stands without it.
Most companies are paying twice for one job and skipping the other entirely. The fix isn’t a louder dashboard or a glossier reporting view. It’s a stage that finally does the work everyone always assumed was already being done.
If this was useful, I would write about hiring, AI tools, and the messy reality of building teams. Follow along.
