Developers ship at AI speed. Quality engineering still ships at human speed.

Why the quality engineer became the bottleneck, and what it takes to stop being one.

There is a quiet shift happening in software teams, and quality engineers feel it before anyone else.

A year ago, a developer might open two or three pull requests in a day. Today the same developer has a coding agent beside them, and the pull requests arrive faster than anyone can read them. Features that took a sprint take an afternoon.

The quality engineer did not get a coding agent for testing. Or if they did, it writes test code just as fast as the developer’s agent writes product code, which only moves the pile.

So here is the week a lot of us are living. Four developers with AI. One quality engineer. Every afternoon a stack of changes to verify. Every morning a stack of red tests. And most of those reds are not bugs at all.

Most red tests are not bugs

Look at what actually breaks an end-to-end suite on a normal day.

• A button got renamed. “Book now” became “Reserve a room”. The product works. The test looked for the old label.
• A field moved. The guest details left their own page and joined the checkout form. Same data, new place.
• A wait was a guess. A step sleeps for two seconds, and today the server took three.
• A helper did too much. One function called proceedToCheckout() clicks through five screens. When screen three changes, the whole thing fails, and the error points at the wrong place.

None of these is a defect. Each of them costs a quality engineer twenty minutes to find, understand and fix. Multiply by the number of developers, multiply by the speed of their agents, and the maintenance work grows faster than any one person can keep up with.

That is the trap. The faster the team gets, the further behind quality falls. Not because quality engineers are slow, but because their job has quietly turned into repairing selectors.

The job was never “type every selector”

Step back and ask what a quality engineer is actually for.

It is not writing page.getByRole('button', { name: 'Book now' }).click(). That is a means. The job is deciding what the product must do, checking that it does, and noticing when it stops. It is judgement.

The shift teams need is not “test faster”. It is to move the quality engineer from the person who types every line to the person who decides what is tested and checks the evidence. Describe the intent. Let a machine handle the path. Keep the verdict human.

That sounds like a slogan, so here is what it looks like in practice.

Tests written as what they mean

Saffron is the tool I built around that idea. A test is written in plain language:

Scenario: Book a room for two nights
Given I am on the rooms page
When I book the sea view room for 2 nights
Then the booking is confirmed

There is no code underneath to maintain. The first time the scenario runs, an AI agent performs each step in a real browser, and Saffron records exactly what it did: every click, every field, every check. That recording is saved as a file in your repository, beside the scenario.

From then on, the AI is out of the loop. Every later run replays the recording with zero AI calls and zero tokens. It is deterministic, it is fast, and it costs nothing per run.

You do not even have to write the scenarios yourself. A coding agent with the Saffron plugin can explore your application, find the flows worth testing, write them as scenarios, and record them. Your part is deciding which ones matter.

When the UI changes, the test repairs itself, and asks first

Back to the renamed button.

When a replay no longer matches the page, Saffron does not just fail. The agent picks up from the step that broke, in the same browser and on the same page, works out what that step/test case was trying to do, and finishes the run. Then it files the repair as a proposal: the new recording beside the old one. Nothing changes in your suite until a person accepts it.

There is one thing it will never repair on its own: the verdict. In a scenario, the Given and When steps are the path, how you get there. The Then steps are the checks that decide pass or fail. By default, Saffron never heals a Then. If “the booking is confirmed” is no longer true, the run is red and stays red, and the report says so in plain words. A self-healing tool that could rewrite its own checks until they pass would be worse than no tool at all.

So the renamed button becomes a proposal you glance at, not a twenty-minute investigation.

The part that actually saves your week: less to review

Here is the honest objection. If an AI writes and repairs hundreds of tests, someone has to review hundreds of recordings. Haven’t we just swapped one pile for another?

Get S Chathuranga Jayasinghe’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This is where Saffron does most of its work, and it does it before you ever see a recording. Every recording the agent makes is tested by machine first, with no AI involved:

• It must replay on its own, in a fresh browser session, with fresh data, without the agent that made it.
• It must pass on every row of a data table, each with its own values, so no row passes on another row’s data.
• Every check must be able to fail. Saffron runs each check on the page as it was before the step it verifies. A check that passes there too was never checking anything.
• Its dates must survive tomorrow, the end of the month and the end of the year.
• A check may not act. While recording, a check is not allowed to click Save or Submit to make itself true.
• A save must finish before the page moves on, or the recording is flagged.

Recordings that pass all of it can be accepted together, in one command. Anything suspicious is held back for a person, with the reason written next to the step it is about.

That changes the shape of the work. You stop reading a hundred recordings to find the three that are wrong. The machine finds the three, and tells you why. Your attention goes where it is actually needed.

We wrote up every one of these checks in detail, in the order they happen, in the Saffron docs (link at the end).

What it looks like at real scale

A client recently put this to the test on a production booking web app. They had a suite of about 400 end-to-end tests, written as code for another automation tool. Nothing could simply be converted: every test had to be understood, rewritten from its intent, and recorded from scratch.

An AI agent with the Saffron plugin did the work. For each test it worked out what was really being checked, explored that flow in the running app, wrote it as plain-language scenarios with shared steps, recorded it, ran it, and verified it.

The result:

• 483 test cases, all green. More than the original 400, for a reason worth telling below.
• 34 feature files, 378 scenarios and 43 shared step sets.
• $119.03 in total AI cost for recording, proving and re-recording, which comes to about $0.25 per test case. Exploring the old suite and writing the scenarios ran on a Claude subscription and is not included in that number.
• About one hour of human time, for the whole migration.
• Zero AI cost for every replay after that. The agent only comes back when the UI changes and a test needs repair.

Why did 400 tests become 483? Many of the old tests called one helper that did far too much at once, something like “proceed to checkout by adding two guests and selecting available packages and rooms”. The agent’s first instinct was to write that as a single plain step: “proceed to checkout”. That is a wish, not a step. It hides too much, and a step that hides too much is flaky. So those were broken into smaller, honest steps, and sometimes into separate tests. More tests, each one clear about what it does.

The migration also found a few gaps in Saffron itself, such as checking a downloaded calendar file and the border colour of an input. They were fixed. That is how a tool gets better: real suites, real edge cases.

Where the machine stops

It would be dishonest to stop there, so here is the limit.

The machine can prove that a recording is a real, working test of the scenario as written. It cannot prove that the scenario is the one your product needs. It does not know that the refund flow matters more than the newsletter sign-up, or that a certain edge case cost your company a week last year.

That part is yours. It always was. It is the part of the job that needs a quality engineer: judgement, context, and knowing what “working” means for your users.

The point of tools like this is not to replace that judgement. It is to give it back its time.

The short version

The speed gap between development and quality is real, and it is not going to close by everyone working longer hours. It closes when quality engineers stop spending their days repairing selectors and start spending them on what only they can do.

Write tests as what they mean. Let a machine record them, repair them, and check its own work. Keep the verdict human.

AI explores and writes. A machine checks. A person decides.

Try it: npm i -D saffron-ai, then npx saffron init --examples installs a small example suite you can run straight away. Saffron is free to use.

• Saffron: https://saffron-ai.io
• How every recording is checked before you review it: https://saffron-ai.io/docs/how-recordings-are-checked
• Two-minute videos of each part: https://saffron-ai.io/videos