Your End-to-End Tests Do Not Need Step Definitions

Two agents and a real browser run a Gherkin file with no step definitions underneath it, and code checks every number

In this example, I deleted the step definitions from an end-to-end test and handed the plain-English file straight to two AI agents and a real browser. Seven steps passed on the first try in under a minute, for about ten cents, and the one number that mattered was checked by ordinary code, never by the model. This was a single recorded run on a small static demo app, so read the numbers as an illustration, not a benchmark.

Here, an end-to-end test, or E2E, is a script that clicks through the real application the way a user would, to prove the whole thing still works. Gherkin writes that test in plain Given/When/Then English, so a product manager can read it and sign it off. Step definitions are the glue code hidden underneath, the part that says “click the button whose internal name is btn-refund-v2, then wait 300 milliseconds.” The English reads well; the glue code breaks every time the frontend team renames something.

So I removed the glue code. Here the .feature file is the whole test. One agent, the executor, reads each step and carries it out in a real browser with no visible window; a second agent, the evaluator, judges what happened; and plain Python does the rest. On the refund scenario below, all seven steps passed on the first attempt in 58.7 seconds, and the final step…