Execution Speed in E2E Test Automation Clarified
What’s the speed bottleneck for your E2E automated tests?
In this article, I’ll cover what you should be looking at for improving/affecting the execution speed of your E2E automated tests.
By E2E test automation execution, I mean running an automated end-to-end test from an end-user perspective. In web testing, for example, this involves watching a browser interact with UI elements — clicking buttons, filling forms, navigating pages — just as a real user would.
Compared to unit tests, E2E tests are undeniably slower. However, they are still far faster than manual testing. Because of this, most software teams naturally want to optimize E2E execution speed — and many attempt to do so. Unfortunately, some approaches are misguided. In this article, I’ll clarify what actually affects execution speed in E2E test automation — and what doesn’t.
1. Script Language Differences Are Negligible
One common misconception is that the choice of programming language significantly impacts execution speed.
In application development, compiled languages like C++ often have a major performance advantage over scripting languages. But in the context of test automation, this difference is almost negligible.
Why? Because the majority of execution time is spent waiting on the application under test — not executing the test script itself. The script simply issues commands (e.g., click, type, wait), browser and network speeds are very fast right now, so the backend app is the most time spent on the application itself. Whether the script is written in Ruby, JavaScript, Java, or Python has minimal impact on overall execution time.
So, instead, choose a language that is flexible and easy to read, e.g. Ruby, a scripting language.
2. Framework Differences Are Also Overstated
Another frequent claim is that certain test frameworks are “much faster” than others. As stated before, it is the app costs the most time.
Get Courtney Zhan’s stories in your inbox
Join Medium for free to get updates from this writer.
In reality, the test automation framework differences are often exaggerated. Some frameworks — such as Cypress — promote faster execution as part of their marketing. However, real-world benchmarks often show a different picture. Many framework demos use extremely simple applications (like a basic TODO app). These scenarios don’t reflect the complexity of real-world systems, where network latency, rendering time, and backend processing dominate execution time.
3. Execution of Individual Test vs Test Suite Execution
In E2E test automation, we usually run in one of two modes:
- Running a single test (for development, refine, stabilise, debugging)
- Running a full test suite (for regression runs)
Some test automation frameworks, e.g. Cypress, claim it has a much faster test execution speed, but various benchmark shows otherwise. Why? One reason is architectural: for example, frameworks relying on chatty communication protocols can introduce overhead that accumulates over time, especially in large test suites.
4. Sum of Individual Test Time vs. Suite Execution Time
If you have a suite of 200 automated E2E tests, and each test takes an average of one minute, the total time adds up to 200 minutes (over three hours). However, this does not have to be the actual suite execution time.
By utilizing parallel execution, tests can run concurrently across multiple machines or environments, significantly reducing the overall runtime. With an efficient parallel setup, what would take hours sequentially can often be completed in a fraction of the time.
For example, below is a test run of the WhenWise test suite, which consists of 626 user-story-level UI automated tests built with Selenium WebDriver and executed on the BuildWise Continuous Testing server
By utilizing five build agents, the total execution time was reduced to 47 minutes — down from nearly three hours when run on a single machine.
