E2E test automation can generally be divided into the following categories:

1. Web Test Automation — Automating web applications using frameworks such as Selenium WebDriver and Playwright.
2. Mobile Test Automation — The de facto framework is Appium, together with platform-specific drivers such as XCUITestDriver for iOS and Espresso or UIAutomator2 Drivers for Android.
3. API Test Automation — Automating the testing of APIs, typically covering protocols and architectures such as SOAP and REST.
4. Desktop Application Test Automation — Automating native desktop applications. Appium, together with platform-specific drivers, is a commonly used approach

In this article series, I will introduce Desktop app test automation on macOS platform.

Why Desktop App Test Automation?

Compared with web, mobile, and API test automation, desktop application test automation is rarely discussed or given much attention. One simple reason is that it is considerably more challenging. If achieving reliable web test automation is already difficult, desktop application test automation can be even more challenging.

Some people may argue that modern application development is increasingly focused on web and mobile applications. So, why bother learning desktop application test automation?

This observation is valid. However, it does not mean desktop application test automation is no longer relevant or useful.

The reality is that a significant proportion of work is still performed on desktop computers. The ability to automate desktop applications and repetitive desktop tasks can therefore deliver substantial benefits.

This is particularly relevant in the age of AI, where combining AI capabilities with desktop automation opens up new opportunities to automate workflows, driving agents, improve productivity, and reduce manual effort.

In other words, desktop application test automation is not just about testing desktop software. It also provides valuable foundations for automating the desktop environment itself.

Just imagine being at work when your colleagues see you click a single button to automate a routine task that they perform manually every day, completing it much faster and more reliably.

Desktop Application Test Automation Is Different from Web Test Automation

Web test automation benefits from a common, universally adopted standard established by the W3C. As a result, although different automation frameworks may be used, the underlying automation approaches are largely similar.

Desktop test automation, however, doesn’t have an equivalent common standard. Consequently, implementation approaches can vary significantly across operating systems and platforms.

Even when using the same automation framework, such as Appium, different platform-specific drivers can introduce substantial differences in capabilities, implementation, and behaviour.

Therefore, discussions about desktop application test automation need to be platform-specific rather than treating desktop automation as a single, unified approach.

The Driver Landscape for Desktop Automation

Even within Windows desktop application automation, several drivers have emerged, including:

• WinAppDriver (deprecated)
• Windows Driver
• NovaWindows Driver
• WinCore Driver

Each driver may offer different capabilities and exhibit different behaviours, meaning that the choice of driver can significantly influence the automation implementation.

The situation is comparatively straightforward for macOS desktop application automation, where there is essentially one established Appium driver: Mac2 Driver.

In the upcoming articles, I’ll show you how to set up Appium and develop automation scripts to test desktop applications on macOS.

Related reading:

• My books:
  - Web Test Automation in Action: Volume 1
  - Effective Web Test Automation with Selenium WebDriver and JUnit
  - Practical Web Test Automation with Playwright and Pytest
• My “Discover Ruby Programming Through Fun Examples” interactive course on Educative

In this article, I’ll cover what you should be looking at for improving/affecting the execution speed of your E2E automated tests.

By E2E test automation execution, I mean running an automated end-to-end test from an end-user perspective. In web testing, for example, this involves watching a browser interact with UI elements — clicking buttons, filling forms, navigating pages — just as a real user would.

Compared to unit tests, E2E tests are undeniably slower. However, they are still far faster than manual testing. Because of this, most software teams naturally want to optimize E2E execution speed — and many attempt to do so. Unfortunately, some approaches are misguided. In this article, I’ll clarify what actually affects execution speed in E2E test automation — and what doesn’t.

1. Script Language Differences Are Negligible

One common misconception is that the choice of programming language significantly impacts execution speed.

In application development, compiled languages like C++ often have a major performance advantage over scripting languages. But in the context of test automation, this difference is almost negligible.

Why? Because the majority of execution time is spent waiting on the application under test — not executing the test script itself. The script simply issues commands (e.g., click, type, wait), browser and network speeds are very fast right now, so the backend app is the most time spent on the application itself. Whether the script is written in Ruby, JavaScript, Java, or Python has minimal impact on overall execution time.

So, instead, choose a language that is flexible and easy to read, e.g. Ruby, a scripting language.

2. Framework Differences Are Also Overstated

Another frequent claim is that certain test frameworks are “much faster” than others. As stated before, it is the app costs the most time.

In reality, the test automation framework differences are often exaggerated. Some frameworks — such as Cypress — promote faster execution as part of their marketing. However, real-world benchmarks often show a different picture. Many framework demos use extremely simple applications (like a basic TODO app). These scenarios don’t reflect the complexity of real-world systems, where network latency, rendering time, and backend processing dominate execution time.

3. Execution of Individual Test vs Test Suite Execution

In E2E test automation, we usually run in one of two modes:

• Running a single test (for development, refine, stabilise, debugging)
• Running a full test suite (for regression runs)

Some test automation frameworks, e.g. Cypress, claim it has a much faster test execution speed, but various benchmark shows otherwise. Why? One reason is architectural: for example, frameworks relying on chatty communication protocols can introduce overhead that accumulates over time, especially in large test suites.

4. Sum of Individual Test Time vs. Suite Execution Time

If you have a suite of 200 automated E2E tests, and each test takes an average of one minute, the total time adds up to 200 minutes (over three hours). However, this does not have to be the actual suite execution time.

By utilizing parallel execution, tests can run concurrently across multiple machines or environments, significantly reducing the overall runtime. With an efficient parallel setup, what would take hours sequentially can often be completed in a fraction of the time.

For example, below is a test run of the WhenWise test suite, which consists of 626 user-story-level UI automated tests built with Selenium WebDriver and executed on the BuildWise Continuous Testing server

By utilizing five build agents, the total execution time was reduced to 47 minutes — down from nearly three hours when run on a single machine.

Non-Medium Members: You can read this article for free on “The Agile Way — Practical Automated E2E Testing” Substack Newsletter.

In mobile test automation, waiting is essential. Mobile applications frequently perform asynchronous operations such as network requests, UI animations, screen transitions, and background rendering. If a test script attempts to interact with an element before the UI is fully ready, the test may fail even though the application itself is functioning correctly, this is known as a ‘flaky’ failure.

Appium follows the WebDriver protocol and therefore inherits the same waiting mechanisms used by Selenium WebDriver. This means mobile tests can use implicit waits, explicit waits, and polling-based synchronization in exactly the same way as web tests.

Waiting difference between Web and Mobile Test Automation

In web automation, waiting is typically required for events such as AJAX requests, DOM updates, or JavaScript execution. Modern browsers expose a relatively stable DOM, and elements usually become interactable quickly once they appear. In contrast, mobile automation interacts with a native UI hierarchy provided by platform automation frameworks, such as UiAutomator2 on Android and XCUITest on iOS. These frameworks may report that an element exists even while the screen is still animating or transitioning, meaning the element may not yet be stable or ready for interaction.

Another difference is device performance variability. Web tests typically run on consistent browser environments or cloud grids, whereas mobile tests run on real devices or emulators with widely varying CPU, memory, and rendering speeds. Network conditions also affect mobile apps more strongly because many mobile apps rely heavily on backend APIs for data loading.

Because of these factors, mobile test automation often requires more careful synchronization strategies than web automation. Instead of waiting only for element presence, reliable Appium tests frequently wait for conditions such as element visibility, enabled state, absence of loading indicators, or completion of screen transitions. Designing proper waiting strategies is therefore a crucial part of building stable mobile automation suites.

For common alternative waiting (such as explicit fixed waits and implicit waits), see this other article:

• Waiting Strategies for Test Steps in Web Test Automation

This article will focus on auto-waiting for Appium specifically.

Catching Specific Errors

My current preferred method to handle effective waiting is to use Selenium WebDriver’s wait but customize it to ignore the Appium ‘element not found” and “stale element reference” errors that are associated with item visibility/loading.

wait = Selenium::WebDriver::Wait.new(
  timeout: 15,
  interval: 0.5,
  ignore: [Selenium::WebDriver::Error::NoSuchElementError, 
           Selenium::WebDriver::Error::StaleElementReferenceError]
)

wait.until do
  el = driver.find_element(:id, 'com.example:id/submit')
  el.click if el.displayed?
end

You can also use Appium’s wait helper method directly, which is a wrapper around the Selenium wait:

wait { driver.find_element(:id, 'my_element').displayed? }

To learn more about Auto Waiting, check out this other article:

Auto-Waiting in Web Test Automation Clarified

My father’s innovative try_for approach works well for Appium too, as long as the test scripts are in Ruby syntax.

• My Innovative Solution to Test Automation: Easy Wait in Web Automation Scripts

Related reading:

• My books:
  - Web Test Automation in Action: Volume 1
  - Effective Web Test Automation with Selenium WebDriver and JUnit
  - Practical Web Test Automation with Playwright and Pytest
• My “Discover Ruby Programming Through Fun Examples” interactive course on Educative

Using macOS to develop automated E2E scripts for my latest book

This is a continuation from Part 1. In this article I’ll be looking outside of specific tech stacks, and more on the benefits of using macOS for testing.

Why I Choose macOS as the platform for my upcoming book? Part-1

1. E2E test automation foundation is scripting, which unix-based Mac is far superior

Currently Windows does include Linux now with WSL2, but some older versions of Windows may not. macOS is Unix-based out of the box and natively supports popular scripting languages and package managers (like Homebrew). This makes environment set up and scripting a lot smoother.

2. Mobile test automation with iOS

Although my recent book focuses on web test automation, I am also interested in mobile test automation. iOS development and testing are locked to macOS devices. So if you plan to expand to mobile testing, having macOS ensures you are not blocked when running and developing iOS test suites.

3. Continuous Testing lab

The Mac mini (M4 and and upcoming M6) are the best choices for a dedicated testing lab as it ticks all of boxes:

• Low Price, from only US$599
• Fast
• Reliable
• Small Form Factor
• Quiet
• Power Efficiency & Environmental Friendly

For detail, check out this article, M4 Mac Mini: A Perfect Build Machine for E2E Test Automation Labs.

4. Easy to add macOS virtual machine

On Apple hardware, it’s easy to set up native VMs, unlike Windows which involves licensing keys and activation. On macOS, with VMware Fusion now available for free, setting up a Windows virtual machine on a Mac is easy, it’s easy to create new virtual machines, which is helpful to use for test build machines.

5. Apple’s built-in Screen Sharing app is great for remote controlling build machines.

macOS also comes with Apple’s “Screen Sharing” app, which lets you view and control another macOS machine remotely. This is really convenient if you have additional build machines, so you can remote in, update configurations and monitor builds from a single central machine.

API tests are generally faster and more reliable than UI-based end-to-end (E2E) tests. For this reason, many ‘agile teams’ focus heavily on API testing. The technical challenges are certainly lower, as software engineers are typically more comfortable writing and maintaining API tests.

However, for applications with a user interface (which are most applications, such as Web and Mobile), API testing alone is not true end-to-end testing. Many defects related to the user experience — including rendering issues, browser compatibility problems, JavaScript errors within the UI layer — cannot be detected through API testing.

Teams often accumulate a large number of API tests because they are granular and straightforward to create. While API testing is significantly faster than UI automation, it is still black-box testing and considerably slower than white-box unit testing. As the test suite grows, it becomes impractical for developers to execute the entire API regression suite repeatedly during day-to-day development.

The logical solution is to execute the full API test suite automatically in a continuous testing environment whenever application changes are made. Some organizations achieve this through a CI/CD pipeline. My father prefers the term Continuous Testing (CT) because many so-called CI/CD pipelines perform little or no meaningful automated regression testing, particularly at the end-to-end level.

The answer to the title question: We should run the whole suite of API tests on a CT server, multiple times a day.

A Continuous Testing environment provides rapid feedback by automatically executing the complete API regression suite whenever code changes are introduced. This helps detect defects early, reduces the cost of maintaining both the application and the test suite, and prevents automated tests from becoming stale, outdated, and rarely executed! After all, an automated test that is not run regularly provides little real value.

Both API testing and functional UI testing are types of black-box testing, meaning they operate independently of the application’s internal implementation. Consequently, the same Continuous Testing (CT) environment can be used to execute both API and end-to-end (UI) test suites, improving infrastructure utilization and simplifying operations.

For example, my father’s web, mobile, and API test suites are all built on the same RSpec-based testing framework, allowing them to share a common syntax and execution model (within the same parallel testing lab, consisting of a BuildWise CT server and multiple build agents).

Related reading:

• My books:
  - Web Test Automation in Action: Volume 1
  - Effective Web Test Automation with Selenium WebDriver and JUnit
  - Announcing my new book, “Practical Web Test Automation with Playwright and Pytest”
• My “Discover Ruby Programming Through Fun Examples” interactive course on Educative
• A Story of 100X API Testing I Implemented in a Matter of Days

Using macOS to develop automated E2E scripts for my latest book

Except for one half-broken Dell laptop that I used during high school, my family has always been a Mac-only household.

The original Practical Web Test Automation book was written by my father in 2009. All of the test scripts were initially developed on his MacBook. They were then rerun in a Windows virtual machine solely to capture the screenshots shown in the book.

For my new book, Practical Web Test Automation with Playwright and Pytest, I decided to eliminate that extra work and use macOS as the only platform presented throughout the book.

First, let me make one point clear. For end-to-end test automation — whether web or mobile — the entire technical stack is cross-platform. Frameworks such as Selenium WebDriver, Playwright, and Appium, along with tools such as VS Code, TestWise IDE and BuildWise CT Server, work the same way across operating systems.

As a result, anyone who learns test automation on macOS can easily apply the same knowledge on Windows or Linux, provided they are familiar with the basics of those operating systems.

My father agrees with this decision. For his future books — and when updating existing titles — he also plans to use macOS as the primary platform.

This reflects a broader recommendation for those who are starting their test automation journey: if you are open for suggestions, I recommend learning on macOS.

For the general public, there is now little reason to choose a Windows PC over a Mac, considering factors such as price, build quality, design, support, longevity, and overall user experience.

On LinkedIn, when software engineers announce that they have started a new job, they often share photos of their welcome packages. From what I’ve seen, the included computer is almost always a MacBook Pro (but maybe if it is not a MacBook, they just don’t post it… 😅)

At my current company, Amazon, software engineers are offered a choice between Windows and Apple laptops. Almost all choose a MacBook Pro.

For test automation engineers — also known as Software Development Engineers in Test (SDETs) — there is an additional reason to choose a Mac: a Mac can be used to test both macOS and Windows applications by running Windows in a virtual machine (such as VMware Fusion, which now is free). However, the reverse is not possible.

In the next article, I will share the specific reasons why macOS is better (than Windows) for E2E test automation. Stay tuned.

In this series:

• Part-1: Syntax
• Part-2: Language Features
• Part-3: My Preference

I have written books on test automation using five different programming languages:

• Web Test Automation in Action: Volume 1: Hands-On Exercises with Selenium WebDriver and Playwright (Ruby and TypeScript)
• Effective Web Test Automation with Selenium WebDriver and JUnit (Java)
• Selenium WebDriver Recipes in C#: Practical Testing Solutions for Selenium WebDriver (C#)
• Practical Web Test Automation with Playwright and pytest (Python)
• Practical Mobile Test Automation (Appium in Ruby) — upcoming

Some readers might wonder:

“Which programming language is the best for E2E test automation?”

Programming language preference is one of the most sensitive topics in software engineering. Every language has its passionate supporters.

My answer is simple: for end-to-end test automation, I prefer Ruby. This doesn’t mean I can’t or won’t use other languages when needed.

A quick summary of the five objective comparisons covered in Parts 1 and 2:

• Ruby is easier to create more readable DSL. Ruby👍
• Ruby keeps your project folder cleaner 👍 , it doesn’t generate an annoying __pycache__ folder 👎🏽
• Calling methods without parentheses: Python 👎🏽; Ruby👍
• Ruby modules are elegant. Ruby👍
• Python’s indentation rules are strict. Python 👎🏽; Ruby👍

Then, some maybe subjective ones.

6. RSpec is more expressive than pytest.

The language itself is only part of the story.

For test automation, the testing framework matters just as much.

Ruby’s RSpec produces extremely more readable test scripts than pytest (the de facto test syntax framework in Python). For differences, check my other article, “Test Syntax Framework Comparison in E2E Test Automation: RSpec vs. Pytest”

7. Ruby Has a Better History and Ecosystem for automation.

There are also several other reasons to consider Ruby:

• RubyGems is a better package management ecosystem than pip.
• Ruby has Rake, a task management and build automation tool. The configuration file (Rakefile) is actually a Ruby script. You can configure Rake targets, that are useful for executing tests in CI/CD.
• Some important advances in web test automation originated in Ruby, such as Watir and Cucumber.
• Ruby programmers tend to love Ruby, not merely like it. This is evidenced by creators intentionally putting Ruby in the framework name, such as Watir (Web application testing in Ruby) and Ruby on Rails. As such, there is a lot of support for Ruby online..

8. TestWise IDE supports Ruby Better

Finally, a personal and subjective one: my favourite Testing IDE TestWise (disclaimer: created by my father) is really a huge productivity booster, and while it supports both Python and Ruby (and JavaScript too), its Ruby support is better.

Related reading:

• My books:
  - Web Test Automation in Action: Volume 1
  - Effective Web Test Automation with Selenium WebDriver and JUnit
  - Announcing my new book, “Practical Web Test Automation with Playwright and Pytest”
• My “Discover Ruby Programming Through Fun Examples” interactive course on Educative

This is a continuation from Part 1 of this series, comparing Ruby and Python for E2E test scripting. In Part 1, we looked at the syntax differences, between Ruby and Python. In this article, we’ll compare some of the language features in the context of E2E test scripts.

3. Ruby does not clutter the project with __pycache__

Python commonly creates __pycache__ directories containing compiled bytecode files. These are harmless and normally excluded from source control, but they add some noise to a project.

Ruby does not create an equivalent __pycache__ directory when you run a test script. A Ruby test project therefore tends to remain relatively clean.

4. Ruby has powerful language features such as blocks and mixins

Ruby has several language features that are particularly useful for building maintainable test automation frameworks.

Blocks allow you to pass a piece of behavior to a method. They are used extensively throughout Ruby and make it possible to create concise and expressive APIs. For example, my father’s invention of try_for is using this feature.

Mixins, implemented through modules, provide a convenient way to share behaviours between classes without relying solely on traditional inheritance.

Below is an example Mixin usage in a test script to take screenshots:

module ScreenshotMixin
  def take_screenshot
    puts "Taking screenshot..."
  end
end

class LoginTest
  include ScreenshotMixin
end
test = LoginTest.new
test.take_screenshot

Adding reusable behaviour is as simple as: include ScreenshotMixin .

Python achieves similar functionality using multiple inheritance.

These features can be particularly useful when building Page Object Models and reusable test helpers. Instead of repeatedly copying code, common behaviour can be encapsulated and composed where it is needed.

The result can be test code that is both concise and expressive without sacrificing maintainability.

5. Calling methods without parentheses: Python 👎; Ruby 👍

Python requires parentheses when calling a method, even if it has no parameters.

page.locator("#testwise").click()  # works

If you accidentally omit them:

page.locator("#testwise").click

Python does not report a syntax error. Instead, it simply returns a method object, meaning the click is never performed.

For beginners to Python, this can be confusing because the script runs without an obvious indication that the intended action never happened.

Ruby is much more forgiving. Both of the following are equivalent:

driver.find_element(:link_text, "TestWise").click
driver.find_element(:link_text, "TestWise").click()

Related reading:

• My books:
  - Web Test Automation in Action: Volume 1
  - Effective Web Test Automation with Selenium WebDriver and JUnit
• My “Discover Ruby Programming Through Fun Examples” interactive course on Educative