<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jenny Keeper on Medium]]></title>
        <description><![CDATA[Stories by Jenny Keeper on Medium]]></description>
        <link>https://medium.com/@jennykeeper?source=rss-53b25a1355b7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Jenny Keeper on Medium</title>
            <link>https://medium.com/@jennykeeper?source=rss-53b25a1355b7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:01:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jennykeeper/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cloud Testing Tools for a More Complete Browser and Device Test Matrix]]></title>
            <link>https://jennykeeper.medium.com/cloud-testing-tools-for-a-more-complete-browser-and-device-test-matrix-354f8ba3094b?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/354f8ba3094b</guid>
            <category><![CDATA[testing-matrix]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[cloud-testing-tools]]></category>
            <category><![CDATA[browser-testing]]></category>
            <category><![CDATA[cloud-testing]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 09:30:38 GMT</pubDate>
            <atom:updated>2026-11-18T09:30:38.292Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*X_VqIVKF8ml28eYs" /><figcaption>Photo by <a href="https://unsplash.com/@growtika?utm_source=medium&amp;utm_medium=referral">Growtika</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Sometimes, the biggest gap in a browser testing strategy is the environments that are untested. A regression suite may pass across Chrome and Firefox on desktop, yet users may encounter issues in Safari on iOS. Adding more test cases won’t solve this problem. Teams need to validate applications across the browsers, operating systems, and devices their customers actually use.</p><p>To do so, you would need constant access to browser-OS and device matrix, without risking considerable infrastructure overhead.</p><p>Cloud testing platforms address this challenge by providing remote access to a wide range of browsers, operating systems, and real or virtual devices without the overhead of maintaining local testing infrastructure.</p><p>In this guide, I look at five cloud testing platforms and how their capabilities, browser and device coverage, and integrations address different testing requirements.</p><h3>Why Use Cloud Testing Tool for Cross-Browser Testing?</h3><p>A cloud-based testing tool <strong>reduces the need to maintain local testing infrastructure</strong> and makes it easier to test across a wider range of environments.</p><p>Cloud testing offers several advantages for cross-browser testing, such as:</p><ul><li><strong>Wider browser coverage:</strong> Test across older and latest browser versions, operating systems (macOS, Windows, Linux, iOS, and Android), screen sizes, and mobile viewports.</li><li><strong>Faster test execution:</strong> Run automated test suites across multiple environments simultaneously using parallel execution, reducing overall testing time.</li><li><strong>Faster debugging:</strong> Use video recordings, network HAR files, DOM snapshots, console logs, and device logs to investigate test failures.</li><li><strong>Lower infrastructure overhead:</strong> Reduce the need for physical hardware procurement, device maintenance, and operating system updates.</li><li><strong>CI/CD integration:</strong> Trigger cross-browser regression suites through Jenkins, GitHub Actions, and GitLab CI after code changes to catch issues before deployment.</li></ul><h3>How I Evaluated The Top 5 Cloud Testing Tools</h3><p>I assessed the five tools against five criteria- <strong>browser and device coverage, testing capabilities, performance and reliability, integrations,</strong> and <strong>pricing</strong>. Each criterion addresses a practical question about whether a tool fits a team’s cross-browser and cross-device testing requirements.</p><ul><li><strong>Browser and device coverage:</strong> Which browser versions, operating systems, device models, and viewport configurations does the product support?</li><li><strong>Testing capabilities:</strong> Which manual and automated testing workflows, frameworks, and debugging features does the platform support?</li><li><strong>Performance and reliability:</strong> What execution capacity, concurrency limits, and reliability information does the vendor document?</li><li><strong>Integrations:</strong> How does the tool connect to CI/CD pipelines, issue trackers, test management systems, and reporting workflows?</li><li><strong>Pricing:</strong> What do subscriptions, execution minutes, parallel sessions, add-ons, and separate licenses contribute to the overall cost?</li></ul><p>Since these tools serve different purposes, I assessed each against its core use case rather than comparing every feature on equal terms. This approach helps identify which tool best fits your application stack, testing needs, and release process.</p><h3>5 Most Popular Cloud Testing Tools in 2026</h3><p>Using the five evaluation criteria outlined above, I assessed each platform based on its browser and device coverage, testing capabilities, performance and reliability, integrations, and pricing.</p><p>The following breakdown examines how each tool addresses these areas and where it fits for your testing requirements:</p><h4>1. BrowserStack Automate</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*se9K6aigogXnAAWv9xaH1A.png" /></figure><p>BrowserStack Automate is a cloud-based browser automation platform that allows teams to run automated tests across different desktop browsers, operating systems, and real mobile browsers. It supports popular automation frameworks and provides parallel test execution, debugging tools, and CI/CD integrations for cross-browser testing.</p><p><strong>=&gt; Device and Browser Coverage:</strong> Provides access to 30,000+ real devices and 3500+ browser-OS combinations across different operating systems.</p><p><strong>=&gt; Testing Capabilities:</strong></p><ul><li><strong>Framework Support:</strong> Run automated tests using Selenium, Playwright, Cypress, and Puppeteer.</li><li><strong>Parallel Test Execution:</strong> Run multiple automated tests simultaneously across different browser and OS combinations.</li><li><strong>Local Testing:</strong> Test locally hosted and staging applications through a secure tunnel.</li><li><strong>Debugging Tools:</strong> Access screenshots, video recordings, browser console logs, network logs, and execution details to investigate failures.</li></ul><p><strong>=&gt; Performance and Reliability:</strong> Supports parallel test execution, allowing teams to run hundreds of tests concurrently. The available parallel sessions depend on the selected plan.</p><p><strong>=&gt; Integrations:</strong> Supports CI/CD integrations with tools such as Jenkins and GitHub Actions, along with integrations for popular automation frameworks.</p><p><strong>=&gt; Pricing:</strong> The Desktop plan starts at<strong> $99/month</strong>, billed annually. The Desktop &amp; Mobile plan starts at <strong>$175/month</strong>, billed annually.</p><h4>2. Browserling</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MgtFs4VwhGqmlYmdNivgvg.png" /></figure><p>Browserling is a cloud-based cross-browser testing platform that gives teams remote access to different browsers and operating systems without installing them locally or maintaining virtual machines. It supports interactive browser testing, responsive layout inspection, and testing of locally hosted websites through secure SSH tunnels.</p><p><strong>=&gt; Device and Browser Coverage:</strong> Provides access to real browsers running on remote machines, including Chrome, Firefox, Safari, Edge, Opera, and Internet Explorer across supported operating systems. Available environments vary by subscription plan.</p><p><strong>=&gt; Testing Capabilities:</strong></p><ul><li><strong>Live Browser Testing:</strong> Interact with remote browsers to test website functionality and identify browser-specific issues.</li><li><strong>Responsive Testing:</strong> Resize browser windows and test websites across different screen resolutions and viewports.</li><li><strong>Screenshot Capture:</strong> Capture, annotate, and share screenshots to document rendering issues.</li><li><strong>Local Testing:</strong> Access locally hosted and staging websites through secure SSH tunnels.</li><li><strong>Browser Testing API:</strong> Launch remote browser sessions and integrate browser access into custom testing workflows.</li></ul><p><strong>=&gt; Performance and Reliability:</strong> Browserling states that remote browser sessions launch within seconds. However, independent performance benchmarks and public uptime figures are not available.</p><p><strong>=&gt; Integrations:</strong> Provides a browser testing API and browser extensions to support testing workflows.</p><p><strong>=&gt; Pricing:</strong> Individual plans start at <strong>$9/month</strong> for the Starter plan and <strong>$19/month</strong> for the Developer plan, billed annually. Developer Pro starts at<strong> $29/month </strong>and adds macOS, iOS, and native Safari testing.</p><h4>3. AWS Device Farm</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KAT9Lr2oSHdlKLQzjCVTtw.png" /></figure><p>AWS Device Farm provides cloud infrastructure for testing mobile applications on real devices and web applications across desktop browsers. It allows teams to execute automated test suites, access devices remotely, and evaluate application behaviour under different device and network conditions.</p><p><strong>=&gt; Device and Browser Coverage:</strong> Offers access to thousands of real iOS and Android devices. Desktop browser testing supports Chrome, Firefox, and Microsoft Edge on Windows.</p><p><strong>=&gt; Testing Capabilities:</strong></p><ul><li><strong>Real-Device Automation:</strong> Execute mobile tests using Appium, Espresso-based instrumentation tests, and XCTest for iOS applications.</li><li><strong>Remote Streaming:</strong> Access physical mobile devices through a browser to manually reproduce issues and inspect application behavior.</li><li><strong>Performance Profiling:</strong> Collect application performance data, including CPU, memory, and frame-rate metrics, during supported test runs.</li><li><strong>Network Emulation:</strong> Simulate network conditions such as bandwidth limitations, latency, jitter, and packet loss using configurable network profiles.</li><li><strong>Diagnostic Logging:</strong> Generate test videos, screenshots, crash reports, Android logcat outputs, and iOS device logs for failure investigation.</li></ul><p><strong>=&gt; Performance and Reliability:</strong> Executes tests concurrently across available devices and browser instances. Tests may remain queued when the requested physical devices are unavailable, although other available devices in the same run can continue testing.</p><p><strong>=&gt; Integrations:</strong> Supports AWS CodePipeline, Jenkins, Gradle, AWS CLI, and REST APIs for integrating test execution into development workflows.</p><p><strong>=&gt; Pricing:</strong> Includes a one-time trial of 1,000 free device minutes. Metered real-device testing costs $0.17 per device minute, while unmetered device subscriptions start at $250/month per device. Desktop browser testing costs $0.005 per instance minute.</p><h4>4. TestingBot</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t67dEm0itM7WD-fSOAErnQ.png" /></figure><p>TestingBot is a cloud-based testing platform for running automated tests across desktop browsers and physical mobile devices. It supports Selenium, Playwright, Cypress, and Puppeteer, with additional capabilities for visual regression and location-based testing.</p><p><strong>=&gt; Device and Browser Coverage:</strong> Offers access to 6,100+ browser environments and devices, including Chrome, Firefox, Safari, Edge, and Internet Explorer across Windows, macOS, and Linux, alongside physical iOS and Android devices.</p><p><strong>=&gt; Testing Capabilities:</strong></p><ul><li><strong>Visual Regression Testing:</strong> Compare screenshots against baseline images to detect unintended UI changes across supported browsers and devices.</li><li><strong>Geolocation Testing:</strong> Run tests from more than 15 geographic locations to verify localized content, regional pricing, and location-dependent application behavior.</li><li><strong>Secure Local Tunneling:</strong> Connect remote browsers and devices to locally hosted or staging applications through an encrypted tunnel.</li></ul><p><strong>=&gt; Performance and Reliability:</strong> Publishes a 99.9% uptime SLA for its cloud testing platform.</p><p><strong>=&gt; Integrations:</strong> Supports CI/CD integrations with GitHub Actions, Jenkins, CircleCI, GitLab CI, Bitbucket Pipelines, Azure DevOps, Travis CI, and TeamCity. Slack integration is also available for test notifications.</p><p><strong>=&gt; Pricing:</strong> Live plans start at<strong> $20/month</strong>, Automated at <strong>$50/month</strong> (1,000 minutes), and Automated Pro at <strong>$90/month</strong> (unlimited minutes), billed annually.</p><h4>5. TestGrid</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rnDgt61gcprENONC77ZG4A.png" /></figure><p>TestGrid is a cloud-based testing platform that combines browser and real-device infrastructure with automated testing capabilities. It supports existing test suites across web and mobile environments, with options for dedicated cloud, private cloud, on-premises, and hybrid deployments.</p><p><strong>=&gt; Device and Browser Coverage:</strong> Provides access to 1,000+ real device and browser combinations across iOS, Android, Windows, macOS, and Linux environments. Supported browsers include Chrome, Firefox, Safari, Edge, Opera, and Samsung Internet.</p><p><strong>=&gt; Testing Capabilities:</strong></p><ul><li><strong>Framework Support:</strong> Run existing Selenium, Playwright, Cypress, Appium, Espresso, and XCUITest test suites without converting them into a proprietary format.</li><li><strong>Dedicated Device Infrastructure:</strong> Reserve specific devices for testing or connect organization-owned devices to a private testing environment.</li><li><strong>Private Network Testing:</strong> Use secure tunnels and VPN connectivity to test applications hosted behind corporate firewalls.</li><li><strong>Flexible Deployment:</strong> Choose between hosted cloud, private cloud, on-premises, and hybrid deployments based on infrastructure and security requirements.</li></ul><p><strong>=&gt; Performance and Reliability:</strong> Supports parallel test execution across multiple browsers and devices. Public uptime figures and independent execution benchmarks are not readily available.</p><p><strong>=&gt; Integrations:</strong> Supports CI/CD integrations with Jenkins, GitHub Actions, GitLab CI, Azure DevOps, and CircleCI, along with API-based integrations for custom workflows.</p><p><strong>=&gt; Pricing:</strong> The Starter plan costs <strong>$249 per seat/month</strong>. Growth and custom device lab deployments have<strong> custom pricing</strong>.</p><h3>What Should You Assess Before Choosing a Cloud Testing Platform?</h3><p>Before choosing a platform, assess how well it fits your testing requirements, existing workflows, and budget. Ask these questions first:</p><ol><li><strong>Does it cover your target environments?</strong> Check support for the browser versions, operating systems, and devices your customers use.</li><li><strong>Does it support your existing test frameworks?</strong> Verify compatibility with your automation frameworks and whether existing test scripts can run without significant changes.</li><li><strong>Can it handle your execution requirements?</strong> Check parallel session limits, usage quotas, and session startup times against your regression suite and release frequency.</li><li><strong>Does it provide sufficient debugging information?</strong> Look for screenshots, videos, browser logs, and network traces that help developers investigate failures.</li><li><strong>Does it integrate with your existing workflows?</strong> Verify compatibility with your CI/CD pipelines, test frameworks, and issue-tracking systems.</li><li><strong>Does the pricing fit your budget at scale?</strong> Account for subscription fees, testing minutes, parallel sessions, add-ons, and licensing costs.</li></ol><p>Before committing, run a representative test suite on shortlisted platforms. Compare execution results, debugging workflows, and estimated costs at your expected testing volume.</p><h3>What are the Most Common Challenges of Cross-Browser Testing?</h3><p>Cross-browser testing introduces challenges related to stability and coverage. The most common ones include:</p><ul><li><strong>Browser rendering differences:</strong> Layouts, font rendering, CSS support, and browser APIs vary across Chromium, WebKit, and Gecko. Reproduce issues on the affected browser and version before changing the implementation.</li><li><strong>An oversized test matrix:</strong> Testing every feature across every browser and device combination increases execution time and cost. Instead, prioritize critical user journeys and environments based on customer usage, supported configurations, and defect history.</li><li><strong>Flaky automation:</strong> Unstable selectors, fixed waits, animations, and shared test data can cause inconsistent results. Therefore use stable locators, condition-based waits, and isolated test data.</li><li><strong>Emulation limitations:</strong> Simulated viewports cannot fully reproduce physical sensors, biometric authentication, camera behavior, or every touch interaction. To avoid this, validate hardware-dependent features on real devices.</li><li><strong>Insufficient debugging evidence:</strong> A failed assertion may not reveal the underlying cause. Instead capture screenshots, browser console output, network traces, execution logs, and video recordings where supported.</li><li><strong>Concurrency and cost limits:</strong> Parallel execution reduces test time, but available concurrency depends on the subscription plan. So, tests must also run independently to avoid conflicts over shared state.</li></ul><p>A sensible approach is to start with a small set of high-value environments, stabilize the test suite, and then expand coverage based on customer usage and observed defects.</p><h3>Conclusion</h3><p><em>The best test matrix reflects the failures customers may encounter. Hundreds of environments add little value if tests miss critical workflows, rely on unstable selectors, or skip physical devices.</em></p><p><em>I would start by identifying gaps in local testing. Use responsive inspection for layout issues, browser automation for repeatable workflows, and physical devices for hardware-dependent behavior. Add test management when case organization and result tracking become difficult.</em></p><p><em>Before choosing a platform, run a representative test journey and review its results. Reproducing and diagnosing failures matters as much as environmental coverage.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=354f8ba3094b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parallel Test Execution in Automation: How to Choose the Right Concurrency]]></title>
            <link>https://jennykeeper.medium.com/parallel-test-execution-in-automation-how-to-choose-the-right-concurrency-4c3a816f091e?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c3a816f091e</guid>
            <category><![CDATA[parallel-test-execution]]></category>
            <category><![CDATA[selenium-grid]]></category>
            <category><![CDATA[cloud-testing]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 05:01:03 GMT</pubDate>
            <atom:updated>2026-11-12T05:01:03.730Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F4adJjaYK5nQvQb4" /><figcaption>Photo by <a href="https://unsplash.com/@haky?utm_source=medium&amp;utm_medium=referral">Shamin Haky</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Imagine a busy supermarket. When the queue gets long, the manager opens more checkout counters, and the line moves faster. Right? But if every counter shares one card machine, or one shopper has a trolley piled high, adding counters stops helping the queue.</p><p>Automation parallel test execution works the same way. More parallel sessions speed up your suite at first. Then your slowest test, your setup time and your shared test data start holding everything back, and some tests even fail.</p><p>In this article, I’ll show you where concurrency is set, how to prepare your tests, and how I find the right number of sessions with a small experiment.</p><h3>What is Parallel Test Execution in Automation?</h3><p>Automation parallel test execution means running several automated tests at the same time, each in its own browser or device session. Without it, tests run one after another, and every new test adds to the total time.</p><p>Here’s the simple maths. If you have 200 tests that take one minute each, one session needs about 200 minutes. With ten sessions, you’d expect about 20 minutes.</p><p>Real suites don’t divide that neatly. A run can’t finish faster than its slowest test, and every session spends time starting a browser. So running automated tests in parallel gives big savings at first and smaller savings after that.</p><h3>Where Do You Set Concurrency?</h3><p>Concurrency is set in four places. The lowest limit of the four decides how many tests really run at once.</p><ul><li><strong>Your test runner:</strong> This sets how many tests run side by side on one machine. In Playwright, it’s the workers setting, which defaults to half your CPU cores. In TestNG, it’s parallel and thread-count in testng.xml, and in pytest, it’s the -n option from pytest-xdist.</li><li><strong>Your CI pipeline:</strong> This sets how many machines run at once. Playwright splits a suite across machines with — shard=1/4. Cypress needs Cypress Cloud and the — record — parallel flags to do the same.</li><li><strong>Your grid or cloud:</strong> This sets how many browsers can open at once. On a self-hosted Selenium Grid, each node’s session limit decides it. On a cloud service, your plan’s parallel limit decides it.</li><li><strong>Your test environment:</strong> Your staging database, API rate limits and shared test accounts also cap how much load your tests can create.</li></ul><p>Say you set Playwright to 8 workers on each of 4 shards, which is 32 tests at a time. If your grid only has room for 10 sessions, 22 tests wait in a queue, and your timing numbers stop meaning anything.</p><p>So before you change your automation parallel test execution setup, write down the limit at each layer. Then keep your runner and CI numbers at or below your grid’s limit.</p><h3>Prepare Automated Tests for Parallel Execution</h3><p>Automation parallel test execution exposes every hidden link between your tests. A suite that passes one test at a time can fail randomly once tests share data, accounts or browsers.</p><p>The Selenium docs put the rule simply: tests must be isolated from one another. Here’s what I check before running automated tests concurrently.</p><ul><li><strong>One browser session per test:</strong> Create a new WebDriver for each test. In TestNG, keep each thread’s driver in a ThreadLocal variable, because WebDriver isn’t thread-safe.</li><li><strong>Data and accounts each test owns:</strong> Have each test create its own data with a unique ID. In Playwright, you can use each worker’s parallelIndex to log it in as its own user, like user-0 and user-1.</li><li><strong>No order dependencies:</strong> If test B only passes after test A, they’ll break in parallel. Playwright’s — shuffle flag runs tests in random order, so these links show up early.</li></ul><p>I run the suite with shuffle turned on a few times before I touch concurrency. If a test fails only in that mode, I fix it first, or every later result will be noisy.</p><h3>Set Session Limits on Your Grid or Cloud</h3><p>Your runner decides how many tests try to start. Your grid or cloud decides how many browsers can actually open, so it sets the ceiling for automation parallel test execution.</p><h4>On a Self-Hosted Selenium Grid</h4><p>With your own Grid, you set the limits yourself.</p><ul><li><strong>Sessions per node:</strong> The — max-sessions flag sets how many browsers one node runs at once. The Selenium docs say the default is the number of CPU cores. You can go higher with — override-max-sessions, but the docs warn that sessions may become unstable.</li><li><strong>Docker nodes:</strong> The official images use SE_NODE_MAX_SESSIONS and SE_NODE_OVERRIDE_MAX_SESSIONS. Each container runs one session by default, so raise that number or start more containers.</li><li><strong>The queue:</strong> When every slot is busy, new requests wait. The — session-request-timeout flag sets how many seconds they wait before timing out.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A1PQXot9V61Y39HJicO7oQ.png" /></figure><p>Here’s a Chrome node that allows four sessions:</p><pre>docker run -d -p 4444:4444 - shm-size=&quot;2g&quot; \<br>-e SE_NODE_MAX_SESSIONS=4 \<br>-e SE_NODE_OVERRIDE_MAX_SESSIONS=true \<br>selenium/standalone-chrome</pre><p>To see how full your Grid is, query its GraphQL endpoint for maxSession, sessionCount and sessionQueueSize. The trade-off is upkeep, because you own the machines, the browser updates and the memory problems.</p><h4>On a Cloud Testing Service</h4><p>Services such as BrowserStack, Sauce Labs and LambdaTest host the browsers and devices for you. Your plan sets how many sessions run at once, and extra tests wait in a queue.</p><p>Before tuning, I look up three things in the provider’s docs. These are the parallel limit, how many tests can queue, and how long a queued test waits before it’s dropped. The trade-off here is cost, because every extra parallel adds to the bill.</p><h3>Find the Right Concurrency With a Controlled Experiment</h3><p>Most teams pick a parallel count once and never check it again. I treat the count in automation parallel test execution like any other setting and measure it.</p><ol><li><strong>Fix everything else:</strong> Use the same commit, environment and tests for every run. Turn retries off, so a flaky pass doesn’t hide a failure.</li><li><strong>Pick a few levels:</strong> I usually test 1, 2, 4, 8 and 16 sessions, or stop at my grid or plan limit.</li><li><strong>Run each level three times:</strong> One run can be unlucky, and three show how much the time moves.</li><li><strong>Record four numbers:</strong> Write down the total time, the slowest test, the failures and any time spent in a queue.</li><li><strong>Find the bend:</strong> Look for the level where doubling the sessions saves little time, or where failures start to climb.</li></ol><p>For a starting point, TestingBot suggests dividing your total suite time by the time you’re willing to wait. A 200-minute suite with a 20-minute target needs about 10 parallels.</p><p>The experiment then tells you if that holds. If 8 sessions finish in 27 minutes and 16 finish in 24, the extra 8 bought only 3 minutes. That usually means one slow test or a shared resource is setting the floor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*U2AjYxpZqNgTI3NErh900A.png" /></figure><h3>Manage Parallel Automation Across Builds and Teams</h3><p>Once several teams share one pool of sessions, automation parallel test execution becomes a sharing problem too. Good parallel automation test management comes down to three habits.</p><ul><li><strong>Split the pool on purpose:</strong> Give a busy team its own Grid nodes, or use a cloud plan that lets admins assign sessions per sub-team. Then one team’s nightly run can’t block another team’s pull request checks.</li><li><strong>Cancel stale builds:</strong> If a developer pushes three commits in a row, only the last one matters. GitHub Actions’ concurrency setting with cancel-in-progress stops the older runs and frees their sessions.</li><li><strong>Give fast checks priority:</strong> Run a short smoke suite on every pull request, and save the full regression for merges or a nightly slot.</li></ul><p>Every parallel automation testing tool shows some view of usage. I check it weekly to see whether teams are waiting on each other.</p><h3>Troubleshoot Slow or Unreliable Parallel Runs</h3><p>When automation parallel test execution goes wrong, the run is usually slow, flaky or stuck in a queue. Here’s what I check.</p><ul><li><strong>The run barely got faster:</strong> Look at your slowest test first. If one test takes 12 minutes, no number of parallels will finish the suite in 5. With Playwright sharding, turn on fullyParallel so shards split the suite by individual test.</li><li><strong>Tests fail only in parallel:</strong> This almost always means shared state, like shared users, records or a static driver. Run with — shuffle to reproduce it.</li><li><strong>Failures rise as you add sessions:</strong> Your staging database, a rate-limited API or an overloaded Grid node may be the bottleneck. Lower the concurrency or add resources.</li><li><strong>Tests sit in the queue or get dropped:</strong> Multiply workers by shards by browsers, and compare the total with your grid or plan limit. You’re likely asking for more sessions than it can hold.</li></ul><p>Playwright’s — max-failures option also helps. It stops a run after a set number of failures, so a broken build doesn’t waste every session.</p><h3>So, How Many Sessions Would I Run?</h3><p>In automation parallel test execution, the number that matters is how long your team waits for a result. The parallel count is just one way to change it.</p><p>Just like checkout counters, extra sessions only help until something else becomes the bottleneck. Whether you run your own Grid or pay for a cloud, I’d start with isolated tests, measure a few levels and stop where the curve flattens.</p><p>Running tests in parallel for automation pays off most when you know what each extra session buys you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c3a816f091e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real Browsers vs Emulators: 4 Ways to Check What Your Tests Run]]></title>
            <link>https://jennykeeper.medium.com/real-browsers-vs-emulators-4-ways-to-check-what-your-tests-run-fbbb67356a0b?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/fbbb67356a0b</guid>
            <category><![CDATA[real-browser-testing]]></category>
            <category><![CDATA[automated-testing]]></category>
            <category><![CDATA[headless-chromium]]></category>
            <category><![CDATA[emulator]]></category>
            <category><![CDATA[test-run]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 08:44:59 GMT</pubDate>
            <atom:updated>2026-11-10T08:44:59.081Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7iHgjURnfq4z9jtb" /><figcaption>Photo by <a href="https://unsplash.com/@glencarrie?utm_source=medium&amp;utm_medium=referral">Glen Carrie</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>About 60% of web traffic now comes from phones, according to StatCounter’s October 2026 figures. Yet a lot of test suites run on a Linux server in CI, inside a browser nobody has looked at in months.</p><p>That gap matters. Your config might say “iPhone” or “Chrome”, while your tests really run in headless Chromium with a phone-sized window. Everything passes, and a bug still reaches real users.</p><p>Don’t worry, it’s easy to check. Here are four quick ways to see what your automated real browser testing really uses.</p><h3>Difference Between Real Browsers and Browser-Like Test Environments</h3><p>In automated real browser testing, most test environments look like a real browser from the outside. They load your page, click buttons and pass tests. But each one copies only part of what a real user’s browser does.</p><p>Here are the four you’re most likely to meet, from the least real to the most real.</p><h4>Emulators Only Change the Screen Size</h4><p>Browser emulation is what you get with Chrome’s Device Mode, Playwright’s devices[‘iPhone 15’] presets or Chrome’s mobileEmulation option in Selenium. It changes the window size, the pixel density, the user agent string and touch events.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9W0ubnRgtQhQpCWW4fODRQ.png" /></figure><p>That’s useful for checking layouts, but it isn’t automated real browser testing. But Chrome’s own docs call Device Mode a “first-order approximation” and say your code doesn’t actually run on a mobile device.</p><p>Microsoft’s Edge docs go further. They say browser emulators don’t copy differences in web API or CSS support between browsers. So an “iPhone” preset running in Chrome is still Chrome’s engine underneath, with Chrome’s features and a desktop CPU.</p><h4>Headless Is Not an Emulator, But There Are Two Kinds</h4><p>Headless means the browser runs without a visible window. That alone doesn’t make it fake, but Chrome has two very different headless modes.</p><p>The new headless mode is the real Chrome browser running without a window. Chrome’s docs describe it as more authentic and reliable, and better for end-to-end testing.</p><p>The old headless mode now ships as a separate, lighter program called chrome-headless-shell. Chrome’s docs say it suits screenshots and web scraping. Since Chrome 132, the old mode is only available as that standalone program.</p><p>Here’s the catch. Playwright’s docs say it uses the headless shell for headless runs by default. You opt into the new headless mode by setting the chromium channel.</p><h4>Chromium Is Not Chrome</h4><p>Chromium is the open-source project that Chrome is built on. Google adds extra parts to Chrome, including licensed media codecs and DRM support for protected video.</p><p>Playwright’s docs say plain Chromium “does not have all the codecs” that Chrome and Edge bundle, because of licensing. Since version 1.57, Playwright downloads Chrome for Testing builds, which add those codecs back. But testing builds still aren’t the exact Chrome your users install, and some Docker images and older setups still use plain Chromium.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fdzFn-JIjOREZeDHd3Ky3w.png" /></figure><p>The same idea applies to Safari. Playwright’s WebKit is built from WebKit’s source code, and its docs say it doesn’t work with the branded Safari app.</p><h4>A Real Browser on a Real Device</h4><p>A real browser on a real device is the version your users actually run. That means the real Chrome or Safari app, on a real phone or computer, with its own CPU, memory, codecs and system settings.</p><p>This is the only setup in automated real browser testing that catches problems tied to the hardware or the full browser. Think of slow scripts on a mid-range phone, a video codec that’s missing, or a keyboard covering a form field. That’s why I treat automated testing on real browsers and real devices as the final check before a release.</p><h3>4 Ways to Check What Your Tests Run On</h3><p>You don’t need special tools to audit your automated real browser testing setup. These four checks take a few minutes each, and you can run them in any framework.</p><p>I run them in this order, because each one rules something out before the next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EKQ3oQC1ITFnZqTEJqlD1w.png" /></figure><h4>Check 1, Look at the Browser in Your CI Config</h4><p>Start with your config files. They often tell you most of the story before you run anything.</p><ul><li><strong>Device presets:</strong> In Playwright, a line like …devices[‘iPhone 15’] means browser emulation. That preset runs in WebKit on your CI machine, which isn’t the Safari app on a real iPhone.</li><li><strong>Browser channel:</strong> In Playwright, no channel setting means Playwright’s own browser build. A setting like channel: ‘chrome’ or channel: ‘msedge’ means the branded browser installed on the machine.</li><li><strong>Headless flags:</strong> Look for headless: true, — headless or — headless=new in your launch options. Then check which headless mode you’re getting, using the section above.</li><li><strong>Mobile emulation in Selenium:</strong> A mobileEmulation entry in ChromeOptions means Chrome is pretending to be a phone.</li><li><strong>Docker images and runners:</strong> Check the image your CI job uses. A Playwright or Selenium image comes with its own browser builds, which may not match what your users run.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sBQ_SzOAAoSxhtdEPcygHA.png" /></figure><h4>Check 2, Print the User Agent From a Running Test</h4><p>Config files can lie, or be overridden somewhere else. So next, I ask the browser directly from inside a running test.</p><pre>const info = await page.evaluate(() =&gt; ({<br>  userAgent: navigator.userAgent,<br>  platform: navigator.platform,<br>  brands: navigator.userAgentData?.brands,<br>  cores: navigator.hardwareConcurrency,<br>}));<br>console.log(info);</pre><p>Here’s what to look for in the output.</p><ul><li><strong>“HeadlessChrome” in the user agent:</strong> This shows the browser runs headless.</li><li><strong>An iPhone user agent with a Linux or Mac platform:</strong> This combination means emulation. A real iPhone doesn’t report a Linux platform.</li><li><strong>The brands list:</strong> Branded Chrome lists “Google Chrome” in navigator.userAgentData.brands. Plain Chromium lists only “Chromium”.</li><li><strong>The CPU core count:</strong> A large number like 16 or 32 points to a CI server. Phones usually report far fewer cores.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cVMqo8xwIp_6lg5vxuzVDw.png" /></figure><h4>Check 3, Play a Video That Needs a Licensed Codec</h4><p>Media support is where plain Chromium and real Chrome differ most clearly in automated real browser testing. This check takes one line of code.</p><pre>const h264 = await page.evaluate(() =&gt;<br>  document.createElement(&#39;video&#39;).canPlayType(&#39;video/mp4; codecs=&quot;avc1.42E01E&quot;&#39;)<br>);<br>console.log(h264); // &quot;probably&quot; or &quot;maybe&quot; means supported, &quot;&quot; means not</pre><p>An empty result for H.264 means you’re on a plain Chromium build without licensed codecs. Real Chrome, Edge and Safari return a supported answer.</p><p>For a stricter check, try DRM. Call navigator.requestMediaKeySystemAccess(‘com.widevine.alpha’, …) with a basic config. Branded Chrome supports Widevine, which streaming sites use for protected video. Testing builds such as Chrome for Testing don’t include it.</p><p>If your app plays video or audio, this check matters a lot. A media bug can pass every test in Chromium and still break for real users.</p><h4>Check 4, Compare a Speed Test in CI With the Same Test on a Phone</h4><p>The last check is about hardware. Emulation can make a page look like a phone, but it can’t give it a phone’s CPU.</p><p>Pick one page and one simple measure, like the time until your main content appears. Run it in CI, and then run the same test on a real mid-range phone.</p><p>If CI is much faster, your tests don’t feel what a phone feels. Chrome’s docs point out that mobile CPUs work very differently from laptop and desktop CPUs. CPU throttling in DevTools helps a little. It slows a desktop chip down, but the chip still works differently from a phone’s.</p><p><em>Screenshot: Add a capture of a performance trace from CI next to the same trace from a real Android phone.</em></p><h3>What Your Results Mean?</h3><p>Once you’ve run the four checks, put the answers together to see how real your automated real browser testing is. Here’s how I read them.</p><ul><li><strong>Emulation:</strong> A phone user agent with a desktop platform and many CPU cores means your “mobile” tests run on a desktop browser.</li><li><strong>Headless shell:</strong> “HeadlessChrome” plus a default Playwright headless run usually means the lightweight shell.</li><li><strong>Plain Chromium:</strong> Missing H.264 support and no “Google Chrome” brand mean you’re testing the open-source engine without Chrome’s extras.</li><li><strong>Testing build:</strong> H.264 works but Widevine doesn’t, so you’re on a testing build like Chrome for Testing. That’s close to Chrome, with a few gaps.</li><li><strong>Real browser on a real device:</strong> Your brands, codecs and speed match what your users see. This is true automated testing across real browsers.</li></ul><p>None of these results is bad on its own. Emulated and headless runs are fast and cheap, and I still use them for most quick checks. The problem only starts when you think you’re testing one thing and you’re really testing another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r3BkGE-OwZKlFDLVhDgC9g.png" /></figure><h3>Which Browsers You Actually Need to Test?</h3><p>Automated real browser testing can’t cover everything, so start with what your users run. StatCounter’s October 2026 numbers show Chrome at about 69% of browser use worldwide and Safari at about 17%. Edge has about 5%, and Opera, Firefox and Samsung Internet each have around 2%.</p><p>Your own analytics matter more than global numbers, though. Here’s the order I usually follow.</p><ul><li><strong>Your top browsers on real desktop machines:</strong> For most sites, that’s Chrome, Edge and Safari on macOS, plus Firefox if your analytics show it.</li><li><strong>Safari on a real iPhone:</strong> On iPhones, most browsers use Apple’s WebKit engine, so Safari covers a big share of your mobile users. WebKit on Linux doesn’t replace it.</li><li><strong>Chrome and Samsung Internet on real Android phones:</strong> Pick a recent flagship and a mid-range phone, because the mid-range one shows your real speed problems.</li><li><strong>Emulation for everything else:</strong> Use device presets and headless runs for fast layout checks on every pull request.</li></ul><p>This way, browser automation with actual browsers covers the setups your users depend on. Emulation handles the long list of screen sizes.</p><h3>Where to Run Real Browser Tests?</h3><p>You have three main options for automated web testing on real browsers. Each one handles real devices and browsers testing in a different way. It comes down to cost against effort.</p><ul><li><strong>Your own machines:</strong> You can install real Chrome, Edge and Firefox on CI runners and point your tests at them, for example with Playwright’s channel: ‘chrome’. For Safari, you need Mac machines running Apple’s safaridriver.</li><li><strong>Your own device lab:</strong> You can buy phones and connect them through Appium. This gives you full control, but you also handle charging, updates, broken screens and flaky USB connections.</li><li><strong>A real device cloud:</strong> Services like BrowserStack Automate run your existing Selenium, Playwright, Cypress or WebdriverIO tests on real browsers and real devices. You change a config file and keep your test code the same.</li></ul><h4>Playwright</h4><ul><li><strong>Best for:</strong> I use Playwright when I want Chromium, Firefox and WebKit behind one API, while keeping installed Chrome and Edge a channel setting away.</li><li><strong>Limitation:</strong> The default browser version follows Playwright’s pinned browser rather than whatever stable Chrome reached this week.</li><li><strong>My take:</strong> I keep a normal Chromium project for reproducibility and add channel: ‘chrome’ when I specifically want the locally installed Chrome channel.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oPWkfBwD294t6KjW9QjyBw.png" /></figure><h4>Selenium WebDriver</h4><ul><li><strong>Best for:</strong> I use Selenium when I want to drive browsers through their vendor WebDriver implementations, including Safari through Apple’s safaridriver.</li><li><strong>Limitation:</strong> Once I scale past a few machines, I have to manage Grid infrastructure or hand that problem to a hosted service.</li><li><strong>My take:</strong> Safari is where Selenium still gives me a very clean answer. Apple ships safaridriver with the OS, and I can enable browser automation with safaridriver — enable.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ockn00DQ8l1FCJwMzS_m4g.png" /></figure><h4>BrowserStack Automate</h4><ul><li><strong>Best for:</strong> I use it when I need existing Playwright, Selenium or Cypress tests to run across real Safari, branded desktop browsers and physical phones without maintaining the hardware myself.</li><li><strong>Limitation:</strong> A hosted grid still runs the tests I give it. It cannot rescue a weak assertion or a bad synchronization strategy.</li><li><strong>My take:</strong> I value the session metadata, video and logs because they make the environment easier to inspect. The Weather Company says BrowserStack helped it run 20,000 tests in roughly two hours while avoiding more than $50,000 per year in device-farm costs.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MrDEEiERYAWb-Ry5Wl1vIQ.png" /></figure><h4>Cypress</h4><p>Cypress currently supports Chrome, Chrome for Testing, Chromium, Edge, Firefox and experimental WebKit.</p><ul><li><strong>Best for:</strong> I use Cypress when front-end teams want browser-first debugging and command-level visibility during a test.</li><li><strong>Limitation:</strong> The bundled Electron browser is now deprecated, and Cypress still falls back to it when I don’t specify another browser. WebKit support also remains experimental.</li><li><strong>My take:</strong> I make the browser explicit in CI with cypress run — browser chrome or set defaultBrowser: ‘chrome’. That removes one more ambiguous label from the pipeline.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KFYDz4w_if3DGj9-19bR6Q.png" /></figure><h3>Know What You’re Testing Before You Trust It</h3><p>Emulators, headless browsers and Chromium builds all have a place in my pipeline. They’re fast, and they catch most layout and logic bugs early.</p><p>But I don’t call a test “real browser” until the four checks say so. Spend ten minutes on them this week, and then you’ll know exactly what your automated real browser testing covers and what it misses.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbbb67356a0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choosing Between 8 Test Automation Tools for Web Apps]]></title>
            <link>https://jennykeeper.medium.com/choosing-between-8-test-automation-tools-for-web-apps-7fd67173dd4e?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7fd67173dd4e</guid>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[web-apps]]></category>
            <category><![CDATA[cypress]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[test-automation-tools]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Fri, 30 Oct 2026 06:49:35 GMT</pubDate>
            <atom:updated>2026-10-30T06:49:35.619Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xyZSdRhvZvpZLa6H" /><figcaption>Photo by <a href="https://unsplash.com/@karlp?utm_source=medium&amp;utm_medium=referral">Karl Pawlowicz</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Selenium, Playwright, Cypress, WebdriverIO, TestCafe, Robot Framework, Katalon Studio, and BrowserStack Automate, all of these tools can test a web app. But the real question is which one fits your team.</p><p>My answer usually depends on four things:</p><ul><li>Languages your team writes in,</li><li>Browsers mix of the end users,</li><li>How many tests/runs do you need,</li><li>How much you’re willing to spend.</li></ul><p>I’d add a fifth point that’s easy to overlook, which is what a tool shows you when a test fails. Some tools save a screenshot, a video, and logs on their own. Others give you only an error message until you set them up.</p><p>So, I compared all the eight test automation tools on these five points. Each tool gets the same subheadings, so you can compare them side by side.</p><h3>What Test Automation Tools Cover?</h3><p>In basic terms, test automation tools do the clicking/typing for the tests. For a web app, they open a browser and act like a user. They type, click, and then check that the page shows the right result.</p><p>Good automated testing tools in software testing also have a second job. When a test fails, they should leave you some evidence. That evidence can be a screenshot, a video, or a log of what happened.</p><p>This is where web application testing tools differ the most. Some collect a lot of evidence by default. Others collect very little until you set them up.</p><h3>Failure Evidence Compared Across 8 Tools</h3><p>When a test fails, three things help me find the cause quickly. A screenshot shows what the page looked like. A video shows the steps that led there. Logs show what the browser and the network were doing at that moment.</p><p>Based on each tool’s own documentation, these test automation tools fall into three groups.</p><ul><li><strong>Evidence on by default:</strong> Robot Framework creates a step-by-step HTML log and takes a screenshot when a browser step fails. BrowserStack Automate records a video and collects text, Selenium, and console logs for every session. Cypress takes a screenshot automatically when a test fails in CI.</li><li><strong>Evidence one setting away:</strong> Playwright can save traces, screenshots, and videos once you add a few lines to its config file. TestCafe takes failure screenshots after you turn on one option. Katalon Studio has settings for failure screenshots and videos.</li><li><strong>Evidence you build yourself:</strong> Selenium gives you a stack trace, and you add screenshots with your own code. WebdriverIO works the same way, and video comes from a community add-on.</li></ul><p>The pattern is clear. Only a few test automation tools collect useful evidence without any setup. With most of them, you need to turn it on yourself.</p><h3>Test Automation Tools for Web Applications</h3><p>Now let’s look at each of the eight test automation tools more closely. For every tool, I cover the same points, so you can compare them easily.</p><p>To simplify, under key features, I look at languages, browsers, parallel runs, and cost. Under when a test fails, I look at what you get by default and what you should turn on. Also, remember, the order in which the tools are added is not based on a ranking.</p><h3>Selenium</h3><p>Selenium has been around since 2004. It’s still the open-source standard among test automation tools for browsers.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You can write tests in Java, Python, C#, Ruby, or JavaScript.</li><li><strong>Browsers:</strong> It works with all major browsers through their browser drivers, including Chrome, Firefox, Edge, and Safari.</li><li><strong>Parallel runs:</strong> Selenium Grid runs your tests on many machines and browsers at the same time.</li><li><strong>Cost:</strong> Selenium is free and open source.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> You mostly get a stack trace. A stack trace is the error message plus the lines of code where the failure happened, so it shows where the test broke, but it doesn’t show the page.</li><li><strong>What to turn on:</strong> You add a few lines of screenshot code to your teardown, which is the code that runs after each test. Selenium 4 also offers WebDriver BiDi, which can collect browser console messages and JavaScript errors once you enable it in your browser settings.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cebcKjeyHWm8DLq-H7hB5Q.png" /></figure><p><strong><em>Verdict:</em></strong><em> Selenium works with almost any language, browser, and grid. That makes it a safe choice for older apps and for teams with mixed skills. Plan to build your own evidence collection on top of it.</em></p><h3>Playwright</h3><p>Playwright is an open-source framework from Microsoft. It’s built for modern web apps, and it comes with its own test runner.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You can write tests in JavaScript or TypeScript, Python, Java, or .NET.</li><li><strong>Browsers:</strong> It runs tests on Chromium, Firefox, and WebKit, which are the engines behind Chrome, Edge, Firefox, and Safari.</li><li><strong>Parallel runs:</strong> Its test runner runs test files in parallel using workers by default. You can also split a suite across several machines with the — shard option.</li><li><strong>Cost:</strong> Playwright is free and open source.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> Playwright doesn’t save screenshots, videos, or traces until you ask it to. Its Trace Viewer is built in, though, and a trace works like a recording of the whole test, with every action, a copy of the page at each step, and the matching console and network logs.</li><li><strong>What to turn on:</strong> I add three settings to the config file, as shown below. You can then open a trace with npx playwright show-trace, or drag it into trace.playwright.dev, which opens it inside your browser.</li></ul><pre>// playwright.config.ts<br>import { defineConfig } from &#39;@playwright/test&#39;;<br>export default defineConfig({<br>retries: 1,<br>use: {<br>trace: &#39;on-first-retry&#39;,<br>screenshot: &#39;only-on-failure&#39;,<br>video: &#39;retain-on-failure&#39;,<br>},<br>});</pre><p>These settings give a failed test one retry and record a trace during that retry. They also save a screenshot when a test fails and keep the video only for failed tests. Playwright’s docs recommend this trace setting for CI, because it keeps normal runs fast.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LExZuCzzXJ3zSeKDdHLl3w.png" /></figure><p><strong><em>Verdict:</em></strong><em> A single trace file tells you a lot about a failure. You need a few lines of config, and your CI has to save the trace files after each run.</em></p><h3>BrowserStack Automate</h3><p>BrowserStack Automate runs your existing tests on real browsers in the cloud. It doesn’t replace your framework. Instead, it gives your current tests more browsers to run on.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> It works with the languages your framework already uses, because it runs your Selenium, Playwright, and Cypress tests as they are.</li><li><strong>Browsers:</strong> BrowserStack says it offers more than 3,500 real desktop and mobile browser and operating system combinations.</li><li><strong>Parallel runs:</strong> You can run many tests at the same time, and the number of parallel tests depends on your plan.</li><li><strong>Cost:</strong> It’s a paid service, and you can start with a free trial.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> Every session records a video, and you get text logs, Selenium logs, and browser console errors without changing any settings. The evidence is collected on BrowserStack’s side, so every project gets the same evidence.</li><li><strong>What to turn on:</strong> Setting debug to true takes a screenshot at each step. Setting networkLogs to true saves a HAR file, which is a record of every network request the page made, and captureContent adds the request and response content.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kYXlrMGQoG0TfN3eVe7wKQ.png" /></figure><p><strong><em>Verdict:</em></strong><em> It’s a good fit for teams that use more than one framework. You get the same kind of evidence for all of them in one place. I’d turn on network logs for any test that calls an API.</em></p><h3>Cypress</h3><p>Cypress runs inside the browser, next to your app. While you write tests, it shows every command in a list, and you can click any step to see the page at that moment.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You write tests in JavaScript or TypeScript.</li><li><strong>Browsers:</strong> It supports Chrome-family browsers, including Edge, and Firefox. WebKit support is still experimental.</li><li><strong>Parallel runs:</strong> Cypress’s docs tie parallel runs to its paid Cypress Cloud service, using the record and parallel flags.</li><li><strong>Cost:</strong> The Cypress app is free and open source, while Cypress Cloud is a paid service.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> When you run tests in CI with cypress run, Cypress takes a screenshot automatically whenever a test fails. Video is off.</li><li><strong>What to turn on:</strong> You can set video to true for recordings. Also check trashAssetsBeforeRuns, because Cypress deletes the screenshots and videos folders before each run unless you set it to false or your CI saves them first.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zpv6gSYcMqoWLbJbV1o_oA.png" /></figure><p><strong><em>Verdict: </em></strong><em>Automatic failure screenshots make Cypress easy to start with. Make sure your CI saves the screenshots folder after each run, or you’ll lose it on the next one.</em></p><h3>WebdriverIO</h3><p>WebdriverIO is an open-source test framework for Node.js. It’s very flexible, and many of its features come from plugins and reporters that you add yourself.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You write tests in JavaScript or TypeScript.</li><li><strong>Browsers:</strong> It controls all major browsers through the WebDriver standard.</li><li><strong>Parallel runs:</strong> The maxInstances setting controls how many tests run at the same time. Its default is 100, which suits cloud grids, and the docs suggest a much lower number on a single machine.</li><li><strong>Cost:</strong> WebdriverIO is free and open source.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> WebdriverIO doesn’t take screenshots on failure, so you rely on the error message and your reporter’s output.</li><li><strong>What to turn on:</strong> Most teams call saveScreenshot in an afterTest hook, which is code that runs automatically after each test. For video, the community package wdio-video-reporter joins screenshots into a video, keeps only failed tests by default, and leaves out popups and alert boxes.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wRPtK_6ZadDne0nRHe1QAg.png" /></figure><p><strong><em>Verdict:</em></strong><em> You can build exactly the evidence you want, so it suits teams that like to customise their setup. Expect to spend some time on hooks and reporters first.</em></p><h3>TestCafe</h3><p>TestCafe is an open-source tool for Node.js. Unlike Selenium, it doesn’t use WebDriver to control the browser, so there are no browser drivers to install.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You write tests in JavaScript or TypeScript.</li><li><strong>Browsers:</strong> It works with most desktop browsers, including Chrome, Firefox, Edge, and Safari.</li><li><strong>Parallel runs:</strong> The concurrency option opens several copies of each browser and splits the tests between them.</li><li><strong>Cost:</strong> TestCafe is free and open source.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> Failure screenshots are off, so you get the test report and error message.</li><li><strong>What to turn on:</strong> The option takeOnFails=true saves a screenshot in an errors folder for each failed test. Video needs a free program called FFmpeg, and it only works in Chrome, Firefox, and Edge running on your own machine.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jQ01aJFeOluDn7bO_j20eA.png" /></figure><p><strong><em>Verdict:</em></strong><em> TestCafe is light to set up, and its screenshot options are simple. If you run tests on remote browsers, remember that video recording won’t work there.</em></p><h3>Robot Framework</h3><p>Robot Framework is an open-source, keyword-driven framework. You write tests using short commands called keywords, so tests read almost like plain English.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> Tests use Robot Framework’s keyword syntax, and you can write your own keywords in Python.</li><li><strong>Browsers:</strong> Libraries do the browser work. SeleniumLibrary uses Selenium, and Browser library uses Playwright.</li><li><strong>Parallel runs:</strong> A separate tool called Pabot runs Robot Framework suites in parallel.</li><li><strong>Cost:</strong> Robot Framework and these libraries are free and open source.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> Every run creates a file called log.html, which shows each step of each test. SeleniumLibrary also takes a screenshot whenever one of its keywords fails, and Browser library does the same.</li><li><strong>What to turn on:</strong> Robot Framework has no built-in video. On its community forum, the usual advice is to start a screen recorder before the tests and stop it afterwards.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5pY6Zu8QZfou5ULQb5d8iA.png" /></figure><p><strong><em>Verdict:</em></strong><em> Its default logs are easy for non-developers to read. That makes it a good choice for teams where testers and analysts work on the same tests.</em></p><h3>Katalon Studio</h3><p>Katalon Studio is a low-code testing tool. You can record your actions and play them back as tests, and it’s built on top of Selenium and Appium.</p><h4>Key features</h4><ul><li><strong>Languages:</strong> You can build tests without code, or write scripts in Groovy when you need more control.</li><li><strong>Browsers:</strong> It supports the main desktop browsers for web testing, and it also covers API and mobile testing.</li><li><strong>Parallel runs:</strong> Katalon’s pricing page lists parallel test execution in its paid plans and not in the free plan.</li><li><strong>Cost:</strong> There’s a free plan for local runs, and paid plans add parallel runs, CI execution, and integrations.</li></ul><h4>When a test fails</h4><ul><li><strong>By default:</strong> Katalon keeps a log of the test run, and you choose the evidence settings in the project settings.</li><li><strong>What to turn on:</strong> One option takes a screenshot when a test fails, for web and mobile tests. A video recorder is also available for web tests, and you can keep videos of failed tests only or of all tests.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Grlbx--xdc8KL7g9pvfJw.png" /></figure><p><strong><em>Verdict:</em></strong><em> You can turn on evidence with a few clicks and no code. The trade-off is that parallel runs and CI need a paid plan, and some teams feel limited as their test suite grows.</em></p><h3>Picking a Regression Test Tool for a Growing Suite</h3><p>A regression test tool reruns your tests after every change, to make sure nothing that used to work has broken. Picking one is easy when you have fifty tests. It gets harder when you have five hundred.</p><p>These are the four questions I ask before choosing between test automation tools for a growing suite.</p><ul><li><strong>Who writes the tests:</strong> If developers write most of the tests, code-based tools like Playwright, Cypress, or WebdriverIO usually feel natural. If testers and analysts also write tests, Robot Framework or Katalon Studio can be easier for everyone to read.</li><li><strong>Which browsers you need:</strong> Playwright covers Chromium, Firefox, and WebKit in one tool. Selenium works with many browsers and grids, including older setups.</li><li><strong>How you’ll run tests in parallel:</strong> Playwright, WebdriverIO, and TestCafe run tests in parallel on their own. Cypress and Katalon tie parallel runs to paid plans, while Selenium and Robot Framework use Grid and Pabot.</li><li><strong>How you will debug failures:</strong> Use the failure evidence groups earlier in this article as a checklist. With hundreds of tests, some will fail every week, so good debugging tools will save your team real time.</li></ul><h3>Why a Passing Suite Can Still Mislead You</h3><p>A green test run feels reassuring. But it doesn’t always mean everything is fine.</p><p>Retries are the main reason. Many teams let a failed test run a second time. If it passes on that second try, the report still shows it as passed.</p><p>Retries help keep a pipeline stable. But a test that needs a retry every few runs is usually telling you something. The cause is often a timing problem, a slow API, or a locator that breaks easily.</p><p>So I look at more than the pass rate. First, I count how many tests passed only after a retry. Then, I keep the traces or videos for those tests. Finally, I make sure the evidence is saved before the next run deletes it.</p><p>The test automation tools that help most here are the ones that save evidence on the very first failure. So the next time your test suite turns green, ask yourself a simple question. If a test passed only on its second try, would you know what went wrong the first time?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7fd67173dd4e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top Mabl Alternatives for Low Code Automation]]></title>
            <link>https://jennykeeper.medium.com/top-mabl-alternatives-for-low-code-automation-a082cf375d54?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a082cf375d54</guid>
            <category><![CDATA[mabl-alternatives]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[low-code-automation]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Sat, 17 Oct 2026 07:21:01 GMT</pubDate>
            <atom:updated>2026-10-17T07:21:01.603Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cover banner of ‘Top Mabl Alternatives for Low Code Automation’" src="https://cdn-images-1.medium.com/max/1024/1*5PoTlY7KcHp_AZTKm-XMBQ.png" /></figure><p>Test automation was once a scripting exercise. Low-code tools changed that. They made it possible for teams to create and maintain automated tests without relying on code for every workflow.</p><p>Mabl is one such tool. It is a popular platform for low-code test automation, giving teams a simpler way to create and maintain automated tests.</p><p>However, as teams grow, they need capabilities beyond what Mabl offers. Some require faster CI/CD execution, while others need deeper conditional logic and better control over complex workflows. Mabl remains capable, but its web-focused approach and customized pricing may not fit every team.</p><p>That makes the choice of an automation platform worth revisiting.</p><p>In this blog, I compare the top Mabl alternatives, covering their features, pricing, testing capabilities, and practical differences. I also explain what to evaluate before switching and how to migrate existing Mabl tests.</p><h3>Why Should You Look for an Alternative to Mabl?</h3><p>Mabl makes test automation easier to adopt. Its low-code workflows and AI capabilities can reduce the amount of manual effort involved in creating and maintaining tests.</p><p>So, why should you consider alternatives?</p><p>Because the right automation platform depends on<em> what your team needs from it.</em> When you’re looking for new testing options, you’re searching for a tool on the basis of the application, testing process, structure of the team, and the budget that you have. All this will influence if the tool is a right fit for your team.</p><p>You would also want that the tool aligns with the organization’s preferences, values, and the ways of working. This will look different for every organization, so having a few alternatives in your bucket is always beneficial.</p><p>If you have used Mabl already, you must have observed that:</p><ul><li>Some <strong>complex test logic requires JavaScript</strong>, which can take you outside the low-code workflow.</li><li><strong>Desktop application testing is limited</strong>, which can become a challenge when your testing scope extends beyond web applications.</li><li><strong>Test execution costs increase with usage</strong>, making the pricing harder to manage as your test suite grows.</li><li>Some <strong>complex workflows require greater control</strong> over conditions, dynamic data, and advanced test logic.</li><li><strong>Mabl-specific test assets can make migration harder</strong>, especially when you have an established suite of tests.</li><li><strong>Broader application coverage may be necessary</strong> when your team needs to test more than web applications.</li></ul><p>These limitations may become more noticeable as your testing requirements grow. At that point, it makes sense to see what other platforms can offer.</p><h3>Some Popular Mabl Alternatives for Low-Code Automation</h3><p>Different teams need different testing capabilities across applications and environments. Let’s take a closer look at the leading mabl alternatives for low-code automation.</p><h4>1. BrowserStack Low Code Automation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ko7nrmNjU-fl5H4MltrCLg.png" /></figure><p>BrowserStack Low Code Automation lets teams create automated tests by recording browser actions, adding validations, variables, modules, AI-powered steps, and JavaScript. Tests run on BrowserStack’s cloud across real desktop and mobile browsers.</p><p><strong>Key Features of BrowserStack Low Code Automation</strong></p><ul><li><strong>Visual test recorder:</strong> Capture browser actions and turn them into reusable automated workflows.</li><li><strong>AI-powered testing:</strong> Generate steps from natural language and use AI-assisted self-healing.</li><li><strong>Real-browser execution:</strong> Run tests across real desktop and mobile browser environments.</li><li><strong>Advanced test logic:</strong> Add variables, modules, validations, loops, APIs, and JavaScript.</li><li><strong>Debugging and reporting:</strong> Review videos, screenshots, console logs, and network activity.</li></ul><p><strong>BrowserStack Low Code Automation Pricing:</strong> It offers a <strong>Free plan.</strong> Its <strong>paid plans </strong>have different pricing based on usage and requirements; you can contact their sales team for the same. It also offers a <strong>14-day trial</strong>.</p><p><strong>How is BrowserStack Low Code Automation Different From Mabl? </strong>BrowserStack combines low-code authoring with real-browser and real-device infrastructure. Its workflows also support JavaScript, APIs, modules, conditions, and data-driven testing for more complex scenarios.</p><p><strong><em>My Take on BrowserStack Low Code Automation: </em></strong><em>What stood out to me when I used BrowserStack was the ease of moving between simple recorded tests and more involved workflows. I can start with a low-code flow and introduce JavaScript, API, condition, or even data-driven steps only when the test actually needs them.</em></p><h4>2. Testim</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rRxy9wNzNSQbSgMJyLMh1g.png" /></figure><p>Testim combines visual test authoring with AI-powered Smart Locators and code extensibility. It supports web, mobile, and Salesforce testing, with reusable groups, loops, conditions, API steps, custom JavaScript, data-driven workflows, and CI/CD integrations.</p><p><strong>Key Features of Testim</strong></p><ul><li><strong>AI Smart Locators:</strong> Analyze multiple attributes to identify elements and adapt to UI changes.</li><li><strong>Custom JavaScript:</strong> Add code inside or outside the browser for advanced test scenarios.</li><li><strong>Reusable groups:</strong> Turn repeated sequences into reusable components across multiple tests.</li><li><strong>Advanced workflows:</strong> Build loops, conditions, validations, API calls, and parameters.</li><li><strong>CI/CD integrations:</strong> Connect tests with Jenkins, GitLab, GitHub, Jira, and other tools.</li></ul><p><strong>Testim Pricing:</strong> Testim offers a <strong>Community free plan</strong> after the trial. Paid plans are available with <strong>custom pricing</strong>, based on your testing requirements.</p><p><strong>How is Testim Different From Mabl? </strong>Testim combines visual automation with Smart Locators and custom JavaScript. Its reusable groups, API steps, and code extensions give teams more control over complex and data-driven workflows.</p><p><strong><em>My Take on Testim:</em></strong><em> I like Testim when I need low-code test creation without giving up engineering control. Smart Locators and reusable groups keep everyday tests manageable, while API actions and custom JavaScript give me room to handle more complex workflows.</em></p><h4>3. ACCELQ</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ajQYHDIiyBBbaZQlAi2nlg.png" /></figure><p>ACCELQ is a codeless continuous testing platform built around business-process automation. It supports web, mobile, API, desktop, packaged applications, databases, middleware, and other enterprise environments through its automation and test-management products.</p><p><strong>Key Features of ACCELQ</strong></p><ul><li><strong>No-code logic editor:</strong> Build automation workflows without conventional programming syntax.</li><li><strong>Business-process modeling:</strong> Design tests around complete application and business workflows.</li><li><strong>API and database testing:</strong> Connect frontend scenarios with backend and database validation.</li></ul><p><strong>ACCELQ Pricing:</strong> ACCELQ offers a <strong>free trial</strong> for its Web, Mobile, and API automation products. <strong>Enterprise</strong> and <strong>Cloud Lab</strong> pricing is available on request.</p><p><strong>How is ACCELQ Different From Mabl? </strong>ACCELQ is designed for broader business-process automation across web, mobile, APIs, databases, desktop, and enterprise applications. This gives it wider application coverage than Mabl’s web-focused approach.</p><p><strong><em>My Take on ACCELQ:</em></strong><em> I find ACCELQ useful when a test flow spans more than just the UI. Its codeless workflows let me connect UI, API, and database testing in the same business flow, which makes it easier to validate an end-to-end scenario without stitching together multiple tools.</em></p><h4>4. TestComplete</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NxoKrKcrZdOr0bzIV5opSA.png" /></figure><p>TestComplete is a functional UI automation platform for desktop, web, and mobile applications. It combines record-and-replay, keyword-driven testing, scripting, object recognition, and CI/CD capabilities. Current documentation applies to TestComplete 15.84.</p><p><strong>Key Features of TestComplete:</strong></p><ul><li><strong>Record-and-replay:</strong> Capture application interactions and convert them into automated tests.</li><li><strong>Keyword-driven testing:</strong> Create visual workflows from reusable testing actions.</li><li><strong>Desktop automation:</strong> Automate native Windows applications and legacy interfaces.</li><li><strong>Script support:</strong> Extend recorded workflows with programming when needed.</li><li><strong>Object recognition:</strong> Identify application controls for repeatable UI automation.</li></ul><p><strong>TestComplete Pricing: </strong>TestComplete offers <strong>Fixed</strong> or <strong>Floating</strong> licenses for Desktop, Web, and Mobile. Annual single-module <strong>plans start at $4,318</strong>, while <strong>Pro</strong> bundles start at<strong> $7,799</strong>. A <strong>14-day trial</strong> is available.</p><p><strong>How TestComplete is Different From Mabl? </strong>TestComplete supports native Windows desktop automation alongside web and mobile testing. Its object repository and scripting capabilities also provide direct control over application elements and test logic.</p><p><strong><em>My Take on TestComplete:</em></strong><em> TestComplete is a strong option when desktop testing is part of your automation scope. Its object repository and scripting support give QA teams more control when browser automation alone is not enough.</em></p><h4>5. Testsigma</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pHNUh2kzKacZpqJhuUXdvQ.jpeg" /></figure><p>Testsigma is a cloud-based test automation platform that uses natural-language authoring for web and mobile testing. Its current Pro and Enterprise plans include AI assistance, parallel execution, auto-healing, integrations, real-device testing, and broader enterprise deployment options.</p><p><strong>Key Features of Testsigma</strong></p><ul><li><strong>Plain-English authoring:</strong> Create automated scenarios using readable, natural-language test steps.</li><li><strong>AI testing companion:</strong> Use AI assistance across test creation and automation workflows.</li><li><strong>Auto-healing:</strong> Adapt automation when application elements change between test runs.</li><li><strong>Parallel execution:</strong> Run tests across multiple browser and device configurations.</li><li><strong>Real-device testing:</strong> Execute Android and iOS scenarios on physical mobile devices.</li></ul><p><strong>Testsigma Pricing:</strong> Testsigma offers <strong>Pro</strong> and <strong>Enterprise</strong> plans with <strong>custom pricing</strong> based on your usage and requirements. A <strong>free trial</strong> is also available.</p><p><strong>How is Testsigma Different From Mabl? </strong>Testsigma emphasizes natural-language test authoring across web and mobile environments. Its enterprise options also support private deployment, local testing, and private grids for controlled environments.</p><p><strong><em>My Take on Testsigma:</em></strong><em> Testsigma makes sense when you have both technical and nontechnical testers contributing to automation. Its natural-language authoring keeps test creation accessible, while API testing, parallel execution, and real-device testing give QA teams more room to expand coverage.</em></p><h4>6. Ranorex Studio</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eq4X0Nvzi4gQ_pVUjYP1sA.png" /></figure><p>Ranorex Studio is a hybrid automation platform for desktop, web, and mobile applications. It combines a no-code recorder, object repository, reusable modules, and C# or VB.NET support, allowing teams to move between visual and coded automation.</p><p><strong>Key Features of Ranorex Studio</strong></p><ul><li><strong>No-code recorder:</strong> Capture application interactions without writing automation scripts.</li><li><strong>Object repository:</strong> Maintain reusable application elements across test modules.</li><li><strong>C#/VB.NET support:</strong> Add programmatic logic for complex automation requirements.</li><li><strong>Reusable modules:</strong> Build shared test components for repeated application workflows.</li><li><strong>Desktop automation:</strong> Support Windows applications alongside web and mobile testing.</li></ul><p><strong>Ranorex Studio Pricing:</strong> Ranorex offers <strong>perpetual and subscription</strong> licenses. A <strong>14-day free trial</strong> is also available, while exact pricing depends on the license type and requirements.</p><p><strong>How is Ranorex Studio Different From Mabl?: </strong>Ranorex Studio combines visual recording with C# and VB.NET automation and supports Windows desktop applications. It is designed for teams that need both low-code and programmatic control.</p><p><strong><em>My Take on Ranorex Studio:</em></strong><em> I like the flexibility Ranorex gives me when I need to switch between visual test creation and coded automation. The recorder handles the initial test flow, while the object repository and C#/VB.NET support gives me more control when the test gets more involved.</em></p><h4>7. Functionize</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mMAVzo7r12ojIyi1VkOtJw.png" /></figure><p>Functionize is an AI-focused test automation platform that uses AI and machine learning across test creation, execution, maintenance, and analysis. Its current plans use monthly credits and parallel-run limits, with enterprise options for larger deployments.</p><p><strong>Key Features of Functionize</strong></p><ul><li><strong>AI test creation:</strong> Generate automated tests with AI assistance from natural-language requirements.</li><li><strong>Intelligent element recognition:</strong> Use multiple signals to identify application elements during tests.</li><li><strong>Self-healing automation:</strong> Adapt tests when application interfaces change over time.</li><li><strong>Parallel execution:</strong> Run multiple tests simultaneously within plan-specific limits.</li><li><strong>Failure analysis:</strong> Use automated diagnostics to investigate failures and test behavior.</li></ul><p><strong>Functionize Pricing:</strong> Functionize offers a <strong>Free plan</strong>, with paid <strong>Growth ($20/month)</strong> and <strong>Scale ($100/month)</strong> plans. <strong>Enterprise</strong> pricing is available on request.</p><p><strong>How Functionize is Different From Mabl? </strong>Functionize places AI across test creation, element recognition, maintenance, and failure analysis. Its usage model also tracks automation through credits and parallel execution limits.</p><p><strong>My Take on Functionize: </strong>Functionize is suitable for QA teams managing large suites where test maintenance is a major concern. Its AI-based element recognition and failure analysis can help reduce repetitive locator maintenance.</p><h4>8. Rainforest QA</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ZGgT6uhN3gZMoEGInq9Tg.png" /></figure><p>Rainforest QA uses a visual, no-code approach to automated testing, with tests created through a visual editor and plain-English steps. Its platform also includes AI test generation, self-healing, debugging, and QA-as-a-service options.</p><p><strong>Key Features of Rainforest QA</strong></p><ul><li><strong>Visual test authoring:</strong> Build tests through an interface that mirrors real user interactions.</li><li><strong>Plain-English steps:</strong> Let technical and nontechnical users review and maintain tests.</li><li><strong>AI test generation:</strong> Generate end-to-end test scenarios from natural-language prompts.</li><li><strong>Self-healing tests:</strong> Automatically adapt automation when application interfaces change.</li><li><strong>Visual debugging:</strong> Review test replays, logs, and execution evidence after failures.</li></ul><p><strong>Rainforest QA Pricing:</strong> Rainforest QA offers <strong>custom pricing</strong> based on your testing needs. A <strong>free trial</strong> is also available.</p><p><strong>How is Rainforest QA Different From Mabl? </strong>Rainforest QA uses visual interaction and plain-English test steps as its core authoring model. This reduces reliance on traditional selector-based workflows for UI-heavy test scenarios.</p><p><strong><em>My Take on Rainforest QA:</em></strong><em> I like Rainforest QA when I need tests that are easy for the whole team to understand and maintain. The visual approach also makes UI-heavy testing less dependent on selectors, which can save time when the interface changes frequently.</em></p><h3>How Do You Choose the Right Mabl Alternative for Your Team?</h3><p>The right automation tool should match your application, workflows, and team skills. Here are the key factors to consider before selecting a mabl alternative.</p><ul><li><strong>Match Your Application Stack:</strong> Check support for web, mobile, desktop, APIs, and enterprise applications. Avoid paying for platforms that offer coverage your team does not need.</li><li><strong>Assess Test Complexity:</strong> Evaluate support for loops, conditions, dynamic data, APIs, and custom logic. Advanced workflows should not require extensive scripting to remain maintainable.</li><li><strong>Evaluate Team Skills:</strong> Consider who will create and maintain tests. Visual authoring suits mixed teams, while JavaScript, C#, or other code support helps engineers handle advanced scenarios.</li><li><strong>Check Execution and CI/CD:</strong> Compare browser and device coverage, parallel execution, local testing, reporting, and CI/CD integrations. These directly affect regression speed and pipeline adoption.</li><li><strong>Compare Total Cost:</strong> Look beyond subscription fees. Assess execution credits, user limits, parallel runs, infrastructure, maintenance effort, and enterprise features to estimate the actual automation cost.</li></ul><h3>How to Migrate From Mabl to Another Low-Code Automation Tool?</h3><p>A poorly planned migration can leave gaps in test coverage and CI/CD workflows. Follow the steps below to move from Mabl with fewer testing disruptions.</p><p><strong>Step 1: Audit Your Mabl Test Suite:</strong> Review every test, remove duplicates and obsolete cases, and flag flaky workflows. Prioritize critical smoke, regression, and release-blocking tests for migration.</p><p><strong>Step 2: Map Data and Reusable Workflows:</strong> Document authentication flows, shared steps, variables, test data, and dependencies. Export portable data where possible and identify Mabl-specific components that need rebuilding.</p><p><strong>Step 3: Rebuild Critical Tests First:</strong> Re-create core user journeys in the Mabl alternative. Validate dynamic locators, assertions, variables, loops, API calls, and custom logic before migrating the wider suite.</p><p><strong>Step 4: Run Both Tools in Parallel:</strong> Execute Mabl and the new low-code automation tool side by side. Compare pass rates, execution time, failure evidence, locator behavior, and CI/CD notifications.</p><p><strong>Step 5: Move CI/CD and Retire Mabl:</strong> Shift deployment gates and scheduled runs after the new suite reaches acceptable coverage and stability. Archive Mabl assets and cancel unused subscriptions or test capacity.</p><h3>Conclusion</h3><p>You cannot judge a low-code automation tool by how little code it makes you write. I care more about what happens when a test becomes complicated. I check if I can extend it without fighting the platform? Can I understand why it failed? Can I maintain it when the application changes?</p><p>That is where the real difference between automation tools shows up. A polished recorder can make the first hundred tests feel effortless. The real test comes later, when the suite grows and the edge cases start piling up.</p><p>For me, that is the point worth keeping in mind when moving away from Mabl. Do not choose the tool that makes the best first impression. Choose the one that feels manageable when your test suite becomes complicated.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a082cf375d54" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Cross-Browser Automation Tools I’d Shortlist for Different QA Workflows]]></title>
            <link>https://jennykeeper.medium.com/10-cross-browser-automation-tools-id-shortlist-for-different-qa-workflows-7b2901d090cc?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b2901d090cc</guid>
            <category><![CDATA[qa-workflow]]></category>
            <category><![CDATA[cross-browser-automation]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[automation-tools]]></category>
            <category><![CDATA[cross-browser-testing]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 18:55:47 GMT</pubDate>
            <atom:updated>2026-10-15T18:55:47.813Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ub15S8Wt32zB_QhbFbsPIQ.png" /></figure><p>People ask me which cross-browser automation tool they should use all the time. I usually answer with another question: <strong>do your users use Safari?</strong></p><p>It sounds like I am avoiding the question, but that answer cuts the list down faster than most feature comparisons.</p><p>Three of the ten tools here cannot open Safari at all. Another runs WebKit, which is useful, but it still is not the Safari browser sitting on a user’s Mac or iPhone.</p><p>That changes how I look at this entire category.</p><p>I do not start with who has the nicest test runner or the longest feature page. I start with what browser the tool can reach. Then I look at where the tests run, what I get when they fail, and how much maintenance I am signing myself up for. So I have grouped these tools around the workflows where I would use them.</p><h3>What Cross-Browser Covers</h3><p>I think the phrase “cross-browser” hides an important distinction.Some tools drive the actual browser your users installed. Others drive the browser engine underneath it. Those are not always the same test.</p><p>Playwright is a good example. It runs Chromium, Firefox and WebKit, but its own documentation says it does not work with the branded version of Safari because its WebKit build relies on patches.</p><p>Cypress takes a similar route with experimental WebKit support. Puppeteer does not support Safari at all.</p><p>If I need real Safari, I need Apple’s browser and driver on macOS. I can enable safaridriver on a Mac and use Selenium to automate it directly. That distinction sounds small until Safari is the browser producing the bug.</p><h3>How I’d Evaluate Cross-Browser Automation Testing Tools</h3><p>Before I look at syntax or execution speed, I ask four questions.</p><ul><li><strong>Which browsers can I reach?</strong> I write down the exact browser or binary. Chromium is not Chrome, and WebKit is not Safari.</li><li><strong>Where will my tests run?</strong> A local framework and a hosted grid solve different problems. They also create very different infrastructure and CI costs.</li><li><strong>What will I get when something fails?</strong> I want to know whether I get a stack trace, screenshot, video, network log or trace.</li><li><strong>What will I have to maintain?</strong> I look at selector churn, browser-driver updates, grid maintenance and configuration overhead. None of those costs appear nicely packaged on a pricing page.</li></ul><p><strong>Disclosure:</strong> I include Nightwatch.js below. Its GitHub repository identifies BrowserStack as where the project was built, so I keep that relationship in mind when evaluating it.</p><h3>10 Cross-Browser Automation Testing Tools I’d Consider</h3><p>The 10 tools below solve different problems, so I look at what each one reaches, where I would use it, what it costs, and where I would stop relying on it.</p><h3>1. BrowserStack Automate</h3><p>BrowserStack Automate lets me run Selenium, Playwright, Cypress and Nightwatch suites on hosted browser and device infrastructure.</p><p>It covers more than 3,500 desktop and mobile browser and OS combinations. I can also use a local tunnel when I need to test staging or an application sitting behind a firewall.</p><p>The debugging layer matters just as much to me. A failed run can give me video, screenshots and unified logs instead of leaving me with only a stack trace.</p><p>Pricing starts at <strong>$59 per month when billed annually</strong>, while the Desktop and Mobile plan starts at <strong>$175 per month</strong>. On G2, BrowserStack sits at <strong>4.4/5 from 3,361 reviews</strong>.</p><p>That review volume matters more to me than the raw star score. Thousands of reviews give me a much broader signal than a perfect rating based on one or two people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xGaOJzxuZDlu3kO9HNwtpQ.png" /></figure><p><strong>My take:</strong> I would use Automate when browser and OS infrastructure becomes the problem.</p><p>Safari is the obvious example. If I need real Safari and macOS coverage but do not want to maintain a Mac fleet, hosted execution solves a practical problem.</p><p>I would still debug the first failure locally whenever possible. Moving a bad test onto remote hardware does not make the test better. It just gives it a more expensive place to fail.</p><h3>2. Playwright</h3><p>Playwright gives me Chromium, WebKit and Firefox in one framework. I can also launch branded Chrome and Edge through channels such as chrome and msedge.</p><p>I like its auto-waiting model because I spend less time manually coordinating basic UI state. Its tracing is also genuinely useful when a failure is difficult to reproduce.</p><p>Playwright is <strong>free and open source</strong>. Its G2 rating sits at <strong>4.7/5</strong>, although that comes from only <strong>13 reviews</strong>.</p><p>I would not read too much into the rating because the sample is small. Open-source frameworks often look underrepresented on G2 simply because teams do not review them like purchased enterprise software. The browser distinction is more important.</p><p>Playwright’s WebKit build gives me WebKit coverage. It does not give me the branded Safari browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UAVIu8lHSvPznIlE3UttYQ.png" /></figure><p><strong>My take:</strong> I would use Playwright when I want fast coverage across browser engines with a modern runner and strong debugging tools.</p><p>If real Safari matters to the release decision, I would still run Safari elsewhere. I would never label a WebKit result as a Safari result just because the two share an engine.</p><h3>3. Selenium WebDriver</h3><p>Selenium still matters because it drives the real browsers people use. That includes Safari through Apple’s safaridriver, alongside Chrome, Edge and Firefox.</p><p>It also has mature Java, Python, C#, Ruby and JavaScript clients. The W3C WebDriver standard gives it an unusually broad ecosystem, which is why so many hosted grids and testing products support Selenium suites.</p><p>Selenium itself is <strong>free and open source</strong>. It holds a <strong>4.5/5 G2 rating from 101 reviews</strong>.</p><p>That is a healthier review sample than most open-source frameworks in this list, but I still care more about its browser reach than its stars.</p><p>The trade-off is that Selenium gives me more building blocks than finished workflow. I still have to decide how I want to structure the runner, reporting, waits and execution infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*adgKl-bIFNrILFLa6WMU6Q.png" /></figure><p><strong>My take:</strong> I keep Selenium on the shortlist when actual branded-browser coverage matters more than getting the most opinionated developer experience.</p><p>It gives me reach. It also gives me more things to assemble and maintain.</p><h3>4. Cypress</h3><p>Cypress gives me a very different development experience. Its runner shows commands as they execute, and time-travel debugging makes it easy to inspect what the application looked like at different points in the test.</p><p>It supports Chrome-family browsers, Edge, Firefox and experimental WebKit.</p><p>The core Cypress app is <strong>open source</strong>, while Cypress Cloud adds paid functionality. On G2, Cypress has a <strong>4.7/5 rating from 107 reviews</strong>.</p><p>That gives me a more meaningful review base than most developer-first frameworks here, although I would still evaluate the actual WebKit gaps myself.</p><p>Its WebKit implementation remains experimental. Features such as cy.origin() do not work there, and Test Replay is unavailable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r2IX32QEMIuT0i1g_0xxKA.png" /></figure><p><strong>My take:</strong> I like Cypress for Chrome-first products where developers own the test suite and fast local iteration matters.</p><p>I would be much more cautious if somebody told me the same suite had to become their Safari compliance strategy. Test the awkward flows in WebKit first and see exactly where the gaps affect the application.</p><h3>5. WebdriverIO</h3><p>WebdriverIO gives me the browser reach of the WebDriver ecosystem but wraps more of the runner around it.</p><p>Version 9 and later needs Node 18.20.0 or newer. It includes a configuration wizard and can also run inside a normal Node script. The ecosystem stretches beyond web testing into mobile, desktop and Flutter automation.</p><p>WebdriverIO is <strong>free and open source</strong>, and it currently has <strong>no meaningful G2 listing</strong> to lean on.</p><p>I do not see that as a negative by itself. For tools like this, GitHub activity, documentation quality, ecosystem size and how easily I can solve an unusual problem often tell me more than procurement-site reviews. A lot of the functionality arrives through reporters, services and other plugins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qfX6g44acOTfV8S6leAT3w.png" /></figure><p><strong>My take:</strong> I reach for WebdriverIO when I want WebDriver’s reach but do not want to assemble every part of the test runner myself.</p><p>The plugin ecosystem is both the attraction and the cost. It gives me flexibility. It can also turn wdio.conf into a small infrastructure project if nobody owns it properly.</p><h3>6. Sauce Labs</h3><p>Sauce Labs gives me hosted web and mobile execution across browsers and devices. It supports the common automation frameworks and provides tunnelling for internal environments.</p><p>Sauce Labs uses <strong>commercial tiered pricing</strong> and offers a <strong>free trial</strong>. Its G2 rating sits at <strong>4.3/5 from 182 reviews</strong>.</p><p>I would pay more attention to those reviews if I were comparing enterprise support, reliability or procurement experience than if I were deciding which underlying automation framework to write tests in.</p><p>The part I would investigate most closely is security and compliance. For some teams, especially in regulated industries, the testing framework is not the blocker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yxMLTBsSJ9hwJInQJRDByg.png" /></figure><p><strong>My take:</strong> I would put Sauce Labs on the shortlist when compliance requirements shape the buying decision.</p><p>I would compare the actual requirements rather than treating “enterprise security” as a useful feature by itself. I would also run the same failed test across shortlisted grids and compare what each one gives me for debugging. Hosted grids can look remarkably similar until something breaks.</p><h3>7. TestingBot</h3><p>TestingBot offers more than 6,100 browser combinations across Windows, macOS and Linux, along with physical iOS and Android devices. It supports Selenium, Playwright, Cypress, Puppeteer, WebdriverIO and Robot Framework.</p><p>TestingBot uses <strong>commercial pricing</strong>, with real devices included across its paid plans. Its G2 score is <strong>4.0/5</strong>, but that currently comes from only <strong>two reviews</strong>.</p><p>I would barely use that rating in a buying decision. Two reviews are closer to anecdotes than a market signal. Its data-hosting options in the EU and Switzerland interest me more because teams with specific GDPR or residency requirements can test that requirement against their own policies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tofrzTmYe0CAaGKjAN5ogw.png" /></figure><p><strong>My take:</strong> I like that the data-hosting difference is concrete.</p><p>I can take it to a security or compliance team and ask whether it matters. For everything else, I would run my own suite during the trial rather than trying to extract wisdom from two G2 reviews.</p><h3>8. Katalon</h3><p>Katalon sits in a different part of the market. It combines full-code, low-code and no-code test creation on top of a Selenium-compatible foundation.</p><p>It also includes record and playback, object spying, self-healing selectors and AI-generated XPath.</p><p>Katalon has a <strong>free tier</strong>, paid plans and a <strong>30-day Studio Enterprise trial</strong>. Its G2 rating is <strong>4.4/5 from 228 reviews</strong>.</p><p>That gives me enough feedback volume to take the rating seriously as one input, especially if I care about usability for mixed-skill QA teams. Still, I would not buy it based on ease-of-use reviews alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x4cyDmVaYN_9Z4ikSfZm6g.png" /></figure><p><strong>My take:</strong> I would evaluate Katalon when the team needs automation but neither the testers nor developers want to build the entire suite from code.</p><p>I would not use a login flow as the trial project. I would give it one of the ugliest workflows in the product. I want to know what the abstraction feels like once the test has conditionals, awkward selectors, pop-ups and a third-party dependency.</p><p>That tells me far more than watching record-and-playback handle a happy path.</p><h3>9. Nightwatch.js</h3><p>Nightwatch is a Node.js end-to-end framework built around the W3C WebDriver API.</p><p>It gives me an integrated runner, assertions and page object support rather than making me assemble those pieces separately.</p><p>I can point it at local browser drivers or hosted grids through the same configuration approach.</p><p>Nightwatch is <strong>free and open source</strong>. It currently has <strong>no G2 listing</strong>, so I would not pretend there is a useful review score to compare. I would look instead at documentation, GitHub activity, integrations and whether the smaller community creates friction for the workflows I care about.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1yGK9JSx-EjEWSnv9IrwCw.png" /></figure><p><strong>My take:</strong> The integrated setup is the reason I would consider Nightwatch.</p><p>I can avoid spending the first week wiring together a runner, assertion library and reporting stack.</p><p>The trade-off is ecosystem size. Its community is smaller than Playwright’s or WebdriverIO’s, and that becomes noticeable when I hit an unusual problem and start searching for somebody who has already solved it.</p><h3>10. Puppeteer</h3><p>Puppeteer supports Chrome for Testing and stable Firefox. That is basically where its cross-browser story stops.</p><p>Version 25.11.0 pairs with Chrome for Testing 153 and Firefox 155, and the project publishes a version map.</p><p>Puppeteer is <strong>free and open source</strong>. Its G2 rating is <strong>5.0/5</strong>, but that figure comes from exactly <strong>one review</strong>.</p><p>I would ignore the star score completely in this case. One person’s excellent experience is useful to that person. It is not much of a benchmark.</p><p>I still like Puppeteer for browser automation built around Chrome DevTools Protocol use cases such as PDF generation, performance work and scraping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vr7EYmA3ZrqNg3hMOepY1A.png" /></figure><p><strong>My take:</strong> I would not choose Puppeteer because I need broad cross-browser coverage.</p><p>I include it because teams often put it in the same comparison anyway.</p><p>For Chrome-specific automation, tracing and PDF workflows, it remains very capable. If the requirement says Safari, Edge and broad browser parity, I can rule it out before writing the first test.</p><h3>Which Tool I’d Choose for Each Testing Workflow</h3><p>I find the workflow usually makes the decision faster than a giant scorecard.</p><ul><li><strong>If I have a Chrome-first product and developers write the tests:</strong> I would start with Cypress or Playwright. I would lean toward Playwright once Firefox or WebKit needs to sit in the same workflow.</li><li><strong>If I need real branded browsers:</strong> I would use Selenium WebDriver, or WebdriverIO when I want more of the runner provided for me. I would put the suite on a hosted grid when macOS and Safari infrastructure becomes expensive to maintain.</li><li><strong>If my testers do not write code:</strong> I would put Katalon through a serious trial using an awkward production workflow rather than a demo flow.</li><li><strong>If I have no Mac fleet and do not want to maintain a grid:</strong> I would compare BrowserStack Automate, Sauce Labs and TestingBot based on the constraint blocking me, whether that is Safari coverage, security requirements or data residency.</li></ul><h3>How I’d Validate My Final Two Choices</h3><p>I would never make the final decision from a comparison article alone, including this one. Consider two tools and give them the same difficult workload.</p><ul><li><strong>Automate the hardest realistic flow I have:</strong> I would pick something with an iframe, file upload, date picker or third-party payment step instead of another login page.</li><li><strong>Deliberately break the test:</strong> I would change a selector and compare how quickly each tool helps me understand why it failed.</li><li><strong>Run the browser my users use:</strong> If Safari matters, I would put a real Safari result beside the WebKit result and see whether the difference affects me.</li><li><strong>Check how hard it is to leave:</strong> I would look at the actual test files and ask how much of the suite survives if I change tools a year later.</li></ul><p>Most teams I talk to, don’t really have a generic browser-coverage problem. They have one or two browsers that makes their decision harder than the rest. Very often, that browser is Safari. Once the constraint is identified, the shortlist gets much smaller and much more useful.</p><p><em>So the question I would answer before choosing anything is simple: </em><strong><em>Am I testing the browser my users opened, or only the engine underneath it?</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b2901d090cc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud-Based Test Automation Grids: Setup, Parallel Execution, and Optimization]]></title>
            <link>https://jennykeeper.medium.com/cloud-based-test-automation-grids-setup-parallel-execution-and-optimization-4eefc789a6b9?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4eefc789a6b9</guid>
            <category><![CDATA[cloud-testing-automation]]></category>
            <category><![CDATA[parallel-testing]]></category>
            <category><![CDATA[automation-grid]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[cloud-based-testing]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 20:08:06 GMT</pubDate>
            <atom:updated>2026-10-14T20:08:06.451Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ThgnB_CRMEyy8cyg7xbbg.png" /></figure><p>A cloud based test automation grid gives me browsers on demand. It does not make my tests run at the same time, and I think that gap is where teams lose the speed they paid for.</p><p>If I point a one-at-a-time test suite at a cloud grid, it still runs one test at a time. Each test now also waits for a remote browser to start. The grid supplies browsers, and my test runner decides how many of them I actually use.</p><p>In this blog, I explain how to build a small suite, run it with three workers, and show how I check that tests really ran side by side. I also compare two self-hosted grid options for teams that want to run their own.</p><h3>How Cloud-Based Test Automation Grids Work?</h3><p>Three parts share the work, and I find that mixing them up is a common reason parallel automation testing on cloud feels slow. The test runner decides which test runs next, the grid finds a browser for it, and the browser session carries out that one test’s commands.</p><p>A few plain definitions help before the details:</p><ul><li><strong>Session:</strong> One browser that the grid opens for one test, identified by a unique session ID until the test closes it.</li><li><strong>Capabilities:</strong> The wish list a test sends with its request, such as Chrome, the latest version and Windows 11.</li><li><strong>Worker:</strong> A separate copy of the test runner that picks up tests on its own, so three workers can drive three browsers at once.</li><li><strong>Slot:</strong> One unit of grid capacity, meaning one browser that can be open at the same moment.</li></ul><p>Selenium’s Grid documentation splits a grid into a Router, Distributor, Session Map, New Session Queue, Event Bus and Node. One test passes through them in this order:</p><ol><li>The test sends POST /session with its capabilities.</li><li>The Router puts that request in the New Session Queue, which works like a waiting line.</li><li>The Distributor looks for a free slot whose browser matches the capabilities.</li><li>The Node starts the browser and returns a session ID, and the Session Map writes down which Node owns it.</li><li>Every later command, such as “click this button”, carries the session ID, so the Router sends it to the right Node.</li><li>driver.quit() sends DELETE /session/&lt;id&gt;, which closes the browser and frees the slot.</li></ol><p><strong>Fun fact:</strong> A self-hosted Selenium Grid does not turn a request away when every slot is busy. It retries every 15 seconds and only gives up with “New session request timed out” after its default wait of 300 seconds.</p><h3>Choose Where Your Test Automation Grid Runs</h3><p>Every option below runs those same six parts. What changes for me is who looks after them, how I add capacity, and how I read how busy the grid is.</p><p>My working example in this article uses BrowserStack Automate, and switching to a self-hosted grid means changing one URL and removing the vendor settings block.</p><h3>1. Selenium Grid on Docker</h3><p>A single container gives a small team its own grid: docker run -d -p 4444:4444 -p 7900:7900 — shm-size=2g -e SE_NODE_MAX_SESSIONS=3 -e SE_NODE_OVERRIDE_MAX_SESSIONS=true selenium/standalone-all-browsers:4.47.0–20260808.</p><p>Each flag has a job. — shm-size=2g gives Chrome enough shared memory, because it crashes tabs when Docker’s default 64 MB fills up. The two SE_NODE_* settings raise the Node from its default of one browser to three, and port 7900 lets you watch a live browser in your own browser.</p><p>The Grid has no login by default. On a cloud VM I keep port 4444 behind a firewall rule, because anyone who reaches it can start browsers on your machine.</p><h3>2. Selenium Grid on Kubernetes With KEDA</h3><p>I find Kubernetes suits several teams sharing one grid. The docker-selenium Helm chart 0.58.0 installs the grid with helm install selenium-grid docker-selenium/selenium-grid — version 0.58.0 — set autoscaling.enabled=true, and that setting also installs KEDA 2.20.1, a tool that adds browsers when the waiting line grows.</p><p>By default KEDA checks the queue every 10 seconds, starts from zero browsers, and creates one short-lived Kubernetes Job per test. That keeps idle cost low, but each test can wait up to 10 seconds to be noticed and then wait again for its container to start.</p><h3>3. BrowserStack Automate</h3><p>This is a managed grid, which means the provider runs all six parts and you get one address: <a href="https://hub.browserstack.com/wd/hub.">https://hub.browserstack.com/wd/hub.</a> You pay for a number of parallels, meaning the number of browsers you may keep open at once across your whole team.</p><p>Its Plan API at <a href="https://api.browserstack.com/automate/plan.json">https://api.browserstack.com/automate/plan.json</a> reports parallel_sessions_running, parallel_sessions_max_allowed and queued_sessions. Plans with 1 to 5 parallels can queue 5 extra tests, larger plans can queue as many as they run, and a queued test is dropped after 15 minutes.</p><p>The limitation I plan around is that the parallel count is shared by everyone on my account. A teammate’s nightly build can take your slots, so your worker count has to leave room for theirs. A private staging app also needs the BrowserStack Local tunnel before a cloud browser can reach it.</p><h3>Configure a Cloud Grid With Selenium and pytest</h3><p>The example is two Python files and one helper script. I run three tests on two browsers, which gives six test runs, and one environment variable tells the code where the grid lives.</p><h3>Install Dependencies and Configure Authentication</h3><p>These versions were current in August 2026: Python 3.14, Selenium 4.47.0, pytest 9.1.0 and pytest-xdist 3.8.0. I use pytest to run the tests, and pytest-xdist is the plugin that adds workers.</p><pre>pip install selenium==4.47.0 pytest==9.1.0 pytest-xdist==3.8.0<br>export BROWSERSTACK_USERNAME=&quot;your_username&quot;<br>export BROWSERSTACK_ACCESS_KEY=&quot;your_access_key&quot;<br>curl -s -u &quot;$BROWSERSTACK_USERNAME:$BROWSERSTACK_ACCESS_KEY&quot; \<br>  https://api.browserstack.com/automate/plan.json</pre><p>Keep the username and key in environment variables or your CI secret store, never in the code. The curl call is a safe first check: if it returns your plan name and parallel count, the credentials work before any browser starts.</p><h3>Define Browser Capabilities and Session Fixtures</h3><p>A fixture is a small piece of setup code that pytest runs before each test and cleans up after it. This one opens a fresh cloud browser for every test, labels it so I can find it later, and records timing data for the checks at the end.</p><p>I give every test its own browser even though startup costs a few seconds each time. Sharing one browser between tests is the first thing I suspect when my suite passes alone and fails in parallel.</p><pre># conftest.py<br>import json, os, time, pytest<br>from selenium import webdriver<br><br>HUB = &quot;https://hub.browserstack.com/wd/hub&quot;<br>BROWSERS = {&quot;chrome&quot;: webdriver.ChromeOptions, &quot;firefox&quot;: webdriver.FirefoxOptions}<br><br>@pytest.hookimpl(hookwrapper=True)<br>def pytest_runtest_makereport(item, call):<br>    outcome = yield<br>    report = outcome.get_result()<br>    setattr(item, f&quot;rep_{report.when}&quot;, report)   # lets the fixture see pass/fail<br><br>@pytest.fixture(params=list(BROWSERS))<br>def driver(request, worker_id):<br>    opts = BROWSERS[request.param]()<br>    opts.browser_version = &quot;latest&quot;<br>    opts.set_capability(&quot;bstack:options&quot;, {<br>        &quot;os&quot;: &quot;Windows&quot;, &quot;osVersion&quot;: &quot;11&quot;,<br>        &quot;userName&quot;: os.environ[&quot;BROWSERSTACK_USERNAME&quot;],<br>        &quot;accessKey&quot;: os.environ[&quot;BROWSERSTACK_ACCESS_KEY&quot;],<br>        &quot;buildName&quot;: os.environ.get(&quot;BUILD_NAME&quot;, &quot;grid-baseline&quot;),<br>        &quot;sessionName&quot;: f&quot;{request.node.name} [{worker_id}]&quot;,<br>        &quot;idleTimeout&quot;: 60})          # frees the slot if a test hangs<br>    asked = time.time()<br>    drv = webdriver.Remote(command_executor=HUB, options=opts)<br>    ready = time.time()<br>    yield drv<br>    rep = getattr(request.node, &quot;rep_call&quot;, None)<br>    status = &quot;passed&quot; if rep and rep.passed else &quot;failed&quot;<br>    drv.execute_script(&quot;browserstack_executor: &quot; + json.dumps({<br>        &quot;action&quot;: &quot;setSessionStatus&quot;, &quot;arguments&quot;: {&quot;status&quot;: status}}))<br>    drv.quit()<br>    with open(&quot;sessions.log&quot;, &quot;a&quot;) as log:<br>        log.write(f&quot;{request.node.name},{worker_id},{drv.session_id},&quot;<br>                  f&quot;{asked:.2f},{ready:.2f},{time.time():.2f}\n&quot;)</pre><p>Each part of the fixture has a reason:</p><ul><li><strong>The report hook:</strong> pytest normally hides a test’s result from its fixtures, so the hook stores it on the test item and the cleanup code can read it.</li><li><strong>The session name:</strong> Adding the worker ID, such as gw1, lets you see in the dashboard which worker drove which browser.</li><li><strong>The idle timeout:</strong> If a test freezes, the grid closes its browser after 60 seconds of silence, so a stuck test cannot hold a paid slot for long.</li><li><strong>The status call:</strong> Without it, the dashboard marks every session as completed, even the ones whose assertions failed.</li><li><strong>The log line:</strong> Each line stores when the browser was requested, when it was ready and when it was closed, which is everything the checks below need.</li></ul><h3>Create Independent Tests for Parallel Execution</h3><p>I treat a test as independent when it passes no matter which worker runs it, in what order, or what else is running at the same time. Each test below opens its own page, logs in or clicks on its own, and checks one visible result.</p><pre># test_smoke.py<br>from selenium.webdriver.common.by import By<br>from selenium.webdriver.support.ui import Select<br><br>BASE = &quot;https://the-internet.herokuapp.com&quot;<br><br>def test_login(driver):<br>    driver.get(f&quot;{BASE}/login&quot;)<br>    driver.find_element(By.ID, &quot;username&quot;).send_keys(&quot;tomsmith&quot;)<br>    driver.find_element(By.ID, &quot;password&quot;).send_keys(&quot;SuperSecretPassword!&quot;)<br>    driver.find_element(By.CSS_SELECTOR, &quot;button[type=submit]&quot;).click()<br>    assert &quot;secure area&quot; in driver.find_element(By.ID, &quot;flash&quot;).text<br><br>def test_checkbox(driver):<br>    driver.get(f&quot;{BASE}/checkboxes&quot;)<br>    box = driver.find_elements(By.CSS_SELECTOR, &quot;#checkboxes input&quot;)[0]<br>    box.click()<br>    assert box.is_selected()<br><br>def test_dropdown(driver):<br>    driver.get(f&quot;{BASE}/dropdown&quot;)<br>    Select(driver.find_element(By.ID, &quot;dropdown&quot;)).select_by_value(&quot;2&quot;)<br>    selected = Select(driver.find_element(By.ID, &quot;dropdown&quot;)).first_selected_option<br>    assert selected.text == &quot;Option 2&quot;<br>The practice site is public, so my cloud browsers reach it without a tunnel. Real suites break independence in a few predictable ways, and the grid cannot fix any of them for me.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G9ra3VF2E7ySHHWTL-eyvg.png" /></figure><h3>Execute Tests Sequentially and in Parallel</h3><p>For each run below, I change one thing only: the number of workers. The browsers, tests and website stay the same, which is the only way I can compare the timings fairly.</p><h3>Establish a Sequential Execution Baseline</h3><p>The baseline is my “before” number. I run the six tests on a single worker, five times, and keep the middle value of pytest’s total run time.</p><pre>for i in 1 2 3 4 5; do BUILD_NAME=run-1w pytest -q test_smoke.py | tail -1; done</pre><p>I use the middle value, the median, because cloud timings wobble from run to run. One slow browser start can distort a single run, and I do not trust a speedup figure built on one sample.</p><h3>Configure Parallel Workers and Verify Concurrent Sessions</h3><p>The -n 3 flag tells pytest-xdist to start three workers. By default it hands each worker a small batch of tests up front, so with only six tests I add — maxschedchunk 1, which makes each free worker take exactly one test at a time.</p><p>To prove the tests really overlapped, I watch the plan’s live numbers during the run with a short helper script. It polls the Plan API every 2 seconds and prints the highest counts it saw.</p><pre># watch_capacity.py<br>import base64, json, os, time, urllib.request<br><br>creds = f&quot;{os.environ[&#39;BROWSERSTACK_USERNAME&#39;]}:{os.environ[&#39;BROWSERSTACK_ACCESS_KEY&#39;]}&quot;<br>headers = {&quot;Authorization&quot;: &quot;Basic &quot; + base64.b64encode(creds.encode()).decode()}<br>peak_running = peak_queued = 0<br>stop_at = time.time() + int(os.environ.get(&quot;WATCH_SECONDS&quot;, &quot;90&quot;))<br>while time.time() &lt; stop_at:<br>    req = urllib.request.Request(&quot;https://api.browserstack.com/automate/plan.json&quot;, headers=headers)<br>    plan = json.load(urllib.request.urlopen(req, timeout=10))<br>    peak_running = max(peak_running, plan[&quot;parallel_sessions_running&quot;])<br>    peak_queued = max(peak_queued, plan[&quot;queued_sessions&quot;])<br>    time.sleep(2)<br>print(f&quot;peak running={peak_running} peak queued={peak_queued} &quot;<br>      f&quot;limit={plan[&#39;parallel_sessions_max_allowed&#39;]}&quot;)</pre><pre>python watch_capacity.py &amp; BUILD_NAME=run-3w pytest -n 3 --maxschedchunk 1 test_smoke.py; wait</pre><p>A clean result for my run shows a peak of 3 running and 0 queued. The Plan API counts my whole team, so if a colleague runs tests at the same moment, the running number includes their browsers too.</p><p>The log file gives me a second, independent check. This script reads sessions.log, counts how many browsers were open at the same instant, and reports the median startup time.</p><pre># overlap.py<br>import csv, statistics<br>events, startups = [], []<br>for name, worker, sid, asked, ready, closed in csv.reader(open(&quot;sessions.log&quot;)):<br>    events += [(float(ready), 1), (float(closed), -1)]<br>    startups.append(float(ready) - float(asked))<br>open_now = peak = 0<br>for _, change in sorted(events):<br>    open_now += change<br>    peak = max(peak, open_now)<br>print(f&quot;peak overlap={peak} median startup={statistics.median(startups):.1f}s&quot;)</pre><p>I delete sessions.log between runs so each result covers one build only. When both scripts report 3, I know the three workers truly ran three browsers side by side.</p><h3>Optimize Worker Count Against Grid Capacity</h3><p>More workers help me until they run out of useful work.</p><p>The table below is a worked model, built from assumed test times of 12 to 18 seconds each including browser startup, 86 seconds in total, plus 3 seconds of pytest overhead. I treat it as the shape to expect, then replace it with my own medians.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mv7uuI5j4PJjHYL0gUpniw.png" /></figure><p>Going from three workers to four saves only 4 seconds and more than doubles the idle time. Six tests cannot be split evenly across four workers, so two workers finish early and sit waiting while the rest catch up.</p><p>Two readings tell me what limits my own suite:</p><ul><li><strong>Peak running below the worker count:</strong> Your plan limit or a teammate’s build is holding slots, so extra workers only wait in the queue.</li><li><strong>One worker still running long after the others:</strong> A single slow test sets the finish line, and — dist worksteal can rebalance the waiting tests but cannot split that one test.</li></ul><p>My rule is simple. I set workers to the plan’s parallel limit minus what teammates need at the same hour. Anything above that sits in a queue that drops tests after 15 minutes, which turns a speed problem into a failure.</p><h3>Cloud Grid Validation Checklist</h3><p>A green build does not yet prove that the grid is set up well. I sign off on an automation testing grid on cloud only when my logs show these four things, because a faster total can hide a skipped test.</p><ul><li><strong>Every combination ran:</strong> sessions.log has six lines per build, one for each test on each browser, with nothing missing or repeated.</li><li><strong>Browsers overlapped:</strong> overlap.py and watch_capacity.py both report a peak equal to the worker count, and the dashboard shows the same session names.</li><li><strong>Browsers were released:</strong> Every log line has a close time, and the Plan API shows zero running sessions once the build ends.</li><li><strong>Speed improved without new failures:</strong> The median total time fell against the one-worker baseline, and the failure count stayed the same.</li></ul><p>My position is that worker count is a budget you measure against your own plan and your own slowest test. A number copied from a blog post or a default setting carries someone else’s limits, and I would rather defend a number I measured.</p><p>I keep coming back to one question: what worker count does my suite run at today, and when did I last measure it?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4eefc789a6b9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Test a Local Mobile Web App on Real Phones]]></title>
            <link>https://jennykeeper.medium.com/how-to-test-a-local-mobile-web-app-on-real-phones-06eaf7b101ed?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/06eaf7b101ed</guid>
            <category><![CDATA[api-verification]]></category>
            <category><![CDATA[real-device-testing]]></category>
            <category><![CDATA[local-development]]></category>
            <category><![CDATA[mobile-web-apps]]></category>
            <category><![CDATA[software-testing]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Sun, 11 Oct 2026 07:56:01 GMT</pubDate>
            <atom:updated>2026-10-11T07:56:01.518Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A developer tests a local mobile web app on multiple iOS and Android devices using a laptop connected to a cloud-based device testing environment." src="https://cdn-images-1.medium.com/max/1024/1*GQQ8agV_WRsH8gbMXHJbkw.png" /></figure><p>When I test a local web app on my phone, opening the page is the easy part. What I care about is what happens after I tap Save.</p><p>Did the request reach my laptop? Did the API store the right data? If I reload, will the record still be there, or was that reassuring confirmation message getting ahead of itself?</p><p>That’s where I’d focus when it comes to local mobile testing. A form can look completely functional while its API requests are going to the wrong machine.</p><p>A success message tells me what the interface believes. I still want to know what the server saved.</p><p>Here’s how I’d connect a real phone to a local app, choose an automation setup, and follow a submission all the way to the backend.</p><h3>Why a Mobile App Can Load While Its Local API Fails</h3><p>The first thing I check is the API URL. On your laptop, localhost points back to your laptop. On your phone, it points back to your phone, unless you’ve configured forwarding to change that route.</p><p>Say you open <a href="http://192.168.1.20:5000/">http://192.168.1.20:5000</a> on your phone. The page loads from your laptop, but its JavaScript sends a request to <a href="http://localhost:5000/api/records.">http://localhost:5000/api/records.</a> That request targets the phone instead.</p><p>This is why I look at the page request and the API request separately. Loading one tells me very little about the other.</p><p>For this example, I’d serve the page and API from the same origin and call a relative path:</p><pre>const response = await fetch(&#39;/api/records&#39;, {<br>  method: &#39;POST&#39;,<br>  headers: { &#39;Content-Type&#39;: &#39;application/json&#39; },<br>  body: JSON.stringify({ note })<br>});<br><br>if (!response.ok) {<br>  throw new Error(`Save failed: ${response.status}`);<br>}<br><br>const record = await response.json();</pre><p>Now the request follows the origin the phone actually opened. There’s no laptop address baked into the browser code.</p><p><strong>Did you know?</strong> crypto.randomUUID() requires a secure context. HTTP localhost can qualify on your laptop, while a plain HTTP LAN address on your phone generally doesn’t. I’d check window.isSecureContext before blaming a UUID-related failure on the device.</p><h3>Prepare a Local App You Can Actually Test</h3><p>I’d keep the example small: one note field, one Save button, and an API that stores the note. That gives me a clear path to debug without dragging an entire application into the first connectivity check.</p><p>These are the behaviors I’d build into it:</p><ul><li>One browser-facing origin: Serve the page and /api/records together, or proxy the API through the frontend’s development server.</li><li>A server-generated ID: Return the ID after storing the note, then put it in the page URL so the record can be retrieved after a reload.</li><li>A fresh read: On reload, fetch the record by ID. For this example, keep API responses out of browser and service-worker caches.</li><li>A useful log entry: Write created record &lt;id&gt; after the record is stored, so the log describes a completed write.</li><li>A reachable server: Expose /health and bind to 0.0.0.0 when testing over Wi-Fi.</li></ul><p>Before opening the app, I’d visit http://&lt;laptop-IP&gt;:5000/health from the phone. If that fails, I’d sort out the network connection first.</p><p>In-memory storage is fine for this exercise. It proves the running server can return the record, but it doesn’t prove the data survives a server restart. For an application that promises durable saves, I’d verify the database-backed read too.</p><h3>Connect a Real Mobile Browser to Your Local Server</h3><p>I separate connecting the phone from automating it. First, I need a route that lets its browser reach the app.</p><ul><li>Same Wi-Fi network: Bind the server to 0.0.0.0, allow the required port through the laptop firewall, and open http://&lt;laptop-IP&gt;:5000 on the phone. A hardcoded localhost API URL still points to the phone. Guest-network isolation can also prevent the devices from reaching each other.</li><li>USB forwarding on Android: Enable USB debugging, authorize the computer, and run adb reverse tcp:5000 tcp:5000. You can then open <a href="http://localhost:5000/">http://localhost:5000</a> on the phone. Chrome DevTools also supports configuring port forwarding through chrome://inspect.</li><li>An HTTPS tunnel: Run a tunnel client against the local server and open its HTTPS URL on the phone. Keep API requests relative so they follow that URL. The tunnel doesn’t automatically repair a hardcoded API address.</li></ul><figure><img alt="Chrome DevTools showing port forwarding settings for a connected Google Pixel 7, mapping device port 5000 to localhost:5000 with port forwarding enabled." src="https://cdn-images-1.medium.com/max/1024/1*KqeVXhi3AzLKHbi1oAJv7g.png" /></figure><p>I’d start with Wi-Fi for a basic connectivity check, then use forwarding or HTTPS where the workflow needs it. A feature that requires a secure context will need more than an HTTP LAN URL.</p><p>I want the connection setup to reveal a bad API address before it makes that address work by accident.</p><p>USB forwarding is useful. I just wouldn’t let a passing forwarded session be my only check of the app’s URL configuration.</p><h3>Choose Where the Automated Test Runs</h3><p>Once the phone can reach the app, I’d choose the runner based on the browsers I need to cover and the setup I’m willing to maintain.</p><p>The test goal stays the same across these options. Session configuration changes, and Playwright needs its own implementation of the assertions.</p><h3>ChromeDriver on a USB-Connected Android Phone</h3><p>For an Android Chrome check, I’d consider ChromeDriver before introducing a larger mobile automation stack. It can target the phone’s Chrome app through ADB using androidPackage:</p><pre>from selenium import webdriver<br><br>options = webdriver.ChromeOptions()<br>options.add_experimental_option(&#39;androidPackage&#39;, &#39;com.android.chrome&#39;)<br>driver = webdriver.Chrome(options=options)</pre><p>The phone must be connected and authorized, and the installed ChromeDriver must be compatible with its Chrome version. With multiple phones, specify androidDeviceSerial.</p><p><strong>My take:</strong> this is a focused option when Android Chrome is the requirement. I’d keep the scope explicit rather than count it as iPhone coverage.</p><h3>safaridriver With Safari on a Connected iPhone</h3><p>For Safari-only web testing, I’d look at Apple’s built-in safaridriver. It supports a connected iPhone through a Mac, with Remote Automation enabled and the device trusted and unlocked.</p><p>The session uses platformName: “ios” to target iOS.</p><p><strong>My take: </strong>I like this option for a contained Safari web flow. It still needs Mac and device setup, and browser automation won’t cover every interaction with the surrounding operating system.</p><h3>BrowserStack Automate With BrowserStack Local</h3><p>I’d consider BrowserStack Automate when I need real device models I don’t have available. BrowserStack Local connects those cloud sessions to the app running on my machine.</p><p>There’s a relevant detail here: its documentation says localhost navigation URLs on Safari and iOS real devices are rewritten to bs-local.com. Explicit localhost references inside application configuration can therefore cause problems.</p><h3>My take: extra device access is useful once the basic flow works. I’d still inspect the actual API destination and verify the backend result in every environment.</h3><h3>Appium With the UiAutomator2 and XCUITest Drivers</h3><p>I’d consider Appium when the suite needs Android Chrome and iPhone Safari, particularly if the team already maintains mobile tests. UiAutomator2 and XCUITest provide the respective platform drivers.</p><p>The iPhone setup deserves its own time allocation. WebDriverAgent needs a valid provisioning profile and the required device settings before it can run on a physical device.</p><p><strong>My take:</strong> the shared WebDriver approach is useful, but it doesn’t make platform setup disappear. I’d get one reliable session working on each platform before expanding the suite.</p><h3>Playwright’s Android Support</h3><p>If the existing suite uses Playwright, its _android API is worth evaluating for Android Chrome. The documentation labels Android support experimental and requires an authenticated ADB connection.</p><p>I’d port the submission and backend checks into Playwright syntax, then verify the interactions the flow actually needs. This Android API doesn’t provide real iPhone Safari coverage.</p><p><strong>My take:</strong> using familiar tooling can reduce setup work, but I’d still give iPhone Safari a separate test path.</p><h3>Automate a Complete Form Submission and Verify the Backend Result</h3><p>This is the part I’d spend the most care on. A well-connected phone can still run a test that checks too little.</p><p>For the example app, I’d use this sequence:</p><ol><li>Check /health from the test runner with a short timeout. This checks server availability; opening it on the phone checks the device’s route.</li><li>Generate a unique note, such as e2e- plus a random suffix.</li><li>Open the app on the phone, enter the note, and submit.</li><li>Wait for the server-generated ID and capture it.</li><li>Reload the record URL and check that the fresh API read returns the same ID and note.</li><li>Find the matching completed-write entry in the local API log.</li><li>In teardown, delete this run’s record, close the session, and remove any forwarding or tunnel created for the test, even if an assertion fails.</li></ol><p>Here’s the verification portion in Python with Selenium. It assumes driver, record_id, and note already exist, the page renders Record &lt;id&gt;: &lt;note&gt; in #out, and the runner can read the server’s api.log.</p><pre>from pathlib import Path<br>from selenium.webdriver.common.by import By<br>from selenium.webdriver.support.ui import WebDriverWait<br><br>expected = f&quot;Record {record_id}: {note}&quot;<br>driver.refresh()<br><br>WebDriverWait(driver, 20).until(<br>    lambda browser: browser.find_element(By.ID, &quot;out&quot;).text == expected<br>)<br><br>log_path = Path(&quot;api.log&quot;)<br>expected_log = f&quot;created record {record_id}&quot;<br><br>WebDriverWait(driver, 5).until(<br>    lambda _: log_path.exists()<br>    and expected_log in log_path.read_text(encoding=&quot;utf-8&quot;).splitlines(),<br>    message=f&quot;No completed write logged for {record_id}&quot;<br>)</pre><p>A reload alone isn’t proof of a fresh server read. Browser storage, response caches, and service workers can preserve data. That’s why the example’s read behavior matters, and why I’d inspect the GET response when adapting it to an existing app.</p><p>I’d also test a rejected save. Make the API return an error in a controlled test and check that the page shows a failure instead of a success message.</p><p>If my test still passes when the save fails, I’ve tested the reassurance, not the result.</p><h3>Fix “Page Loads, Submission Fails” Errors</h3><p>When this happens, I’d check the request URL and response before changing the automation code. These are the first possibilities I’d work through:</p><ul><li>The phone reports Failed to fetch: Inspect the destination. A hardcoded localhost URL is one possible cause, but the message alone doesn’t establish it. Check connectivity, browser errors, and the actual API address.</li><li>USB passes but Wi-Fi fails: Look for a localhost dependency that forwarding made reachable. Switch browser-side calls to relative paths and repeat the Wi-Fi check.</li><li>The frontend and API use different ports: Ports are part of an origin. With a frontend on :5173 and an API on :5000, I’d proxy /api through the frontend server or configure the required CORS behavior deliberately.</li><li>Vite rejects the tunnel hostname: Add the specific hostname to server.allowedHosts. I’d keep an explicit list rather than disable host checking.</li></ul><p>My debugging order is simple: destination, reachability, browser rejection, then application response. Each check narrows down where the request stopped.</p><h3>Confirm the Same Flow Works on iPhone Safari</h3><p>I’d repeat the full flow on iPhone Safari even if Android passes. For this setup, I’d use the laptop’s Wi-Fi address, an HTTPS tunnel, or a cloud device connection.</p><p>To inspect the request on a connected iPhone:</p><ol><li>Open Safari’s settings on the phone and enable Web Inspector under Advanced.</li><li>On the Mac, open Safari’s Advanced settings and enable its web developer features.</li><li>Connect and trust the iPhone, then open the app in the phone’s Safari browser.</li><li>Select the phone and page from Safari’s Develop menu on the Mac.</li><li>Inspect the submission and subsequent record fetch in the Network tab.</li></ol><p>I want to see the request URL, status, and returned record. If the POST succeeds but the page shows an error, I’d inspect how the client handles the response before assuming the save failed.</p><figure><img alt="Safari Web Inspector showing a form submission and matching record fetch from a connected iPhone." src="https://cdn-images-1.medium.com/max/1024/1*_I331mtRWUThBwootj6Jjw.png" /></figure><h3>Conclusion: A Working Local Setup Includes the Backend</h3><p>“It loads on my phone” is where I’d start the investigation. I’d call the save flow working once the phone submits the note, the server stores it, and a fresh read returns the same record.</p><p>The runner helps me repeat that check. The assertions decide whether the result means anything.</p><p>Start with one form and one phone. Follow a unique submission into the backend, check what happens when the save fails, then repeat the flow on your other target browser.</p><p>What does your suite accept as proof of a save: the message on the screen, or the record on the server?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06eaf7b101ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating Tests on Localhost and Staging Environments]]></title>
            <link>https://jennykeeper.medium.com/automating-tests-on-localhost-and-staging-environments-6ed808abcbcf?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ed808abcbcf</guid>
            <category><![CDATA[low-code-automation]]></category>
            <category><![CDATA[localhost]]></category>
            <category><![CDATA[staging-environments]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[automated-testing]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Fri, 09 Oct 2026 11:11:57 GMT</pubDate>
            <atom:updated>2026-10-09T11:11:57.484Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MYlZeMDRpmbKLeMWTJzXHg.png" /></figure><p>A browser test can open a page, submit a form, and still tell you very little about the deployment you meant to test.</p><p>When I run automated tests on staging websites, I want evidence of three things: the browser reached the intended environment, the expected build loaded, and the server completed the action. A connected tunnel is only part of that evidence.</p><p>In this walkthrough, we’ll use a Python test for localhost and private staging. You can run it in local Chrome, on Selenium Grid, or through a managed browser connection.</p><h3>Why Localhost and Private Staging Need a Different Connection Setup</h3><p>localhost refers to the machine that is making the request. For a browser running on your laptop, that means your laptop is the localhost.</p><p>In the case of a remote browser, the remote machine is the one making the request. A remote browser cannot reach your development server just because both use port 8000.</p><p>Private staging introduces another boundary. Its hostname may resolve only through internal DNS, and its server may accept traffic only from a corporate network or VPN.</p><p>So there are 3 practical ways to handle this situation:</p><ul><li><strong>Run browsers inside the network: </strong>Local browsers or Selenium Grid nodes can access staging directly when their network permits it. Your team maintains browser versions, machine capacity, and connectivity.</li><li><strong>Create a private connection for cloud browsers: </strong>A vendor tunnel, such as BrowserStack Local, which forwards supported requests through a machine that can reach the application. The tunnel can only reach what that machine can already reach.</li><li><strong>Expose a temporary preview:</strong> Cloudflare Quick Tunnels can publish a local server through a generated URL. This method changes the exposure of the application and is suitable only when that public preview is acceptable.</li></ul><p>I would choose the route from the application’s network requirements before choosing a provider. Whichever route you use, check the runner-to-app path, the browser-to-app path, and the browser’s access to any private dependencies separately.</p><h3>Set Up One Test Project for Both Environments</h3><p>Use Python 3.12 and create <strong>demo/app.py, demo/wait_ready.py, tests/conftest.py, and tests/test_form.py</strong> in one project. The examples below supply their contents.</p><p>Create a virtual environment and install the core dependencies:</p><pre>python3 -m venv .venv<br>source .venv/bin/activate<br>python -m pip install flask selenium pytest</pre><p>Install browserstack-local, if you choose that tunnel, and install gunicorn on the Linux staging host for the deployment example. Record the resolved versions in your project’s dependency lockfile after validating the setup.</p><p>Keep the configuration separate from the test logic:</p><ul><li>BASE_URL identifies the application: Use <a href="http://127.0.0.1:8000/">http://127.0.0.1:8000</a> locally and the deployed HTTPS address for staging.</li><li>EXPECTED_BUILD identifies the intended deployment: Use the build identifier supplied by the deployment job, rather than reading whichever identifier staging currently returns.</li><li>TEST_EMAIL and TEST_TOKEN supply dedicated test credentials: The demo checks these on submission. They are separate from any cloud-provider credentials.</li><li>BROWSER_TARGET selects execution: Set it to local, grid, or browserstack. Grid also needs SELENIUM_REMOTE_URL.</li></ul><p>The demo is deliberately small. Its credential check demonstrates authenticated submission; it does not model a production login system, session management, or SSO.</p><h3>Run Your First End-to-End Test on Localhost</h3><p>Start with the application and browser on one machine. Once that passes, change the connection route while keeping the same test. This separates application failures from remote-network failures.</p><h3>Start a Minimal Demo App and Wait for Readiness</h3><p>Save the following as demo/app.py. The health endpoint reports the running build, and the submission route validates the test account before returning a result.</p><pre>import hmac<br>import os<br>from flask import Flask, abort, render_template_string, request<br><br>app = Flask(__name__)<br>BUILD = os.environ[&quot;BUILD_ID&quot;]<br>EMAIL = os.environ[&quot;APP_TEST_EMAIL&quot;]<br>TOKEN = os.environ[&quot;APP_TEST_TOKEN&quot;]<br><br>PAGE = &quot;&quot;&quot;&lt;!doctype html&gt;&lt;html&gt;&lt;body&gt;<br>&lt;p id=&quot;build&quot;&gt;{{ build }}&lt;/p&gt;<br>{% if email %}<br>&lt;p id=&quot;result&quot;&gt;Received {{ email }}&lt;/p&gt;<br>{% else %}<br>&lt;form method=&quot;post&quot; action=&quot;/submit&quot;&gt;<br>  &lt;input id=&quot;email&quot; name=&quot;email&quot; type=&quot;email&quot; required&gt;<br>  &lt;input id=&quot;token&quot; name=&quot;token&quot; type=&quot;password&quot; required&gt;<br>  &lt;button id=&quot;submit&quot; type=&quot;submit&quot;&gt;Send&lt;/button&gt;<br>&lt;/form&gt;<br>{% endif %}<br>&lt;/body&gt;&lt;/html&gt;&quot;&quot;&quot;<br><br>@app.get(&quot;/healthz&quot;)<br>def health():<br>    return {&quot;status&quot;: &quot;ok&quot;, &quot;build&quot;: BUILD}<br><br>@app.get(&quot;/&quot;)<br>def index():<br>    return render_template_string(PAGE, build=BUILD, email=None)<br><br>@app.post(&quot;/submit&quot;)<br>def submit():<br>    email = request.form.get(&quot;email&quot;, &quot;&quot;)<br>    token = request.form.get(&quot;token&quot;, &quot;&quot;)<br>    if email != EMAIL or not hmac.compare_digest(<br>        token.encode(), TOKEN.encode()<br>    ):<br>        abort(403)<br>    return render_template_string(PAGE, build=BUILD, email=email)<br><br>if __name__ == &quot;__main__&quot;:<br>    app.run(host=&quot;127.0.0.1&quot;, port=8000)</pre><p>The form uses a relative action, so it submits to the same origin that served the page. It does not accidentally send a staging submission back to a hard-coded localhost address.</p><p>Save this readiness check as demo/wait_ready.py:</p><pre>import json<br>import sys<br>import time<br>from urllib.request import urlopen<br><br>url, expected = sys.argv[1:3]<br>deadline = time.monotonic() + 30<br>last_error = &quot;No response was received.&quot;<br>while time.monotonic() &lt; deadline:<br>    try:<br>        with urlopen(url, timeout=2) as response:<br>            body = json.load(response)<br>        if body.get(&quot;status&quot;) == &quot;ok&quot; and body.get(&quot;build&quot;) == expected:<br>            break<br>        last_error = f&quot;Unexpected health response: {body}&quot;<br>    except (OSError, ValueError) as error:<br>        last_error = str(error)<br>    time.sleep(0.5)<br>else:<br>    raise SystemExit(f&quot;Readiness failed: {last_error}&quot;)</pre><p>For a local-only demonstration, start the app in one terminal:</p><pre>export BUILD_ID=local-dev<br>export APP_TEST_EMAIL=qa-local@example.com<br>export APP_TEST_TOKEN=local-demo-only<br>python demo/app.py</pre><p>In another terminal, activate the virtual environment and check readiness:</p><pre>python demo/wait_ready.py http://127.0.0.1:8000/healthz local-dev</pre><p>Checking the identifier prevents a stale server on the same port from satisfying readiness. It proves access from the runner, however; the browser still needs its own build assertion.</p><h3>Connect the Browser Through Your Chosen Network Route</h3><p>The following fixture makes local execution the default. Its Grid branch connects to infrastructure you manage, while its optional cloud branch starts one tunnel for the test session. Save it as tests/conftest.py.</p><pre>import os<br>from uuid import uuid4<br>import pytest<br>from selenium import webdriver<br><br>@pytest.fixture(scope=&quot;session&quot;)<br>def route():<br>    target = os.environ.get(&quot;BROWSER_TARGET&quot;, &quot;local&quot;)<br>    if target not in {&quot;local&quot;, &quot;grid&quot;, &quot;browserstack&quot;}:<br>        raise ValueError(f&quot;Unknown browser target: {target}&quot;)<br>    if target != &quot;browserstack&quot;:<br>        yield target, None<br>        return<br><br>    from browserstack.local import Local<br>    identifier = f&quot;e2e-{uuid4().hex}&quot;<br>    tunnel = Local()<br>    args = {<br>        &quot;key&quot;: os.environ[&quot;BROWSERSTACK_ACCESS_KEY&quot;],<br>        &quot;localIdentifier&quot;: identifier,<br>    }<br>    if os.environ.get(&quot;TUNNEL_FORCE_LOCAL&quot;) == &quot;true&quot;:<br>        args[&quot;forcelocal&quot;] = &quot;true&quot;<br>    try:<br>        tunnel.start(**args)<br>        if not tunnel.isRunning():<br>            raise RuntimeError(&quot;The tunnel did not report ready.&quot;)<br>        print(f&quot;Tunnel ready: {identifier}&quot;)<br>        yield target, identifier<br>    finally:<br>        if tunnel.isRunning():<br>            tunnel.stop()<br><br>@pytest.fixture<br>def driver(route):<br>    target, identifier = route<br>    options = webdriver.ChromeOptions()<br>    if target == &quot;local&quot;:<br>        options.add_argument(&quot;--headless=new&quot;)<br>        browser = webdriver.Chrome(options=options)<br>    elif target == &quot;grid&quot;:<br>        browser = webdriver.Remote(<br>            command_executor=os.environ[&quot;SELENIUM_REMOTE_URL&quot;],<br>            options=options,<br>        )<br>    else:<br>        options.browser_version = os.environ.get(&quot;BROWSER_VERSION&quot;, &quot;latest&quot;)<br>        options.set_capability(&quot;bstack:options&quot;, {<br>            &quot;os&quot;: &quot;Windows&quot;, &quot;osVersion&quot;: &quot;11&quot;,<br>            &quot;userName&quot;: os.environ[&quot;BROWSERSTACK_USERNAME&quot;],<br>            &quot;accessKey&quot;: os.environ[&quot;BROWSERSTACK_ACCESS_KEY&quot;],<br>            &quot;local&quot;: True, &quot;localIdentifier&quot;: identifier,<br>            &quot;buildName&quot;: os.environ[&quot;EXPECTED_BUILD&quot;],<br>        })<br>        browser = webdriver.Remote(<br>            command_executor=&quot;https://hub-cloud.browserstack.com/wd/hub&quot;,<br>            options=options,<br>        )<br>    try:<br>        browser.set_page_load_timeout(30)<br>        yield browser<br>    finally:<br>        browser.quit()</pre><p>Local execution requires Chrome on the runner. For Grid, the nodes must have Chrome capacity and be able to reach BASE_URL. A Grid container’s loopback address is not the runner’s loopback address; use reachable service names and server bindings when crossing containers. Selenium Grid documentation</p><p>For the cloud branch, install browserstack-local and supply the two provider credentials from your secret store. The generated identifier is passed to both the tunnel and browser capabilities so the session selects the intended connection. Multiple-connection documentation</p><p>isRunning() checks tunnel status, not application access. The library exposes explicit start and stop operations, and forcelocal routes traffic through the tunnel host when required. Python tunnel binding</p><p>A public preview is another route for this synthetic demo. With cloudflared installed, run cloudflared tunnel — url <a href="http://127.0.0.1:8000/">http://127.0.0.1:8000</a> and use its generated HTTPS address as BASE_URL. Remote browsers still need a separate execution environment. This publishes the demo; it does not automatically make other private API hosts reachable.</p><h3>Verify the Build, Submit the Form, and Assert the Result</h3><p>Save the shared test as tests/test_form.py. It verifies the build before submission and checks both the response and build afterward.</p><pre>import os<br>from selenium.webdriver.common.by import By<br>from selenium.webdriver.support import expected_conditions as EC<br>from selenium.webdriver.support.ui import WebDriverWait<br><br>def test_form_submission(driver):<br>    expected = os.environ[&quot;EXPECTED_BUILD&quot;]<br>    email = os.environ[&quot;TEST_EMAIL&quot;]<br>    base = os.environ[&quot;BASE_URL&quot;].rstrip(&quot;/&quot;)<br>    wait = WebDriverWait(driver, 15)<br>    driver.get(base + &quot;/&quot;)<br>    build = wait.until(EC.visibility_of_element_located((By.ID, &quot;build&quot;)))<br>    assert build.text == expected, f&quot;Wrong build: {build.text}&quot;<br>    wait.until(EC.element_to_be_clickable((By.ID, &quot;email&quot;))).send_keys(email)<br>    driver.find_element(By.ID, &quot;token&quot;).send_keys(os.environ[&quot;TEST_TOKEN&quot;])<br>    driver.find_element(By.ID, &quot;submit&quot;).click()<br>    result = wait.until(EC.visibility_of_element_located((By.ID, &quot;result&quot;)))<br>    assert result.text == f&quot;Received {email}&quot;<br>    assert driver.find_element(By.ID, &quot;build&quot;).text == expected</pre><p>Run it from the second terminal:</p><pre>export BASE_URL=http://127.0.0.1:8000<br>export EXPECTED_BUILD=local-dev<br>export TEST_EMAIL=qa-local@example.com<br>export TEST_TOKEN=local-demo-only<br>BROWSER_TARGET=local python -m pytest -s tests/test_form.py</pre><p>Select grid or browserstack after configuring that route. The test remains unchanged. A successful submission proves the demo server validated the credentials and returned the expected response; it does not claim database persistence because the demo stores nothing.</p><p>The browser fixture closes its session after failures, and the session fixture stops its tunnel afterward. Forced process termination can bypass teardown, so CI timeouts and remote-session expiry are still needed. Stop the manually started app with Ctrl+C after the run.</p><h3>Run the Same Test Against Private Staging</h3><p>Staging environment automation testing should exercise the deployment you intended to validate. Use the build identifier produced by deployment, not an unrelated test-repository commit.</p><h3>Deploy the Same Demo and Change the Environment Configuration</h3><p>On a Linux staging host, serve the same app with a WSGI server behind your existing HTTPS reverse proxy. Flask’s development server is intended for development, not deployed operation. Flask deployment guidance.</p><pre># Run on the staging host after deploying the project files.<br>python -m pip install flask gunicorn<br>export BUILD_ID=&quot;$DEPLOYED_SHA&quot;<br>export APP_TEST_EMAIL=&quot;$STAGING_TEST_EMAIL&quot;<br>export APP_TEST_TOKEN=&quot;$STAGING_TEST_TOKEN&quot;<br>gunicorn --chdir demo --bind 127.0.0.1:8000 app:app</pre><p>The reverse proxy must forward the staging hostname to that listener. This command starts the application; your deployment system remains responsible for transferring files, TLS, process supervision, and rollback.</p><p>On a runner with staging access, change the test configuration:</p><pre>export BASE_URL=https://staging.internal.example.com<br>export EXPECTED_BUILD=&quot;$DEPLOYED_SHA&quot;<br>export TEST_EMAIL=&quot;$STAGING_TEST_EMAIL&quot;<br>export TEST_TOKEN=&quot;$STAGING_TEST_TOKEN&quot;<br>python demo/wait_ready.py &quot;$BASE_URL/healthz&quot; &quot;$EXPECTED_BUILD&quot;<br>BROWSER_TARGET=local python -m pytest -s tests/test_form.py</pre><p>Replace the example hostname with yours. Inject the same dedicated staging credentials into the deployment and test job through secrets, and use HTTPS. They should differ from the local demonstration values.</p><p>This example runs Chrome on a network-connected runner. You can instead select Grid or the tunnel branch when their browser or tunnel hosts have the necessary access.</p><h3>Verify Access to the Page and Its Private Dependencies</h3><p>Automation testing behind a firewall involves more than reaching the HTML document. A frontend can load successfully while its API requests, identity provider, or static assets remain inaccessible.</p><p>Check each boundary separately:</p><ul><li>Check internal DNS from the relevant host: Resolve the staging name from the browser node or tunnel host. A successful lookup on your laptop says nothing about the CI machine.</li><li>Check private dependencies: Verify that required API hosts are reachable, then assert their user-visible outcomes in the browser. Runner-side curl cannot prove browser CORS, cookie, or permission behaviour.</li><li>Check authentication independently: A tunnel provides connectivity. Your real application still needs its login or SSO flow; the demo’s submission token does not bypass those controls.</li><li>Check split DNS and proxies: If a hostname serves different content internally, ensure requests follow the intended route. Configure proxy and routing settings for the selected connection method.</li><li>Check certificate trust: Install the intended CA certificates where appropriate. Disabling certificate checks can hide a deployment problem and should not become the default fix.</li></ul><p>I would investigate the first failed boundary before changing the test’s timeout. Waiting longer cannot correct the wrong DNS answer or missing credentials.</p><h3>Run the Workflow in CI</h3><p>Use the same sequence in CI: start or deploy the app, verify its build, establish the selected route, run assertions, and clean up resources owned by the job.</p><p>The following Bash script runs on a Linux CI agent with the project environment and Chrome installed. Save it as ci-local.sh and invoke it with bash ci-local.sh after checkout and dependency installation.</p><pre>set -euo pipefail<br>export BUILD_ID=&quot;${CI_BUILD_ID:?Set the checked-out build identifier}&quot;<br>export EXPECTED_BUILD=&quot;$BUILD_ID&quot;<br>export BASE_URL=http://127.0.0.1:8000<br>export APP_TEST_EMAIL=&quot;${TEST_EMAIL:?Set the demo test account}&quot;<br>export APP_TEST_TOKEN=&quot;${TEST_TOKEN:?Set the demo test token}&quot;<br>export BROWSER_TARGET=&quot;${BROWSER_TARGET:-local}&quot;<br>python demo/app.py &gt; app.log 2&gt;&amp;1 &amp;<br>app_pid=$!<br>cleanup() {<br>  kill &quot;$app_pid&quot; 2&gt;/dev/null || true<br>  wait &quot;$app_pid&quot; 2&gt;/dev/null || true<br>}<br>trap cleanup EXIT<br>python demo/wait_ready.py &quot;$BASE_URL/healthz&quot; &quot;$EXPECTED_BUILD&quot;<br>python -m pytest -s tests/test_form.py --junitxml=results.xml</pre><p>Local execution needs no cloud credentials. The optional tunnel branch uses the same script with its dependency and secrets supplied, and the fixture owns tunnel cleanup. This loopback-based script does not directly support a separate Grid host; provide a reachable app address and listener for that topology.</p><p>For staging, trigger the test job after successful deployment, supply its deployed SHA, and omit local app startup. Use a runner with approved private-network access. Do not terminate a shared staging application during test cleanup.</p><p>Configure your CI system to retain app.log and results.xml after failures. Capture browser screenshots before fixture teardown when investigating UI failures, and avoid printing credentials or storing unsanitised secrets in artifacts.</p><h3>Troubleshoot Localhost and Staging Failures</h3><p>The first useful question is which process made the failing request. That identifies where to inspect DNS, routing, and credentials.</p><ul><li><strong>The remote browser cannot open localhost:</strong> It may be reaching its own loopback interface. Use a network-reachable application address or the correctly configured tunnel.</li><li><strong>The tunnel is ready but navigation fails: </strong>Check application reachability from the tunnel host, then verify that the browser selected the same connection identifier.</li><li><strong>The page loads but submission fails: </strong>Check credentials, the server response, private API connectivity, and browser console errors before increasing timeouts.</li><li><strong>The build assertion fails: </strong>Verify the deployed artifact, reverse-proxy destination, and concurrent deployments. A stale build is an environment mismatch, not evidence that the new code failed.</li><li><strong>Chrome works but another browser does not: I</strong>nspect origin changes, redirects, cookies, and certificate trust. Some tunnel integrations rewrite localhost hostnames on particular browsers; avoid hard-coded absolute localhost URLs.</li><li><strong>Parallel jobs interfere with each other: </strong>Give tunnels and mutable test records unique identities. Separate ports or isolated runners are also necessary when jobs launch local applications.</li><li><strong>Resources remain after cancellation: </strong>Inspect teardown failures and remote-session expiry. Hard termination can prevent both Python finalizers and shell cleanup from running.</li></ul><h3>Conclusion</h3><p>A reliable test should identify the build it reached and verify the server-side outcome. Application readiness, network connectivity, and browser assertions each answer a different question.</p><p>Start with the simplest connection that meets your browser and network requirements. Keep environment settings separate from the test, and move to Grid, a private tunnel, or a preview URL when the topology calls for it.</p><p>Does your staging suite verify the deployed build, or does it assume the last deployment finished?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ed808abcbcf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Locator Strategy in Selenium and Why XPath Breaks]]></title>
            <link>https://jennykeeper.medium.com/locator-strategy-in-selenium-and-why-xpath-breaks-7371a99ab86f?source=rss-53b25a1355b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7371a99ab86f</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[xpath-breaks]]></category>
            <category><![CDATA[xpath]]></category>
            <category><![CDATA[locator-strategies]]></category>
            <category><![CDATA[selenium]]></category>
            <dc:creator><![CDATA[Jenny Keeper]]></dc:creator>
            <pubDate>Fri, 02 Oct 2026 20:32:44 GMT</pubDate>
            <atom:updated>2026-10-02T20:32:44.075Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0yaEFJkBGAMhf4a5" /><figcaption>Photo by <a href="https://unsplash.com/@ffstop?utm_source=medium&amp;utm_medium=referral">Fotis Fotopoulos</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Selenium can automate almost any interaction on a web page, but first it needs to identify the element that it’s going to interact with. A locator provides that identification. The choice of locator strategy directly affects how reliable and maintainable a Selenium test becomes.</p><p>As applications grow, their UI elements and underlying DOM can change. A locator that works well for your test today may end up failing after a minor UI change, especially if you’re using XPath. XPath is prone to this when it relies on unstable attributes or complex element relationships.</p><p>In this guide, I’ll walk you through what locators are, what locator strategies are available in Selenium, how to choose one, and why XPath is susceptible to breakage in test automation.</p><h3>What is a Locator?</h3><p>A locator is an<strong> identifier </strong>used to find a specific web element on a page. In Selenium, locators help automated tests identify elements such as buttons, text fields, links, and checkboxes.</p><p>A web page can contain hundreds of elements. Selenium needs a way to distinguish the element a test needs from all the others.</p><p>For example, a login test needs to identify the username field before entering text into it. A locator provides the information Selenium needs to find that field in the page’s DOM.</p><h3>What is a Locator Strategy in Selenium?</h3><p>A locator strategy is the <strong>method</strong> Selenium uses to identify a web element. It determines what property or characteristic of the element Selenium should use for locating it.</p><p>Different elements can expose different identifying properties. An element may have a unique ID, a name, visible text, or other attributes that Selenium can use.</p><p>It is important that you choose the right strategy because the same element you choose may be easy to identify with one approach and difficult to identify with another.</p><h3>How Selenium Uses Locators to Find Web Elements</h3><p>Selenium uses the selected locator strategy to search the page’s DOM and identify the required element. Once Selenium finds the element, the test can perform an action such as clicking it or entering text.</p><p>For example:</p><pre>driver.findElement(By.id(&quot;username&quot;)).sendKeys(&quot;John&quot;);</pre><p>Here, Selenium uses the ID strategy to find the username field and then enters the specified text.</p><p>The process is:</p><p><strong>Choose a locator strategy → find the web element → perform the action</strong></p><p>This process works reliably when the locator points to a stable element. If the attributes or structure used by the locator change, the test may fail to find the element.</p><h3>Types of Locator Strategies in Selenium</h3><p>Selenium provides eight locator strategies for identifying web elements. Each strategy uses a different property or characteristic of an element. The right choice depends on the element’s attributes, uniqueness, and stability.</p><h4>1. ID</h4><p>The ID locator uses an element’s id attribute. When an element has a unique and stable ID, this is usually a straightforward choice.</p><pre>driver.findElement(By.id(&quot;username&quot;));</pre><p>Here, Selenium searches for the element with id=”username”. An ID that changes dynamically is less useful for automation.</p><h4>2. Name</h4><p>The Name locator uses the name attribute. It is commonly used with form elements such as input fields, radio buttons, and checkboxes.</p><pre>driver.findElement(By.name(&quot;email&quot;));</pre><p>The name value does not have to be unique, so verify that it identifies the intended element.</p><h4>3. Class Name</h4><p>The Class Name locator identifies an element using its class attribute.</p><pre>driver.findElement(By.className(&quot;login-button&quot;));</pre><p>Class names are often shared across multiple elements. They work best when the class value uniquely identifies the required element.</p><h4>4. Tag Name</h4><p>The Tag Name locator uses the element’s HTML tag, such as input, button, or select.</p><pre>driver.findElement(By.tagName(&quot;button&quot;));</pre><p>Tag names are rarely unique. They are more useful when working with multiple elements of the same type, especially with findElements().</p><h4>5. Link Text</h4><p>The Link Text locator finds an anchor element using its complete visible text.</p><pre>driver.findElement(By.linkText(&quot;Forgot Password&quot;));</pre><p>It is useful when the link text is unique and stable. Changes to the visible text can make the locator fail.</p><h4>6. Partial Link Text</h4><p>Partial Link Text works similarly but matches part of an anchor’s visible text.</p><pre>driver.findElement(By.partialLinkText(&quot;Forgot&quot;));</pre><p>This can help when the complete link text is long or changes partially. However, overly broad text can match the wrong link.</p><h4>7. CSS Selector</h4><p>CSS Selector uses CSS patterns to identify elements through IDs, classes, attributes, and relationships.</p><pre>driver.findElement(By.cssSelector(&quot;#username&quot;));</pre><p>CSS selectors are concise and flexible, making them a common choice when a simple ID or class locator is not sufficient.</p><h4>8. XPath</h4><p>XPath identifies elements using their attributes, text, position, and relationships within the DOM.</p><pre>driver.findElement(By.xpath(&quot;//input[@name=&#39;username&#39;]&quot;));</pre><p>XPath is particularly useful when an element does not have a suitable ID, name, or class. It can also handle more complex relationships between elements.</p><p>However, XPath can become difficult to maintain when it depends heavily on changing attributes or DOM structure. That is why choosing how XPath is written matters as much as choosing XPath itself.</p><h3>How to Choose the Right Locator Strategy for Your Workflow?</h3><p>The right locator strategy depends on the element, its attributes, and how stable those attributes are.</p><p>Start with a locator that is unique, predictable, and unlikely to change with routine UI updates.</p><p>Most times, a stable ID is a good starting point. If the application provides dedicated test attributes such as data-testid, those can also provide reliable hooks for automation. When these options are unavailable, CSS selectors and XPath provide more flexibility.</p><p>The key is to avoid choosing a locator simply because it works. A locator should also remain maintainable as the application evolves.</p><h4>When Should You Use CSS Selectors or XPath?</h4><p>CSS selectors and XPath can both locate elements using attributes and relationships. The better choice depends on what you need to express in the locator.</p><p><strong>Use CSS selectors when:</strong></p><ul><li>The element can be identified through IDs, classes, or attributes.</li><li>You need a concise and readable selector.</li><li>The relationship is primarily between a parent and its descendants.</li></ul><pre>driver.findElement(By.cssSelector(&quot;input[name=&#39;username&#39;]&quot;));</pre><p><strong>Use XPath when:</strong></p><ul><li>You need to locate an element using its text.</li><li>You need to navigate complex relationships between elements.</li><li>You need to move beyond straightforward CSS-style selection.</li></ul><pre>driver.findElement(By.xpath(&quot;//button[contains(text(),&#39;Login&#39;)]&quot;));</pre><p>Neither strategy is inherently better. The more important consideration is whether the resulting locator is <strong>unique, stable, readable, and resilient to UI changes</strong>.</p><p><strong>Can XPath Break in Test Automation?</strong></p><p>Yes, XPath can break in test automation when the page structure or attributes used in the XPath change. The element may still exist and work correctly, but Selenium may no longer find it using the same XPath.</p><p>Common reasons XPath becomes fragile include:</p><ul><li><strong>Dynamic attributes:</strong> IDs and class names can change between builds or page loads.</li><li><strong>Complex paths:</strong> Long XPath expressions can depend on several levels of the DOM structure.</li><li><strong>Position-based locators:</strong> Changes in the order or number of elements can affect position-based XPath.</li><li><strong>Text changes:</strong> Updates to visible text, content, or language can make text-based XPath fail.</li></ul><p>A reliable XPath should always depend on attributes or relationships that are expected to remain stable.</p><h3>How to Build More Stable Selenium Locators?</h3><p>A stable locator should identify the intended element without depending heavily on details that are likely to change. Ultimately, all you want is to make the locator reliable as the application evolves.</p><p>A few practices can help:</p><ul><li><strong>Prefer stable attributes:</strong> Use unique IDs or other attributes that remain consistent across UI changes.</li><li><strong>Use dedicated test attributes:</strong> Attributes such as data-testid or data-test can provide reliable hooks when suitable user-facing attributes are unavailable.</li><li><strong>Keep locators simple:</strong> Short, readable locators are easier to understand and maintain than deeply nested selectors.</li><li><strong>Avoid positional selectors:</strong> Selectors based on positions such as div:nth-child(3) can break when elements are added or reordered.</li><li><strong>Limit DOM dependency:</strong> Avoid building locators around long chains of parent-child relationships when a simpler attribute can identify the element.</li><li><strong>Review locators when the UI changes:</strong> A locator that worked before a major UI change should be checked rather than assumed to remain valid.</li></ul><p>A useful locator should be <strong>unique, stable, readable, and specific</strong> enough to identify the intended element.</p><h3>Conclusion</h3><p>A locator strategy should be treated as part of the application’s testability, not as a collection of clever XPath tricks.</p><p>If your tester repeatedly needs long XPath expressions to find an ordinary button or a link that is a concern, you should be raising it up with your development team. A stable id or dedicated data-test attribute can often eliminate an entire class of test maintenance problems.</p><p>I firmly believe that developers and testers should agree on stable automation hooks as part of UI development. Test automation becomes much easier when the application gives tests something reliable to hold on to.</p><p>The strongest locator is never the most clever one. It is the one the application can afford to keep unchanged.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7371a99ab86f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>