Utilizing Automation for End to End Testing
Gustavo Cavero, Ray Regno / July 13, 2022
Introduction
With gym owners and customers relying on PushPress software to structure their fitness flow, we need to guarantee that the system is always operating at its fullest potential, doing so without any glitches or issues. On the engineering end, changes are constantly being made to the system’s code; due to this, it is essential for us to have a fast and adept method of testing our PushPress software. At PushPress, we have shifted from being a manual QA team to an automated QA team. By doing so, we can ensure that the programs are all running well and working properly for our customers.
What is QA?
In order to ensure that our software is running up to speed, we must have a reliable Quality Assurance team. The QA team is the gatekeeper for our releases; code is not released until they approve it first. They are people that document and understand the current functions of each application, whatever it may be. The QA team tests cases, documents test cases for coverage, and documents functionality.
When we are trying to deploy code, there is always back and forth between engineers, QA, and product. The product people request something to be built. The engineers build the product, which the QA team checks for errors and regression bugs. Their goal is to validate problems, and to understand what should and should not work to resolve them.
Background
Every time our team makes a change to our code, we risk adverse, unexpected effects. Sometimes an update will solve one issue while simultaneously introducing new issues elsewhere in the system; we call these errors regression bugs. A change made to even one line of code can implement a cascading effect of unforeseeable problems that need to be identified as quickly and efficiently as possible.
The best ways to avoid regression bugs are via unit tests, integration tests, and automated testing for an end to end test. Unit tests are non-functional, the word “functional” referring to a broad outcome. An end-to-end test has to test multiple things; different sequences must occur in order for it to work. End-to-end tests help engineers understand how much of the application is made up of main paths, or main user journeys, which must be working for the customer.
Thorough manual testing for these potential scenarios and outcomes, or test cases, would require a member of our team to go into the code and test each potential sequence a user could click through. As one could imagine, this would be extremely tedious and time consuming, especially considering the amount of changes our team makes to our software to keep it up to par. Additionally, people are fallible; mistakes can easily be made when conducting a series of tests on code. It is important that testing remains consistent across the field to garner reliable results.
In contrast to manual testing, automated end-to-end testing is a faster, more systematic way to combat the risks posed by changing codes, ensuring that a change made to one portion doesn’t cause conflicting effects in another. It does this by first running tests on all the existing code, and then doing so with the new changes to guarantee that there haven’t been any new issues introduced. As an automated test, end-to-end testing eliminates the lag often posed by manual testing, and we can ensure that all of the test cases are running perfectly.
In a fast-paced working environment, efficient and reliable testing processes are imperative to maximize the speed at which development teams can resolve problems within the system. The less time engineers are required to spend mitigating issues, the more they can spend on innovation and helping customers maximize their usage of our software.
Strategy
At PushPress, we utilize the Cucumber BDD Testing Framework to streamline and automate our brute force regression testing, as well as to test common end to end workflows. Rather than requiring a human to manually test each individual scenario, Cucumber automatically tests all possible clicking sequences within the code to make certain that changes have not resulted in a cascading effect.
Get PushPress Engineering’s stories in your inbox
Join Medium for free to get updates from this writer.
In addition to testing, Cucumber also provides engineers with a document that details the sequences and steps it took to get to each result. Often in the world of development, engineers will be working on software without any documentation as to how and why certain flows are created. This document combats that issue by providing engineers with a detailed map to the code that they can follow with ease; even a non-technical person could read the document! With this information, engineers can go back and trace the exact steps that Cucumber took in its testing pattern, and fix any rising issues if necessary.
In addition to Cucumber, Selenium WebDriver is another tool that we use at PushPress to ensure that our system is running smoothly. Selenium works as the interface between codes; the software opens a new web browser through which the viewer can watch the test clicks as they occur. While Cucumber completes all the tests and then provides engineers with a written report, Selenium allows the engineers to watch the sequential testing in real time. The progression typically begins at the system’s login page, as we know that this is the first step every gym owner and customer takes when they are entering our software. Selenium will test the typing in of a username, password, and so on through all of the potential sequences in our system.
Results
With the combination of Cucumber and Selenium, our team is able to certify that whenever we implement a change to our software, each of the use cases are tested and retested with consistency. Automating tedious test workflows gives engineers the ability to guarantee accurate results while drastically reducing the time it takes to conduct a full suite of regression and end to end tests. They can run in parallel; a human may take 8 hours to run all the necessary test cases, whereas the automation suite does so in a matter of minutes. We have invested time and resources to build an expansive testing framework that is measurable, verifiable, and automated.
Using the Cucumber Testing Framework also has the built-in benefit of providing engineers and stakeholders with a non-technical form of communicating software specs, use cases, and workflows. This, in turn, will be immensely useful for any new personnel who join the team later in a project’s life cycle.
As a company composed of gym owners serving other gym owners, we have an intimate understanding of how our tools are utilized on a daily basis. Increasing the speed and accuracy of our testing process helps us fix issues faster; the faster these issues are fixed, the faster our customers can return to business as usual. This allows our engineering team to continue working on new innovations to upgrade our current software, elevating the customer experience.
Lessons and Impact
One of our PushPress core values is to “Give Gyms the Tools to Dominate.” Automated testing helps our team make certain that our software is up to this standard as a high-quality product. When our tools are operating as they should, it allows our customers to continue focusing more on growing their business, rather than being concerned with the system’s flow. While implementing technologies such as Cucumber requires a significant investment on our part, it allows us to serve our customers better in the long run. Putting customers first helps us build long lasting relationships with content and loyal clientele.
Lord Kelvin, also known as William Thomson, once noted, “If you cannot measure it, you cannot improve it.” At PushPress, we strive to be a data-driven organization that utilizes measured tests to improve our system. By automating our testing and thus minimizing the risk of human error, our data becomes more reliable and repeatable. Issues within the code are much easier to identify and address when our team can rely on this consistency.
Quality Assurance is our primary vehicle to achieve incremental perfection. In a world where technology is constantly moving forward, our software needs to evolve as well, or we risk stunting our growth and scalability. By investing in automation, our engineering team has been able to mitigate issues more adeptly, allowing for growth and development to flourish across the system.
Why do we use automation?
When a software project gets to the point in which doing test cases execution is a tedious work that consumes too much time and resources, it is time to start thinking about automation.
But why do we use automation, and for what?
Automation helps the engineering team reduce the execution time of testing by yielding quick results within a short time frame. While staging different parts of production used to take two days to complete, the task can now be completed within an hour. By providing a framework to quickly test all the changes made, engineers can have more confidence that the system is running properly and giving customers a satisfactory user experience.
Automation helps the team optimize the repetitive tasks that the team needs to complete every time a new build or release is completed. For example, if the due nature of a project has two daily builds, the QA team should execute manual tests to validate the system’s flow for each build. This will require at least one member of the team to dedicate time to this task everyday.
By utilizing automation, the team only needs to execute one automated test to ensure the system’s flows were not affected by new changes, allowing our team to focus on different QA tasks.
Team Shout-Outs
Our automated end to end testing is an ongoing effort and has included the work of many different collaborators. It would not have been possible without the contributions from Gustavo Cavero, Brandon Cavero, Daniel Gonzales, Ray Regno and our amazing Customer Experience Team.
