By Vikas and Abhinav from UX-Engineering.

Developers and designers are entities who always try to do their best to build a beautiful product for users. But this is not always enough to ensure that the carefully-crafted designs look exactly the same on a user’s device.

To make sure developers build UI close to the designs, we have to understand the designs first, discuss with designers, and double check on Zeplin after development. We also have to make sure the UI doesn’t break in different scenarios, code changes etc.

This all takes a tremendous amount of time and is not always feasible.

Wouldn’t it be great if we could automatically check if everything is as the designer intended?

In this post, we’re going to discuss one way of solving this gap between designed UI and developed UI efficiently. (In other words, the gap between developers and designers.)

Asphalt — Maintain Consistency

We’ve been talking about our design language system, Asphalt, for some time now. It was built to ensure that our UI is consistent and easy to use across GOJEK’s vast array of products and services.

Recap: Asphalt is GO-JEK’s very own design language system. It aims to unite developers, designers, and all other functions across the org for consistency and efficiency. It is the foundation to be built upon.

The problem?

In Asphalt, we treat components like reusable Lego blocks. This helps in building components at a fast pace.

Build UI like Lego

In the process, developers tend to update or extend the functionality of a component, and they need to be able to do this without causing side effects to other components.

We felt the need for a monitoring system to check UI components with each commit/change a developer makes, that informs the developer of the impact of the change. We also needed to run this system on autopilot mode, check our components frequently and detect any anomalies. The objective was not to replace human operators, but to assist them. This is what we seek to achieve with our UI Components Library.

To achieve this goal, we settled on snapshot tests.

Snapshot Testing

Conceptually, snapshot testing is a simple idea: take a screenshot of your user interface and store it as a reference image. Then, make your tests take screenshots of the same UI and compare it to the reference image. Once the test is complete, let the developer running the test see the difference and the affected changes, and decide if the changes are correct.

Implementing Snapshot tests on iOS

We use XCUITest framework that ships with Xcode, which allows you to easily develop UI tests that reflect users’ interaction with the application.

Finding the diff between two screenshots

We use Core Image filters (CISubtractBlendMode and CIAreaAverage) to find the diff between a reference screenshot and a screenshot of the current state of a component.

In the code sample above, we use a UIImage extension where DifferenceFilter subtracts the background image sample colour from the source image sample colour and gives us the difference image. Both the input and output image we get is a CIImage.

We inject the difference image to AverageFilter, which returns a single-pixel image that contains the average colour for the region of interest.

We run the averageFilterOuputImage through the firstPixelColor function shown above. This returns UIColor which we use to conclude whether it is all black or not. If it is not completely black, then we’ve got a difference in our source image.

In our UI tests, we launch the screen with the component we need to test in our demo app and call this image diff algorithm. If there is a difference, we fail the test and generate a report showing the diff.

Observing the results

Here’s an example that we recently experienced.

We have this button class, TextButton, which is given a PrimaryState and is responsible for updating its background colours, text colours, enabled state, and text. We include this button on most of our views in the app for primary interactions. We wanted to change some of the rendering logic in the button, but we inadvertently broke another part of our app that we didn’t realise was using TextButton too.

As we can see from the screenshot, the test highlighted that the border colour was different between the two images which was not what we intended. Snapshot tests saved us from committing a mistake that would have led to bugs across the app in different products and caused delays in our shipping cycle.

Managing reference screenshots

We keep our screenshots segregated by device type. We have a naming convention which goes device_resolution+os_version. Above, you’ll see the 640+1136+10 folder and reference images for TextButtons. These images are generated by running the tests on iPhone 5s( which has a device resolution of 640x1136 and runs iOS 10). Similarly, we generate images for multiple available devices.

Sample code for ButtonTests above has a folderName variable and testPrimaryButton() function. When we run the test above for the first time after building the component, we generate these images from identifier TextButton-primary_cover_normal_state appending the screen dimensions and save them under the folder named ButtonTests. This way, we’re generating reference images from our UITest code only. After adding these folders in the bundle we run the tests again and this time screenshotHelper has a reference image in the bundle to test them against the UI screenshot. This way we ensure a consistent pattern (like the naming convention, bundle structure etc) for all reference images and make the process faster.

So does it work?

The advantages are that you can easily test complex structures without writing much code, you get good warnings when something changed, and you can easily update this test.

The disadvantages are that from just reading the test, it is not always clear what is tested and what the expected behaviour is. Sometimes, the created snapshot is so complex that you overlook wrong assumptions that then end up as expected results. As it is so easy to update snapshots, wrong stuff can sneak in.

One challenge is, at around test count 170, Xcode or xcodebuildwelcomed us with the following error:

Waiting for accessibility to load
Wait for app to idle
    App event loop idle notification not received, will attempt to continue.
    App animations complete notification not received, will attempt to continue.

After this error, all the subsequent tests fail with another message:

Waiting for accessibility to load Assertion Failure: testUITests.swift:21: UI Testing Failure — App accessibility isn’t loaded

We’ve looked around Google and found here the details about this issue. We’ve tried multiple workarounds mentioned in the blog, also tried to make multiple targets and schemes for a number of tests, but in vain.

Until Apple finally fixes this issue, we’re keeping the count of tests lower than 170 😑 (If you know why this happens, feel free to leave a comment below).

One more challenge is to run the tests faster. Currently, we’ve disabled all the animations by setting the following in the Appdelegate didFinishLaunchingWithOptions function. Disabling all animations makes the tests more reliable and makes them run quicker.

if ProcessInfo.processInfo.arguments.contains("AsphaltUITests") {

UIView.setAnimationsEnabled(false)

}

Flaky Tests also need a special mention. As our test suite is expanding we’re starting to face the problem of one or two tests failing randomly. To solve this kind of misbehaviour, we’re currently experimenting with fastlane-plugin-test_center multi-scan functionality to rerun only those fragile tests, saving us time taken to rerun the complete suite.

Note: this feature does not support Xcode parallel tests functionality yet but it is WIP here.

What’s next in Snapshot Testing?

We’re planning to auto-generate UITest classes by analyzing the UIComponents classes. Most of the times the code for UITest class is dumb. Reach up to the component, take a screenshot of the view assigned accessibility identifier and match it with the reference image(whose filename is generated by class name). Our first step is to autogenerate these dumb steps and then move up to write stencils for different behaviours of components like swipe, buttons, max data, min data and many more. Of course, we’ll not be able to cover all testing scenarios with this process, but this will remove a lot of generic testing scenarios we write for a UIComponent.

In Summary

Overall, Snapshot tests have served us well, but a tool is only as good as its user. So it’s important to make them granular, readable and treat them just like you would treat normal tests. Combining these with a culture of thorough code reviews has paid great dividends for us.

I hope this gives you a clear picture of what snapshot tests are and how they can be used as a part of a comprehensive testing strategy.

As designers and developers, there are always things to be improved and processes to be streamlined. At GOJEK, this is something we realise and attempt to overcome with a culture of close collaboration between teams. If this culture is something you can contribute to, check out gojek.jobs — we’re looking for smart minds to help us build Southeast Asia’s SuperApp 🖖

Pixel Perfect UI at Scale was originally published in Gojek Product + Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.

GO-JEK. “An Ojek for Every Need“.

Like literally, EVERY need.

From food, transport, massage, cleaning, payment, shopping, logistics, delivery, medicine, entertainment, you name it, we have it.

Oh, and in case you’re wondering, an “ojek” is a motorcycle taxi, or cart bike. It’s a licensed form of transport in some countries.

We have around 18 services up and running across our platforms and more are coming. The amount of services GO-JEK helps you with is nothing short of phenomenal.

This is the story of how we built our iOS app:

The Beginning

Our iOS codebase was Monolithic. By nature of our vastness, a lot of code was jumbled across products. The code was so convoluted, no one knew how a small change in their code would affect the others.

Inevitably, there was a ripple effect; solving a bug would raise a new one. Debugging was not easy. It took almost 45 minutes to build, whether we were building from scratch or just changing a single line of code.

The bizarre amount of time it took to test the smallest changes was proving to be cumbersome. This time wasted was worse on our Continuous Integration pipeline. And it was fairly obvious because we were handling nearly 18 products inside a single repository, and separate teams were simultaneously working on the same codebase. So much so, we used to have a team whose only responsibility was to fix bugs for the entire app.

Every new iOS developer at GO-JEK spent a few weeks with this team, fixing bugs and learning the codebase, before being assigned to a different product team. If we wanted to improve the codebase, we first need to figure out how to spend less time on bugs.

The VIPER Phase

We needed to reinvent ourself. After analysing our main issues, we decided to remove all cross-product dependencies and restructure our products using VIPER.

We decided to build GO-PAY and GO-POINTS, with this new structure. These products would lay a foundation for future products and become a template for refactoring existing ones. It was important for us to choose the architecture best suited to handle these many products together. Initially we chose VIPER, and here’s why:

Pros:

• Code becomes much more testable
• Separation of components really well
• Very high encapsulation
• Easy removal of any feature

Cons:

• Two-way communication between modules is not possible
• Difficult to step through while debugging since everything is abstracted behind a protocol

Irrespective of the cons, we accepted this architecture and continued the transition for other products as well. But to start this for existing products, we needed a firebreak.

Firebreak — Modularisation

An obstacle to the spread of fire

We pinned down the architecture and got the implementation right in products. But, we needed to modularise the whole approach. We were still counting 45 mins on average of compile time. This firebreak exercise was going to literally stop feature development for a few weeks and let us just focus on modularisation tasks. It was a hard decision at the time, but to get this right, we had to get things right internally. Features, and product development comes only after practicing what we preach — adhering to the principles of good software development practices.

“It’s not about being the first to market, it’s about being the first to get it right.” — Benny Xian

This one step did wonders for us in iOS. On Mobile Consumer App, our main goal for Firebreak, was to break down the single repo into multiple product repos as a part of modularisation task.

Every iOS developer or a small team became POCs for each product. We identified each product’s files, assets, fonts, resources and moved them out to separate repositories. Sometimes we had to make duplicate files in separate repos as both products were using them. But we were ready to hold this duplicity for now. Home, Settings, Help, History, Tabs, Navigation stack, Analytics were still at the core of GO-JEK holding 18 products through CocoaPods.

Advantages of breaking down the repos gave us the following benefits:

• Pushes for platformization of the Consumer App. All essential services to be extracted out in platform library. This includes Network, Notification, Remote Config, Analytics, UI Components,
• Gives more independence to each product on moving quickly without being affected by other product changes
• Easier to track quality and impact by a product on the integrated app. E.g. app size, code quality can be tracked at product repo level.
• Easier rollback of product specific changes
• Enforces coming up with better API contracts/generalised solutions, because of module separation.

Some of the retrospective points we collected from our team:

• Satisfied with Firebreak
• Compile time down on iOS
• Many tech clean up tasks pending, need more time on Firebreak/Tech clean up activities
• [Action] have a small team continue on pending critical tasks
• Build process with pods is getting complex

Clearly, developers were happy with the progress and they don’t have to wait for the main codebase to compile and test the product. Now, developers host apps to run and test their products in complete isolation.

This gave tremendous freedom fo the teams to kick start feature development in full throttle.

But we were not done yet. The main GO-JEK app was still taking about 40 mins on average to compile all product’s pods and the main code. We can’t get the leverage to have another firebreak to fix more things, so we continued with it. But we knew the problem was at core.

Launchpad

Our main goal for the iOS app was to boost developer productivity. All the measures we took in this endeavour helped us reduce the time taken to build, test and ship out new features and ensured quick bug fixes.

The main motivation behind separating product codebases was to make product teams independent of each other. This also resulted in immensely reducing the time taken to build individual products. Since each team was no longer dependant on other products, we were able to ship independently.

During this time, we introduced the concept of dummy launcher apps. These were single screen apps which sort of had an index of a product’s various screens. Every product repository had its own dummy app and product developers used these apps to test the features they were building.

There were few drawbacks to this approach:

• You cannot test a product in complete isolation from other products in the real GO-JEK environment
• Testing and debugging analytics events were not possible
• Testing and debugging remote notifications and deeplinks were not possible
• Testing a complete booking flow — from placing an order to viewing the history were not possible
• Testing logout/login events were not possible

This meant in order to fully test a feature, developers had to build the actual GO-JEK app (with all products) with their product code in it. And debugging and fixing bugs required multiple app builds. Preparing production releases required a synchronised effort from all teams since it was complicated to automatically merge many codebases together in a reliable way.

To fix these, we came up with Launchpad. It was a 6-week project where we re-wrote the GO-JEK app (barring actual product code).

The main vision and goals behind Launchpad were as follows:

• Sandbox all product codebases into their own repositories
• Make GO-JEK fully independent of products
• Wrap the GO-JEK app into a framework that can be imported into products
• Allow products to be tested in isolation in the real GO-JEK environment
• Come up with unified APIs to integrate a product into GO-JEK
• Allow for easy removal of products from GO-JEK
• Allow a product or a group of products to be submitted as a stand-alone app to the App Store without major code changes
• Allow products to be individually profiled and audited for quality and performance
• Export APIs, allowing products to show dynamic components in the home screen without modifying the GO-JEK codebase

A clean build succeeded in just 8 minutes. We’re actively improving this further with compiler optimisation tweaks.

Throughout the past year or so we’ve seen the codebase transforming radically and maturing gradually. This helped us shift our focus to build some cool features which improves the overall experience of the app. To sum up the post, as John Johnson famously said,

“First, solve the problem. Then, write the code”

And yes We’re hiring! For any roles in Engineering, Design or Product Management, or anything at all, visit gojek.jobs. Come solve amazing engineering challenges.

Happy hacking :)

1 app, 18 products — A journey from monolith to a microapps codebase was originally published in Gojek Product + Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.