<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by sakshi kakkar on Medium]]></title>
        <description><![CDATA[Stories by sakshi kakkar on Medium]]></description>
        <link>https://medium.com/@sakshikakkar12?source=rss-cdc1ca87e790------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*3oszHYBfcj-yl3kt</url>
            <title>Stories by sakshi kakkar on Medium</title>
            <link>https://medium.com/@sakshikakkar12?source=rss-cdc1ca87e790------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 19:20:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sakshikakkar12/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Health check for Test Automation: Blue green deployment]]></title>
            <link>https://medium.com/vmware-end-user-computing/health-check-for-test-automation-blue-green-deployment-4b365595582e?source=rss-cdc1ca87e790------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b365595582e</guid>
            <category><![CDATA[blue-green-deployment]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[quality]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[quality-management]]></category>
            <dc:creator><![CDATA[sakshi kakkar]]></dc:creator>
            <pubDate>Tue, 07 Sep 2021 19:56:10 GMT</pubDate>
            <atom:updated>2024-08-15T20:59:02.116Z</atom:updated>
            <content:encoded><![CDATA[<p>With a product’s evolution and exponential growth over time, adding new features to make user experience of the application better release by release, we as software engineers want to make sure the product’s delivery is faster without compromising quality. It is the responsibility of the product developers to not just focus on product health, but to also think about the health of the automated tests. We create different types of tests like unit tests, UI tests, instrumentation tests, and end-to-end tests to evaluate the health and the quality of the product, but we are missing out on the fact that we also need to maintain the health of these tests.</p><blockquote>“Also common is the test automation group zombie. This zombie is the practice of assigning test automation to a dedicated team of test automators. The appeal is that we can keep developers focused on writing new code instead of writing and maintaining automated tests. The danger is that test automation inevitably lags development, so feedback from testing is delayed in a way that significantly reduces its value.”<strong> — </strong><a href="http://dhemery.com/"><strong>Dale Emery</strong></a>.</blockquote><h3>How can we achieve faster delivery of the product?</h3><h3>Approach 1: CI/CD Pipeline</h3><figure><img alt="Continuous Integration Continuous Deployment" src="https://cdn-images-1.medium.com/max/800/1*DhDTs5WmZdXHfjj6XcrzeA.gif" /><figcaption>CI/CD Pipeline | Photo by <a href="https://dribbble.com/lpah/collections/1624771-Tufin">Tufin by Alpha Sadcopen | Dribble</a></figcaption></figure><h4>What is CI/CD Pipeline</h4><p>Continuous Integration/ Continuous Deployment, also known as CI/CD, is a method used to efficiently deliver apps to customers by introducing automation into all stages of app development.</p><p>Our team started by building automation test coverage and plugging them into the CI/CD automation test pipelines, which would run every night. In the initial stage, it was working fine, but as the products evolved, we started seeing different challenges.</p><h4>Challenges with this approach:</h4><ul><li><strong>Automation reliability</strong>: Flakiness of the tests decreased the automation reliability; tests would pass the first time but fail on subsequent runs despite the same application being tested. Individual tests ran locally but failed when combined because of the stale data being left after the run of the previous test.</li><li><strong>Device Issues</strong>: Sometimes the internet connectivity on the device would go off, the VPN would disconnect automatically, or the Google Play store login would throw an error and require reinstallation of the application.</li><li><strong>Environment Issues/Reliable Data Issues</strong>: Changes of the configuration in the test environment by a different team.</li></ul><p>To eliminate some of the issues that we faced with CI/CD pipeline, we went ahead with a new approach.</p><h3>Approach 2: CI/CD pipeline with a robin role (this is the slight enhancement of the first approach)</h3><figure><img alt="Robin Role : Aperson is dedicated for a week to look at the test failures, triage the test executions, analyze the failure points, and update the results to the dashboard" src="https://cdn-images-1.medium.com/max/480/1*MlAsECTlboBYleJfiPjijw.gif" /><figcaption>CI/CD with Robin Role | Photo by <a href="https://giphy.com/explore/teen-titains-go">Teen Titains Go Giphy</a></figcaption></figure><h4>What is robin role?</h4><p>In this approach, a person is dedicated for a week to look at the test failures, triage the test executions, analyze the failure points, and update the results to the dashboard.</p><p>With this in place, we now had an idea of the reason behind the failures, but there wasn’t anyone who is fixing these issues in parallel, so the number of issues like product change, existing failures, or flakiness were still piling up. We as a team spent a good amount of time building a robust automation suite that could help in our release train by analyzing the quality of the product; however, we were now spending more time maintaining those automated tests instead of identifying the product defects.</p><h4>Challenges with this approach:</h4><ul><li>Inordinate time spent triaging</li><li>Upcoming releases become bottlenecked on the robin role</li><li>No targeted approach to resolve the identified issues</li></ul><p>This was the initial method for maintaining the quality of the tests and the code, but with the fast-paced growth of the product and their corresponding tests, it was no longer sufficient. This opens the platform to introduce our next approach.</p><h3>Approach 3: Blue Green Deployment Model</h3><figure><img alt="Automated Green Blue Deployment Model" src="https://cdn-images-1.medium.com/max/600/1*ddXUucI5tUslLgggP80YGQ.png" /><figcaption>Fully Automated Blue Green Deployment | Photo by <a href="https://codefresh.io/kubernetes-tutorial/fully-automated-blue-green-deployments-kubernetes-codefresh/">Codefresh</a></figcaption></figure><h4>What is Blue Green Deployment?</h4><p>Blue green deployment is a very well-known term in the world of continuous delivery. It involves having two identical instances up and running, with any new change being pushed to the blue deployment first. Once every minimum acceptance criterion checks out, then it is pushed to the green deployment.</p><p>After the <a href="https://developers.google.com/google-test-automation-conference/2014/presentations?hl=en">2014 Google Test Conference presentation</a> by Roy Williams “<a href="https://docs.google.com/presentation/d/1FobpyXuNHK4-2yPrBGAlHnhJll3xFl84_OyP0HVqvKc/edit#slide=id.g3f5cc2519_6148">Never Send a Human to do a Machine’s Job — How Facebook uses bots to manage tests</a>”, we thought of adopting this approach to our product by creating two suites: primary and secondary. With this, we started eliminating intermittent failures by not adding the tests directly to the primary suite until they proved themselves a fixed number of times. A Test Warden Service is created, which would be responsible for tracking the health of the tests. The time taken for the test to prove its quality and move to the primary test suite is considered its probation period.</p><h4>Adoption of Blue Green Deployment for the existing tests</h4><ul><li>Created a small number of tests, ran them constantly</li><li>With these consecutive runs, flaky tests were identified</li><li>Then these tests were separated out from the stable tests</li><li>Since now we have two suites: primary suite serving as the Green Pipeline and secondary suite (unstable) serving as the Blue Pipeline</li><li>Created issues for the flaky tests, which are part of Blue Pipeline</li><li>Now started tackling the items one by one when we have our Green Pipeline always in place and running smoothly</li></ul><h4>Adoption of Blue Green Deployment for the new tests</h4><ul><li>New tests are added to the blue pipeline</li><li>Tests prove themselves for 5 consecutive runs (one week) in the Blue Pipeline and then moved onto the Green Pipeline once the health of the tests is established</li><li>If flakiness is detected in the Blue Pipeline, a priority 1 (critical) product bug is created and assigned to the responsible user. A strict timeline of 3 days is followed to eliminate the flakiness of the tests, or else the tests will be removed from the test suite and the test coverage for the module will be decreased</li><li>Green Pipeline ensures that we can easily identify product defects</li></ul><p>Now that the flaky tests are separated from the automation test suite, it is much easier to identify any product defect. With this, our execution time of the automated tests gets reduced and very few tests fail</p><h4>Benefits of Blue Green Deployment:</h4><ul><li>Test execution time is reduced from around 6 hours to 3 hours.</li><li>Robin role: Flacky tests resolution task is deprioritized which in turn frees up time for the robin to help with other tasks in the sprint. Earlier, it was considered a high weightage task, but with this approach, it’s reduced to a low one.</li><li>Turnaround time for categorizing product issue is dramatically reduced with higher accuracy.</li></ul><h3><strong>Conclusion</strong></h3><p>The software development process is a learning curve, and changes happen with experience. There is no universally correct way of doing things; one must experiment and find what’s right for their product. With a product that is widely used by millions of customers, we cannot compromise on quality at all.</p><blockquote>“When the product is right, you don’t have to be a great Marketer” — <strong><em>Lee Iacocca</em></strong></blockquote><h4>References:</h4><ul><li><a href="https://kevintuck.co.uk/why-your-tests-are-failing-and-how-to-combat-unreliable-automation/">https://kevintuck.co.uk/why-your-tests-are-failing-and-how-to-combat-unreliable-automation/</a></li><li><a href="https://devqa.io/common-myths-test-automation/">https://devqa.io/common-myths-test-automation/</a></li><li>Amit Gupta, “Beyond the Script: Elevating Software Product Quality with Structured Ad-Hoc / Exploratory Testing — A Case Study”, International Journal of Science and Research (IJSR), Volume 9 Issue 9, September 2020, pp. 1644–1648, <a href="https://www.ijsr.net/getabstract.php?paperid=SR24608152029">https://www.ijsr.net/getabstract.php?paperid=SR24608152029</a></li></ul><p>I would like to extend my gratitude to <a href="https://medium.com/@reenasingh.aw"><strong>Reena Singh Kshatriya</strong></a> in co-authoring this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b365595582e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vmware-end-user-computing/health-check-for-test-automation-blue-green-deployment-4b365595582e">Health check for Test Automation: Blue green deployment</a> was originally published in <a href="https://medium.com/vmware-end-user-computing">VMware 360</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Increasing Quality Efficiency with Exploratory Testing]]></title>
            <link>https://medium.com/vmware-end-user-computing/exploratory-testing-362f654729e8?source=rss-cdc1ca87e790------2</link>
            <guid isPermaLink="false">https://medium.com/p/362f654729e8</guid>
            <category><![CDATA[test-driven-development]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <dc:creator><![CDATA[sakshi kakkar]]></dc:creator>
            <pubDate>Fri, 30 Apr 2021 21:09:34 GMT</pubDate>
            <atom:updated>2021-11-22T20:51:02.171Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Explore and break your own product — Guide for better testing coverage</em></p><blockquote>“Perfection has to do with the end product, but excellence has to do with the process” — Jerry Moran</blockquote><p>In this article, I’ll be talking about how my team changed the way we do exploratory testing to improve product quality. Before going into the details of it, let me first explain you what exactly exploratory testing is.</p><p><strong>Exploratory testing</strong> is a type of testing in which there are no initial scripted tests written, but the testing is based on the thought process on the fly. It is mainly a ‘thinking’ activity. It is the key to recognizing gaps and making changes in the product for improving its overall design, performance, and functioning without the need to undergo a long and planned <strong>testing</strong> approach.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*692qG0xlei5N4lKNw0Xshg.jpeg" /><figcaption>Exploratory Testing Framework Layout | Photo by <a href="https://testovation.com/BlogView?b=Exploratory-Testing-techniques">Tetovation</a></figcaption></figure><p>Thumb rule for exploratory testing: <strong>minimum planning and maximum execution</strong>. The more you explore together, the better you work as a team. Exploratory testing is not just about finding bugs —it’s a way to add a shield in the process of quality over your product before it reaches customers. We have to think about engineering excellence while reducing errors in the product.</p><p>Let’s give humans some value over machines. We could decrease the testing time by automating more tests and filling out the gaps, but <em>we still need humans for catching these in and out issues.</em></p><p>Initially, we were performing “exploratory testing” in a different way, testing the feature that was being released in that version of the app. However, it was not effective as we had hoped. We continued to find issues in production. I was tasked with driving the efforts in improving exploratory testing.</p><p>As a quality lead of the team, I made some changes in the process, worked around, did some analysis, and adapted a new approach to exploratory testing. The main target with this new approach was not just the feature being added in that particular release, but also some other main modules where we were getting escalations. We started with pairing each author with a helper, whose main focus was on breaking the product as much as they can without just relying on the scripted test cases like we do in regression. ​</p><p>The main criteria that our team adopted was hit it hard, be crazy, and break the product in every possible angle you can.</p><h3>Framework we used for “New Approach”</h3><blockquote>Exploratory Framework</blockquote><p><strong>How can you get started?</strong></p><p>You would need the tools and techniques mentioned below to get started with the process.</p><p><strong>Testing Environments<br></strong>Environment plays the foundational role that you are going to use for testing. Internal testing environment rarely mimic what customer use. Hence it is important to tests against an environment which includes actual customer use cases as well some involving different combinations of data instead of direct feature use cases.</p><p><strong>Configuration<br></strong>Figure out all the different configurations the customer would be using. There could be combination of different features, new operating system version in the market or a new device model. Once the data is collected, setting up an environment with the configuration to be used for testing would be ideal.</p><p><strong>User Personas<br></strong>Putting yourself in the shoes of the end-user or the admin and then trying to analyze and test the flow will make a huge difference in the testing. Giving up your knowledge for those 2 hours and try to determine the use case a customer will execute. Think it from the customer perspective.</p><p><strong>Charter<br></strong>Laying down the specific areas that need to be focused on during testing for a particular release. Rotating different modules per release will help the team gain their knowledge and also make sure that the feature is tested well. For example, if a person wants to learn about any existing feature in the product they can choose that because with exploratory it will help them increase their knowledge in that domain as well.</p><p><strong>Who can participate<br></strong>Team members (including DevOps engineers as well as developers), Product Managers, UI/UX team, Integration teams (teams getting affected by some of your modules). People from different team often look the product from a different perspective.</p><p><strong>Scope<br></strong>Priority is given to the features in scope per release but it is open-ended for anyone to pick any module; modules that were most affected and have customer escalations</p><p><strong>Time Limit<br></strong>Making testing time-specific gets your entire focus on one particular task instead of multi-tasking. We started by blocking two hours per release where everyone gets on a Zoom call so that if anyone had any questions, they could ask right away instead of chasing them over slack. With some changes in the future releases, we are also thinking about splitting the two hours over two days so that testing does not eat up too much of one day.</p><p><strong>Tactics<br></strong>Identify the type of specification for the feature/module and risk a new feature could have on the stakeholders; which type of testing to be performed, and which entry and exit criteria apply (like devices and the OS versions required for a feature). The more widely testing is done for any feature, the lower the chances of getting any customer-found bugs.</p><p><strong>Document and Process<br></strong>Documenting the steps so that it can be used for introspection, future improvements and determining the effectiveness of this approach.</p><p><strong>Advantages of Exploratory Testing</strong></p><ul><li>More quality-driven product</li><li>Intelligent testing more than scripted testing</li><li>Important bugs are found</li><li>Team members get to know more about the product</li></ul><p><strong>Future improvements in the framework</strong></p><ul><li>Including other team members like UI/UX teams and PM</li><li>Save the Date invite ahead of time to block everyone’s calendars</li><li>1 hour each on two different days to accommodate everyone’s schedule</li><li>RCCA on bugs found</li></ul><h3>Conclusion</h3><p>Without doubt, exploratory testing brings out the creativity of each person and has an adequate effect on improving the quality of the product. This technique works well because no two sets of eyes can have the same perspective in life; our team’s results speak for themselves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*p6Dqm8RovI2hjG1odzGXcw.png" /><figcaption>Internal Exploratory Testing Results</figcaption></figure><p>Hopefully, with the results it can encourage other teams to start using this approach; it takes somewhat more time and effort, but it’s far more effective from a product quality standpoint.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=362f654729e8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vmware-end-user-computing/exploratory-testing-362f654729e8">Increasing Quality Efficiency with Exploratory Testing</a> was originally published in <a href="https://medium.com/vmware-end-user-computing">VMware 360</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>