<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jeff Sing on Medium]]></title>
        <description><![CDATA[Stories by Jeff Sing on Medium]]></description>
        <link>https://medium.com/@jeffsing?source=rss-a21816f868bc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*IZBKrymP613fdcXl</url>
            <title>Stories by Jeff Sing on Medium</title>
            <link>https://medium.com/@jeffsing?source=rss-a21816f868bc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 19:02:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jeffsing/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Modern Quality Program from the Ground Up]]></title>
            <link>https://medium.com/@jeffsing/building-a-modern-quality-program-from-the-ground-up-2f94953eb460?source=rss-a21816f868bc------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f94953eb460</guid>
            <category><![CDATA[qa-testing]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Jeff Sing]]></dc:creator>
            <pubDate>Tue, 04 May 2021 20:18:40 GMT</pubDate>
            <atom:updated>2021-05-04T20:18:40.846Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ibrRMIQqesVlzHsSNACOEw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ballonandon?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ben Allan</a> on <a href="https://unsplash.com/s/photos/construction?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>If you review job boards for QA Manager or QA Tech Lead roles, you will find an increasing number of companies looking to create or “build a Quality Engineering” program. Talking to some of the engineering leadership at these companies, you often hear conflicting requirements and asks for this role (<em>Automation Architect, Quality Program Manager, Testing Coach, Quality Agile Leadership, our first Quality Engineer, etc… this list goes on forever</em>). This is because building a Quality Engineering from scratch these days is a completely different experience than that of Quality Programs from the last decade.</p><p>Gone are the days where companies will hire a horde of testing specialists who act as a safety net, waiting for code to be tossed over the proverbial wall by developers to perform validation before a release can happen. Nowadays, a modern Quality Program is significantly smaller and more specialized. They rely on technology, push a concept of “Quality Culture,” and are pseudo Quality Program Managers to ensure that engineering organizations successfully deliver value to their customers.</p><p>So, to understand why this change has happened, I’ll be exploring why the old Quality Programs of yore no longer make sense, how newer modern Quality Programs address these shortcomings, and what tips and tricks I have found immensely useful in building a Modern Quality Program from the ground up.</p><h3>Why Software Companies Moved Away from QA</h3><p>Why have software companies moved away from QA Departments to dedicated QA Engineers? With modern software development practices, <strong><em>traditional</em></strong> QA departments are quickly becoming obsolete.</p><p><strong>Startups are on the rise</strong>. The number of tech startups have increased by 47% since 2007 (<a href="https://itif.org/">Information Technology and Innovation Foundation</a>). In startups, especially in their infancy, the most important goal is to find product market fit.</p><p>Engineering is tasked with solving this and delivering at high velocity. Quality often takes a back seat until the company matures. At this point, engineering teams often learn to operate without a dedicated QA Program.</p><p><strong>Code is deployed frequently</strong>. Facebook’s motto was famously “<a href="https://www.businessinsider.com/mark-zuckerberg-innovation-2009-10?r=US&amp;IR=T">Move Fast and Break Things</a>.” Software companies around the world have adopted the concept of constant code deploys based on the idea that mature tooling allows developers to get code quickly out to customers for instant feedback, and quick fixes.</p><p>This is a stark change from when companies had long cycles before they cut a release, and therefore needed multiple validation steps before the product was safe for consumption.</p><p><strong>Shipping a defect isn’t as significant</strong>. In many startups, the impact of releasing a defect is low — especially when compared to the need to ship features at high velocity.</p><p>Unless your software company deals with medical history for patients, sensitive financial information, or some financial penalty if a product doesn’t meet SLA, the cost of a defect is not as significant.</p><p><strong>Automation and tooling has matured</strong>. Improved tooling and the maturity of deployment pipelines has created the expectation that developers begin to test their own code earlier in the development lifecycle (<a href="https://en.wikipedia.org/wiki/Shift-left_testing">shift-left testing</a>). As a result, engineering leaders now sometimes question the need and value for testers near the end of the deployment lifecycle.</p><h3>Build a Modern Quality Engineering Program</h3><p>Modern Quality Engineering Programs address many of the shortcomings mentioned above and tackle some of the more complex problems that can’t be solved by automation or a mature deployment process.</p><p>Here are my keys to building a modern Quality Engineering Program:</p><p><strong>Be the customer advocate.</strong> Automation testing does a great job of answering the question of“ Did I build the product right?”, but it does not answer, “Did I build the right product?” Modern Quality Engineering Programs drive processes that collect customer usage data and incorporate them into testing and release validation.</p><p><strong>Build it bespoke</strong>. It’s essential to understand what your engineering organization excels at and where it needs to improve. Good programs are agile at adapting to Quality needs, and adjust day to day operations to serve the engineering team. For example, your engineering organization may have a very mature automation framework, but the governance of certain processes is weak. You may find yourself spending more time fixing these processes than coding.</p><p><strong>Coach on Quality</strong>. It’s highly unlikely that you will have a large team of testers. Instead, each Quality Engineer should be a specialist who partners closely with development teams to build and maintain the right testing frameworks. A Quality Engineer may build out the framework and libraries, but have the developers build the test cases for their own features. Quality Engineers may also be driving forward testing in gaps of coverage (see <a href="https://devops.com/shift-right-testing-the-emergence-of-testops/">shift-right</a> testing paradigm).</p><p><strong>Audit Engineering Decisions through Data Driven Process</strong>. In early 2014, Facebook changed their famous motto to “<a href="https://mashable.com/2014/04/30/facebooks-new-mantra-move-fast-with-stability/">Move Fast with Stable Infra</a>.” They “<em>…realized over time is that it wasn’t helping us to move faster because we had to slow down to fix these bugs and it wasn’t improving our speed.</em>” An engineering organization can easily find itself going down the wrong path. It’s important for a Quality Program to constantly audit the health and quality of an engineering organization through data to determine if the initial hypothesis is correct. Sometimes a catchy mantra isn’t enough, and a little more thoughtfulness will actually increase developer velocity.</p><h3>Tips and Tricks that I found invaluable to building it fast and right:</h3><p><strong>Find the value in every decision. </strong>For every decision — whether it’s a new process or new tool — it’s important to define the value add for your customers or engineers. Industry best practices and vendor recommendations may not fit your team.</p><p><strong>Establish KPIs</strong>. It’s impossible to quantify improvement without good data. Know the metrics and KPIs you’re measured by, and create hypotheses for improvements based on them. After a few cycles, analyze your data. Did you make the impact you’d hoped? If not, adjust and repeat.</p><p><strong>Ask uncomfortable questions.</strong> Just because things have always been a certain way, doesn’t mean it’s the only way or the right way. Part of the Quality mindset is asking “why.” If you think something can be improved upon, be ready to bring a solution to the table and drive the discussion forward.</p><p><strong>Take on low-hanging fruit.</strong> In the beginning, you may want to take on a big moonshot project but be bogged down with smaller cleanup tasks. Worry not. You’re driving improvement and creating value. Eventually, your momentum will carry you through to your larger goals.</p><p><strong>Reach outside of engineering.</strong> It’s important to understand the challenges faced by stakeholders outside of engineering: customer service engineers, solution engineers, account executives, marketing teams, etc. Learn their stories. They’ll help you build and steer the right programs. Also, share the initiatives that Quality is running to delight customers. This will help build confidence in engineering and strengthen your company’s overall ability to serve its customers.</p><p>These are a few of the high level points to building a Modern Quality Engineering program. If you remember nothing else, remember to build a bespoke Quality Program for your company and customers. No two programs will ever look the same. The needs of your engineers and customers will vary.</p><p>I would love to hear feedback or stories from your journey. You can reach me at jeffsing@gmail.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f94953eb460" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Happy Birthday Optimizely’s QA Team]]></title>
            <link>https://medium.com/engineers-optimizely/happy-birthday-optimizelys-qa-team-7f1ee5372d8f?source=rss-a21816f868bc------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f1ee5372d8f</guid>
            <category><![CDATA[quality-assurance]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Jeff Sing]]></dc:creator>
            <pubDate>Thu, 02 Jan 2020 19:17:16 GMT</pubDate>
            <atom:updated>2020-01-02T19:17:16.444Z</atom:updated>
            <content:encoded><![CDATA[<p>Happy Birthday Optimizely’s QA Team</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ES6jilsFPGZ6xIE9" /><figcaption>Happy 2nd Birthday to Optimizely’s Quality Engineering Team!</figcaption></figure><p>Optimizely’s Quality Engineering team was born on January 1st, 2018. It’s been quite a journey since that first day. To mark the anniversary, I want to take some time to reflect on important lessons we learned along the way about how to raise the quality bar of an existing product and build a successful QE program.</p><p>But first, a look back. Prior to having a QE team, engineers at Optimizely were directly responsible for the quality of their code. They wrote their own unit and integration level tests, contributed to a global end-to-end framework that was jointly maintained by all developers at the company, and each team developed and executed their own test processes.</p><p>This model, called “total ownership,” served to hold all engineers accountable for quality. That worked — for a while. But as the complexity of the product increased, there was a general feeling that the quality of the product should be improved.</p><p>Two years ago, I was brought on board to understand a key question: “Where in our product is quality low — and why?”</p><p>Today, we’ve grown the team from a single engineer (me!) to five, full-time Software Quality Engineers. In that time, we’ve done the following:</p><ul><li>Created a new automation framework using Cypress.io for a product that previously had no automation.</li><li>Produced a monthly Quality Report that details the state of our product, the high severity bugs that have occurred, and the traceability/analyses of each one. This report also includes the strategies that we’d employ for prevention.</li><li>Host Quarterly Reviews for leadership to review the state of the product and drill down into data that describes trends and traceability.</li><li>Categorized regression test cases that weren’t covered by the current regression automation framework and added them.</li><li>Improved the Bug Bash program for product launches.</li><li>Created a “Sanity Check” stage in our deployment cycle and staffed it.</li><li>Provided domain expertise to help all customer-facing teams.</li></ul><p>Additionally, we’ve embedded Software Quality Engineers in our development teams. Each Quality Engineer regularly performs the following for their team:</p><ul><li>Creates an Acceptance Criteria process that clearly defines test deliverables.</li><li>Generates test cases and test plans for each product team.</li><li>Partners with developers to create automation tests for new releases.</li><li>Audits and triages incoming tickets.</li><li>Tracks and audits the feature flags for the team.</li></ul><p>How did we decide on this path? Well, if you go online and search “how to start a QA program,” you’re likely to find two types of articles: how to build an automation program (think, testing pyramid) and an encyclopedia of all the different process types that you should implement (including regression tests, sprint tests, story workshops, etc.). These resources are great for understanding what different processes are. But every company has a different view of what they want in a QA program — and for many, the suggestions just won’t fit.</p><p>For example:</p><ol><li>You may not need to set up an automation testing framework. Optimizely already has its own developers contributing to its own internal framework.</li><li>You may not be able to implement every single quality-related process because the trade-off between running each process and development velocity may not pay off.</li><li>You may hit a cultural wall where different development teams have a total ownership culture and do not want to farm out these tasks to a QE team.</li></ol><p>So, here are the most essential components that we used to start and run a successful QE program at Optimizely.</p><p><strong>Understand your Quality mission statement</strong></p><p>Every company that I’ve worked for has had a QE team, but each one needed something different. Before one comes in and begins work, it’s important to understand what you plan to build and why.</p><p>At Optimizely, the goal isn’t just to build an awesome new automation framework, run manual tests, or write a million test cases. Our Mission Statement is holistic: “To make our developers’ lives easier, so they can build and create a product with high quality.” With this in mind, every tool we build, every process we create, and every program we run <em>must</em> answer this question. The mission statement helps us make sure that our choices are right for Optimizely — and that we don’t build frameworks or processes just because it’s what the book says.</p><p><strong>Solve your problem using different tools</strong></p><p>This perspective has been covered extensively in many articles online, so I won’t go over it in depth. But we find that it’s important to make sure to use all three Quality Engineering processes (Quality Assurance, Quality Control, and Robust Testing) to solve our problems. Some problems do often require building automation tools, but sometimes the right manual process can be more cost efficient and useful.</p><p><strong>Build milestones and visibility</strong></p><p>Starting a new program is the equivalent of being thrown in the middle of the ocean and told to swim to shore. There isn’t a lot of direction, especially since you’re supposed to be the one giving directions. Make sure to ground yourself by building milestones toward what you plan to do, and build visibility in the organization when you hit them. Milestones allow you to measure your success, prove ROI of what you’re doing to leadership, and figure out what to do next.</p><p>Often, if I am unsure of the next big project to tackle, I look at the reports that I’ve generated and search for trends in problems that we’re seeing. This allows me to create the right project proposals to tackle these issues and show ROI. Having the before and after, and seeing the improvement helps me justify that each project that we take on is the right one.</p><p><strong>Make sure you have an awesome champion</strong></p><p>No Quality Program will ever succeed without an awesome champion from leadership to stand behind it. I am blessed with two awesome Directors of Engineering who were all in behind me (Joy Scharmen and Keith Nottonson), and our Senior Vice President of Engineering (Bill Press). Make sure you have a champion who understands what you’re trying to do and is supportive. Reach out to them when you encounter roadblocks and share your strategies so they can support you.</p><p>Finally, keep reading, attending conferences, and learning. I am always blown away by how our industry is constantly changing and what new approaches are being developed. With this new year just beginning, I’m excited to see how Optimizely’s QE team will continue to grow and I look forward to celebrating many more Quality birthdays to come!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f1ee5372d8f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineers-optimizely/happy-birthday-optimizelys-qa-team-7f1ee5372d8f">Happy Birthday Optimizely’s QA Team</a> was originally published in <a href="https://medium.com/engineers-optimizely">Engineers @ Optimizely</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>