<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Olga Beregnaya on Medium]]></title>
        <description><![CDATA[Stories by Olga Beregnaya on Medium]]></description>
        <link>https://medium.com/@olgaberegnaya?source=rss-1998667b1425------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*RuWXrEya3_5vvSqj2Ijydw.jpeg</url>
            <title>Stories by Olga Beregnaya on Medium</title>
            <link>https://medium.com/@olgaberegnaya?source=rss-1998667b1425------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:27:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@olgaberegnaya/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What is in your Testing Scope?]]></title>
            <link>https://medium.com/wix-engineering/what-is-in-your-testing-scope-8846714d4358?source=rss-1998667b1425------2</link>
            <guid isPermaLink="false">https://medium.com/p/8846714d4358</guid>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[scopes]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[test]]></category>
            <dc:creator><![CDATA[Olga Beregnaya]]></dc:creator>
            <pubDate>Wed, 25 Aug 2021 18:50:09 GMT</pubDate>
            <atom:updated>2021-08-30T13:01:13.108Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lxeyCe6eNS0OZ9WOpCwhFg.jpeg" /></figure><p>I decided to talk about the testing scope since I realized that not everyone clearly understands what it is and how to define it. What should be included and what should be ignored? How detailed should the scope be? Do I have to align it with my stakeholders? What if the deadline is coming, and I don’t have enough time to test even the business-critical areas?</p><p>Before starting our journey into the testing scope depths, I would like to mention that the testing scope it’s not a “must-have” as a QA base skill set. I chose to use it because I found it fits the best to my testing needs. In this article I’m sharing my personal thoughts and the vision of the testing scope.</p><p>Our role as testers tells us that in order to gain the accurate assessment of a product’s quality we have to test simply everything. In an ideal world that would be the case. However, as our reality constantly changes, the best we can do in order to provide a result without losing those important moments is to perform short test cycles with fast feedback about the state of the product.</p><p>But how can I deliver a quick result if, potentially, I have an endless list for testing, you may ask. The answer is to create a testing scope that focuses on the most critical product aspects based on the business needs.</p><p>Before we discuss how to design the testing scope, let’s first define what it is.</p><p><strong><em>A Testing Scope</em></strong><em> is a list of product features, product parts, or product-related integrations that must be tested in order to build a reliable assessment of a product’s quality.</em></p><p>The Testing Scope doesn’t include details of the testing strategy, the automation approach or the tools that are going to be used. I won’t list who will perform the testing or when it will happen. The document that represents the details above is called the Testing Plan. It answers the questions “Who”, “When”, “Why”:</p><ul><li>who will perform the testing,</li><li>when to perform the testing,</li><li>why to test / not to test specific product parts, etc.</li></ul><p>The Testing Scope itself answers the question “What”: what to test.</p><p>Let’s take a look at a small example. Imagine we have a feature, ‘<strong>a new layout for a site component’</strong>. Let’s think about which parts of that feature should be tested in order to conclude whether it meets the expectations or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*5yoCoNz71LW3gY2NB_JAUw.png" /></figure><p>In the example above you can find the different testing types like UI, functional, performance, accessibility. Each product item is defined on a very high level.</p><p><strong>How to find out which testing types to apply?<br></strong>To answer this question, explore your feature from different angles:</p><ul><li>feature has a UI part — think of adding UI/UX, functional, accessibility, performance testing to your testing scope;</li><li>feature uses an API, it makes sense to test server-side validation and error handling;</li><li>feature reads/writes to a database — security testing helps ensure there won’t be unaccounted DB access for malicious reasons;</li><li>your product has a representation on mobile devices — mobile testing will come in handy.</li></ul><p>I prefer to talk to the product manager and the developers before starting to work on the testing scope. This gives me the most up-to-date information. Documentation usually is written in the early stage and not all the changes that happen during the development are reflected there.</p><p>After I gather the information, there is a good practise in my team to do a collective brainstorming session. I work with 9 people, great experts, whom I can invite to this session where we think of any possible user scenarios and examine failure cases. The result of such a session is a draft of the testing scope that later on will be systematized and optimized according to actual resources that I have.</p><p>We don’t do brainstorming for every single feature. For small features usually it’s clear what has to be tested. But for large complex projects with a lot of integrations and dependencies, having a second pair of eyes helps a lot. At this stage, when you just explore your product or feature, there is no need to worry whether you have enough time to test everything you plan or not. Be agnostic to your environment, try to produce as many ideas for testing as possible and later on do another talk with your developer and/or product manager to figure out what is critical and what can be skipped.</p><p><strong>What should a testing scope level of detail be?</strong></p><p>The level of detail of the testing scope should be selected according to the needs of the tester. For instance, If I write the testing scope only for myself (I’ll be the only QA on the project), the left part from the example above should be enough for me to understand the task. In case I create the scope for a group of testers that are new to the product, I will most likely add more explanations and even split each item into subtasks.</p><p>It is possible to list the feature parts that won’t be tested in the Testing Scope. Or to consider that everything that is not listed in testing scope (out-of-testing-scope) will be omitted during testing.</p><p><strong>Which stakeholders should be aware of the testing scope?</strong></p><p>There are several reasons that may incline you to notify some of the stakeholders and ask them to review or approve your testing scope:</p><ul><li>When a feature has parts that for some reason cannot be tested, it’s highly recommended to let the manager and the developer know. It always leads to some risks that should be discussed and accepted;</li><li>When you find the use case scenarios that are not covered in the design or the documentation, it makes sense to raise an issue about it. In my practise designers might forget to add an error screen (general error like 404 or 500 or a specific server-side error);</li><li>Sometimes, when working on the test scope you may find the illogical user scenario or the missing requirements. And again, it’s better to raise this issue at the early stage, rather than after the development is completed.</li></ul><p>Since we defined what the testing scope is, let’s consider what can help us optimize its content and focus on the very critical areas for the business.</p><p><strong>How does the business influence the testing scope?</strong></p><p>Let’s examine particular products and features according to their business goals and figure out how they affect the testing scope:</p><ul><li><strong>Killer Feature or a Game Changer</strong>: an innovative feature that potentially can take your product to market leadership. In this case time plays the most important role. The sooner the killer feature is released, the more users it engages. Quality becomes secondary: once the happy flow is working, it’s acceptable to release the feature to the customers even with a large number of non-critical bugs. For testers such business goal means performing frequent test cycles focusing on the critical business areas;</li><li><strong>Refactoring of a critical business area</strong>. Refactoring means that the user experience should not change. The goal of such a job, in most cases, is to improve performance, strengthen security, migrate to a new API, and so on. Thus, the task for the tester is to prevent regression. If during the release the main feature flows work as expected, but we produce a large number of new bugs, the users most likely will be unhappy and may tend to leave our product. In the testing scope, it makes sense to thoroughly cover each product part with tests and, in addition, add a scenario that covers the feature’s intent: execute performance testing, verify new API, examine security;</li><li><strong>PoC (Proof of Concept)</strong>. PoC is an idea or a solution that the company wants to try and explore before releasing to a large audience. PoC can also be used to choose the best from several similar approaches or technologies. The main goal of a PoC product is to demonstrate the particular solution, no matter how buggy it is. Often such a feature doesn’t require any testing at all, but in particular cases, testers may be involved as domain experts. In this matter the testing scope should include use cases that help reveal the weak spots of the solution chosen.</li></ul><p>Examples above illustrate how the business changes the testing focus and directly defines the testing scope. That’s why it’s important to ask WHY: why we want to release a particular product, which user issue it aims to solve, how it helps the company to achieve its goals. Many times I noticed testers that while working on a big project weren’t able to answer a simple question — ‘why this project and why now?’. Not knowing Why creates a risk of missing the deadline (if you are working on a killer feature), or doing extra work (if it’s a PoC approach).</p><p><strong>How can people in your team help minimise your testing scope?</strong></p><p>What you should definitely avoid while testing is performing double work. If you have a UX designer in your team that always verifies the user interface before a feature release, you can use it as a chance to shorten your testing scope. Of course, the designer can miss some client issues (nobody does testing better than you), but that is the risk, the justified risk.</p><p>As testers we cannot afford ourselves to take enough time to cover every single piece of our product. We are limited in time, resources and budget. That’s why we have to manage our assets carefully. Thus, it’s good to know our team members, what they are doing and not to be afraid of delegating a part of our responsibilities. Of course, if that doesn’t contradict with people’s roles (simply, don’t expect a business analyst to be able to perform proper API testing :)).</p><p>In a situation when you have a tough deadline and you don’t have the time to cover even the critical product areas, it might be a good idea to ask your team for help. Do a bug hunt session, take this chance to learn how the people you are working with explore the product, what is important to them in the first place.</p><p><strong>Conclusion</strong></p><p>The testing scope is a high-level list of product parts that have to be tested in order to gain a reliable assessment of a product’s quality.</p><p>To figure out which testing types to apply, explore your product from different angles. Choose the level of details that fits your needs best: think who will perform the testing, how well the executors know the SUT (System Under Testing).</p><p>As an additional step, check if your testing scope has to be reviewed by your stakeholders. This helps align the deadlines and clarify all the controversial points.</p><p>Finding out the business goal of your product or feature will give you a clue about what to focus on. Talk to the owner of the product, get to know why the company wants this feature and who will be its audience.</p><p>And finally, consider delegating some parts of testing to your teammates. Often we are limited in time or human resources, and it’s totally OK to ask for help and collaborate to achieve the one goal — ensure the product’s quality.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8846714d4358" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wix-engineering/what-is-in-your-testing-scope-8846714d4358">What is in your Testing Scope?</a> was originally published in <a href="https://medium.com/wix-engineering">Wix Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Survive Production Incidents in a Company with No-blame Culture]]></title>
            <link>https://medium.com/wix-engineering/how-to-survive-production-incidents-in-a-company-with-no-blame-culture-c16d36635b23?source=rss-1998667b1425------2</link>
            <guid isPermaLink="false">https://medium.com/p/c16d36635b23</guid>
            <category><![CDATA[wix]]></category>
            <category><![CDATA[company-culture]]></category>
            <category><![CDATA[wix-engineering]]></category>
            <category><![CDATA[incident-management]]></category>
            <dc:creator><![CDATA[Olga Beregnaya]]></dc:creator>
            <pubDate>Wed, 13 Jan 2021 08:08:05 GMT</pubDate>
            <atom:updated>2021-01-13T08:08:05.849Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xhQ3kMvjIxq5X-mEm2rKfQ.jpeg" /></figure><p>Every so often in our lives we all face a truly terrifying creature — a production incident. I am not referring to, say, pharma or aircraft manufacturing industries where a software bug can cost human lives, of course. Yet even for web services minutes of downtime can hurt users’ businesses, ruin company’s reputation, or damage people’s careers.</p><p><em>A </em><strong><em>Production Incident</em></strong><em> is a massive system failure that is accompanied by significant damage for consumers of the product.</em></p><p><em>Examples of production incidents: system downtime, payment transactions are not going through, user data corruption, etc.</em></p><p>Does this mean that once a production incident takes place, it’s always someone’s fault? Was there a QA who didn’t do their job thoroughly enough? Was there a developer who implemented something at the last moment, or a product manager made a decision to release a particular feature too early?</p><p>Certain companies have a policy of looking for someone to blame when bad things happen, and when found — punish them as a warning to others. Can an approach like that motivate people to work better? I don’t think so. Fear of making a mistake, public dispraise can only kill creativity and train people to avoid hard decisions and thoughtlessly follow the rules.</p><p>If the penalty tactic doesn’t work, should we just turn a blind eye to production incidents? Nobody can avoid making mistakes, so why not treat them as part of a working process? Might sound logical, except for one small detail — a production incident is not a trivial bug. Having a production incident means there’s a major issue with the system, with the processes or with communication. To ignore such an issue would be to consciously let it happen again in the future, which, in my opinion, should be unacceptable for any successful company.</p><p>What might help here is promoting a company culture that would encourage people to have open discussions about any type of failures, and at the same time teach people about personal responsibility. This would also help to create a safe environment for the company employees, where they have room for error.</p><p>This is referred to as ‘no-blame culture’. I’m lucky to work in a company with this exact culture — Wix — and want to share with you how it looks from the inside.</p><h3>Let’s consider the phases of an incident:</h3><ol><li>Incident discovery (system alert, user complaints, self-identification)</li><li>Data collection (find what doesn’t work, who is affected, steps to reproduce, etc.)</li><li>Criticality assessment (which side of the business is affected, types of users, calculate possible aftermath)</li><li>Actionable steps to recover the system (fix, rollback, switching traffic, etc.)</li><li>Minimization of damage done to users (run a script to restore user data, contact users if needed)</li><li>Root cause analysis &amp; discussion</li><li>Postmortem sum up</li><li>Work on action items to improve the system</li></ol><p>Once an incident has been detected, the first thing that should be done is to collect as much data as possible to localize the buggy scenarios, understand user impact and work out a plan to recover the system. It is impossible to do it all alone: it requires involving customer care agents, business analytics, operation managers, tech leads, etc. Not to mention that the more coordination there is within a team like that, the faster and more seamlessly the incident will be extinguished.</p><p>Don’t let emotions take over you — whether it’s anger, irritation, or disappointment. It’s better to put all the efforts into minimizing damage done to users — and when that’s taken care of, with a cooled off heart, to do the research to find the root cause and the underlying factors of the issue. It’s time for discussion; not for the individual journey, but a collective brainstorm. The trigger of the incident might as well be a specific action by an employee, but the real cause, in a complex system, in the majority of cases is actually a combination of a variety of factors.</p><p><strong><em>Let’s take a look at a very common example:</em></strong><em> a service ran out of memory and stopped responding. The trigger for the incident was a commit by a developer. The fix is simple — roll back to the last production version.</em></p><p><em>In this case, the incident analysis might be:</em></p><p><em>- check with devs why that service ran out of memory</em></p><p><em>- check the deployment process with DevOps</em></p><p><em>- check with QA why it wasn’t noticed during testing</em></p><p><em>- check test coverage with the automation team</em></p><p><em>- check with support agents the production monitoring</em></p><p><em>- etc.</em></p><p>I hope you agree with me that a complex analysis like that can hardly be done by just one person. You know, it’s like going to a doctor with a persistent headache. A neurologist would be looking for the problem in the nervous system, an endocrinologist — in the endocrine system, an ophthalmologist would check your eyes first. And what are the chances their diagnosis is correct? Often a symptom caused by the nervous system is triggered by a disease of the endocrine system and vice versa.</p><p>What if instead of reviewing all of them one by one you could have them all in one place? There is a good term for it — a synergy, when the result of the group work significantly exceeds the result of each of the participants working separately. In medicine, synergy is represented by doctors’ concilium which allows to perform a comprehensive examination of the patient’s issue and usually helps properly diagnose the actual disease.</p><p>The same is applicable when investigating a production incident. Use the synergy power and a variety of people with expertise in different fields to build an all-around picture that would cover much more ground.</p><p>All the above refers to the main point behind a no-blame culture:</p><ul><li>Work together on fixing the issue</li><li>Have a group discussion to learn about the cause of the issue</li><li>Perform action items to improve the system</li><li>Move on</li></ul><p>You may say now that no-blame culture looks good, but does it provide enough awareness and a sense of responsibility for major failures? My answer is yes — and the tool for such ‘educational practice’ is a Postmortem. Postmortem is a document where we publish all the information regarding the incident, including the timeline, the detailed reasons, user impact, people who were involved, how we fixed it, and the action items that have to be performed in order to not let it happen again. This document is published for the whole company to see, allowing anyone to learn about the incident.</p><p>There is always a responsible person or group of people who own a Postmortem. I participated in writing one a few times and I can tell that once you gather information related to user impact, you will do anything to not let it happen again. When you see the impact an issue, a system failure has had on users, it really stays with you and becomes a huge factor in drawing conclusions and reflecting on the situation, learning lessons from it.</p><p>At last, I want to tell what no-blame culture means to me personally:</p><ul><li>for me as an RnD engineer, a culture like that provides a safe environment where I can learn, experiment, and play with my product knowing that if I make a mistake, there will be people for help</li><li>as a QA leader, I can rely on it and encourage my team to try new things, to be open and supportive to others when needed, to allow them to be responsible for their decisions, and make sure they have room to grow;</li><li>as a manager, I learn how to deal with crises and how to communicate with different departments and provide feedback.</li></ul><h3>To sum it up</h3><p>There is no way to avoid production incidents completely. Using a penalty system or simply ignoring incidents is bad practice in the sense that it kills people’s creativity and doesn’t let them learn from what happened.</p><p>An alternative method to handle production incidents is based on the no-blame company culture. In a nutshell:</p><ul><li>leave emotions aside, focus on fixing production issue</li><li>do not blame the person who triggered the incident, instead, do a group discussion to build an all-round picture and reveal all aspects of an incident;</li><li>work out and execute action items to improve the system;</li><li>assign an owner for the Postmortem sum up, let people take the responsibility for their actions;</li><li>move on.</li></ul><p>This culture means a lot to me personally. I can safely play and experiment with my product, teach my team to be supportive and responsible, and grow as a manager by learning how to deal with crises.</p><p>I hope you now have a few ideas of your own on how to survive a hard time like the one we just discussed. Stay safe and don’t be afraid to make a mistake!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c16d36635b23" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wix-engineering/how-to-survive-production-incidents-in-a-company-with-no-blame-culture-c16d36635b23">How to Survive Production Incidents in a Company with No-blame Culture</a> was originally published in <a href="https://medium.com/wix-engineering">Wix Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hidden Depths of Product Requirements]]></title>
            <link>https://medium.com/wix-engineering/hidden-depths-of-product-requirements-892c5ec055e6?source=rss-1998667b1425------2</link>
            <guid isPermaLink="false">https://medium.com/p/892c5ec055e6</guid>
            <category><![CDATA[requirements]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[product-requirements]]></category>
            <dc:creator><![CDATA[Olga Beregnaya]]></dc:creator>
            <pubDate>Fri, 07 Aug 2020 13:00:15 GMT</pubDate>
            <atom:updated>2020-08-07T13:00:15.957Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BKkwzK6W3aK6uyR4IheD5A.jpeg" /></figure><p>When a testing task comes in, the first thing we usually think about is what the requirements are. If I have to perform a test, I need to know the expectations in order to evaluate the extent to which my system under test (SUT) corresponds to them. The more requirements I have, the more test cases I can think of.</p><p>But what if I only get high-level requirements, a short description of the core feature’s functionality, or even worse — a Jira ticket with just the title and a totally empty description field. What to do in this case? I see 3 possible scenarios here:</p><ol><li>send the task back to a task owner and ask to provide more details;</li><li>take what you have and do your best to test the product;</li><li>start on a requirements journey of your own.</li></ol><p>The first two scenarios are quite feasible. It’s totally OK to request the feature owner to provide more information. And I for sure understand someone who would do their job based on the task assigned. However, if you want to be more than a person executing written steps, it’s your mission to gain enough knowledge to be able to create a test coverage that is able to reveal the majority of bugs hidden in the product.</p><p>A missing product requirement is a potential production bug. The cost of such a bug, found after the release, is about ten times bigger than if it were found at an early development stage.</p><p>If you want to discover more about the requirements, stay with me. Let’s examine the requirements together, learn where to look for them, and why QA should question each piece of information they receive.</p><p><strong>The requirements</strong> are the set of expectations that SUT should meet. The requirements can be explicit — formally documented in a certain tracking system, or implicit — based on common knowledge about the usage, appearance, properties of a particular feature. Let’s get into details.</p><p><strong>Explicit requirements</strong> represent the core idea of the feature. This is what we, testers, use to check the positive flow, to perform a sanity test cycle. If we’re talking about a business feature and not a technical task, explicit requirements are usually described by product managers as a User Story.</p><p>Example: “<em>As a user, I want to add a product to a shopping cart.</em>”</p><p>This simple use case gives me a clue about my main test case: add a product to cart, check that the product has appeared in cart. Then I can think about more cases like:</p><ul><li>add same product to cart;</li><li>add several items of the same product to cart;</li><li>add a few different products to cart;</li><li>etc.</li></ul><p>Let’s imagine that we delivered an online store website with an ‘add product to the shopping cart’ feature to the end-user. What if some of our users already have experience with the logic of online stores and expect our application to work in a similar manner? For instance:</p><ul><li>when adding a product to the cart, there is a cart icon component that shows how many products there are in the cart (we can call it ‘best practice’ for online store sites);</li><li>what if there’s a law in our target country which forbids the sale of certain products to persons under 18 y.o. (this requirement comes from legal);</li><li>it could be that we’re doing a redesign of some kind and our users expect the new product to support the ‘old’ feature, like cart preview (this requirement is called product history);</li><li>and so on.</li></ul><p>The set of requirements above was not documented, but they are all important for the product to be competitive, conform to legal requirements, and consistent with previous versions. These are the<strong> implicit requirements.</strong></p><p>You can ask, how would QA know about these requirements? Where to look for them? Here are a few tips about possible sources:</p><p><strong>Products History</strong> — does your product have a ‘history’? If yes, learn thoroughly the previous version(s) and pay attention to when the new product misses some of the ‘old’ features. That ‘old’ feature might be missing on purpose, but it’s a QA job to verify it.</p><p><strong>Similar products inside your company</strong> — my product is an eCommerce platform, however, at Wix, there are a bunch of other verticals that also have the Checkout Flow. In this case a ‘vertical’ is a software product that solves business problems of a particular group of users, like store owners, fitness trainers, event managers, etc. What if each vertical has a different UI, different navigation, and some custom features implemented in the Checkout Flow? I bet that’ll drive your users crazy! Good user experience practices suggest that similar products should be predictable and do exactly what users would expect.</p><p><strong>Similar products on the market </strong>— same as the previous, but instead of comparing similar products in your company, take a look at products of the same type and kind on the market.</p><p><strong>Developer code / System Architecture</strong> — read the code, talk to developers, ask them to draw a model of the system, check what they expect from it. Sometimes you may find hidden features you would never find unless you knew exactly where to look. It could be error handling, limitations, big data-related flows, etc.</p><p><strong>World wide known standards, regulations</strong> — Accessibility, GDPR, performance standards. Apply whatever is applicable to your product.</p><p><strong>Analytics data</strong> — if you have access to the company’s analytics data, you have everything you need. There is no better source of info on your product and your users:</p><ul><li>Most popular browsers, operating systems, mobile devices;</li><li>GEO breakdown;</li><li>Product funnels.</li></ul><p><strong>User feedback</strong> — since our main goal is to satisfy our customers, we have to take care in listening to what they have to say. Take a look at their feedback: what are their main pain points, do they clearly understand the product, is there anything they’d like to add? This should give you some good ideas about what to consider as requirements.</p><p>Just be mindful of the fact that sometimes a user can complain about how a product behaves when what they are lacking is a specific feature. It’s like when you ask someone “why do you need a glass?”. People usually say that they need it to drink water. What if they don’t have a glass? They might say they will use their hands. So the reason they need a glass is to make it convenient to drink water, which means it doesn’t have to be a glass per se.</p><p>The last thing I want to talk about is that you shouldn’t perceive each requirement you get or find as the source of truth. Since we are testers, we should question our requirements the same way we question our product. Requirements may also contain bugs: misinterpretations, uncertainty, impaired integration with different product parts, conflicting conditions, etc.</p><p>Often problems with requirements arise when those come from different sources: product manager and technical lead, designer and developer, worldwide known practice and custom solutions.</p><p>The final set of requirements should represent a system of rules and conditions that clearly describes all product aspects and unequivocally states if certain product appearance or behavior is desirable or a bug.</p><p>As a conclusion.</p><p>Requirements represent people’s expectations about a given product. They are divided into <strong>explicit</strong> — formally documented in a certain tracking system, and <strong>implicit</strong> — based on common knowledge about the usage, appearance or properties of a particular feature.</p><p>It’s QA mission to gain as much knowledge about SUT as they can in order to create a test coverage which would be able to reveal the majority of bugs hidden in the product.</p><p>Here are some good tips on where to look for implicit requirements:</p><ul><li>Products History</li><li>Similar products inside your company</li><li>Similar products on the market</li><li>Developer code / System Architecture</li><li>World wide known standards, regulations</li><li>Analytics data</li><li>User feedback</li></ul><p>Finally, each requirement you obtain should be questioned and tested the same way we test the product, because requirements may also contain bugs.</p><p>Start your requirements journey, I hope you will enjoy it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=892c5ec055e6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wix-engineering/hidden-depths-of-product-requirements-892c5ec055e6">Hidden Depths of Product Requirements</a> was originally published in <a href="https://medium.com/wix-engineering">Wix Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to write an efficient QA Report]]></title>
            <link>https://medium.com/wix-engineering/how-to-write-an-efficient-qa-report-61d0ded730e1?source=rss-1998667b1425------2</link>
            <guid isPermaLink="false">https://medium.com/p/61d0ded730e1</guid>
            <category><![CDATA[report]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Olga Beregnaya]]></dc:creator>
            <pubDate>Fri, 29 May 2020 14:20:06 GMT</pubDate>
            <atom:updated>2020-05-29T14:20:06.572Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xPWwG26tcc57fFhMBxIONw.jpeg" /></figure><p>In our daily lives, information plays a very powerful role. Ability to provide the right kind of information to the right people is one of the vital skills for any QA engineer. In this article, I want to share my thoughts on what an efficient QA Report (also referred here as Status Report or Test Report) is, how to write one and how it influences the QA professional appearance.</p><p>Get a cup of your favorite tea, we are starting…</p><p>I’ve been in testing for 9.5 years, working at a product company, leading a group of 6 QA engineers and have expertise in testing eCommerce solution products.</p><p>Many times I’ve noticed that when asking QA to provide a status report, they answer with something like: ‘all good’, ‘in progress’, ‘I’ve passed 30 test cases’, etc. To tell the truth, to me as a manager that just said nothing. If you say ‘all good’ what do you mean? You tested everything and the feature Passed QA? You know what to do and how to follow the test plan? How much have you done? When do you plan to finish? Is there anything that is blocking you? All these questions come into my mind every time I hear a report like that. This is my indicator that signals to me that the test report misleads or is missing important information.</p><p>If saying ‘all good’ is not enough, then what should be added to the Test Report? Why can’t QA just tell how many test cases are passed, how many are left and send a link with found bugs? Let’s think together.</p><p><strong>Test Report</strong> represents how much a tester considers an SUT (system under testing) to correspond to expected requirements (implicit and explicit). A Test Report should be sufficient, but not excessive. To understand what information to include in the report, its author should think first about its recipient. Who are the ‘customers’ of the report.</p><p>Imagine a product manager that wants to deliver a new feature to her users. The feature was well-designed, developers did their part and passed the feature to QA for testing. Now, let’s take a talented QA engineer that thoroughly tested that feature and was then asked to provide a QA status. A Test Report will look this way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4gd3iXlzjmGskOcXP7dPCA.png" /></figure><p>A report like this might seem quite professional and now the product manager has to make a decision if she is free to deploy the feature to the users. But she is confused, because as she looks at the report, she cannot answer simple questions like:</p><ul><li>Did QA finish their work (it looks that yes, but it wasn’t mentioned, what if not)?</li><li>Did the feature Pass QA or Fail QA (there are defects in the report, but what if all of them are minor)?</li><li>What should she do with the ‘defect density’ note? Is this ‘defect density’ — ‘2.5 per day’ same as on other projects or here it’s worse and she needs to do something about it?</li><li>Where can she find the open bugs?</li></ul><p>Probably this will all result in a need for a face-to-face meeting with QA to clarify all missing things.</p><p>This was an example of the situation when QA reports the status based on test-related activities that were done in relation to the feature. A QA manager would probably understand it, since one has the same set of skills, but the product manager usually has only general ideas about test approaches and thus needs a “decryption” of such a report.</p><p>To help the product manager understand the report without a “decryption”, we can rewrite it to be more role-specific (in this case by ‘role’ I mean product manager’s role):</p><p><em>“I’ve completed a full QA test cycle. 15 bugs have been found, 3 of them are blockers, 5 look minor, but please review and make a decision if they are release-blockers. Also, I’ve noticed that localization is only partially done — please take it into account as I don’t know if it is required for the phase1 or not. All the bugs have been passed to developers, I’m waiting for fixes to run the new test cycle.”</em></p><p>Whoa! Now the product manager knows where QA has landed, she doesn’t need to speak with developers, as they are already updated, she only needs to wait for the fixes to proceed to the next step. The process is clear and transparent.</p><p>The example above is of a role-specific report. Sometimes our reports should be even person-specific. The main thing is that the report has to be useful to the receiver. The idea is to think about the customers of the report, what information they need, and how they are going to use our report. These are the 3 main questions I ask myself before writing any Test Report.</p><p>Let’s go one by one of these.</p><p><strong>Who are the customers of my report? </strong>Besides the product manager, a QA Report can have other customers: developers, QA manager, dev lead, project manager, another QA that’s working with you.</p><p><strong>Which information does the ‘customer’ need? </strong>A developer is probably interested in bugs you found. It might be relevant to send them a link to a bug tracking system with a filter applied and mention the most ‘dangerous’ bugs.</p><p>A QA manager needs to know your capacity and plan which tasks to give you next. It might be relevant to tell that manager how much work you’ve done, when you plan to finish and if you need any help with the task.</p><p>A project manager has to know the state we’re in: product, dev, qa, customer care, etc. Here it’s relevant to provide a high-level status report with small explanations.</p><p><strong>How do ‘customers’ use my report?</strong></p><ul><li>Product manager — to make a decision about feature release;</li><li>QA Manager — to plan QA engineer’s capacity;</li><li>Developer — to plan their tasks considering the time needed to fix bugs found.</li></ul><p>Taking into consideration our customers and their needs, and by knowing how they are going to use our report, we can plan the filling of the report.</p><p>Of course, we don’t always send the report to just one specific customer. Frequently we have to send a report to the whole team to update everyone. In this case a QA Report Template, which contains the most important information, will come in handy.</p><p><strong>Structure of a QA Report template</strong></p><ul><li><strong>Name of the feature, link to the task with the requirements</strong></li></ul><p>Identify the feature you’re testing to eliminate misunderstanding in case you’re working on several different tasks.</p><ul><li><strong>Work progress</strong></li></ul><p>Examples: Haven’t Started | QA in Process | Blocked | Completed.<br>Answer the question ‘where do you stand with the testing?’. Let people plan their job according to your working progress.</p><ul><li><strong>Feature Status</strong></li></ul><p>Keep it simple — Passed QA or Failed QA, but use it wisely. If you haven’t finished testing yet, leave this field empty. If there are some blocker bugs, set the status to “Failed QA”. Draw stakeholders’ attention, ask for their help in unblocking issues.</p><ul><li><strong>List of blocker bugs with links</strong></li></ul><p>These kinds of bugs should be addressed as soon as possible, that’s why it’s important to list them separately from other issues found.</p><p>One note: if you haven’t found blocker bugs, don’t skip. Write ‘none’ for the sake of transparency.</p><ul><li><strong>Number of all open bugs</strong></li></ul><p>Add a link to a page with those bugs listed (filtered, for example). This information helps developers and product managers prioritize bugs and estimate the time needed for fixes.</p><ul><li><strong>Next steps</strong></li></ul><p>Explains what you plan to do next. For instance, if a feature is big, you can divide it into parts and tell what the next part you’re going to work on is. Or if you complete the testing according to your plan and the feature is ‘Pass QA’, but you want to run a sanity check before the release, specify this step here. Otherwise stakeholders might make a decision to deploy this feature and you won’t have time for sanity.</p><ul><li><strong>Potentially risky areas</strong></li></ul><p>Raise any issues you think are potentially risky. It can be something that’s not directly related to QA responsibilities, but worth to be discussed before a release. Don’t forget that you are the first customer of a feature. You might notice something other people missed.</p><p>Example: missing localizations. In my company we have a specific team that’s responsible for localization. However, If I see that it’s not ready, or partially ready and there are some issues, I can raise a flag to my manager, so she can check it.</p><ul><li><strong>Parts of the feature that won’t be tested (or cannot be tested)</strong></li></ul><p>It goes without saying that it’s very important to keep your team informed if there are certain parts of the feature that were not / are not going to be tested. People should consider risks involved and plan to eliminate potential issues.</p><ul><li><strong>Link to a test plan</strong></li></ul><p>Add a link to your test plan, do not leave even a small chance for miscommunication to occur.</p><p>Last thing I want to talk about is how writing efficient QA Reports influences our professional appearance as QA engineers. My company (unit within Wix with engineers and other employees working on the same set of projects united by a certain function) is large, with around 70 amazing people including product managers, developers, UX designers, operation managers, and a translation team. I don’t work with all of them directly. They do not see my day-today work and cannot know if I’m a good professional. They see me through the documents I’m sending, assess my knowledge and my skill set via the information I provide. It’s important how I phrase my notes, if we speak the same language, if they can rely on my findings and make educated decisions as a result.</p><p>Think about all of that next time you write an email to your managers. Remember that you’re building your professional brand not only when speaking at conferences, but also when doing such a seemingly routine task as writing your QA Report.</p><p><strong>To summarize</strong>: writing an efficient QA Report is a super important skill to have, one that will definitely help you:</p><ul><li>better relate information significant to the business, keep your teammates focused and allow them to plan their next steps,</li><li>build your professional appearance.</li></ul><p>Before writing a Test Report think of who the customers of your report are, what information they need and how they will be using your report. If a report is sent to the whole team, use a designed template with nine easy parts described above, or work out your own template.</p><p>I hope I’ve given you something handy today. Don’t forget to have fun and enjoy writing your Test Reports!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61d0ded730e1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wix-engineering/how-to-write-an-efficient-qa-report-61d0ded730e1">How to write an efficient QA Report</a> was originally published in <a href="https://medium.com/wix-engineering">Wix Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>