<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Itzik Shabtay on Medium]]></title>
        <description><![CDATA[Stories by Itzik Shabtay on Medium]]></description>
        <link>https://medium.com/@itzik-shabtay-qa?source=rss-4951edfa1d19------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*sXpe8-5RkXGg1a5wzmvvUg.png</url>
            <title>Stories by Itzik Shabtay on Medium</title>
            <link>https://medium.com/@itzik-shabtay-qa?source=rss-4951edfa1d19------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 22:20:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@itzik-shabtay-qa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Agentic QA Automation: Developers Write the Tests, QE Owns the Guardrails]]></title>
            <link>https://medium.com/@itzik-shabtay-qa/agentic-qa-automation-developers-write-the-tests-qe-owns-the-guardrails-3eacc331e615?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/3eacc331e615</guid>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Sun, 27 Sep 2026 07:49:06 GMT</pubDate>
            <atom:updated>2026-09-27T07:49:06.281Z</atom:updated>
            <content:encoded><![CDATA[<p>How our agentic QA-automation plugin lets the developer who builds a feature also write its Playwright tests — to the same standard every time — inside a process that QE designs, gates and maintains. Thirteen steps took it from a bare pipeline to a governed system, and each one either took work off the developer’s plate or put a guardrail under QE’s control.</p><p>When shifting left an organization SDLC, tests written ad hoc by developers drift: different fixture usage, different locator strategies, helpers duplicated per spec, nothing traceable to a plan. The plugin’s answer is to move authorship to the developer — the person who already knows the feature — while keeping quality where QE can control it: in the conventions, the gates and the review rules that every agent is required to follow. The developer supplies the feature; the plugin supplies the discipline; QE maintains the discipline.</p><p>Concretely, the plugin turns a feature — a PRD, a Jira ticket, a branch — into Playwright tests that live in automation-pw, keeps them green when the product moves, and does it without a human babysitting a terminal. What follows is how that split of responsibilities came to be, in thirteen steps grouped into four eras. Read each step by asking two questions: what did it remove from the developer’s plate, and what did it put under QE’s control? The numbering is chronological and matters — each era assumes what the previous one put in place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ypgZNKe4fN5G5ytdfTMo5w.png" /></figure><p>Era I · Foundation Steps 1–3</p><h3>A pipeline a developer can run, then a reason to trust what it produces</h3><p>Version one was a straight line a developer could point at a feature. A pw-test-planner read the feature and wrote scenarios; a pw-test-generator turned them into Playwright specs; a pw-test-healer fixed whatever broke. All three were baselined on Playwright’s own reference agents — sensible, because those agents already knew the framework — but they knew nothing about our product, and nothing about how QE wanted tests written.</p><p><strong>1. Three agents as a serial pipeline</strong>: planner → generator → healer, baselined on Playwright’s reference agents. Established the shape everything else hangs on.</p><p><strong>2. Static knowledge base</strong>: Built from Customer Support documentation and the automation framework itself. Gave every agent a shared, versioned understanding of the product and of the POM/fixture conventions, instead of each one inferring them from scratch. This is the first place QE’s knowledge became the agents’ knowledge.</p><p><strong>3. pw-test-code-review as the last node</strong>. A reviewer agent appended as the terminal gate — after healing, before the PR. Deliberately not between generate and heal: the healer’s force-green edits (weakened assertions, selector hacks, a sneaky fixme) are exactly what the review exists to catch, and blocking findings loop back to the healer. Because the reviewer applies QE’s standards, a developer’s test is held to the same bar as one QE would have written.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m9e2B6cZpKnwL_lHolwxDQ.png" /></figure><p>Era II · Shared knowledge &amp; reach Steps 4–6</p><h3>QE’s conventions become the rulebook every agent follows</h3><p>The second era is where standardization happens. Four agents each carrying their own copy of “how we write Playwright here” is four copies that disagree within a month — and none of them is the one QE maintains. Moving the conventions into injected skills means QE edits one file and every developer’s next test follows it. The era also extends the plugin’s reach to the systems QE already governs with: data-test-id attributes in frontend repos, and test plans in Jira.</p><p><strong>4. Internal skills:</strong> pw-conventions and Jira ticket creation. Conventions became a single injected skill consumed by the generator and the reviewer (and later the whole pipeline). The Jira skill mints a real Test issue whose key becomes the test’s @QA-XXXX id — every test traceable to a ticket. QE owns the skill; developers inherit it without reading it.</p><p><strong>5. fe-locator-injector:</strong> When an element has no data-test-id, the healer no longer guesses a brittle locator. It delegates to an agent that finds the owning frontend repo, injects a conventional id, and opens a PR there. The plugin now changes the product to make it testable.</p><p><strong>6. Jira (Xray) Test Plan skill:</strong> Every saved plan is backed by a real Xray Test Plan with member Test issues. The plan QE and PM can edit in Jira — not the planner’s markdown — becomes the generator’s input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BwrMYW2ukJF58kDHyzKmSw.png" /></figure><p>Era III · Humans in the loop Steps 7–9</p><h3>The guardrails: a hard stop, a diagnosis before the fix, and visibility</h3><p>By now a developer could run the pipeline end to end on their own — which is exactly when autonomy becomes a liability. Era III adds the guardrails that let QE stay in control of a process they no longer execute by hand: a mandatory QE/PM sign-off between planning and generation, a triage agent so the healer stops treating every red test as a test bug, and a notification channel so nobody has to watch a console.</p><p><strong>7. Test plan requires manual approval hard gate:</strong> The pipeline stops at sign-off and never starts generation in the same run — even in AUTO mode. QE and PM review the real Test issues in Jira; deleting one structurally descopes the scenario. The contract is the member list, not a chat reply.</p><p><strong>8. pw-test-analysis agent </strong>reads a finished run’s Allure report and works top-down: groups failures that share an API/network/rate-limit signature into <em>one</em> systemic issue, then account issues (expired plan, exhausted credits), then re-runs each remaining failure on the exact account the run used and correlates with the last 7 days of merges in the relevant repos. Only what is confirmed as a test problem reaches the healer — with the analysis attached.</p><p><strong>9. qa-notify skill: </strong>Every main agent sends one Slack DM to the person who launched it the moment its mission finishes: bold headline rows, then the same phase report shown in the console. Long runs can be launched and walked away from.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ias894GCZkuBCzwo3yU4uA.png" /></figure><p>Why the gate is unconditional: A flag that skips sign-off in AUTO mode would eventually be flipped for a hurried release, and generation would then run on a plan nobody read. Making the stop structural — the run <em>ends</em> at hand-off — means the only way to get tests is to have QE look at the member list first. The developer moves fast up to the gate; QE decides what crosses it.</p><p>Era IV · Hardening Steps 10–13</p><h3>Precision, preflight, isolation, and taking things away</h3><p>The last era is the least glamorous and the one that decides whether developers keep using the tool. Nothing new is bolted on the front; instead each agent is tightened so its output is easier for a developer to trust and revert, and safe to run next to their own feature work.</p><p><strong>10. Precision improvements:</strong> Analysis downloads the full Allure data rather than scraping a summary. The planner marks core acceptance (certification) tests — their names start with Acceptance: so the certification set reads straight off the member list. The healer lands <em>one commit per fix</em>, so an unwanted change is one git revert.</p><p><strong>11. Access preflight and a fixme rule: </strong>Analysis opens with an access check (Vault, AWS, DB) and never attempts an interactive login itself — an expired login is handed back as exact terminal commands. The healer never applies test.fixme() on its own; it proposes it with evidence and the user decides.</p><p><strong>12. Worktree isolation, parallelism, headless: </strong>Every agent that mutates a repo works in a git worktree beside your checkout — a named heal/gen branch for writers, a throwaway detached tree for reproductions — so your branch, IDE and console stay free. Runs use 4 workers and go headless.</p><p><strong>13. Cleanup:</strong> No new helper files: a one-test helper lives in the spec, a sharable one in the domain’s existing common-actions file. Existing tests are reused rather than re-suggested; overkill is removed. The plugin now has opinions about what <em>not</em> to write.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G9tv_3ngV5oZ0c8gcodCTQ.png" /></figure><h3>The system as it stands</h3><p>Put the four eras together and the shape is no longer a line. It is two loops around one repository — a creation loop that runs plan → sign-off → generate → heal → review → PR, and a maintenance loop that runs CI → analysis → heal → (locator PR) → CI — with six skills injected into the agents so the rules live in exactly one place, and a Slack DM leaving every agent when it finishes.</p><p>Two things about how it is driven are worth stating, because they are design choices rather than omissions. There are no slash commands: you address an agent by name with a feature source, and each agent ends by <em>offering</em> the next hop — the order is encoded in the agents, and every hand-off waits for your go-ahead. And the intended user is the developer writing the feature, not a QA specialist: the agents supply QE’s framework conventions, the gates supply QE’s judgement, and the same peer who reviews your service code reviews your test code. QE’s job shifts from writing tests to maintaining the skills, gates and knowledge base that shape every test anyone writes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wnKE7V_VBxVcmn-foBuMqQ.png" /></figure><h3>Who owns what</h3><p>The split is deliberate and narrow. Developers own everything specific to their feature; QE owns everything that should be the same across features. Neither side has to do the other’s job to get a good test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*79WShcbjHkaDzie-RXuPmg.png" /></figure><h3>Reading the maturity</h3><p>A quick way to see how far this moved is to hold each dimension of the first version against the current one. Every row is a place where either the developer’s job got easier or QE’s control became structural, and each corresponds to a step you can point at.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H_bHPGs3Zfa7ZA_g011mbw.png" /></figure><h3>Why this works for both sides</h3><p>It would be easy to read thirteen steps as thirteen features. The stronger reading is that the plugin kept choosing <em>structure over instruction</em> — the safe, standard behaviour is the only behaviour, so a developer does not need to know QE’s rules to follow them, and QE does not need to police each test to enforce them.</p><ul><li>Standard output regardless of author. Two developers on two teams get tests with the same POM usage, fixtures, tags, clean-state preconditions and @QA ids, because the generator reads QE’s conventions, not the developer’s habits. Standardization is a property of the pipeline, not of the person.</li><li>The contract is a Jira object, not a prompt. Sign-off happens by editing or deleting Test issues. A deleted member is a descoped scenario by construction; the generator cannot generate what is not in the list. QE’s sign-off is a real artifact with an audit trail, and no prompt engineering is needed to make descoping stick.</li><li>Diagnosis is separated from repair. Before the analysis agent, an expired subscription on a pooled account looked like twelve broken tests. Now it is one line in a report, and the healer never sees it. That is the difference between a tool that adds noise and one that removes it.</li><li>Every irreversible action is either human-gated or one-step reversible. Generation waits for people; heals are one commit each; fixme is proposed, never applied; locator changes go out as PRs. There is no path where the plugin silently changes something you cannot undo in one command.</li><li>It fixes the product instead of working around it. A missing data-test-id becomes a PR against the owning frontend repo, not a brittle text locator that will flake next sprint. And because that id is not live until the PR merges and deploys, the caller keeps a resilient interim locator meanwhile — it never reports green on an undeployed id.</li><li>It never goes quiet. All seven agents carry the same progress contract: a standalone update at every phase boundary and a heartbeat at least every three minutes, so a suite run or an account sweep never looks like a hang. The Slack DM at the end is the summary, not the only signal.</li><li>Rules live in one place, and QE holds the pen. Skills are injected, not copied. When the convention for helper placement changed (step 13), QE changed one file and it changed for the generator, the reviewer and the healer in the same edit — no developer had to be told.</li><li>It stays out of your way. Worktrees, headless runs (PW_HEADLESS=1), four workers, occupied_by locking on the automation account pool, and a Slack DM at the end mean the natural way to use it is to launch it in the background and keep working. Because a worktree shares the repo’s .git, every heal commit is visible from your checkout without visiting the worktree. A tool that demands attention while it runs does not get run.</li><li>It learned to say no. The planner drops scenarios an existing test already covers, excludes what cannot be automated in the framework as it stands, and refuses to plan without a grounded source. Restraint is the last thing most automation acquires and the first thing that makes its output reviewable.</li></ul><h3>What to watch next</h3><ol><li>The static knowledge base was a good decision at an early stage due to superior performance and the automation framework baseline, but it will eventually be connected to the main brain.</li><li>Not all features are created the same, and we’re working on feature-scope logic: a small, medium or large feature scope with a particular risk level affects the plan’s size and depth.</li><li>Even before test plan creation, developers should be able to get the existing affected tests as additional context for their development. We’ll incorporate this capability.</li><li>We’ll be building CI linkage between the development task and the testing task, with smart reasoning about when it is needed and when it is not.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3eacc331e615" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Velocity Without Regret: Transforming Feature Lifecycle with an Agentic Bug Bash]]></title>
            <link>https://medium.com/@itzik-shabtay-qa/velocity-without-regret-transforming-feature-lifecycle-with-an-agentic-bug-bash-dba5ca050176?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/dba5ca050176</guid>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Tue, 07 Jul 2026 06:38:32 GMT</pubDate>
            <atom:updated>2026-07-07T06:38:32.685Z</atom:updated>
            <content:encoded><![CDATA[<p>Every QA and Engineering leader knows the bittersweet reality of a feature-level bug bash. As a new capability nears completion in its lifecycle, you gather a cross-functional group for a focused burst of testing. But instead of a streamlined quality gate, you often get a chaotic scramble: scheduling conflicts, incomplete bug reports, duplicate tickets, and a realization that people naturally drift toward the paths they are most comfortable testing.</p><p>What if you could run a comprehensive feature bug bash on-demand, the moment a branch is ready, with zero scheduling friction and six distinct domain experts attacking the code simultaneously?</p><p>We are currently building and refining an Agentic Bug Bash process to do exactly that. To be clear, this is not a replacement for our Continuous Integration (CI/CD) pipelines or our core QA automation regression suites. Instead, it replaces the manual, ad-hoc human bug bashes traditionally done at the tail end of a feature’s development lifecycle.</p><p>It is very much a work in progress, but the evolution of this system is fundamentally shifting how we approach feature validation. Here is the blueprint of how we are building it, step by step.</p><h3>Evolution Step 1: The Multi-Persona Orchestrator Blueprint</h3><p>Our initial concept centers on a hub-and-spoke model. At the core sits an Orchestrator Agent that connects directly to our engineering ecosystem using Model Context Protocol (MCP) servers. For any given feature, the Orchestrator ingests its specific context: a Confluence HLD or PRD (or up to five Jira epics/stories in lieu of those pages) via an Atlassian MCP, optional UI mockups via a Figma MCP, and optional code changes from a GitHub branch, PR, or commit range.</p><p>Instead of treating the AI as a single generic tester, the Orchestrator first performs live browser verification of the feature via a Playwright MCP, then provisions six specialized sub-agent personas — briefing them with that shared evidence and fanning them out to reason over it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5uYtWjBQTT6AWfSo" /></figure><p>QA Happy Paths: Focuses purely on core business logic. Does the feature achieve its primary goals exactly as explicitly stated in the PRD?</p><p>QA Boundaries &amp; Edge Cases: The destructive tester. It inputs negative values, attempts to break string limits, forces rapid API calls, and tries to expose unhandled errors.</p><p>Power User: Simulates a highly proficient user. It clicks aggressively, bypasses UI wizards, utilizes keyboard shortcuts, and tests the speed of the interface.</p><p>UX Expert: Compares the live feature branch against the Figma design schemas to call out layout shifts, padding mismatches, and visual alignment bugs.</p><p>Security Expert: Inspects network payloads for over-fetching data and checks client-side state vulnerabilities or basic input injection risks.</p><p>Customer Support Expert: Flags potential friction points. Where will a user get stuck? Are error messages friendly and clear, or will they trigger an influx of support tickets?</p><p>Once the run is complete, the Orchestrator aggregates the results, filters out duplicates, and assembles a central report. With the operator’s confirmation, it then publishes that report to Confluence as a parent page plus one child page per persona. <em>(When Playwright isn’t available, personas instead produce a structured test plan rather than live results.)</em></p><h3>Evolution Step 2: Fine-Tuning, Noise Control &amp; Scope Refinement</h3><p>When you give an AI agent an open-ended mandate to test a new browser view, it will find something to report. Early on, we realized that an uncalibrated agent acts like a hyper-sensitive smoke detector. This ongoing phase is focused on tightening the scope:</p><p>Establishing “In-Scope” Guardrails: We are teaching the agents to understand what truly matters for a feature-level gate. For example, a minor alignment issue in a legacy global footer shouldn’t dilute the focus of a new checkout feature bash. (Other examples now baked in: “Coming soon” UI and mobile/responsive layouts are out of scope unless explicitly requested, and pre-existing baseline behavior is never failed as a regression of the feature under test.)</p><p>Report Structure Refinement: Developers don’t want a wall of text. We are refining the output into a scannable format highlighting Impact Severity, exact Steps to Reproduce, and screenshots captured specifically on failing steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nXcJWQiCnayaIYcc" /></figure><h3>Evolution Step 3: Establishing the Product Baseline as a Distilled Knowledge Skill</h3><p>An agent cannot reliably flag anomalous behavior if it doesn’t know what “normal” looks like across the rest of the application. To ground the system, we distilled our public product Help Center into a structured Agent Skill — a curated knowledge file covering terminology, modules, default flows, role/permission rules, credits, plan gates, and account-settings layout as they exist <em>before</em> any new feature ships.</p><p>Rather than fetching live docs at runtime, this baseline is captured offline and loaded on demand, giving the Orchestrator and personas an immediate, stable understanding of macro-level app behavior with no network dependency. A built-in override rule keeps it honest: if the live UI ever contradicts the baseline, the agents trust the UI and flag the Skill as needing a refresh — because a bug bash judges the <em>delta</em> introduced by the feature, not the baseline itself. This step drastically reduces false positives during feature evaluation.</p><h3>Evolution Step 4: Scaling Deep Context with Domain-Specific Skills</h3><p>High-level baseline knowledge is great, but real features live in specific domains.</p><p>To give the sub-agents deep testing context, we authored a library of domain-specific Skills, each distilling how a given area actually behaves: state pre-conditions, validation rules, common error modes, and per-integration quirks.</p><p>These Skills draw on two sources: our public documentation <em>and</em> the accumulated knowledge encoded in our existing QA automation suites. The high-level Orchestrator dynamically routes a persona to the relevant domain Skill based on the spec.</p><h3>Evolution Step 5: Defining the Operational Blueprint &amp; Stakeholder Expectations</h3><p>For an engineering process to succeed, it needs trust and a clear operational definition. The final part of our ongoing evolution is aligning with Product Managers, Tech Leads, and QA Stakeholders to establish a Revised HLD for the Agentic Bug Bash process itself.</p><p>We are standardizing exactly what the system needs as inputs and what the organization can expect as an output:</p><h3>What goes in, what comes out</h3><p>A bug bash is only trustworthy if everyone agrees on its contract. We’ve standardized the system around a small set of required inputs and a predictable set of outputs — so a Product Manager, Tech Lead, or QA stakeholder knows exactly what to hand over, and exactly what they’ll get back.</p><p>What the tool requires</p><ul><li>A spec oracle (required) — an Atlassian PRD/HLD link, <em>or</em> up to five Jira epics/stories</li><li>A target environment (required) — the app URL, pulled from local account config</li><li>Code context (optional) — a GitHub branch, PR, or commit range</li><li>Design reference (optional) — Figma frame IDs for design-parity checks</li><li>Custom focus (optional) — your own testing instructions or areas of emphasis</li></ul><p>What stakeholders can expect</p><ul><li>A clean, deduped Confluence report — one parent page plus a child page per persona — in under ~30 minutes</li><li>Near-zero noise from minor, out-of-scope styling issues</li><li>Failure screenshots and reproducible steps for immediate developer debugging</li><li>A triage-ready, severity-sorted “Suggested Tickets” list for confirmed Critical/Major bugs</li><li>Human-in-the-loop publishing — the report reaches Confluence only on operator confirmation</li></ul><p>In short: two required inputs (a spec source and a place to test), three optional ones that sharpen the run, and a report whose shape never varies — which is what lets teams treat it as a dependable gate rather than a one-off experiment.</p><h3>The Vision Ahead</h3><p>Replacing manual, lifecycle-specific human bug bashes with an automated agentic framework is challenging, non-linear work. We are constantly adjusting model prompts, tweaking our MCP connections, and narrowing scopes. Future directions we’re exploring include automatic Jira ticket creation for confirmed Critical/Major bugs (today the system produces a triage-ready suggestion table that a human files).</p><p>However, by creating a team of distinct AI personas powered by our own documentation and automated testing footprints, we are proving that feature validation can be fast, objective, and deeply comprehensive — leaving our human teams with more time to focus on complex, creative exploratory testing.</p><h3>Meta Note on the Evolution</h3><p>As this article is a Work in Progress, the titles and phases above reflect our current trajectory as we transition this concept from an experimental pipeline into a mature, everyday gate for our feature lifecycles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dba5ca050176" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Future Proof Quality Assurance — Leveraging AI Tools 2026]]></title>
            <link>https://medium.com/@itzik-shabtay-qa/future-proof-quality-assurance-leveraging-ai-tools-2026-131a41a2c9c7?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/131a41a2c9c7</guid>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Fri, 23 Jan 2026 18:45:45 GMT</pubDate>
            <atom:updated>2026-01-23T18:45:45.442Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Future Proof Quality Assurance — Leveraging AI Tools 2026</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gH1ZCIV0B9MVVtPEPlBq_g.png" /></figure><p>The landscape of Software QA is undergoing a profound transformation, driven by the strategic adoption of AI. The modern QA leader’s operational blueprint is defined by successfully integrating and using AI tools across the entire software delivery lifecycle, from planning and documentation to code-level automation and incident response. The goal is a methodological shift that leverages generative and analytical AI to maximize efficiency and elevate the quality of testing.</p><p>Here is a breakdown of some AI tools I use to enhance my productivity:</p><p><strong>1. Generative AI for Test Planning, Processes &amp; Methodologies (e.g., ChatGPT)</strong></p><p>Tools like ChatGPT serve as powerful accelerators in the foundational phases of testing. Their primary usages revolve around establishing best-in-class frameworks and artifacts:</p><ul><li><strong>Best Practices and Strategy Alignment:</strong> Generating comprehensive, up-to-date best practices that align with evolving architectural patterns.</li><li><strong>KPI and Metric Definition:</strong> Defining clear, measurable Key Performance Indicators (KPIs) and quality metrics for new projects or feature releases.</li><li><strong>Test Plan Generation:</strong> Drafting foundational test plans and test scenarios, significantly reducing the initial time investment required by a Test Architect.</li></ul><p><strong>2. Operational Efficiency and Context (e.g., Google Gemini for Workspace)</strong></p><p>Integration of AI assistants directly into collaboration platforms drastically improves organizational context and information flow for QA teams:</p><ul><li><strong>Information Synthesis:</strong> Providing instant summaries of slides, spreadsheets, and long-form documents (e.g., requirements, design specs), allowing QA to quickly onboard or understand changes.</li><li><strong>Structured Content Creation:</strong> Rapidly generating presentation slides complete with diagrams and schemes for knowledge sharing or test strategy presentations, all without leaving the presentation editor.</li><li><strong>Focused Communication:</strong> Streamlining inbox management by summarizing email threads, enabling QA leaders to quickly grasp the essential context of critical decisions or reported issues.</li></ul><p><strong>3. Browser-based LLM Assistance for Deep Analysis (e.g., Perplexity’s Comet)</strong></p><p>Leveraging AI assistance within the browser elevates the analysis of external and internal systems by QA engineers:</p><ul><li><strong>JIRA and Project Insights:</strong> Summarizing complex JIRA query results, identifying common criteria in bug reports, or converting aggregated data into visual formats like pie charts for faster reporting.</li><li><strong>Documentation Synthesis:</strong> Rapidly summarizing large Confluence pages or internal Wiki documentation, highlighting the essential changes relevant to a test cycle.</li><li><strong>CI/CD Log Analysis:</strong> Interrogating Continuous Integration (CI) logs (e.g., Jenkins) to pinpoint the root cause of build failures, identify performance bottlenecks, and categorize error types, significantly accelerating the feedback loop.</li><li><strong>External API &amp; Git Documentation:</strong> Answering specific technical questions about new features, API usage, or breaking changes documented in 3rd party Git repositories or release notes.</li></ul><p><strong>4. Video Conferencing AI Assistant (e.g., Zoom/Google Meet AI Assistant)</strong></p><ul><li><strong>Call Transcription:</strong> Providing accurate, real-time transcriptions of all critical project and incident calls.</li><li><strong>Automated MoM Distribution:</strong> Creating and distributing Minutes of Meeting (MoM), capturing decisions, action items, and owners for instant dissemination across the team.</li></ul><p><strong>5. Advanced Automation and Code Refinement (e.g., AI Cursor with Playwright)</strong></p><p>AI in the Integrated Development Environment (IDE) and automation framework is a game-changer for code quality and test maintenance:</p><ul><li><strong>Framework Governance:</strong> Building customized automation rules and best-practice checks (<strong>Cursorrules</strong>) according to repository standards and framework guidelines (e.g., Playwright).</li><li><strong>Code Interface Synchronization:</strong> Reading application code interfaces and types from the repository and automatically mirroring the structures within the automation framework, ensuring data consistency.</li><li><strong>UI Resilience &amp; Data-Test-IDs:</strong> Identifying missing front-end data-test-ids and creating Pull Requests (PRs) to inject them into the app codebase, drastically improving the resilience of UI tests.</li><li><strong>Test Generation and Healing:</strong> Employing <strong>Playwright agents</strong> to create end-to-end test plans, generate the actual test scripts, and automatically heal broken UI locators.</li><li><strong>End-to-End Development Cycle:</strong> Utilizing GitHub and Atlassian extensions (<strong>MCP</strong>) to streamline all git operations, perform automated test plan exports to JIRA for centralized test management, and maintain a valid Testing Tree by comparing test IDs across the repository and JIRA.</li><li><strong>Production Monitoring:</strong> Building robust Cloudflare workers with authenticated API call health checks to actively monitor critical production endpoints and trigger Network Operations Center (NOC) alerts on failure, moving QA into the realm of production incident prevention.</li></ul><p><strong>6. Future-Forward: RAG-Supported Reporting Analysis</strong></p><p>The next frontier is integrating Retrieval-Augmented Generation (RAG) capabilities with reporting tools (e.g., Amazon Bedrock Knowledge Base and n8n). This move aims to make test reports smarter and clearer by adding learning capabilities — allowing the reports themselves to provide context-aware insights, root-cause correlations, and predictive analysis, moving beyond static metrics to a living data source.</p><p>The rapid pace of AI integration demands that QA leaders not only adopt these tools but strategically embed them into their core methodologies. The focus has decisively shifted from <em>if</em> to <em>how</em> we leverage these advancements to meet the increasing demands of modern software development.</p><p>What about you? What tools and methodologies do you use and instruct?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=131a41a2c9c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Software Testing Strategies for Senior QA Leaders]]></title>
            <link>https://medium.com/@itzik-shabtay-qa/software-testing-strategies-for-senior-qa-leaders-c87d8bee543a?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/c87d8bee543a</guid>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[tdd]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Wed, 01 May 2024 09:18:03 GMT</pubDate>
            <atom:updated>2024-05-01T12:58:05.232Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*42LlIAKapLgPd4GZHyHPjA.jpeg" /></figure><p>If you’re a QA director / head of QA / QA professional / QA architect — this article is for you.</p><p>Your job in the company is to enhance software product quality &amp; improve development processes, through educating and implementing methodologies, capturing KPIs &amp; metrics, and monitoring reports. This will result in better products, more cost-effective development &amp; approval processes and faster time-to-market. Regardless of the existence of dedicated QA resources or not.</p><p>Let’s discuss the ways to achieve these goals:</p><h3><strong>Implementing Quality Enhancement Tools</strong></h3><ol><li><strong>Version Control </strong>— using a version control system (such as GitHub) to manage code changes, track history, and facilitate collaboration among team members, using the right permissions is essential to maintain clean and manageable code. Branching and having strict timeline is important to minimize merging conflicts. <em>No code should be merged without being tested </em>(to avoid complicated rollbacks).</li><li><strong>Dependency Management</strong> — making scheduled checks that dependent libraries are up-to-date will ensure they work flawlessly without surprises. deprecated libraries result in multiple deployment warnings, security vulnerabilities and at sometimes broken functionalities.</li><li><strong>Static Code Analysis</strong> — tools used to analyze potential code invalidities, vulnerabilities &amp; performance bottlenecks <em>before even compiling the code</em>. They can run in command line (either locally or through Continuous Integration (CI) as build gate keepers).</li><li><strong>Lint</strong> — lint tools are used to analyze style, structure/hierarchy &amp; pattern issues within code. They’re often defined as part of static code analysis. Some lint tools have extensive approach guidelines and can save significant amount of time during code reviews. These tools can be both implemented in the IDE (Integrated Development Environment) as plugins and also in the CI process itself.</li><li><strong>Security Testing</strong> — a tool or combination of tools, aimed to catch code vulnerabilities, exploit breaches and explicit server references. They are often defined as part of static code analysis. In the last few years it became mandatory for every company to protect its intelectual property, users’ privacy and and avoid all types of cyber crimes. It can be implemented as part of CI process or run in command line.</li><li><strong>Unit Testing</strong> — The first layer of tests (mostly written by the original code developer) aimed to <em>verify the correctness of individual</em> units of code through ensuring their basic functionality. Writing unit tests encourages Test Driven Development (TDD). A QA function should review unit tests and give feedback. Unit testing strategies should focus on:<br>* Logic checks (happy path) — Does the system perform the right calculations and follow the right path through the code given a correct, expected input? Are all paths through the code covered by the given inputs?<br>* Boundary checks — For the given inputs, how does the system respond? How does it respond to typical inputs, edge cases, or invalid inputs?<br>* Error handling — When there are errors in inputs, how does the system respond? Is the user prompted for another input? Does the app crash?<br>* Object-oriented checks — If the state of any persistent objects is changed by running the code, is the object updated correctly?</li><li><strong>Code Coverage</strong> — Code coverage tools will use one or more criteria to determine <em>how your code was exercised</em> or not during the execution of your unit tests. The common metrics that you might see mentioned in your coverage reports include: <br>- Function coverage: how many of the functions defined have been called.<br>- Statement coverage: how many of the statements in the program have been executed.<br>- Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.<br>- Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.<br>- Line coverage: how many of lines of source code have been tested. this is the most visual criteria.</li><li><strong>Code Review</strong> — Each code pull request should be <em>reviewed before merging</em>. Code review guidelines:<br>- Clear Objectives: Ensure that all code changes meet our quality standards and are aligned with project requirements.<br>- Review Criteria: Evaluate code based on factors such as readability, maintainability, performance, security, adherence to coding standards, and compliance with project requirements.<br>- Reviewers: reviewers should be familiar with the relevant parts of the codebase and have expertise in the technology stack being used. Reviewers are usually more senior/experienced than the requesters to guide them.<br>- Keep Reviews Small and Focused: Breaking down this large feature into smaller, more manageable tasks for review.<br>- Provide Context: Before reviewing this code, please refer to the user story in Jira (link provided) to understand the requirements and objectives of this feature.<br>- Focus on High-Impact Issues: Example priority: “While reviewing this code, let’s prioritize addressing the SQL injection vulnerability reported by the security team.<br>- Promote Knowledge Sharing: Example sharing: “During the code review, let’s discuss different approaches to handling error states to share knowledge and best practices.<br>- Follow Up on Feedback: Confirming that all issues have been resolved before merging the changes.<br>- Document Review Decisions: Use the code review to take structural/implementation decisions.</li><li><strong>Continuous Integration (CI) / Continuous Deployment (CD)</strong> — CI/CD systems are used to <em>automate the process</em> of building, testing and deploying the code. They can include:<br>- Unit tests.<br>- Static code analysis (linters, security tests and others).<br>- Integration tests.<br>- Regression tests (Functional, Usability, Accessibility, Localization).<br>- Smoke (acceptance) tests.<br>- Deploy code (web service or app build).</li><li><strong>Performance Monitoring and logging</strong> — tools used to monitor overall performance of products (real time with comparative past data) <br>- Screen load times, latency, hangs, UI hangs.<br>- (App) crash reports — crash free sessions/users ratio.<br>- Errors dashboards, e-commerce success rates &amp; relevant actions durations.<br>- Event tracking and analysis (useful to observe conversion drops).<br>- Server health alerts, error notifying bots, scalability scripts.<br>- Load and Performance tests.<br>- Memory Leak Detection.</li></ol><h3><strong>Keeping developers committed to the process</strong></h3><p>Getting full commitment and participation from developers to embrace Test-Driven Development (TDD) and keeping testing in mind throughout the development process requires a combination of communication, education, collaboration, and support.</p><p>Above all, you need the <em>management’s full commitment to quality</em> and to back you up in times of conflict between quality and time constrains. If you have this support, here are some strategies to help you achieve commitment to the process:</p><ol><li><strong>Training and Education</strong><br>Provide detailed training sessions or workshops on principles, best practices, and techniques. Ensure developers understand the benefits, such as improved code quality, faster feedback loops, and reduced debugging time.</li><li><strong>Lead by Example</strong><br>Encourage experienced proficient developers to mentor and pair-program with those who are new to the practice, providing guidance and support as needed.</li><li><strong>Provide Support and Resources</strong><br>Supply relevant references. Establish a knowledge-sharing platform or wiki, where developers can ask questions and seek advice.</li><li><strong>Set Clear Expectations and Goals</strong><br>Clearly communicate expectations regarding TDD adoption. Set specific goals and milestones for implementing practices and track progress periodically.</li><li><strong>Integrate Testing In The Development Workflow</strong><br>Encourage developers to <em>write unit tests before writing code </em>(following the red-green-refactor cycle) and to refactor code to improve testability.<br><em>Automate as much of the testing process as possible</em> to make it seamless and less burdensome. Provide tools and frameworks that support automated testing and continuous integration.</li><li><strong>Create a Positive Environment</strong><br>Foster a culture that values quality and encourages experimentation and learning. Create a safe space where developers feel comfortable asking questions, sharing ideas, and experimenting with new techniques.<br>Recognize and reward individuals and teams who demonstrate a commitment to TDD and excellence in testing practices. Celebrate successes and share success stories to inspire others.</li><li><strong>Provide Feedback and Continuous Improvement</strong><br>Provide constructive feedback on testing practices and code quality, helping developers identify areas for improvement and grow their skills. Offer code reviews and pair-programming sessions focused on testing.<br>Regularly review and refine testing practices, incorporate lessons learned from past experiences, and stay up-to-date with emerging trends and best practices -if needed, introduce new tools and processes.</li></ol><p>By implementing these strategies, you can create an environment where developers are motivated and empowered to embrace TDD and ultimately leading to higher-quality software products and more satisfied customers.</p><p>You will also need to collect metrics and Key Performance Indicators (KPIs), monitor their trends and accordingly push the necessary changes in the company to make a difference:</p><h3>Collecting Metrics and Strategies</h3><p><strong>Performance:</strong><br>Metric: response time, launch time, network request/response time.<br>Improvement Strategies: Optimize performance by reducing unnecessary network requests, minimizing CPU and memory usage, caching data where applicable, and optimizing image and resource loading. Use performance monitoring tools to identify and address performance bottlenecks.</p><p><strong>Security Vulnerabilities:</strong><br>Metric: Number of security vulnerabilities identified, time to fix security vulnerabilities.<br>Improvement Strategies: Conduct regular security audits and penetration testing to identify potential vulnerabilities in the app. Implement secure coding practices, adhere to industry-standard security guidelines, and leverage security tools like static code analyzers, vulnerability scanners, and security testing frameworks.</p><p><strong>Release Cycle Efficiency:</strong><br>Metric: Time to release new features or updates, release cadence.<br>Improvement Strategies: Streamline the release process via Continuous Integration (CI) and Continuous Deployment (CD) practices. Automate build, test, and deployment pipelines to reduce manual effort and minimize release cycle times. Implement feature flagging to enable controlled rollouts and phased releases.</p><p><strong>User Engagement and Retention:</strong><br>Metric: Daily active users (DAU), monthly active users (MAU), user retention rate.<br>Improvement Strategies: Analyze user behavior patterns to understand user engagement levels and factors influencing retention. Implement features to enhance user engagement, such as push notifications, personalized recommendations, and social sharing functionality. Conduct A/B testing to evaluate the effectiveness of new features and optimizations.</p><p><strong>Customer Support Metrics:</strong><br>Metric: Average response time to customer inquiries, customer satisfaction score (CSAT).<br>Improvement Strategies: Establish efficient customer support channels, such as in-app chat support or dedicated support email addresses. Provide comprehensive documentation, FAQs, and troubleshooting guides to empower users to resolve common issues independently. Monitor and analyze customer support interactions to identify recurring issues and address them proactively.</p><p><strong>Compliance and Accessibility:</strong><br>Metric: Compliance with industry regulations (e.g., GDPR, CCPA), accessibility conformance (e.g., WCAG).<br>Improvement Strategies: Ensure compliance with relevant regulatory requirements and accessibility standards throughout the development process. Conduct accessibility audits and usability testing with diverse user groups to identify and address accessibility barriers. Provide accessible features such as screen reader support, keyboard navigation, and high-contrast mode.</p><p><strong>(App) Store Ratings and Reviews</strong>:<br>Metric: Average app store rating, number of positive reviews, number of negative reviews.<br>Improvement Strategies: review feedback, address reported issues promptly, and continuously enhance the app based on user suggestions. Implement automated feedback collection mechanisms within the app to capture user sentiments and preferences.</p><p><strong>(App) Crashes and Stability:</strong><br>Metric: Crash-free session percentage, crash-free user percentage.<br>Improvement Strategies: Implement robust error handling mechanisms, conduct thorough testing across different devices and operating system versions, prioritize and address crash-causing issues promptly based on crash analytics and crash reporting.</p><h4><strong>Defects and Regression Metrics:</strong></h4><p><strong>Defect Density </strong>(Bugs opened per code/time/FTR): The number of defects identified per unit of code (e.g., per thousand lines of code), indicating the quality of the codebase. A lower defect density typically signifies higher code quality.</p><p><strong>Defect Rejection Rate (Bugs rejected):</strong> The percentage of reported defects that are rejected by development or not deemed valid. A high rejection rate may indicate ineffective defect reporting or misunderstanding of requirements.</p><p><strong>Defect Leakage/ Escape (Bugs escaped to production): </strong>The percentage of defects found by customers or end-users after the software has been released to production. Higher defect leakage suggests inadequate testing or poor release quality.</p><p><strong>Test Case Coverage:</strong> The percentage of requirements or functionalities covered by test cases. Higher test case coverage indicates a more comprehensive testing effort.</p><p><strong>Test Automation Coverage:</strong> The percentage of test cases automated out of the total test cases. Higher test automation coverage can lead to faster testing cycles, reduced manual effort, and increased test coverage. Automation coverage increase rate.</p><p><strong>Test Execution Efficiency</strong> (Automation Pass-Fail rate): The ratio of passed tests to total tests executed during a specific period, reflecting the effectiveness of test execution. Higher test execution efficiency suggests better test case design and execution.</p><p><strong>Release Quality</strong> (number of critical bugs found on release candidate/production): The number of high-priority or critical defects found in release candidates. Lower release quality metrics indicate better release readiness and lower customer impact.</p><p><strong>Mean Time to Detect (MTTD)</strong>: The average time taken to detect defects from the moment they are introduced until they are identified. A lower MTTD indicates faster defect detection and response.</p><p><strong>Mean Time to Repair (MTTR)</strong>: The average time taken to fix defects or issues once they are detected. A lower MTTR suggests efficient defect resolution processes.</p><h3>Final thoughts</h3><p>Always deeply understand the product, limitations, constraints and implications of each process change or new guideline. Consult with your peers to get their perspectives and check professional resources to stay up-to-date and educated. Find out what are the best proven practices and tools that fit the technologies used in your company.</p><p>Regularly question choices and decisions made, that are the basis of existing processes and try to simplify what seems to be unnecessarily cluttered.</p><p>Use generative AI tools to research data, summarize it and get references to detailed guidelines.</p><p>Offer help and make yourself available for anyone with any TDD / quality query.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c87d8bee543a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning Test Plan Creation — from an idea to a working process]]></title>
            <link>https://medium.com/@itzik-shabtay-qa/machine-learning-test-plan-creation-from-an-idea-to-a-working-process-d4d39a262ff1?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4d39a262ff1</guid>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[sqa]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Sat, 03 Dec 2022 08:34:10 GMT</pubDate>
            <atom:updated>2022-12-03T08:34:10.576Z</atom:updated>
            <content:encoded><![CDATA[<h3>Machine Learning Test Plan Creation — from an idea to a working process</h3><p>As a QA director at Shutterfly, who leads manual and automation QA engineers working on mobile and web components, I found myself frequently asking what <strong>MORE</strong> can we do to push our quality assurance forward?</p><p>How can we make an interesting project, that will be innovative, exciting and interesting, as well as benefit the world of quality testing and improve company products quality? What technology is accessible and boundaries-breaking that we can leverage to our needs? That’s when the <strong>ML</strong> buzz words started popping up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/0*L2QedMNMUhn5F8US.jpeg" /></figure><p>Every time you have a high level idea to be implemented in practical usage, there’s a risk of imposing the idea without answering any real need. So, what are the needs we’re trying to answer?</p><p><strong>The Needs</strong></p><ol><li>Automatically identify and classify areas touched in the product during each release. Visualize and analyze them.</li><li>Reduce and maybe eliminate the need to manually create a release test plan (manual &amp; automation).</li><li>Automatically trigger the tests once a test plan is ready.</li><li>Automatically add coverage to critical issues on-the-fly.</li></ol><p>Now that we have defined our needs, what do we have to do to get there?</p><p><strong>Pre-conditions</strong></p><ol><li>A dedicated technical engineer with a clear plan.</li><li>Machine Learning knowledge.</li><li>Time and resources. Need to train the system and run few iterations.</li></ol><p><strong>The Architecture</strong></p><p>Yaroslav Kirstein, an automation engineer lead in my team, took the challenge. We met and defined our needs (as written above) and what data we can use.</p><p>Next, we registered Yaroslav to a python ML training in one of the local technological colleges. The training was 150 hours long. When he completed the training he already knew what ML model fits our needs the best.</p><p>He started the implementation using a Linear SVC model (which is best fit for classification needs), as we gather the information from JIRA using the JIRA API queries. We trained the system by manually inputting an “area” value (using JIRA’s ‘Component’ field) in more than 1000 tickets (Thanks Halyna Slobodian).</p><p><strong>Iterations</strong></p><p>We started running the scripts while fixing the wrong decisions of the system. Sooner than later, the system reached a level it can recognize and classify a ticket area by its subject and description texts. Accuracy was then improved from 80% to 86% only by iterating.</p><p>This is how the action blocks diagram looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*iWWJlAvtfoOGjaFG7LaXHw.png" /></figure><p>A Jenkins job is either manually triggered or runs by schedule. It initiates a python JIRA API query that searches for the current release features. Each feature story is classified to an area and the system links it to relevant area test suites in the testing repo. JIRA tasks to run the the test suites are created and the tests are launched. During the process also each release critical bug coverage task is being created.</p><p><strong>The Outcome</strong></p><p>By using this process, we are able to classify all features in a release by product areas (components) &amp; visualize (pie chart) and analyze the areas by version or time. Specific feature test plans are automatically created and are consolidated to a release test plan which is automatically launched.</p><ul><li>It saves us hours spent on weekly manual test plan creation.</li><li>It assures all critical bugs get covered.</li><li>It shrinks the time to production, due to full automation of the processes.</li><li>It shows us visually what ares need more attention and improve the product quality.</li></ul><p>The repo, the code, the results and all the relevant mechanism is used by and for Shutterfly inc.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4d39a262ff1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mobile Automation Frameworks]]></title>
            <link>https://medium.com/swlh/mobile-automation-frameworks-80ef7789711c?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/80ef7789711c</guid>
            <category><![CDATA[mobile-automation-testing]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[appium]]></category>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Wed, 21 Oct 2020 07:38:39 GMT</pubDate>
            <atom:updated>2020-10-21T07:38:39.412Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Advantages and disadvantages — getting the best of each</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bCRw6uyMXGxAg1iRBtC2dA.jpeg" /></figure><p><strong>Intro</strong></p><p>We live in a rapid world, where everything becomes mobile. If you are any kind of service provider or seller, you need to be available for your customers on mobile platforms or you will lose your relevance.</p><p>Mobile devices can run mobile (web) sites, native apps or hybrid apps. each of these has its place and justification, but native apps utilize the device operating system resources and features the best way, because they are directly communicating with them. Most big companies or mobile oriented startups develop native mobile apps, i.e. iOS and Android apps.</p><p>when you test mobile apps, you need to have access to (physical) devices (simulators are simply not accurate enough for customer scenarios — having limited and sterile storage) and testing scenarios can take quite some time.</p><p><strong>User Interface Automation</strong></p><p>As web automation made a revolution in UI (User Interface) testing of web sites, the same thing happened to mobile apps UI testing in recent years.</p><p><a href="https://www.selenium.dev/">Selenium</a> was a big success and still is one of the biggest WEB UI automation solutions. It allows you to develop a super quick and effective UI automation using open source implementations, while getting the support of a worldwide community of developers. Selenium can read and interact with the DOM (Data Object Model) of the web page and perform actions on the page web elements. There is no dependency on the web site developers/releases and you can use your preferred programming language.</p><p>On the basis of Selenium, a mobile solution was built and called <a href="http://appium.io/">Appium</a>. Appium can test all types of mobile apps: web based, native and hybrid. Similar to Selenium you can code in almost any language you want, you get a worldwide community support and you have no critical dependency on app developers/releases, therefore making a black box testing, interacting with the mobile elements it can read.</p><p>Mobile operating system providers (Apple — iOS and Google — Android) are providing their own native testing framework (iOS’s is called <a href="https://developer.apple.com/documentation/xctest">XCTest/XCUITes</a>t &amp; Android’s is called <a href="https://developer.android.com/training/testing/espresso">espresso</a>). It “talks” directly with the mobile elements and is integrated as part of the development environment, therefore test code needs to be written in the app development language (iOS in Swift, Android in Java/Kotlin).</p><p>The way Appium works is that it performs as a mediation layer between the client running the test code and the native testing framework performing the actions. Appium client is installed on your development environment (i.e laptop) and Appium server is installed on the mobile device and then “talks” with the native automation framework.</p><p><strong><em>So, which automation framework should you use, Appium or native?</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*EBn7Uq_XQDokNUpWZiw-3w.png" /></figure><p><strong>Why use Appium and not Native?</strong></p><ol><li>Open source + worldwide community support. Others in the world have the same struggle you have, so most likely, you will find solutions and workarounds on the web (just Google it). Java is the most common language so you will find many results for it.</li><li>Use your preferred language (Java, JavaScript, Python, php, you name it). No need to invest time in learning a new language.</li><li>You can have one Appium repository that handles the testing of both iOS and Android (at least the basic Appium driver layer can be the shared).</li><li><a href="https://en.wikipedia.org/wiki/Black-box_testing">Black box testing</a> — advantages:</li></ol><ul><li>You have no dependency on app releases and you’re not touching the app code repository.</li><li>The Appium repository is completely in your (QE) responsibility — so you can utilize it as you want. You can add API verifications as part of tests and you can add SDKs to any service. Sky&#39;s the limit.</li></ul><p><strong>Why use Native and not Appium?</strong></p><ol><li>Because Appium setup is hard and it takes some time to establish a reliable Appium setup. You need to establish at least 95% stability before you can calmly depend on its results and run it night and day. Native setup is almost seamless.</li><li>Because Native testing is MUCH MUCH faster. how fast? 2–8 times faster depending on type of test. It happens because the way it works is so different. Apart from Appium being another layer in the middle, it actually scans the DOM and waits for an element to be found. Native testing checks for a coded attribute condition. it doesn’t scan the screen elements over and over again like Appium does.</li><li>Because Native testing can make significant shortcuts: It can start a test at a specific page without going through the entire flow to get to it and it can run on mock data and not need to test both backend and frontend all the time. These 2 options make tests super fast and reliable.</li><li><a href="https://en.wikipedia.org/wiki/White-box_testing">White box testing</a> — advantages:</li></ol><ul><li>Since app developers need to be involved, they need to supply IDs for elements and sometimes even add coded attribute conditions. This makes test writing much easier and improves the quality of the app through test driven design &amp; development.</li><li>Tests can be part of the app development repository. This means every app developer can run tests as part of his/hers final round before sending the ticket to QA. A test can run in a simple right click, as simple as running unit tests.</li><li>Because tests can be part of the app repository, it is easier to integrate them into a CI-CD (<a href="https://en.wikipedia.org/wiki/Continuous_integration">Continuous Integration</a>/Delivery) flow.</li></ul><p><strong>What is the right thing for me?</strong></p><p>You didn’t expect to get a clear and definite recommendation, did you?</p><p>Appium allows you to own and control everything, including the ability to have deep testing of backend along with testing the interface, but when you have difficulties to reach an element and need app development help, the app developers are basically doing you a favor, since it’s your toy and not theirs. This independence can also cause reduced code quality and efficiency if you’re developing alone. Tests could be too long/unreliable to be part of a non human interaction CI acceptance level gate.</p><p>On the other hand, native testing requires you to strongly collaborate with app development teams with super clear definition of who does what and a clear test modification and approval process. You get many more people involved and can get higher coverage easier, but there could be control and responsibility arguments.</p><p>Some companies prefer test automation to be done by app developers and have no dedicated automation team. It’s definitely doable, but it brings questions as of how good can the same execution organization criticize/check itself. Clear procedures are needed to address these issues and it doesn’t necessarily say that developers will QA the product better than people who specialize in QA-ing the product.</p><p><strong>To conclude</strong>,</p><p>If you’re part of a small development team, that wishes to support multi-layer testing (with/without backend/mock data) and wants all developers to get involved in testing, if you have just one platform to support and you’d like a full package CI-CD — I’d go with native testing.</p><p>If you need to make API level verifications along with UI testing, you can’t easily get mock data and you need to support multi-platform OS — I’d go with Appium.</p><p>Big companies with a variety of needs and conditions can do both and get both benefits. Appium as a nightly heavy duty full end to end utility and native as a CI-CD simplified solution that involves main app developers and maintains test driven design and development.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80ef7789711c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/mobile-automation-frameworks-80ef7789711c">Mobile Automation Frameworks</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Regression Testing — Time Efficiency & Risk Analysis]]></title>
            <link>https://medium.com/swlh/regression-testing-time-efficiency-risk-analysis-5da3bc8f2279?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/5da3bc8f2279</guid>
            <category><![CDATA[regression-testing]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[qa]]></category>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Wed, 21 Oct 2020 07:21:58 GMT</pubDate>
            <atom:updated>2022-06-06T17:14:41.487Z</atom:updated>
            <content:encoded><![CDATA[<h3>Regression Testing — Time Efficiency &amp; Risk Analysis</h3><blockquote>Optimizing your product regression testing plan</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3R7CD9D_bYY1duZgVcwGAw.jpeg" /></figure><p>This post is written with manual testing in mind, but can be applicable to automation testing as well.</p><p>Whenever you join a new team or inherit a new product, you and your colleagues may find yourselves with a huge <a href="https://en.wikipedia.org/wiki/Regression_testing">regression</a> test cases inventory. It may take you a lot of time to understand the volume of the inventory and relevance of each test case and suite.</p><p>First, make sure only the relevant test suites and cases are kept active. You most probably should back up the irrelevant tests (just in case), but taking them out of the active inventory is step one in making some order.</p><p>Second, if you can validate that test cases are still valid according to the current product flow &amp; behavior it would be super important. It will save a lot of confusion, because these test cases may be executed by new testers who will not be able to run them and precious time will be wasted, exactly when you don’t have it. So, pro-actively handling them when you want to, will be the smartest thing. Best case scenario should be asking a junior QE to run them as part of his/hers on-boarding process.</p><p>Third, look for duplications. You’d be surprised to find that due to different projects and different people, there are almost identical test cases in the inventory, that with little to no work can be unified or dismissed.</p><p>Now, that we have only <strong>relevant, valid and unique tests</strong> in place, you need to examine the current regression testing process. If your team was used to run ALL tests before each release cycle — we need to check:</p><ol><li><strong>Do we really need to run all the tests?</strong></li><li><strong>How much time does it take to complete a regression testing cycle?</strong></li><li><strong>Has this process proved itself and will it be relevant in 1–2 years from now?</strong></li></ol><p>Sometimes, just the force of habit is a strong human nature and sometimes, circumstances have changed throughout time. The fact something was done in a certain way before, doesn’t necessarily mean it shouldn’t be changed. So question the process until you’re proven wrong.</p><p>Long tests that barely find bugs can and should be fine-tuned. Real customer scenarios and issues we found out at the last moment or even in production must find their way into our regression testing.</p><p><strong>Make sure you run the regression final sanity tests on the exact product build that user gets, including set up feature flags, A/B tests and specific customer configuration. In case of a B2B customer, having real customer data as part of testing will be very beneficial (database, models etc.)</strong></p><p>If the test inventory is well built, it probably has functionality based sorting. This is super important to maintain, because sometimes you will need to run specific function regression instead of a full regression and even repeat it to support development. Tagging/labeling/annotating the test with the relevant functionality/area can be very useful for both tester and developer. Specific automation tests can be triggered by the build (as part of continuous integration cycle).</p><p>If you think you’re running too many regression tests or it takes too much time to run them before each release and the business would like to release more often, then you need to start practicing <strong>Risk Analysis</strong>.</p><p>The release regression plan should be consisted of:</p><ol><li>Tests that must be included <strong>in each plan</strong>: high level acceptance tests, final sanity, important customer scenarios.</li><li>Tests that are needed for <strong>this specific plan</strong>: tests of new features and new bug fixes (you should be able to know all these beforehand).</li></ol><p>New features should have been tested thoroughly before being merged to the main development branch, so you probably don’t need to run all their certification tests as part of the regression and going forward.</p><p>Another way to look at regression testing risk analysis is to simply examine the number of tests included in the regression plan.</p><p><strong>You can set a threshold, a limit of tests per plan (assuming most tests run at similar durations).</strong></p><p>For example, you can decide each regression plan should not surpass 200 test cases. In this case, you force yourself and your team to plan accordingly, make the educated prioritization as part of plan building and keep up the time frame.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5da3bc8f2279" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/regression-testing-time-efficiency-risk-analysis-5da3bc8f2279">Regression Testing — Time Efficiency &amp; Risk Analysis</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Becoming a Software QA Engineer]]></title>
            <link>https://medium.com/swlh/becoming-a-software-qa-engineer-5452bdecd2e3?source=rss-4951edfa1d19------2</link>
            <guid isPermaLink="false">https://medium.com/p/5452bdecd2e3</guid>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[high-tech]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[career-paths]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <dc:creator><![CDATA[Itzik Shabtay]]></dc:creator>
            <pubDate>Wed, 21 Oct 2020 07:08:42 GMT</pubDate>
            <atom:updated>2022-06-11T07:30:29.956Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>A quick guideline to navigate your QE career</em></blockquote><figure><img alt="Image taken from Pixabay.com" src="https://cdn-images-1.medium.com/max/1024/1*p-kLHkQv0ReU5tVN6zyYlg.jpeg" /></figure><p>So you decided you want to become a software QA engineer. Congratulations!</p><p>Now, ask yourself <strong>why</strong> did you choose to become a SQA engineer?</p><p>If it’s because “it’s the easiest way to get into the high-tech world”, think again. Everything in our world, whether it’s low-tech or high-tech, requires your most respectful and serious approach to be executed at the highest level. You can’t fake it and your colleagues will notice it. So be sure this is what you ACTUALLY want and be ready to do whatever it takes.</p><p><strong>Being a SQA is mostly about caring for the product quality more than anybody else.</strong></p><p>On the other hand, if:</p><ul><li>Finding flaws in perfectly seeming products makes you happy. (who said OCD?)</li><li>You like to break things 😂</li><li>You enjoy handling things under-the-hood (and accidentally break them first…)</li><li>You enjoy team work and have a technical intuition.</li><li>You’re excited about software &amp; testing (aren’t we all???)</li></ul><p>carry on. You can do it and I can assure you — <strong>YOU WILL ENJOY IT</strong>.</p><p>Ask yourself if you’re seeking a short-term career in QA or a long-term. If you want a taste of what this world has to offer and consider the possibilities on-the-go, you’ll need to grab the opportunity you get, and persuade your potential employer that you’re super bright and dynamic person, that has a high learning curve and can adjust to changes. In this case, no one will expect you to have vast experience, but you’ll need to make a good impression:</p><ul><li><strong>Make some homework and get to know the company you’re considering to work for.</strong></li><li><strong>Know the basics of client-server architecture.</strong></li><li><strong>Keep an open mind and stay focused to get the hints you may get during the interview.</strong></li><li><strong>Let them know you’re someone they can trust.</strong></li></ul><p>In case you’re seeking a long-term career in SQA, you need to bring more to the equation.</p><p><strong>Your potential employer tries to envision how you’ll be performing a year and more from now. So, get yourself educated.</strong></p><p>It can be an online course, a real classroom or a lot of technical reading . Whether you’re already an engineer, a practical engineer or have no technological background, you’ll need to demonstrate that you know the terminology, the working processes, the mindset and understanding of software QA, development processes, common tools and bug’s life cycle.</p><p>Even with good understanding, there’s nothing like work experience, so if you find out that you can’t break this glass wall and get the chance that you need, <strong>get yourself a working experience thorough crowdsourced testing</strong>, such as uTest. This is a real SQA working experience you can utilize to get yourself acquainted with the world of testing and something that can push you to the next level.</p><p>Once you’re already a SQA, build your skills and strengths. Make sure you master the tools needed for your job and always learn what can be done more efficiently. Ask many questions, research and doubt everything you’re not sure of.</p><p><strong>Position yourself as a professional and thorough QA engineer and periodically test yourself for professional evolvement.</strong></p><p>After establishing yourself as a good SQA, ask your manager what you need to improve. Be open for criticism and take initiative, whilst helping your colleagues achieve success with you. Remember that a lone wolf remains alone in times of trouble, so helping each other will help all parties involved.</p><p><strong>At the end of the day, we all want collaboration and acknowledgement.</strong></p><p>As for automation engineers, being an automation engineer is more than just writing a compilable and running code, it’s a pure development role, so prepare yourself to be doing just that — develop code all day long. If that’s what you want — train yourself to be an excellent coder, no less than any full time developer you see around you. The rest is up to you.</p><p>Good Luck!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5452bdecd2e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/becoming-a-software-qa-engineer-5452bdecd2e3">Becoming a Software QA Engineer</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>