<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sreeaus on Medium]]></title>
        <description><![CDATA[Stories by Sreeaus on Medium]]></description>
        <link>https://medium.com/@sreeaus030289?source=rss-b8d0018a0f95------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*lO6gmWpocLVKgh3X</url>
            <title>Stories by Sreeaus on Medium</title>
            <link>https://medium.com/@sreeaus030289?source=rss-b8d0018a0f95------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 03:05:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sreeaus030289/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What your Playwright suite isn’t testing (and how to see it in 5 seconds) ”All tests passed” is the…]]></title>
            <link>https://medium.com/@sreeaus030289/what-your-playwright-suite-isnt-testing-and-how-to-see-it-in-5-seconds-all-tests-passed-is-the-948f98b9e4a4?source=rss-b8d0018a0f95------2</link>
            <guid isPermaLink="false">https://medium.com/p/948f98b9e4a4</guid>
            <category><![CDATA[code-coverage]]></category>
            <category><![CDATA[playwright-automation]]></category>
            <category><![CDATA[npm-package]]></category>
            <category><![CDATA[qa-automation]]></category>
            <dc:creator><![CDATA[Sreeaus]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 10:56:02 GMT</pubDate>
            <atom:updated>2026-10-14T10:56:02.536Z</atom:updated>
            <content:encoded><![CDATA[<h3>What your Playwright suite isn’t testing (and how to see it in 5 seconds) ”All tests passed” is the most misleading sentence in QA</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLz8j33vBgbW8i1xUk5YhQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*puaD7jTXd_eykChT92_X6Q.png" /></figure><h3>Why I built a boring, AI-free tool to show what a Playwright suite actually checks</h3><h4><strong>Every Playwright run ends with a number: <em>142 passed</em>. It’s a good feeling, but it answers the wrong question.</strong></h4><p>The questions I actually get asked as a QA engineer are different:</p><ul><li>Which API endpoints have no tests at all?</li><li>We document 400 and 404 for this endpoint. Does any test check them?</li><li>Are we testing the same thing in five places?</li><li>That test.skip from last spring: is it still skipped, and why?</li><li>A new joiner asks “what does this suite cover?” Where do I point them?</li></ul><p>A green run can’t answer any of these. A test that sends a request and never asserts anything also shows up as <em>passed</em>.</p><p>So I built a small tool to answer them: <a href="https://www.npmjs.com/package/pw-coverage-map">pw-coverage-map</a>.</p><h3>One command</h3><pre>npx pw-coverage-map</pre><p>That’s it. Run it in any Playwright project (TypeScript or JavaScript). It needs no config and no AI. It doesn’t run your tests, so no browsers, no webServer and no test environment are needed. A few seconds later you have coverage-map/index.html: one file you can open locally, attach to a CI job or publish on GitHub/GitLab Pages.</p><h3>What it shows</h3><p>Every test, every step, every check. Suites, test cases and their test.steps (nested ones too), with the actual expect() assertions listed under the step they belong to:</p><pre>When  I create a user with an empty name</pre><pre>↳ createUser (POST /api/users) expects HTTP 400</pre><pre>Then  the user gets the member role</pre><pre>✓ user.role is &quot;member&quot;</pre><p>Tests generated in loops are listed one by one, with ${placeholders} in step titles filled in from the real test title.</p><p>API coverage against your OpenAPI spec. Add --openapi openapi.json and you get a table of every operation in the spec, the tests that call it, and which documented response codes are actually asserted. Endpoints with no tests are shown in red. So are endpoints that are only &quot;covered&quot; by a skipped test, because a skipped test covers nothing.</p><p>Tests that check nothing. A test with no expect(), no assertion helper and no expected-status argument is flagged. These are the most dangerous tests in a suite: always green, never useful.</p><p>Tests that overlap. If two tests in different files send the same requests and make the same checks, they’re flagged as candidates to merge, or to turn into one data-driven test.</p><p>Parked tests. Every test.skip, test.fixme and test.fail, with the reason written in the code, in one list. Skipped tests have a way of becoming permanent.</p><p>Tags × test type. A small matrix that shows things like “@smoke has 12 API tests and 0 UI tests&quot;.</p><p>Last run results (optional). Pass a Playwright JSON report with --results results.json to add passed/failed/flaky, durations, error messages and the slowest tests.</p><h3>How it works (no magic)</h3><ol><li>It runs playwright test --list --reporter=json. That gives the real list of tests: titles (with loops expanded), tags, projects, files and line numbers, without running anything.</li><li>It parses each spec with the TypeScript compiler API, matches each listed test to its test() call by line number, and walks the code: steps, expect chains, request.get/post/... calls, page.goto, beforeEach hooks, and the reasons given in skip/fixme.</li><li>It follows your relative imports to find API client methods on its own. If you have a method like createUser(body, expectedStatus = 201) that makes a POST /api/users, a test calling users.createUser(body, 400) is understood as &quot;POST /api/users, expects 400&quot;. No config needed.</li></ol><p>That’s why it’s fast and deterministic: the same code always gives the same report.</p><h3>Use it in CI</h3><p>Fail the build when things slip:</p><pre>npx pw-coverage-map --openapi openapi.json --min-api-coverage 80 --fail-on-no-assertions</pre><p>Post a summary on GitHub Actions:</p><pre>- run: npx pw-coverage-map --openapi openapi.json --summary &quot;$GITHUB_STEP_SUMMARY&quot;</pre><p>And for code review, pw-coverage-map diff before.json produces Markdown listing tests added, removed, changed and newly skipped. It&#39;s handy for PR/MR comments.</p><h3>Honest limits</h3><p>It’s static analysis, so:</p><ul><li>Steps written inside page-object or helper methods aren’t followed yet (that’s the next feature I’m working on).</li><li>A loop <em>inside</em> one test shows as a single step with a ‹placeholder›.</li><li>Values only known at run time stay as placeholders.</li></ul><h3>Try it</h3><pre>npx pw-coverage-map --openapi path/to/openapi.json</pre><ul><li>npm: <a href="https://www.npmjs.com/package/pw-coverage-map">https://www.npmjs.com/package/pw-coverage-map</a></li><li>GitHub: <a href="https://github.com/sree030289/pw-coverage-map">https://github.com/sree030289/pw-coverage-map</a></li></ul><p>It’s MIT-licensed, has one dependency (TypeScript), no install scripts and makes no network calls, so it should be easy to get approved in company registries.</p><p>I’d really like to hear what it finds in your suite, and what you’d want it to show next. Issues and stars on GitHub are very welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=948f98b9e4a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>