<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lucy Diaz on Medium]]></title>
        <description><![CDATA[Stories by Lucy Diaz on Medium]]></description>
        <link>https://medium.com/@oldiazg?source=rss-ea0274874f58------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Lucy Diaz on Medium</title>
            <link>https://medium.com/@oldiazg?source=rss-ea0274874f58------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:40:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@oldiazg/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Visual Component Testing with Storybook]]></title>
            <link>https://medium.com/juliusbaerengineering/visual-component-testing-with-storybook-fc4e81d9bf12?source=rss-ea0274874f58------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc4e81d9bf12</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[ui-testing]]></category>
            <category><![CDATA[components]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Lucy Diaz]]></dc:creator>
            <pubDate>Tue, 12 Dec 2023 18:04:10 GMT</pubDate>
            <atom:updated>2023-12-15T08:32:04.436Z</atom:updated>
            <content:encoded><![CDATA[<p>I have been actively involved in frontend development for over six years, primarily working with <a href="https://angular.io/">Angular</a>, and I have also gained about a year of experience with <a href="https://react.dev/">React</a>. Throughout this extensive period, we have employed <a href="https://jasmine.github.io/">Jasmine</a> and <a href="https://karma-runner.github.io/latest/index.html">Karma</a> for Angular and <a href="https://jestjs.io/">Jest</a> for React when testing our components. A common limitation shared by both of these testing libraries is that they do not provide a rendering capability that allows developers to visually inspect what they are testing or how the component behaves during the test execution. Consequently, I have observed instances where tests fail to assess the intended aspects, and in some cases, tests end up examining nothing at all.</p><p>Testing components has proven to be a significant challenge for our backend developers who wish to contribute to the frontend. While they are eager to engage in frontend development and complete end-to-end (E2E) user stories that span both back- and frontends, they often struggle with composing component tests. This difficulty leads even our frontend developers to test component logic within the context of E2E tests rather than isolating it, mainly because crafting E2E tests is more straightforward due to the visual feedback they provide.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B83DvOewjBJatWnqHBscbw.png" /><figcaption>Simple Storybook test</figcaption></figure><h3>Why Storybook?</h3><p>This year, we initiated the development of <a href="https://module-federation.github.io/">module-federated</a> web-components to seamlessly integrate with products from other teams in real-time. This necessitated the creation of comprehensive documentation for our web components, ensuring that other teams could integrate them without the need to contact us for clarifications. Given that <a href="https://storybook.js.org/">Storybook</a> was already a widely adopted tool within our organisation, we opted to continue using it for this purpose.</p><p>We were simultaneously in the process of introducing <a href="https://playwright.dev/">Playwright</a> for end-to-end (E2E) testing. Interestingly, we discovered that Storybook incorporated visual component testing with Playwright as part of its functionalities, referred to as <a href="https://storybook.js.org/docs/react/writing-tests/interaction-testing">interaction testing</a>. This serendipitous discovery aligned perfectly with our requirements.</p><p>At the time, we had also considered two other options for visual component testing: <a href="https://www.cypress.io/">Cypress</a> and <a href="https://playwright.dev/">Playwright</a>. However, Cypress fell out of favour due to past experiences that I have detailed in another <a href="https://medium.com/@oldiazg/why-i-switched-from-cypress-to-playwright-dc41ce4d5e1b">article</a>. Additionally, Playwright’s component testing feature was still in an experimental phase, which made us apprehensive about introducing it into an enterprise-level application.</p><p>Ultimately, we concluded that it made more sense to combine web component documentation and testing. This approach ensured that no developer would overlook the documentation for new components, as it would naturally emerge as a by-product of the testing process.</p><h3>Interaction testing benefits</h3><p>Visual component testing with Storybook has proven to be remarkably straightforward and user-friendly. We have harnessed its capabilities for a wide spectrum of components, ranging from lean and straightforward UI elements to more substantial ones, which involve intricate UI and logic. During the past few months, we have reaped several advantages from this approach, which we would like to highlight:</p><ul><li><strong>Ease of writing tests: </strong>Storybook renders the component in the same manner as a typical web application would, which simplifies the process of writing tests and use cases, referred to as “stories.” This approach allows for straightforward, isolated testing of each component using Jest syntax, akin to Playwright, all while providing real-time visualisation of the component’s behaviour.</li><li><strong>Component documentation: </strong>The input and output attributes of a component are well-documented, and they can be conveniently modified from the Storybook user interface. Any changes made take immediate effect, and the component is rendered based on its attribute inputs.<br>You have the flexibility to let Storybook auto-detect your component’s inputs and outputs, which leads to the automatic generation of the controls, as shown in the screenshot provided. We have adopted this approach for our internal components, which are exclusively used within our team. However, for web components used by other teams, we have opted to provide more extensive documentation. In such cases, Storybook enables you to add additional documentation to each control and even override its type, should that level of customisation be necessary.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sJ4I7xGEsGp2N3Bz4hQtgg.png" /><figcaption>Storybook component documentation</figcaption></figure><ul><li><strong>Live interaction: </strong>Since the attributes can be modified, and the component is rendered and can be interacted with, we also have a view of all events (outputs) that are triggered by the component.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lgVhhp4mS27zBQNVZxOgxA.png" /><figcaption>Storybook actions show events triggered</figcaption></figure><ul><li><strong>Debugging: </strong>Running through the test step-by-step is very easy, using the back-and-forth arrows provided in the Interactions tab. We also have our normal browser developer tools available for debugging.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1wJCVePAe-GoaU4m81SMkQ.png" /><figcaption>Storybook interactions debugging</figcaption></figure><ul><li><strong>Mocking APIs: </strong>You can choose to let API calls be executed as normal, or you can mock them. There are a few different libraries available to do this. We are currently using a library called <a href="https://github.com/linearlabs-workspace/storybook-addon-mock">storybook-addon-mock</a> that shows what is being mocked as part of the UI; this provides a fantastic overview of your mocks. You can also modify the API response or HTTP code, the response delay, and enable/disable the mock via the UI. We mostly use this for our module-federated web components. In our case, these are complex components that perform HTTP calls.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LQADe1k-HBF22Cw1-MteUg.png" /><figcaption>Storybook API call mocks</figcaption></figure><ul><li><strong>Headless execution:</strong> While it is convenient that all of these tests run on a browser while you are developing and debugging, you will most likely want to run your tests from the command line in your CI pipeline. The library <a href="https://github.com/storybookjs/test-runner">@storybook/test-runner</a> provides everything you need.</li><li><strong>Parallel runs:</strong> Storybook’s test-runner library supports parallelisation out-of-the-box. While I have not performed a full benchmark test on its performance, I can confidently say that our tests, at least, are almost twice as fast when we use double the workers in our pipeline (which our slow <a href="https://www.jenkins.io/">Jenkins</a> seems to be able to handle).</li><li><strong>Test coverage:</strong> Storybook’s test-runner library generates lcov reports for tests if you use the --coverage flag when running your tests. We then merge this report with our other test coverage reports to provide a unified report using <a href="https://github.com/ljharb/istanbul-merge">istanbul-merge</a>.</li><li><strong>Console logs:</strong> Storybook’s test-runner library shows all browser console logs in the terminal. This helps with debugging tests that may fail only in the pipeline.</li></ul><h3>Interaction testing shortcomings</h3><p>Although Storybook has made significant strides, and we would not revert to our previous methods of crafting visual component tests, there are certain facets of Storybook that could benefit from further enhancement. Here are a few areas that could be improved:</p><ul><li><strong>Error testing:</strong> Storybook currently silently swallows application exceptions. This means that if you would like to test an <a href="https://react.dev/reference/react/Component#catching-rendering-errors-with-an-error-boundary">ErrorBoundary</a> based on some exception being thrown, you are out of luck. We were also not able to test how our web components behave if an HTTP call fails (allowing the real HTTP call to occur); instead, we had to mock the response as shown on the screenshot below. This is because if we let the real HTTP call be executed and the backend were to respond with anything other than a 2xx HTTP status code, the request promise would never complete, and the test would fail with a timeout.<br>This is a fairly major issue for us because we would have liked to test the error use case integration with the backend.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LQADe1k-HBF22Cw1-MteUg.png" /><figcaption>Storybook API call mocks</figcaption></figure><ul><li><strong>Storybook syntax: </strong>The syntax for creating stories, documentation, and tests in Storybook is quite distinctive and does not adhere to the more conventional describe/it syntax employed by other widely recognised testing libraries. It is essential to note that adapting to this novel syntax can be a learning curve and may take some time to become familiar with. This adjustment period should be factored into your evaluation when considering whether Storybook is a suitable tool for your team’s needs.</li></ul><pre>// ---------------------------------------------------------------------------<br>// Define decorators for your stories, this one adds your app theme to<br>// the components in Storybook<br>export const withTheme: Decorator = (StoryFn) =&gt; {<br>  return (<br>    &lt;ThemeProvider theme={theme}&gt;<br>      &lt;StoryFn /&gt;<br>    &lt;/ThemeProvider&gt;<br>  );<br>};<br><br>// ---------------------------------------------------------------------------<br>// Define the story as the default export<br>// With this configuration, Storybook will auto-detect and create the controls<br>// based on the component props<br>export default {<br>  title: &#39;Internal-Components/ThousandSeparatorNumberFormat&#39;,<br>  component: ThousandSeparatorNumberFormat,<br>  decorators: [withTheme],<br>} as Meta&lt;typeof ThousandSeparatorNumberFormat&gt;;<br><br>type Story = StoryObj&lt;typeof ThousandSeparatorNumberFormat&gt;;<br><br>// This is the template for all upcoming stories<br>const Template: StoryFn&lt;typeof ThousandSeparatorNumberFormat&gt; = (args) =&gt; (<br>  &lt;TextField<br>    inputProps={{ ...(args as object) }}<br>    InputProps={{<br>      inputComponent: ThousandSeparatorNumberFormat as any,<br>    }}<br>  /&gt;<br>);<br><br>// ---------------------------------------------------------------------------<br>// Use-case: thousands formatting with suffix without max value<br>export const WithoutMaxValue: Story = Template.bind({});<br>WithoutMaxValue.args = {<br>  name: &#39;Thousands Number Formatter&#39;,<br>  suffix: &#39;$&#39;,<br>  customTextIndent: &#39;2px&#39;,<br>  disableMaxValue: true,<br>  customDataTestId: &#39;thousand-separator-number-format-input&#39;,<br>};<br>WithoutMaxValue.play = async ({ args, canvasElement, step }) =&gt; {<br>  // your component will be rendered within the storybook canvas area<br>  // if you are testing modals, this will not work and you will need to use<br>  // screen.getByTestId(...) instead of canvas.getByTestId(...) where:<br>  // import { screen } from &#39;@storybook/testing-library&#39;;<br>  const canvas = within(canvasElement);<br><br>  // we use steps to group tests in small components like this one, or<br>  // in more complex components, to group logical test steps<br>  await step(&#39;Each digit triggers a change event&#39;, async () =&gt; {<br>    userEvent.clear(canvas.getByTestId(args.customDataTestId ?? &#39;&#39;));<br>    userEvent.type(canvas.getByTestId(args.customDataTestId ?? &#39;&#39;), &#39;123456&#39;);<br><br>    expect(canvas.getByDisplayValue(&#39;123,456 $&#39;)).toBeInTheDocument();<br>    await waitFor(() =&gt; expect(args.onChange).toHaveBeenCalledTimes(6));<br>    await waitFor(() =&gt;<br>      expect(args.onChange).toHaveBeenLastCalledWith({<br>        target: {<br>          name: args.name,<br>          value: &#39;123456&#39;,<br>        },<br>      })<br>    );<br>  });<br>};</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AMbUiY5VSMKRHm2gEKsUXQ.png" /><figcaption>Storybook test from the above code</figcaption></figure><ul><li><strong>Lack of beforeEach/afterEach:</strong> Storybook lacks a built-in mechanism for executing code before and/or after each test. However, there are workarounds available that involve using decorators to encapsulate and customise your component under test, enabling you to implement code to run before each test as needed. It is important to note that while workarounds exist for executing code before each test, there is currently no known workaround for running code after each test in Storybook.</li><li><strong>Reporting:</strong> While Storybook’s test-runner does offer coverage reports and JUnit reports, these provide only limited support for debugging when failures occur in the CI pipeline. What sets Playwright apart is its comprehensive and detailed out-of-the-box report, encompassing screenshots, videos, and the complete DOM for effective debugging. Storybook currently lacks a report that consolidates all the ongoing tests, their execution times, and — most crucially — detailed information for failed test cases. This information could include screenshots, videos, access to all Storybook tabs, and more. Incorporating such a comprehensive reporting feature would greatly enhance the debugging capabilities of Storybook.</li><li><strong>Test run retry:</strong> Storybook does not provide a way of retrying individual failed tests. Both Jest and Playwright do provide configuration options for this, but Storybook does not.</li><li><strong>Speed:</strong> As anticipated, rendering the components in your tests does indeed result in increased test execution times. In our pipelines, we have observed that these tests take approximately three times longer to run compared to Jest tests, although we have not conducted in-depth benchmark tests and we are allowing HTTP calls to be executed in some of these Storybook tests. Despite the decrease in speed, our team finds that the advantages brought about by visualisation outweigh this performance trade-off.</li><li><strong>Headless run needs Storybook to be running: </strong>When executing Storybook tests via the command line, it is currently necessary to manually start the Storybook application and confirm that it is running smoothly before running the tests. It would be more convenient and efficient if the process of spawning Storybook were integrated into the runner command itself, simplifying the workflow for developers.</li></ul><h3>Conclusion</h3><p>Testing UI components can be a challenging endeavour when you lack the ability to visually inspect their appearance at each stage of the testing process. This level of abstraction can lead to situations where developers inadvertently test aspects other than what they originally intended. The advent of visual component testing frameworks has significantly improved the developer experience when it comes to testing components. Even if these tests may run slower, the increased accessibility of testing motivates developers to write more comprehensive tests, enhancing the overall quality of the software.</p><p>We will continue to keep an eye on the progress in this field to see how it evolves, especially considering that the current visual component testing frameworks still have significant room for improvement. For now, we are content with Storybook and plan to persist with its adoption for our new frontend repositories, even though not all of them incorporate web components.</p><p>Note: This article refers to Storybook version 7.5</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc4e81d9bf12" width="1" height="1" alt=""><hr><p><a href="https://medium.com/juliusbaerengineering/visual-component-testing-with-storybook-fc4e81d9bf12">Visual Component Testing with Storybook</a> was originally published in <a href="https://medium.com/juliusbaerengineering">Julius Baer Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why I switched from Cypress to Playwright]]></title>
            <link>https://medium.com/@oldiazg/why-i-switched-from-cypress-to-playwright-dc41ce4d5e1b?source=rss-ea0274874f58------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc41ce4d5e1b</guid>
            <category><![CDATA[playwright-test]]></category>
            <category><![CDATA[cypress]]></category>
            <category><![CDATA[test-automation]]></category>
            <dc:creator><![CDATA[Lucy Diaz]]></dc:creator>
            <pubDate>Wed, 02 Aug 2023 07:52:23 GMT</pubDate>
            <atom:updated>2023-11-07T15:30:19.519Z</atom:updated>
            <content:encoded><![CDATA[<p>I started using <a href="https://www.cypress.io/">Cypress</a> back in 2019 when the company I worked for decided to move away from <a href="https://www.protractortest.org/">Protractor</a> for new projects. At that time, <a href="https://angular.io/">Angular</a> was the framework I was using, and I got the opportunity to implement the Cypress PoC. Recently, I switched jobs and am now working with <a href="https://react.dev/">React</a>, where I also got the chance to implement the <a href="https://playwright.dev/">Playwright</a> PoC.</p><p>Having experience with both Angular and React, the use of data-testid attributes for testing has been my implementation preference. This has allowed me to maintain a consistent approach to UI end-to-end testing, and I haven’t observed any significant differences in testing between Angular and React applications.</p><p>NOTE: I use a React app as an example for both Cypress and Playwright in this article.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*8FsR8lzrCfAf8W2J.jpg" /><figcaption>Playwright vs Cypress</figcaption></figure><h3>What do both testing frameworks offer?</h3><p>Both Cypress and Playwright offer a fantastic UI testing experience (you can also test APIs). They make it easy for developers to write tests because of their auto-wait functionality, they provide a UI to visualise your tests as they run, also generate screenshots and videos of your tests and support typescript.</p><p>Both frameworks also support visual component testing, but I will not be covering this topic in this article.</p><p>Since both frameworks offer very similar functionality, I will dissect how each achieves the following aspects and how they affect the developer experience/productivity:</p><ul><li>The syntax used to write tests plays a crucial role in the learning curve, and overall ease of use of each framework. Including but not limited to extending the framework with custom commands and recording tests.</li><li>Test execution and maintainability are essential factors that impact the confidence developers have in their tests and the amount of time spent debugging them, particularly concerning speed and stability.</li><li>Test reports play a critical role in the testing process, and evaluating their ease of setup and the level of information they provide is essential for both frameworks.</li></ul><h3>My Cypress Experience</h3><p>Installing Cypress was easy enough, mostly just an NPM dependency and off you go. However, it did not take long to encounter the intricate details that you need to know about Cypress to be able to work with it. Here are some of the predominant ones:</p><ul><li><strong>Syntax used to write tests: </strong>Cypress uses a promise-like syntax to write tests. This may not seem confusing at first, but developers tend to think that because their API looks like a promise, it behaves like a promise (async/await). The sad reality is that it does not, and this causes devs to spend a lot of time learning how to work with the Cypress API and make it fit their particular testing scenario.<br>If you need to work with async/await, you have 2 options: either wrap it in Cypress commands to be able to add it to the <em>Cypress command chain</em>, or use a library like <a href="https://www.npmjs.com/package/cypress-promise">cypress-promise</a>. Here is an example of each option:</li></ul><pre>// async-await.spec.ts<br>import promisify from &#39;cypress-promise&#39;;<br><br>function sleep(milliseconds: number) {<br>  return new Promise((resolve) =&gt; setTimeout(resolve, milliseconds));<br>}<br><br>async function asyncFunction(text: string) {<br>  console.log(&#39;started asyncFunction &#39; + text);<br>  await sleep(3000);<br>  console.log(&#39;finalized asyncFunction &#39; + text);<br>}<br><br>context(&#39;Async/Await Test&#39;, () =&gt; {<br>  beforeEach(() =&gt; {<br>    cy.visit(&#39;/&#39;);<br>  });<br><br>  it(&#39;convert promises into cypress commands, do not write tests using async/await&#39;, () =&gt; {<br>    cy.wrap(null).then(() =&gt; asyncFunction(&#39;first&#39;));<br>    cy.wrap(null).then(() =&gt; asyncFunction(&#39;second&#39;));<br>  });<br><br>  it(&#39;convert cypress commands to promises, should be able to code with async/await&#39;, async () =&gt; {<br>    const foo = await promisify(cy.wrap(&#39;foo&#39;));<br>    const bar = await promisify(cy.wrap(&#39;bar&#39;));<br>    expect(foo).to.equal(&#39;foo&#39;);<br>    expect(bar).to.equal(&#39;bar&#39;);<br>  });<br>});</pre><ul><li><strong>Test Reports:</strong> Cypress lacks a built-in test report feature, which means obtaining a comprehensive summary of all tests, their durations, and additional elements like screenshots and linked videos for failures can be a challenging and cumbersome process. Despite investing considerable effort in configuring the reports, there might still be instances where screenshots are missing from the generated reports.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QgXR5prXZLNJ8HBm3tLfqg.png" /><figcaption>Basic Cypress Mocha test report for failed test.</figcaption></figure><pre> // package.json: example of cumbersome test reporting setup<br>  &quot;scripts&quot;: {<br>    &quot;-------------------- E2E Commands --------------------&quot;: &quot;&quot;,<br>    &quot;cypress:open&quot;: &quot;nyc cypress open&quot;,<br>    &quot;cypress:run&quot;: &quot;npm-run-all -s --continue-on-error _clean _cypress-run:run _cypress-run:html&quot;,<br>    &quot;-------------------- Supporting Commands --------------------&quot;: &quot;&quot;,<br>    &quot;_clean&quot;: &quot;npx rimraf coverage cypress/output/junit cypress/output/mocha-json cypress/output/mocha-html/*.html cypress/output/mocha-html/*.json .nyc_output&quot;,<br>    &quot;_coverage-report&quot;: &quot;npx nyc report --reporter html&quot;,<br>    &quot;_cypress-run:run&quot;: &quot;nyc cypress run --headless --browser chrome&quot;,<br>    &quot;_cypress-run:html:merge-json&quot;: &quot;mochawesome-merge cypress/output/mocha-json/*.json &gt; cypress/output/mocha-html/merged-mochawesome.json&quot;,<br>    &quot;_cypress-run:html:gen-html-from-json&quot;: &quot;marge cypress/output/mocha-html/merged-mochawesome.json -f cypress -o cypress/output/mocha-html -i true --charts true&quot;,<br>    &quot;_cypress-run:html&quot;: &quot;npm-run-all -s _cypress-run:html:merge-json _cypress-run:html:gen-html-from-json _coverage-report&quot;<br>  },</pre><ul><li><strong>How to run headed tests:</strong> Cypress runs your web-app inside of its own browser app. It first opens a page where it lists all of your specs, you can then click on a spec to run all tests inside of that spec. I might add that all of this is quite slow. Furthermore, if you would like to only run one test inside of your spec, you need to mark it as only, save and then only that test will auto reload.</li></ul><figure><img alt="App running inside of Cypress browser." src="https://cdn-images-1.medium.com/max/1024/1*DA01oS_nrO9ZdiSIb37IRw.png" /><figcaption>App running inside of Cypress browser.</figcaption></figure><ul><li><strong>How to access browser console logs in headless runs: </strong>Browser console logs are only visible when you run the tests in headed form (with DevTools open). This means that if your tests run successfully locally but fail on your CI pipeline, then you are in for a fun time trying to get those logs to the pipeline console. Luckily there is a plugin that facilitates this, but it was not just a plug-and-play job due to the way Cypress plugins work.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pBBISRJOKqAbxp0kxmQlXg.png" /><figcaption>Browser console logs visible only when DevTools are open.</figcaption></figure><pre>// cypress/plugins/index.ts<br>interface Browsers {<br>  family: string;<br>  name: string;<br>}<br><br>interface LaunchOptions {<br>  args: string[];<br>}<br><br>const cypressPLugins = (on: unknown, config: unknown) =&gt; {<br>  require(&#39;@cypress/code-coverage/task&#39;)(on, config);<br>  // log console.* messages to the cypress console,<br>  // it helps when there are errors on the CI/CD pipeline<br>  require(&#39;cypress-log-to-output&#39;).install(on, consoleToLogConfig);<br>  require(&#39;@cypress/react/plugins/react-scripts&#39;)(on, config);<br>  // @ts-ignore<br>  on(&#39;before:browser:launch&#39;, (browser: Browsers, launchOptions: LaunchOptions) =&gt; {<br>    if (browser.family === &#39;chrome&#39; || browser.name === &#39;chrome&#39;) {<br>      console.log(&#39;Adding chrome config...&#39;);<br>      launchOptions.args.push(&#39;--disable-dev-shm-usage&#39;);<br>      launchOptions.args.push(&#39;--lang=en&#39;);<br>    }<br>    return launchOptions;<br>  });<br>  return config;<br>};<br><br>export const consoleToLogConfig = (_type: unknown, event: { level: string; type: string }) =&gt; {<br>  // return true or false from this plugin to control if the event is logged on the cypress console<br>  // `type` is either `console` or `browser`<br>  // if `type` is `browser`, `event` is an object of the type `LogEntry`:<br>  //  https://chromedevtools.github.io/devtools-protocol/tot/Log/#type-LogEntry<br>  // if `type` is `console`, `event` is an object of the type passed to `Runtime.consoleAPICalled`:<br>  //  https://chromedevtools.github.io/devtools-protocol/tot/Runtime/#event-consoleAPICalled<br><br>  // only show error events:<br>  return event.level === &#39;error&#39; || event.type === &#39;error&#39;;<br>};<br><br>export default cypressPLugins;</pre><ul><li><strong>Debugging Tests: </strong>There are a few different options to debug in Cypress and there are many articles about this topic out there. The main take away is that because Cypress commands run asynchronously, you cannot just add a debugger command. <br>Essentially, Cypress provides you with 2 useful commands: debug() and pause(). The first one adds a debugger command to the Cypress runner ,while the later one stops your test run at that point. Both provide you with the ability to inspect the DOM, but only with pause() can you step through each of the upcoming Cypress commands in your test.<br>Given that Cypress uses a promise-like chaining API, you can chain either of those commands with any Cypress command:</li></ul><pre>cy.get([data-testid=&quot;username-input&quot;]).type(&quot;my-username&quot;).pause();</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z1gGMf2DP-xhqNg8YIQDkA.png" /><figcaption>Debugging in Cypress.</figcaption></figure><ul><li><strong>Test Parallelisation: </strong>Cypress does not support running tests in parallel locally. There are plenty of libraries out there to help you achieve parallelisation, however, this is no easy feat. In my previous company we tried to parallelise our tests in order to improve pipeline runtime, but the effort turned out to be so great that we de-prioritised it.<br>The side effect, is that developers rarely run the full Cypress tests suit locally, they just waited for feedback from the pipeline.</li><li><strong>Speed: </strong>Cypress is slow when it runs both in headless and headed browser format. This is the main reason we wanted to parallelise tests. Even running tests locally is incredibly slow. When developing a test, the test auto-reloads when you modify the file, this can take a couple of seconds even on my MacBook Pro.</li><li><strong>Tests Stability: </strong>Cypress is known for being flaky, mostly on CI/CD pipelines and it causes devs to spend a lot of time chasing ghosts. The usual <em>hacky solution </em>I have seen devs implement, is to add a cy.wait(&lt;ms&gt;) to the test and adding a retry to the CI/CD pipeline.</li><li><strong>Custom Commands: </strong>If you have common functionality that you use often in your tests, you will most likely want to create a custom Cypress command for it. Unfortunately, adding such commands is not intuitive nor type-safe (unless you go the extra mile):</li></ul><pre>// cypress/support/commands.ts<br>Cypress.Commands.add(&#39;getByTestId&#39;, (selector: string, ...args: unknown[]) =&gt; {<br>  return cy.get(`[data-testid=${selector}]`, ...args)<br>});<br><br>// cypress/typings/cypress.d.ts<br>declare namespace Cypress {<br>  interface Chainable {<br>    getByTestId(selector: string, ...args: unknown[]): Chainable;<br>  }<br>}<br><br>// you can now use this command in your test:<br>it(&#39;should type username in username input&#39;, () =&gt; {<br>  cy.getByTestId(&#39;username-input&#39;).type(&#39;my-username&#39;);<br>});</pre><ul><li><strong>Recording Tests: </strong>Cypress now offers the <em>Cypress Studio</em>, which is currently an experimental feature that I have personally not tried yet.</li></ul><h3>My Playwright Experience</h3><p>I started working with React at the start of 2023 and my team did not have any UI tests yet. Therefore I decided to look into introducing Cypress. However, we had also started introducing web-components and we planned to use <a href="https://storybook.js.org/">Storybook</a> to document and test our web-components. Since Storybook uses Playwright under the hood I investigated what Playwright could offer. The initial appeal was: one single framework that my team would have to learn to test both web-components and our UI.</p><p>Installing Playwright was a walk in the park: I had full tests running in less than an hour.</p><ul><li><strong>Syntax used to write tests: </strong>Writing Playwright tests is as simple as writing normal typescript code, there is no special API to learn. The best part is that you can write normal async/await code so you can use all of your normal typescript supporting functions. Example:</li></ul><pre>import { test } from &#39;@playwright/test&#39;;<br><br>test.describe(&#39;Playwright test with async/await&#39;, () =&gt; {<br>  function sleep(milliseconds: number) {<br>    return new Promise((resolve) =&gt; setTimeout(resolve, milliseconds));<br>  }<br><br>  async function asyncFunction(text: string) {<br>    console.log(&#39;started asyncFunction &#39; + text);<br>    await sleep(3000);<br>    console.log(&#39;finalized asyncFunction &#39; + text);<br>  }<br><br>  test(&#39;should open storybook and click around&#39;, async ({ page }) =&gt; {<br>    await page.goto(&#39;http://localhost:6006/&#39;);<br>    await page<br>      .getByRole(&#39;link&#39;, {<br>        name: &#39;Storybook 7.1.1 is available! Your current version is: 7.0.2 Dismiss notification&#39;,<br>      })<br>      .click();<br>    await page.getByRole(&#39;link&#39;, { name: &#39;Storybook&#39; }).click();<br>    await page.locator(&#39;#internal-components-overview--docs&#39;).click();<br>  });<br><br>  test(&#39;should test the async methods used in previous example&#39;, async ({<br>    page,<br>  }) =&gt; {<br>    await asyncFunction(&#39;first&#39;);<br>    await asyncFunction(&#39;second&#39;);<br>  });<br>});</pre><ul><li><strong>Test reports: </strong>This was such a pleasant surprise: I did not have to do anything! The report that Playwright generates out-of-the-box is fantastic. You are also able to run your tests on multiple browsers very easily (just change the config, Playwright will install the browsers it needs for you).<br>I have it configured to only include trace-information, screenshots and videos for failing tests; otherwise, just like Cypress, test runs are slow. In the example below, our first test failed. If we click on it, we get the following info: the exception causing the failure, which steps of the test failed, a video of the test, and full trace of the test.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*i7YqewHSh9Pu93jRBCioew.png" /><figcaption>Playwright HTML Report Overview</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*VVDm5lT4OEQbhKZ0HCoscA.png" /><figcaption>Failed Test Playwright Report Summary</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WOa3ZZLGfQGUR7GTfRxPLw.png" /><figcaption>Playwright Test Trace</figcaption></figure><ul><li><strong>How to run headed tests: </strong>Playwright is developed by Microsoft, so of course they provide a plugin for <a href="https://code.visualstudio.com/">VS Code</a>. I am an IntelliJ user myself, but I find myself switching to VS Code when I want to run Playwright tests.<br>Unlike Cypress, you dont have to start anything to be able to run one or multiple tests. Just open your code in VS Code and click on the usual unit test green triangle, or the green-tick/red-X if you have already run the test in the past. You can also choose if you would like the browser to open or the trace viewer.<br>The developer experience here is fantastic. Tests open fast and are very responsive.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TNZ7F8L-5g02deAoA09xGg.png" /></figure><ul><li><strong>How to access browser console logs in headless runs: </strong>Like Cypress, Playwright does not show browser logs in the console where Playwright is running. In order to achieve this, you can extend the Playwright Page object to forward browser logs to the console log (there are other ways of doing this, for example, extending the test object itself).</li></ul><pre>// utils/console.util.ts<br>export const configureLogForwarding = (page: Page) =&gt; {<br>  page.on(&#39;console&#39;, (msg) =&gt; {<br>    if (process.env.PLAYWRIGHT_LOG_TO_CONSOLE === &#39;true&#39;) {<br>      switch (msg.type()) {<br>        case &#39;info&#39;:<br>        case &#39;log&#39;: {<br>          // eslint-disable-next-line no-console<br>          console.log(`Log: &quot;${msg.text()}&quot;`);<br>          break;<br>        }<br>        case &#39;warning&#39;: {<br>          // eslint-disable-next-line no-console<br>          console.log(`Warning: &quot;${msg.text()}&quot;`);<br>          break;<br>        }<br>        case &#39;assert&#39;:<br>        case &#39;error&#39;: {<br>          // eslint-disable-next-line no-console<br>          console.log(`Error: &quot;${msg.text()}&quot;`);<br>          break;<br>        }<br>      }<br>    }<br>  });<br>};<br><br>// tests/demo-tests.spec.ts<br>test.describe(&#39;Playwright test with async/await&#39;, () =&gt; {<br>  test.beforeEach(async ({ page }) =&gt; {<br>    configureLogForwarding(page);<br>    hostAppNavigationPo = new HostAppNavigationPo(page);<br>    genericAssetModalPo = new GenericAssetModalPo(page);<br>  });<br>// ...</pre><ul><li><strong>Debugging Tests: </strong>This is one of my favourite features. To debug a test, set a break-point and right-click on the green-triangle/green-tick/red-x and click on <em>Debug Test</em>. The browser will open and the run will stop at your break-point. From here on, it is a normal debugging session that devs are used to.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*YpMNIAHnhHhMDNG7JWX1zA.png" /><figcaption>Start Debugging Session in Playwright</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*018sTeChOw29tDN9uGVuJg.png" /><figcaption>Playwright On-Going Debugging Session</figcaption></figure><ul><li><strong>Test Parallelisation: </strong>Playwright supports parallelisation out-of-the-box (your system under test must of course be able to support tests running in parallel). Given that our CI/CD tool does not have a lot of resources and is therefore slow, we have turned parallel testing off in our pipeline. However, when I am coding, I am able to run all of our tests locally very fast. Granted, we have only been using Playwright for about 5 months so we only have ~80 tests. Here are some non-scientific numbers: ~80 tests, 5m in CI/CD (1 worker), 2m locally (1 worker), 55s locally (5 workers).</li></ul><pre>// playwright.config.ts<br>import { defineConfig, devices } from &#39;@playwright/test&#39;;<br><br>export default defineConfig({<br>  /* other configuration... */<br>  /* Opt out of parallel tests on CI. */<br>  workers: process.env.CI ? 1 : undefined,<br>  /* Run tests in spec files in parallel */<br>  fullyParallel: true,<br>});</pre><ul><li><strong>Speed: </strong>When running equivalent tests in headless form (as you would in a CI/CD pipeline), Playwright is ~1.5 faster than Cypress. <br>Furthermore, Playwright is able to start tests on the browser in less than a second (on my MacBook Pro). This means that I can run any test, in any spec, at any time without having to start a <em>Playwright server</em>, etc. Unlike Cypress, for which you first have to start the Cypress UI and then navigate to the test you would like to run within that UI.<br>This makes for a fantastic developer experience!</li><li><strong>Tests Stability: </strong>We have been writing tests for the last 5 months, and we are yet to experience any flakiness and have not had had to use any waits or sleeps in our tests. Since I am no longer working at the previous company where we had Cypress, it is difficult for me to ascertain if migrating those flaky tests to Playwright would stabilise them.</li><li><strong>Custom commands: </strong>Playwright offers a way of <a href="https://playwright.dev/docs/api/class-test#test-extend">extending the base </a><a href="https://playwright.dev/docs/api/class-test#test-extend">test</a> so that you can have custom commands and/or page objects easily accessible during your tests. This pattern is simple to follow, intuitive and type-safe. Example from Playwright:</li></ul><pre>// my-test.ts<br>import { test as base } from &#39;@playwright/test&#39;;<br>import { TodoPage } from &#39;./todo-page&#39;;<br><br>export type Options = { defaultItem: string };<br><br>// Extend basic test by providing a &quot;defaultItem&quot; option and a &quot;todoPage&quot; fixture.<br>export const test = base.extend&lt;Options &amp; { todoPage: TodoPage }&gt;({<br>  // Define an option and provide a default value.<br>  // We can later override it in the config.<br>  defaultItem: [&#39;Do stuff&#39;, { option: true }],<br><br>  // Define a fixture. Note that it can use built-in fixture &quot;page&quot;<br>  // and a new option &quot;defaultItem&quot;.<br>  todoPage: async ({ page, defaultItem }, use) =&gt; {<br>    const todoPage = new TodoPage(page);<br>    await todoPage.goto();<br>    await todoPage.addToDo(defaultItem);<br>    await use(todoPage);<br>    await todoPage.removeAll();<br>  },<br>});<br><br>// example.spec.ts<br>import { test } from &#39;./my-test&#39;;<br><br>test(&#39;test 1&#39;, async ({ todoPage }) =&gt; {<br>  await todoPage.addToDo(&#39;my todo&#39;);<br>  // ...<br>});</pre><ul><li><strong>Recording Tests: </strong>Playwright provides the option to record UI tests using their VS Code plugin. All you need to do is press on the Record new button and Playwright will open a browser for you. Then enter the URL of the web-app and start clicking. The best part about Playwright recording is that it will pick the data-testid selector ( await page.getByTestId(&#39;dialog-title&#39;); ) if one exists.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*clsfh46nro1biK36LZC0RQ.png" /></figure><p>There are many other advantages to Playwright that I did not cover in this article, mostly because I have not used those aspects heavily, but here are some examples:</p><ul><li><strong>iFrames: </strong>Playwright works with iFrames out-of-the-box and the experience is incredibly smooth. We are using iFrames in my current job but not in my previous job (when I was working with Cypress). It is my understanding, however, that you need to install a Cypress plugin to be able to test iFrames. This <a href="https://www.cypress.io/blog/2020/02/12/working-with-iframes-in-cypress/">article</a> explains in great detail why Cypress struggles with iFrames and how to test them.</li><li><strong>Web-app server: </strong>Both Cypress and Playwright require your app to be running for tests to be able to run. A popular request for Cypress was for the framework to be able to start the system under test before running the tests. I personally always have my app running when I run my tests, both locally and in the CI/CD pipeline. However, developers wishes have been heard (by Playwright):</li></ul><pre>// playwright.config.ts<br>import { defineConfig, devices } from &#39;@playwright/test&#39;;<br><br>export default defineConfig({<br>  // Run your local dev server before starting the tests.<br>  webServer: {<br>    command: &#39;npm run start&#39;,<br>    url: &#39;http://127.0.0.1:3000&#39;,<br>    reuseExistingServer: !process.env.CI,<br>  },<br>});</pre><ul><li><strong>Forbid </strong><strong>only Keyword: </strong>A common use-case, is that when a developer wants to run a single test, they mark it as only. Unfortunately, they often forget to remove it before committing. Playwright comes with a configuration to detect this, while with Cypress we had to add eslint rules.</li></ul><pre>// playwright.config.ts<br>import { defineConfig, devices } from &#39;@playwright/test&#39;;<br><br>export default defineConfig({<br>  /* Fail the build on CI if you accidentally left test.only in the source code. */<br>  forbidOnly: !!process.env.CI,<br>});<br><br>// example.spec.ts<br>test.describe(&#39;Playwright test with async/await&#39;, () =&gt; {<br><br>  test.only(&#39;should test the async methods used in previous example&#39;, async ({<br>    page,<br>  }) =&gt; {<br>    await asyncFunction1();<br>    await asyncFunction2();<br>  });<br><br>// more tests...</pre><ul><li><strong>Multiple Pages: </strong>Cypress runs your web-app inside of its own web-app, this imposes the restriction that Cypress can only run tests on one page at the time. Playwright on the other hand, can open multiple <a href="https://playwright.dev/docs/api/class-browsercontext">BrowserContext</a> and/or multiple Page in each browser. all having different websites/domains/etc. and run a test across them all.</li><li><strong>Multi-language Support: </strong>In Cypress, you have the option to write tests using either JavaScript or TypeScript. However, in Playwright, the language options for writing tests are more diverse. You can choose to write tests in JavaScript, TypeScript, Java, Python, or .NET, providing developers with greater flexibility in their preferred programming language for test automation.</li></ul><h3>Conclusion</h3><p>After having been a fan of Cypress for many years, and a strong advocate of UI testing, it was not an easy decision to start using a new framework that I was not sure would be as good as Cypress is. Having said that, it was such a pleasant surprise to discover that Playwright not only does what Cypress can do but exceeds it in all angles that I can think of.</p><p>Lastly, the developer community for both Cypress and Playwright are considerably large. Albeit Cypress still being much more popular, the Playwright community is growing fast as more developers discover how amazing the developer experience is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*SVWOcLzoQEBagQYLDzYR_g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*0gVvLVUmDT1aarl0J1pTXA.png" /><figcaption>Numbers as of 29th July 2023</figcaption></figure><p>I would recommend to anyone who is already using Cypress or is considering getting into UI testing, to consider Playwright as a worthy option.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc41ce4d5e1b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>