<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dilum Pathiraja on Medium]]></title>
        <description><![CDATA[Stories by Dilum Pathiraja on Medium]]></description>
        <link>https://medium.com/@dilumpathi?source=rss-1e78fd7783e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1tFTKnpPKQc26RdyJfXHjA.jpeg</url>
            <title>Stories by Dilum Pathiraja on Medium</title>
            <link>https://medium.com/@dilumpathi?source=rss-1e78fd7783e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 07:19:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dilumpathi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Front End Performance Testing With Datadog(RUM)]]></title>
            <link>https://dilumpathi.medium.com/front-end-performance-testing-with-datadog-rum-c6a12449152a?source=rss-1e78fd7783e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6a12449152a</guid>
            <category><![CDATA[front-end-performance]]></category>
            <category><![CDATA[frontend-performance]]></category>
            <category><![CDATA[datadog]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[performance-testing]]></category>
            <dc:creator><![CDATA[Dilum Pathiraja]]></dc:creator>
            <pubDate>Thu, 29 Jul 2021 07:22:36 GMT</pubDate>
            <atom:updated>2021-07-29T07:22:36.575Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/860/0*xxrhijEpt7bLGzAo.jpg" /></figure><p>In my previous article about <a href="https://dilumpathi.medium.com/front-end-performance-testing-with-chrome-devtools-e51eb9e00d51">Front End Performance Testing With Chrome DevTools</a>, I discuss the importance of Front End Performance Testing, why we need it, and its benefits. Also in there, I mentioned it’s a manual effort and will take some considerable time to do the test.</p><p>In this article, I’m going to explain how to test the Front End Performance using <strong><em>DataDog </em></strong>which will overcome the above-mentioned problems. This will be a very reliable approach and allows us to integrate into the CI pipeline and enable continuous frontend SLA verifications. Here I will be explaining how to use Datadog’s <a href="https://docs.datadoghq.com/real_user_monitoring/"><strong>Real User Monitoring (RUM)</strong></a> capabilities to test the Frontend Performance of a web application​.</p><h3>Why DataDog RUM?</h3><p>Datadog’s RUM provides end-to-end visibility into individual users&#39; real-time activity and experience having many more great features. Out of many, below are some of the key features that fulfilled the technical feasibility.</p><ul><li><strong>Performance</strong>: Track the performance of web pages, user actions, network requests, and your front-end code.</li><li><strong>Error Management</strong>: Monitor the ongoing bugs and issues and track them over time and in versions.</li><li><strong>Analytics / Usage</strong>: Understand who is using your application (country, device, OS), monitor individual users’ journeys, and analyze how users interact with your application (most common page visited, clicks, interactions, feature usage).</li><li><strong>Support</strong>: Retrieve all of the information related to one user session to troubleshoot an issue (session duration, pages visited, interactions, resources loaded, errors).</li><li><strong>Dashboards</strong>: Analyze information about your user journeys, performance, network requests, and errors collected automatically with <a href="https://docs.datadoghq.com/real_user_monitoring/dashboards">out-of-the-box dashboards</a>.</li><li><strong>Tracking user actions via Datadog Custom Actions</strong>: Most user actions like user “clicks” on the application are collected, some actions like scrolling and keyboard tabbing are not automatically collected by RUM SDK. Those user actions or interactions can be collected by added <a href="https://docs.datadoghq.com/real_user_monitoring/browser/tracking_user_actions/?tab=npm#custom-actions">Custom Actions</a> in the application code. Custom actions are user actions declared and sent manually by using the addAction API. They are used to send information relative to an event occurring during a user journey.</li><li><strong>Single-page applications (SPAs) support</strong>:</li></ul><ol><li>For <a href="https://docs.datadoghq.com/real_user_monitoring/browser/monitoring_page_performance/#monitoring-single-page-applications-spa">single-page applications (SPAs)</a> the RUM SDK differentiates between initial_load and route_change navigations with the loading_type attribute. Initial load is a traditional URL change, stemming from a complete load or reload of a URL. This is indicated in the browser when a page<em> </em>load event fires (the window.onload event). Initial page loads appear along with route changes in the browser UI.</li><li>If a click on a web page leads to a new page without a full refresh of the page, the RUM SDK starts a new <strong>view event</strong> with loading_type:route_change. RUM tracks page changes using the <a href="https://developer.mozilla.org/en-US/docs/Web/API/History">History API</a>. Datadog provides a unique performance metric, loading_timewhich calculates the time needed for a page to load. This metric works both for initial_load and route_change navigations. To account for modern web applications, loading time watches for network requests and DOM mutations.</li></ol><blockquote><strong>Initial Load</strong>: Loading Time is equal to whichever<em> </em>is longer: The difference between navigationStart and loadEventEnd.Or the difference between navigationStart and the first time the page has no activity for more than 100ms (activity is defined as ongoing network requests or a DOM mutation).</blockquote><blockquote><strong>Route Change</strong>: Loading Time is equal to the difference between the user click and the first time the page has no activity for more than 100ms (activity is defined as ongoing network requests or a DOM mutation).</blockquote><h3>Frontend performance testing with Datadog RUM</h3><p>Although RUM is mostly used for real user monitoring in PRODUCTION applications, my focus is to test/monitor our application’s user actions via Datadog RUM. Below are the primary steps that enable testing frontend performance via Datadog RUM:</p><ol><li>Setting up Datadog RUM SDK in frontend codebase</li><li>Setting up data collection</li><li>Perform user actions in the browser</li><li>Visualize <a href="https://docs.datadoghq.com/real_user_monitoring/data_collected/">collected data </a>in Datadog dashboard</li></ol><h3>Setting up Datadog RUM SDK in Frontend Codebase</h3><p>This will initialize the Datadog RUM in Frontend and enable Datadog to collect the user actions happening in client-side browsers. There are three methods to install SDK in the codebase.</p><ol><li><strong>npm (node package manager) </strong>—<strong> </strong>The RUM SDK gets packaged with the rest of your front-end javascript code.</li><li><strong>CDN async </strong>— The RUM SDK is loaded from our CDN asynchronously: this method ensures the SDK download does not impact page load performance.</li><li><strong>CDN sync </strong>— The RUM SDK is loaded from our CDN synchronously: this method ensures the SDK is loaded first and collects all errors, resources and user actions.</li></ol><p>You can get more understanding on this by referring to <a href="https://docs.datadoghq.com/real_user_monitoring/browser/#setup">this</a>.</p><h3>Setting up data collection (<strong>Client-Side Browser Data)</strong></h3><p>Once RUM SDK is set up and deployed to Frontend codebase, Datadog collects browser data inline to 6 key <strong>Event Types</strong> (RUM Events) described below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Kp_GqV3DC4e0cyn6VwsBw.png" /></figure><p>Out of these RUM events, <strong>View</strong> and <strong>Action</strong> (User Actions) are what we mainly use to extract the <strong>Loading Time,</strong> Performance Indicator (KPI)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fm4R_mezoPxmOKIhQw4VtA.png" /></figure><p>The following diagram illustrates the RUM event hierarchy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*EDK63gMP8dZgu7k0HwY-fQ.png" /></figure><p>NOTE: If you need you can create a separate environment(<a href="https://aws.amazon.com/ec2/instance-types/">AWS EC2</a>) to execute the tests and capture all the events coming from that environment to RUM graphical interfaces.</p><h3>Perform user actions in the browser</h3><p>To perform the automated user actions like clicks, scrolls and navigations we are using an automated selenium suite. You can create an automated suite with the test scenarios that you want to test the performances. These automated Frontend Performance test scenarios will execute frequently via a Jenkins job.</p><h3>Tracking user actions via Datadog Custom Actions</h3><p>Datadog <a href="https://docs.datadoghq.com/real_user_monitoring/">Real User Monitoring (RUM)</a> Browser SDK automatically detects user interactions performed during a user journey.</p><p>The automatic <a href="https://docs.datadoghq.com/real_user_monitoring/browser/tracking_user_actions/?tab=npm">collection of user actions</a> provides insights into user behavior, without having to manually instrument every single click in your application. It helps you achieve the following objectives:</p><ul><li>Understand the performance of key interactions (for example, a click on a button)</li><li>Quantify feature adoption</li><li>Identify the steps that led to a specific browser error</li></ul><p>Although most user actions like user “clicks” on the application are collected, some actions like scrolling and keyboard tabbing are not automatically collected by RUM SDK. Those user actions or interactions can be collected by added <a href="https://docs.datadoghq.com/real_user_monitoring/browser/tracking_user_actions/?tab=npm#custom-actions">Custom Actions</a> in the application code.</p><h4>What are <a href="https://docs.datadoghq.com/real_user_monitoring/browser/tracking_user_actions/?tab=npm#custom-actions">Custom Actions</a>?</h4><p>Custom actions are user actions declared and sent manually by using the addAction API. They are used to send information relative to an event occurring during a user journey. In the following example, the RUM SDK collects visitor’s data when they hit the button.</p><pre>import { datadogRum } from &#39;<a href="http://twitter.com/datadog/browser-rum">@datadog/browser-rum</a>&#39;;</pre><pre>datadogRum.addAction(&#39;&lt;NAME&gt;&#39;, &#39;&lt;JSON_OBJECT&gt;&#39;);</pre><pre>// Code example<br>datadogRum.addAction(&#39;&lt;NAME&gt;&#39;, {<br>    &lt;JSON_OBJECT&gt;: {<br>        amount: 42,<br>        currency: &#39;$&#39;,<br>        nb_items: 2,<br>        items: [&#39;socks&#39;, &#39;t-shirt&#39;],<br>    },<br>});</pre><h3>Visualize collected data in Datadog dashboard</h3><p>The data collected during the performed user actions are visualized in a Datadog Dashboard that builds custom to showcase the Frontend Performance statistics.</p><p>Let’s look at how we can create a Dashboard in DataDog:</p><ol><li>In Datadog, navigate to the <a href="https://app.datadoghq.com/rum/list?_gl=1*kwnjlf*_ga*Mzc3NjcwMzgxLjE2MjQwMjI0MzI.*_ga_KN80RDFSQK*MTYyNzQ4NjAxMy42LjEuMTYyNzQ4NjQxMy4w">Real User Monitoring page</a> and click the New Application button.</li><li>Enter a name for your application and click Generate Client Token. This generates a clientToken and an applicationId for your application.</li></ol><blockquote>Note: Your application shows up on the application list page as “<strong>pending</strong>” until Datadog starts receiving data.</blockquote><p>For more details on setting up the <a href="https://docs.datadoghq.com/real_user_monitoring/browser/#setup">Dashboard</a>.</p><p>The user actions performed via frequently executed automated frontend performance test scenarios will pump the client-side browser data into time-series graphs. This will help us to understand the frontend performance details with a greater number of samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s_rrp-VBryAYsqFzOSq8hw.jpeg" /><figcaption>Sample Dashboard</figcaption></figure><blockquote>Every time the selenium suite is executed, the graph will be updated with the times that took for the relevant actions. If the graph exceeds the SLA line then there is a Performance breach and you can investigate what causes this degradation.</blockquote><p>By clicking on any point in the time series line, the dashboard will present a list menu of different views that narrow down or illustrates different statistics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tSvLylG9qgN8BMZh3sgJ0g.jpeg" /></figure><p>Out of which, the “<strong>View RUM events</strong>” option highlighted above will further elaborate the <strong>Action</strong>/<strong>View</strong> events that took place at the point you clicked on, in a new window.</p><p>By selecting a particular <strong>Action</strong>/<strong>View</strong> event in the newly opened window, you will be presented with the full trace of calls that contributes to the load time at the point in the time series line you clicked. Datadog dashboards will present all the available information captured pertaining to the particular <strong>Action</strong>/<strong>View</strong> event which will help us understand the happenings under the hood.</p><p>Hope you learn something useful. See you in the next one ✌️</p><h3>References</h3><ul><li><a href="https://docs.datadoghq.com/real_user_monitoring/">https://docs.datadoghq.com/real_user_monitoring/</a></li><li><a href="https://docs.datadoghq.com/real_user_monitoring/browser/">https://docs.datadoghq.com/real_user_monitoring/browser/</a></li><li><a href="https://docs.datadoghq.com/real_user_monitoring/browser/monitoring_resource_performance/">https://docs.datadoghq.com/real_user_monitoring/browser/monitoring_resource_performance/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6a12449152a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Front End Performance Testing With Chrome DevTools]]></title>
            <link>https://dilumpathi.medium.com/front-end-performance-testing-with-chrome-devtools-e51eb9e00d51?source=rss-1e78fd7783e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e51eb9e00d51</guid>
            <category><![CDATA[web-performance-testing]]></category>
            <category><![CDATA[manual-testing]]></category>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[front-end-performance]]></category>
            <category><![CDATA[chrome-dev-tools]]></category>
            <dc:creator><![CDATA[Dilum Pathiraja]]></dc:creator>
            <pubDate>Tue, 06 Jul 2021 12:47:55 GMT</pubDate>
            <atom:updated>2021-07-06T12:47:55.940Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wjST4xqxE7LQ2nwo" /></figure><p>In the current world, about 200 million active websites are providing various services across different domains. Out of that 200 million, some websites fall into the same domain. So there is huge competition among those to attract people to their website. When considering the people&#39;s attraction the performance of the website is a very important thing. If the website is taking too much time to load or to satisfy the user&#39;s expectation that’s not a good or well-developed website. Front End Performance testing can be used to easily identify these loading issues, and developers can take care of those issues.</p><p>There are numbers of paid and non-paid(open source) tools in the modern world to test the FE Performance of a website. In this article, I’m going to discuss how to test the FE Performance of a website using the Chrome Developer tools(using the Performance tab in the Chrome Developer tools).</p><p><strong>NOTE:</strong> <em>This a manual test that should be done manually. For a Continuous Delivery Project, this is not a good approach. This will take some considerable time according to the test scenario carried out. Also, as this will be a manual effort, the accuracy of the test results may be not 100% correct.</em></p><h3>What is Front-End Testing?</h3><p>Front-end testing is a way for QA specialists to assess the performance of a product from a user’s standpoint. Several decades ago, front-end testing was not a necessity for QA teams. As pages used to run by simply loading an HTML file, little-to-nothing could go wrong. With the adoption of CSS and Ajax, a web page has become a series of external references and stringed operations. Now, neglecting front-end performance testing will bring tons of damage in terms of user retention and the extent to which the product adheres to business needs.</p><h3>Benefits of Front-End Performance Testing</h3><p>As front-end performance testing is connected to client-side issues, its impact is more visible to the end-user and directly affects the browsing experience. After having tested the performance of the entire front-end, in theory a QA team should end up with a stable and fully functional build ready to be brought to market. Following are some significant benefits front-end performance testing brings to the table.</p><ul><li>Detecting client-side performance issues</li><li>Tests are simple and cost-effective</li><li>Improved quality of user interactions</li><li>Missing out on front-end end testing does lots of damage</li><li>Can identify bottlenecks observed by front-end users on the browser level</li></ul><h3>What is the Chrome Developer tool?</h3><p>Chrome DevTools is a set of web developer tools built directly into the Google Chrome browser. You can get more details and an idea on this by referring to the <a href="https://developer.chrome.com/docs/devtools/overview/">document</a> by the Chrome developer team. In that, they have clearly mentioned how we can open the dev tools window and the use of each tab in there.</p><h3>Prerequisites for a new test cycle</h3><ul><li>Clear browser cache, cookies, and history from your Chrome browser(Better to take an incognito browser for the test).</li><li>Make sure all other applications that consume system resources are terminated (eg: virus scanners, etc.).</li><li>CPU usage should not be utilized excessively during this test. I have observed that when only the chrome browser is opened, the CPU utilization usually lingers around 30% — 40%.</li><li>Make sure only one Chrome browser window is open with no other tabs, except the tab with AUT.</li></ul><h3>How to measure the FE Performance using DevTools</h3><p>Before going into the test steps let me explain the test scenario first. I’ll take the navigation to a category(smartphones) on <a href="https://www.ebay.com/">eBay</a> as my test scenario.</p><h4>Prerequisites</h4><ul><li>Navigate to the <a href="https://www.ebay.com/">eBay</a> home page.</li><li>Before we do the test, open up dev-tools, go to the Performance tab, and hit the record button.</li></ul><p><strong><em>IMPORTANT</em></strong>: You should stay 15–20 seconds after hitting the record button before you start your scenario. Chrome takes some considerable processing power to initialize the recording, which could affect your readings if you start right away.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ThRbcaK34L_xXo48UgPnEQ.jpeg" /><figcaption>Click Record Button</figcaption></figure><h4>Test Steps</h4><ul><li>Ok, now the test has started. We are going to calculate the loading time of the <a href="https://www.ebay.com/b/Cell-Phones-Smart-Watches-Accessories/15032/bn_1865441">Smartphones</a> category page after clicking on the Smartphones link on the eBay home page. After the recording is on Click on the Smartphones link on the eBay home page.</li><li>After the page is loaded wait for ~5 seconds in the end and stop the recording.</li><li>The next step is analyzing the performance recording profile.</li></ul><blockquote><strong>NOTE:</strong> You would need to repeat the same scenario 3–5 times and get the average value as the final result. If there are any outliers exclude them when you take the average.</blockquote><h3>Analyzing the profile</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z7wDPlS8d8PiVxnJWyyxgA.jpeg" /><figcaption>Profile Analysis</figcaption></figure><h4>Explanation on the numbers in the diagram</h4><ol><li>First, narrow down the timeline selection to start &amp; end positions</li><li>Event Click can be considered as the scenario starting position. The end position can be decided by looking at the screenshots in the timeline.</li><li>These are the network calls that are executed in loading the page. If you want you can reduce the time taken for a specific network call(s) from the total time. For example, you can deduct the synchronous (blocking) backend API calls. By clicking on them you can get the time spend for each of these blocking calls.</li><li>This summary view says how much time has taken for the current selection (also it has a nice breakdown of time taken for different FE tasks). You can reduce the time taken for blocking API calls (measured from step #3 above) from this total duration to get the pure FE time for this scenario.</li></ol><h3>Some Important Notes</h3><ul><li>For a better accurate result, every test Scenario should be measured at least 3 times to retrieve the average. If there are any outliers exclude them when you take the average.</li><li>CPU usage should not be utilized highly during the test. The CPU utilization was between 30% — 40% when only the chrome browser is open.</li></ul><blockquote>NOTE: As I explained earlier this is a manual test that should be done by an QE. So repeating this test multiple times would be a time consuming one. Also the accuracy of the test might be not 100% correct. An automated Front End Performance test will overcome this issue. In my next article I’ll explain on how to measure the FE Performnace with <strong>DataDog</strong> using an automated test suite.</blockquote><p>Hope you learn something useful. See you in the next one ✌️</p><h3>References</h3><ul><li><a href="https://developer.chrome.com/docs/devtools/evaluate-performance/">https://developer.chrome.com/docs/devtools/evaluate-performance/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e51eb9e00d51" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Performance Testing with K6]]></title>
            <link>https://dilumpathi.medium.com/performance-testing-with-k6-6985f74b5a51?source=rss-1e78fd7783e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6985f74b5a51</guid>
            <category><![CDATA[non-functional-testing]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[performance-testing-tools]]></category>
            <category><![CDATA[k6]]></category>
            <dc:creator><![CDATA[Dilum Pathiraja]]></dc:creator>
            <pubDate>Tue, 29 Jun 2021 12:29:21 GMT</pubDate>
            <atom:updated>2021-06-29T12:29:21.892Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sjQat_EcRgsJdWVl" /></figure><p>k6 is an open-source load testing tool for testing the performance of APIs, microservices, and websites and it’s a developer-centric load testing tool built for making performance testing a productive and enjoyable experience. Using k6, you’ll be able to catch performance regression and problems earlier, allowing you to build resilient systems and robust applications.</p><p>k6 is a JavaScript-based load testing tool that has two operational modes:</p><ul><li><a href="https://k6.io/open-source">k6 Open Source </a>— tool for users to self manage (Free)</li><li><a href="https://k6.io/cloud">k6 Cloud</a> — SaaS for engineering teams (Paid)</li></ul><h3>Why k6?</h3><p>k6 supports a set of built-in and custom metrics that can be used to measure various things. Below are some of the most important features that k6 can be satisfied with.</p><ul><li>Scriptability​ &amp; Automatability</li><li>Low learning complexity​</li><li>Easy integration with CI/CD, reporting, and APM platforms</li><li>High accuracy and performance​</li><li>Comprehensive documentation</li><li>High community support</li></ul><p>Ragnar Lönn, who contributed to the early development of k6, has published a comprehensive <a href="https://k6.io/blog/comparing-best-open-source-load-testing-tools/#history-and-status">load testing tool evaluation in 2020</a> which, even though may seem a little biased towards k6, has some very detailed reviews and comparisons on famous open source tools we have today.</p><h3>Pros &amp; Cons of usingk6</h3><p>Following are some advantages and disadvantages of k6.</p><p>Pros:</p><ul><li>Provides a seamless dev experience through its immersive scripting APIs.</li><li>Lightweight, fast, and very easy to use.</li><li>Real-time test results can be published to various platforms including Grafana (via InfluxDB), DataDog, and AWS CloudWatch.</li><li>Can define <a href="https://k6.io/docs/using-k6/thresholds">thresholds</a> within the script, which would be ideal to automate certain SLA-based pass/fail criteria in CI/CD pipelines.</li><li><a href="https://k6.io/docs/integrations">Can integrate with numerous external systems</a>.</li><li>So many <a href="https://k6.io/docs/using-k6/scenarios#executors">out-of-the-box options to control the throughput</a>.</li><li>k6 <a href="https://k6.io/docs/testing-guides/running-large-tests">claims</a> that a single load generator can sustain up to 30,000–40,000 simultaneous users (VUs). This would usually generate closer to 300,000 requests per second (RPS). This is much higher than the test with the highest RPS we have right now.</li></ul><p>Cons:</p><ul><li>Not generating a comprehensive HTML report like in JMeter.</li><li>Even though the scripting syntax is similar to NodeJS, some key fundamental<a href="https://k6.io/docs/#what-k6-does-not"> differences</a> are due to the k6 architecture. Because of this, if someone jumps straight into k6 scripting, without a proper understanding of k6 architecture, chances are high for him/her to end up with an inefficient perf script even if that person is an expert of Node/JS.</li><li><a href="https://github.com/loadimpact/k6/issues/140">Distributed load generation is not yet possible out-of-the-box</a>. However, we can implement that using a single Grafana + InfluxDB setup. Also, k6 claims that its architecture enables a single node to generate large volumes of load.</li></ul><h3>Installation of k6</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ExfTgNnkjgVWCSUjJqpZcQ.png" /></figure><p>The easiest way to install k6 on your machine is to follow this <a href="https://k6.io/docs/getting-started/installation">k6 guide</a> provided by the k6 team. They have provided the instructions OS-wise and you can choose the installation guide according to your OS.</p><p>There is also <a href="https://hub.docker.com/r/loadimpact/k6/">an official docker image</a>. It is not recommended for beginners to build from the source, but you always have the option to do that.</p><p>To enhance your scripting experience I recommend you to install k6 plugins/extensions available for your IDE. Below are a couple of handy extensions if you are using <a href="https://code.visualstudio.com/">Visual Studio Code</a>:</p><ul><li>mjacobson.snippets-for-k6</li><li>k6.k6</li></ul><h3>Sample Folder Structure</h3><p>You can create your own folder structure in k6 and there is no rule for the folder structure. With k6, one main advantage we can gain is the ability to locate the performance test scripts with the application codebase itself. I have used <a href="https://petstore.swagger.io/">PetStore</a> which is an open-source swagger to conduct a sample test scenario using k6. Below shown is a sample folder structure that I have used in it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*n5ALtUHvQhGEMWNBm2cOng.png" /><figcaption>Sample Folder Structure</figcaption></figure><p><strong>data</strong> is where we can keep all our test data such as <strong><em>.csv</em></strong> files needed for the load test.</p><p><strong>lib</strong> is where we can keep utility files needed for k6 scripting.</p><p><strong>tests</strong> is where we can keep individual k6 test files. Each test file should act like a “Thread Group“ in JMeter’s context where typically, it should contain one or more related endpoints that are controlled by the same test conditions and configurations.</p><p><strong>config.js</strong> is the file responsible to hold test configurations. The configurations like Ramp-up, Hold duration, Teardown, and Service host can be stored here.</p><p><strong>main.js</strong> is like a test suite that combines all tests from the <strong>tests</strong> folder for k6 to run all of them together in a single load test. In a way, this acts similar to the Test Plan we had in JMeter. Typically, this <strong>main.js</strong> should contain “All Tests” for the corresponding component. You may create more such “Test Suite” files for different test combinations if required.</p><h3>k6 Test Life-cycle</h3><p>k6 has its own test life cycle and the test should be written in a way that complies with the given modal by the k6. Following is the test life cycle modal given by the k6 team. It’s better to get a clear idea of this before going for the scripting at once.</p><pre>// 1. init code</pre><pre>export let options = {<br>  // 2. options object<br>}</pre><pre>export function setup() {<br>  // 3. setup code<br>}</pre><pre>export default function (data) {<br>  // 4. VU code<br>}</pre><pre>export function teardown(data) {<br>  // 5. teardown code<br>}</pre><ul><li><strong>init code</strong> — contain imports and variable declarations that typically happen before everything else. Here you can import <a href="https://docs.k6.io/docs/init-context">k6 Script API</a> and other <a href="https://docs.k6.io/docs/modules">JavaScript modules</a> (libraries) that you desire to make use of.</li><li><strong>Objects object</strong> — is where the test configurations are defined for k6 such as several virtual users (VUs), test duration, throughput, thresholds, etc.</li><li><strong>setup code</strong> — is executed once before the test enters the VU code. The state can be passed through to the VU code via the data object.</li><li><strong>VU code</strong> — is the function that is getting iterated during the load test. This is where we place our API requests and their assertions, etc. There can be multiple VU code functions in a single test file. k6 supports a feature called virtual users. This means that you can use separate “smart” virtual users to test your system. The code in this section which is inside the exported “<em>default</em>” function, is run over and over inside each VU and the aggregated results of all these VUs are processed and reported by k6.</li><li><strong>teardown code</strong> — is executed once after the VU code execution is completed. Any housekeeping tasks can be placed here.</li></ul><p>For more information regarding the test life cycle of a k6 script read through the <a href="https://k6.io/docs/using-k6/test-life-cycle">k6 documentation</a> to understand each segment’s role in a k6 script.</p><p><strong><em>NOTE:</em></strong> Even though k6 is a JavaScript-based tool, it is not NodeJS, nor is it a browser. Packages that rely on APIs provided by NodeJS, for instance, the <strong>os</strong> and <strong><em>fs</em></strong> modules, will not work in k6. The same goes for browser-specific APIs like the <strong>window</strong> object. Please go through the <a href="https://k6.io/docs/using-k6/modules">k6 Modules</a> which give more understanding on this.</p><p>You can use <strong><em>console.log()</em></strong> to print various attributes available through the <a href="https://k6.io/docs/javascript-api/k6-http/response">response object</a> for debugging purposes. Below is a common and useful console log:</p><pre>console.log(response.request.method + ‘ ‘+ response.request.url + ‘ -&gt; ‘ + response.status + ‘ ‘ + response.body.toString());</pre><h4>A Sample VU Code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*DEsPeRmi2WrNcRyAjmBWGA.png" /><figcaption>Sample test</figcaption></figure><ul><li>By using the <strong><em>papaparse library</em></strong> we can import our test data files to the test.</li><li>Inside the <strong><em>getPet method, </em></strong>we are creating the parameters that we need to send the request.</li><li>In the <strong><em>request(res), </em></strong>the<strong><em> URL</em></strong> should be sent as the first parameter and the <strong><em>parameters(params)</em></strong> should be the second one.</li><li>In the <strong><em>URL</em></strong> I have used a random value as the pet_id which is taken from the <strong><em>pets.csv</em></strong>.</li><li>Using the <strong><em>check</em></strong> I have asserted whether our response is equal to 200.</li></ul><h3>Do k6 have the Test Suites?</h3><p>k6 is built to run one test file at a time and doesn’t support the concept of “test suites“ out-of-the-box. But if you are writing only one or few related endpoints (for the sake of simplicity and maintainability) in a single test file, then you might want to run all of them together to simulate the load test. To overcome this I have come up with my own strategy to implement the concept of “test-suites“ for our tests as you can see below.</p><ul><li>Here, we are importing our individual tests into a single k6 (JavaScript) file as in the below sample code.</li><li>Then we can create the options object on the fly by consolidating scenarios, thresholds, etc. defined in individual tests.</li><li>Next, we can build the VU code segment using the same coming from each imported test file.</li></ul><p>This way, we will end up with another executable k6 script that resembles a “test suite“. This is similar to a <strong>Test Plan</strong> in <strong>Jmeter</strong>.</p><p>But for the above “test suite“ concept to work, you’ll need to ensure the following in your individual test files:</p><ul><li>All test files should have a similar options format as expected by the “test suite“</li><li>VU function in each test file should have its own unique name</li></ul><h4>A Sample Test Suite</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*ME-D9Wb7Oq1qX6rouwdpEQ.png" /><figcaption>main.js</figcaption></figure><h3>Test Configurations</h3><p>In k6, test configurations are set up in the form of a JavaScript object called <strong><em>options</em></strong> in each test file. k6 has a detailed <a href="https://k6.io/docs/using-k6/options">document</a> on this.</p><p>Here the <strong><em>executor</em></strong> is used to control our throughput as we want. In my example, I have used the <strong><em>Ramping Arrival Rate </em></strong>as the<strong><em> </em></strong><em>executor. </em>For more details on these executors, you can have a look at <a href="https://k6.io/docs/using-k6/scenarios/executors/">this k6 document</a>.</p><p>Below is a sample <strong><em>options</em></strong> object that I used in my tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*6g0ZwU6IsowU6eIqgS-yzw.png" /><figcaption>Sample Option object</figcaption></figure><ul><li>Our <strong><em>start rate</em></strong> is zero. That means when the test is starting we don’t have any requests processing.</li><li><strong><em>timeUnit</em></strong> is used to generate the given throughput with the given time period. For example, in <strong><em>stages,</em></strong> I have given the target as <strong>20</strong> and the <strong><em>timeUnit</em></strong> as <strong>10s</strong>. That means 20 transactions will be happening at a time period of 10seconds.</li><li><strong><em>preAllocatedVUs </em></strong>is something like threads. I have given it as 10 and that means 10 threads(virtual users) will be allocated when the test is starting.</li><li><strong><em>maxVUs </em></strong>is the maximum number of threads(virtual users) that the test can have. The test will achieve the given throughput with the <strong><em>preAllocatedVUs. </em></strong>If that amount is not enough to achieve the throughput, the test will take more <strong><em>VUs</em></strong>(virtual users)<strong><em> </em></strong>up to 100<strong><em>.</em></strong></li><li><strong><em>stages</em></strong> is used to control our throughput. From the start to 30seconds which is the <strong><em>RampUp period,</em></strong> we want only a throughput of 20. From 31s to 300s which is the <strong><em>Hold Duration,</em></strong> we want a throughput of 20. When the time reached to 310seconds after starting the test which is the <strong><em>Tear Down period</em></strong>, the throughput will be gradually decreased to 0.</li><li><strong><em>thresholds </em></strong>can be used to see the status of our test, whether it has passed or failed. After a test in Jmeter, we manually check our test results and analyze the results like 90th response time, average, etc. But in k6 we don’t need to do that manually. We can configure that using the <strong><em>thresholds. </em></strong>In the above example, I have checked the 90th value of the <strong>latency</strong> is below 750. If not the test will be failed. Also, I have checked the overall <strong>failure rate</strong> is below 0.001.</li></ul><h3>Test Execution</h3><p>Ok now after scripting the test cases all of you might be looking at how to execute the test cases. If you have done everything correctly(installation, scripting) you just need do is issuing the below command, from a CLI, from the same directory where your script is located:</p><pre>$ k6 run script.js</pre><p>Also, you can run it with 10 <em>virtual users </em>(VU) and over a period of 30 seconds, as follows:</p><pre>$ k6 run -u 10 -d 30s script.js</pre><p>It basically runs k6 on your machine using 10 virtual users over the course of 30 seconds, and it checks to see if the test URL returns <em>200 (OK)</em> in all tests. For more details, you can refer to the <a href="https://k6.io/docs/getting-started/running-k6#running-local-tests">Executing k6 scripts locally</a>.</p><h4>The terminal output of a sample test script for k6</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AtYkODOOS7XKnuJQ.png" /><figcaption>The terminal output of a sample test script for k6</figcaption></figure><h3>Automating Pass/Fail Criteria</h3><p>By using the <strong><em>Thresholds</em></strong> feature in k6, we can automate the pass/fail criteria of a test in the k6 test script itself. In JMeter (and many other tools) we don’t have this feature. For more details, you can look at <a href="https://k6.io/docs/using-k6/thresholds">Thresholds</a> documentation given by the k6 team which clearly describes this.</p><h3>Integrations</h3><p>Another major advantage that has in k6 is the easy <a href="https://k6.io/docs/integrations">integration</a> with a large array of external platforms. This will ease the things in execution as well as results analysis. Let’s look at few tools that the k6 can be integrated.</p><h4>Jenkins</h4><p>There are two common ways to run k6 tests as part of a CI process:</p><ul><li>k6 run to run a test locally on the CI server(You can use the k6 docker image to run the load tests in Jenkins.).</li><li>k6 cloud to run a test on the <a href="https://k6.io/cloud">k6 Cloud</a> from one or multiple geographic locations.</li></ul><p>For more details on this, you can refer to the documentation on load testing with <a href="https://k6.io/blog/integrating-load-testing-with-jenkins/">Jenkins</a> given by the k6 team.</p><h4>Grafana</h4><p>All of you know Grafana is open-source analytics &amp; monitoring solution, we can use Grafana to monitor our test results. It is very easy to <a href="https://k6.io/docs/results-visualization/influxdb-+-grafana">integrate k6 with Grafana + InfluxDB</a> setup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*A6lcMSm5QsQL8NgO.png" /><figcaption>Sample Grafana Dashboard</figcaption></figure><h4>DataDog</h4><p>DataDog is another monitoring tool that we can use to monitor our tests. <a href="https://k6.io/docs/results-visualization/datadog">Integrating k6 with DataDog</a>, which is given by the k6 team is also easy and straightforward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rKQwj_3FBQvyd6yjPLf1vA.png" /><figcaption>Sample DataDog Dashboard</figcaption></figure><h3>Distributed Load Generation</h3><p>k6 <a href="https://k6.io/docs/testing-guides/running-large-tests#distributed-execution">claims</a> its architecture is so improved that a single load generator can handle an enormous load compared to other tools.</p><p><em>“k6 is different from many other load testing tools in the way it handles hardware resources. A single k6 process will efficiently use all CPU cores on a load generator machine. A single instance of k6 is often enough to generate a load of 30.000–40.000 simultaneous users (VUs). This amount of VUs can generate upwards of 300,000 requests per second (RPS).</em></p><p><em>Unless you need more than 100,000–300,000 requests per second (6–12M requests per minute), a single instance of k6 will likely be sufficient for your needs.“</em></p><p>Hope you learn something useful. See you in the next one ✌️</p><h3>References</h3><ul><li><a href="https://k6.io/">https://k6.io/</a></li><li><a href="https://k6.io/blog/comparing-best-open-source-load-testing-tools/#history-and-status">https://k6.io/blog/comparing-best-open-source-load-testing-tools/#history-and-status</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6985f74b5a51" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Test Automation with Cypress]]></title>
            <link>https://dilumpathi.medium.com/test-automation-with-cypress-addf46eac11a?source=rss-1e78fd7783e------2</link>
            <guid isPermaLink="false">https://medium.com/p/addf46eac11a</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[cypressio]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Dilum Pathiraja]]></dc:creator>
            <pubDate>Tue, 05 Jan 2021 16:03:30 GMT</pubDate>
            <atom:updated>2021-01-05T16:03:30.783Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dHPJKZXpWFoRffx1o9UV8g.jpeg" /></figure><h4>What is Cypress and Why Cypress?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfvzAnqxN_x9ItROTE1NpQ.png" /></figure><blockquote>Cypress is a tool for reliably testing anything that runs in a web browser.</blockquote><p>Also, Cypress is a JavaScript-based test automation tool that rapidly gains popularity in modern industry due to:</p><ul><li>Fast and reliable execution</li><li>Lightweight and easy adoption</li><li>Embedded stubbing mechanisms</li><li><a href="https://docs.cypress.io/guides/overview/why-cypress.html#Features">Some rich and unique features</a></li></ul><p>Ability to make tests run fast build-time, in-built reliable features that reduce coding and gain simplicity was tailor-made. Also, being a JavaScript tool, allows us to co-locate this test suite in our React frontend application code-bases straightaway.</p><h4>How to install Cypress?</h4><figure><img alt="Installing cypress" src="https://cdn-images-1.medium.com/max/745/1*7k7ELJqOQ7QO6EjGKFEsrw.png" /></figure><p><strong>If you want to install Cypress and create a new project:</strong></p><ol><li>Navigate to the folder you want to install Cypress in a terminal</li><li>Create a <strong>package.json</strong> file</li></ol><p>$ npm init</p><p>3. Install Cypress with npm</p><p>$ npm install cypress --save-dev</p><p>Note: This will install Cypress locally as a dev dependency for your project. Furthermore, you can refer to <a href="https://docs.cypress.io/guides/getting-started/installing-cypress.html#System-requirements">Cypress installing</a> steps put together by the Cypress team.</p><p><strong>If you want to use an existing Cypress project</strong></p><ol><li>Clone the Cypress project from Git</li><li>Navigate to the project folder in a terminal</li><li>Install the dependencies in the local <strong>node_modules</strong> folder</li></ol><p>$ npm install</p><p>Note: this will download dependencies defined in the <strong>package.json</strong> file and generate required modules in the <strong>node_modules</strong> folder</p><p><strong>Installing Cypress via yarn</strong></p><p>$ cd/your/project/path</p><p>$ yarn add cypress --dev</p><h4>cypress.json</h4><p>This is the <a href="https://docs.cypress.io/guides/references/configuration.html#Options">Cypress configuration file</a> and the main entry point to a Cypress project. In order to use the below-mentioned folder structure, you must add the following custom configurations in this file after the initial Cypress installation.</p><pre>&quot;pluginsFile&quot;: &quot;test/functional/framework/plugins/index.js&quot;,     &quot;supportFile&quot;: &quot;test/functional/framework/support/index.js&quot;,     &quot;fixturesFolder&quot;: &quot;test/functional/fixtures&quot;,     &quot;integrationFolder&quot;: &quot;test/functional/tests&quot;,     <br>&quot;testFiles&quot;: &quot;**/*.spec.js&quot;,     <br>&quot;videosFolder&quot;: &quot;test/functional/results/videos&quot;,     &quot;screenshotsFolder&quot;: &quot;test/functional/results/screenshots&quot;,     &quot;video&quot;: false,</pre><h4>Opening Cypress</h4><pre>$ yarn run cypress open</pre><p>or by using npx</p><pre>$ npx cypress open</pre><p>After a moment, the Cypress Test Runner will launch.</p><h4>Sample Project Structure</h4><figure><img alt="Sample folder structure" src="https://cdn-images-1.medium.com/max/436/1*vkaY0-KlMJMfK__t85eHTg.jpeg" /></figure><ul><li><strong>config</strong></li></ul><p>Domain-specific test configurations should go here</p><ul><li><strong>fixtures</strong></li></ul><p>Test data must go here. Usually, these test data can be stored in JSON format</p><ul><li><strong>framework</strong></li></ul><p>Cypress framework related folders should go here. You should implement any <a href="https://docs.cypress.io/api/cypress-api/custom-commands.html#Syntax">custom Cypress commands</a> here in this file. These must not be domain-specific commands though, which should go somewhere under the <strong>lib</strong> folder instead.</p><ul><li><strong>lib</strong></li></ul><p>Test functions, helpers, and utils should go here</p><ul><li><strong>results</strong></li></ul><p>Test reports, screenshots, videos, etc. are generated inside</p><ul><li><strong>tests</strong></li></ul><p>Test files must go here</p><p><strong>lib/functions</strong></p><p>You can place your application-specific function methods here. These methods will be used to script your functional tests. Hence, it is important to name these methods in a more self-descriptive, readable, and meaningful manner.</p><p>Typically, a function method is something that performs some user action in the page (ie: interacts with the application) and potentially results in a state change. You may create subfolders to modularize these functions as needed.</p><p><strong>lib/helpers</strong></p><p>You can place your application domain-specific helper methods here. These could be some domain-specific under-the-hood tasks such as local storage manipulation, session cookie generation, etc., or a combination of function methods that results in a comprehensive user scenario that you can use as a pre-step in your test to avoid code duplication.</p><p><strong>lib/utils</strong></p><p>You can place any common utility methods, which non-domain-specific, in this folder.</p><h4>Sample test file</h4><pre>// cypress/integration/login.spec.js<br>describe(&#39;login&#39;, () =&gt; {<br>  beforeEach(() =&gt; {<br>    visitLoginPage()<br>  })<br><br>  it(&#39;A User logs in and sees a welcome message&#39;, () =&gt; {<br>    loginWith(&#39;michael@example.com&#39;, &#39;passsword&#39;)<br><br>    expect(cy.contains(&#39;Welcome back Michael&#39;)).to.exist<br>  })<br><br>  it(&#39;A User logs off and sees a goodbye message&#39;, () =&gt; {<br>    loginWith(&#39;michael@example.com&#39;, &#39;password&#39;)<br>    logout()<br><br>    expect(cy.contains(&#39;Goodbye! See you soon!&#39;))<br>  })<br>})<br><br>const visitLoginPage = () =&gt; {<br>  cy.visit(&#39;http://localhost:3000&#39;)<br>}<br><br>const loginWith = (email, password) =&gt; {<br>  cy.get(&#39;[name=&quot;email&quot;]&#39;).type(email)<br>  cy.get(&#39;[name=&quot;password&quot;]&#39;).type(password)<br>  cy.get(&#39;button&#39;).click()<br>}<br>const logout = () =&gt; {<br>  cy.get(&#39;button&#39;).click()<br>}</pre><p>The above tests if a user can successfully log in and off our platform. On successful login, they see a message that says: “Welcome back {firstName}”. On a successful log off they see a goodbye text.</p><p>All of Cypress’s functionality is available under the global cy object you can see above. There is no need for any async/await or any asynchronous non-sense. Tests execute one step at a time, waiting for the previous step to complete.</p><p>There are functions to retrieve DOM elements like get() as well as to find text on the page using contains().</p><h4>Test Execution</h4><ol><li><strong>GUI Mode</strong></li></ol><p>If you used npm to install, Cypress has now been installed to your ./node_modules folder, with its binary executable accessible from ./node_modules/.bin.</p><p>Now you can open Cypress from your <strong>project root</strong> where <strong>node_modules</strong> folder exists, in one of the following ways:</p><p><em>The long way with the full path:</em></p><p>$ ./node_modules/.bin/cypress open</p><p>Or<em> </em>with the shortcut using<strong> </strong>npm bin</p><p>$ $(npm bin)/cypress open</p><p>After a moment, the <strong>Cypress app </strong>will launch.</p><figure><img alt="Test execution window" src="https://cdn-images-1.medium.com/max/920/1*ZcEfCBvjVuOo43rVfWxw3A.png" /></figure><p>Note that Cypress will display your .spec.js test files in the <strong>Cypress app </strong>window</p><p>Clicking on a .spec.js the test file will launch the <a href="https://docs.cypress.io/guides/core-concepts/test-runner.html"><strong>Cypress Test Runner</strong></a> in the selected browser which is installed on your computer and Cypress will start executing the tests in your .spec.js test file.</p><p>2. <strong>Non-GUI Mode</strong></p><p>You can run your Cypress tests in non-GUI mode using a terminal window with the below commands.</p><ul><li>Run all tests in the Chrome browser in headless mode:</li></ul><p>$ ./node_modules/.bin/cypress run --browser chrome --headless</p><ul><li>Run a specific test file:</li></ul><pre>$ ./node_modules/.bin/cypress run — spec “test/functional/test_file_name.spec.js”</pre><ul><li>To enable video recording feature:</li></ul><p>$ ./node_modules/.bin/cypress run --config video=true</p><figure><img alt="Test report with the execution using the tesrminal" src="https://cdn-images-1.medium.com/max/1024/1*XV3BKfjJBGXQKKR-E-71oA.png" /></figure><h4>Errors</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*RjNZeVc02GLJe-KJhxKAxA.png" /><figcaption>Error Window</figcaption></figure><ol><li>Error name: This is the type of error (e.g. AssertionError, CypressError)</li><li>Error message: This generally tells you what went wrong. It can vary in length.</li><li>Learn more: Some error messages contain a Learn more link that will take you to relevant Cypress documentation.</li><li>Code frame file: This is usually the top line of the stack trace and it shows the file, line number, and column number that is highlighted in the code frame below. Clicking on this link will open the file in your <a href="https://on.cypress.io/IDE-integration#File-Opener-Preference">preferred file opener</a> and highlight the line and column in editors that support it.</li><li>Code frame: This shows a snippet of code where the failure occurred, with the relevant line and column highlighted.</li><li>View stack trace: Clicking this toggles the visibility of the stack trace.</li><li>Print to console button: Click this to print the full error to your DevTools console. This will usually allow you to click on lines in the stack trace and open files in your DevTools.</li></ol><h4>Debugging</h4><p>Your Cypress test code runs in the same run loop as your application. This means you have access to the code running on the page, as well as the things the browser makes available to you, like document, window, and debugger.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*HGOIai0rNpAk7GBiYdBLsQ.png" /><figcaption>Debug using debugger</figcaption></figure><p>Is it not working?? This may not work exactly as you are expecting as both <a href="https://docs.cypress.io/api/commands/visit.html">cy.visit()</a> and <a href="https://docs.cypress.io/api/commands/get.html">cy.get()</a> will return immediately, having enqueued their work to be done later, and debugger will be executed before any of the commands have actually run.</p><p>Use <a href="https://docs.cypress.io/api/commands/then.html">.then()</a> to tap into the Cypress command during execution and add a debugger at the appropriate time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*wpPxDHEwqHcnVU1K123q_w.png" /><figcaption>Debug using then()</figcaption></figure><p><strong>Using </strong><a href="https://docs.cypress.io/api/commands/debug.html"><strong>.debug()</strong></a></p><p>Cypress also exposes a shortcut for debugging commands, <a href="https://docs.cypress.io/api/commands/debug.html">.debug()</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*OsaaAArsHCPgxm134OOivQ.png" /><figcaption>Debug using debug()</figcaption></figure><h4>Frequently Installed Together</h4><p>Below are some useful npm packages that most users install together with Cypress:</p><ul><li><a href="https://www.npmjs.com/package/cypress-wait-until">cypress-wait-until</a> — allows you to repeatedly check for something to happen</li><li><a href="https://www.npmjs.com/package/cypress-xpath">cypress-xpath</a> — adds xpath command to Cypress test runner</li><li><a href="https://www.npmjs.com/package/cypress-plugin-retries">cypress-plugin-retries</a> — provides the ability to retry failed tests</li><li><a href="https://www.npmjs.com/package/mochawesome">mochawesome</a> — custom reporter for Cypress</li></ul><p>Cypress is equipped with so many <a href="https://docs.cypress.io/guides/guides/command-line.html#Installation">Command Line</a> options and features that you can explore by visiting the documentation put together by the Cypress team.</p><p>Hope you learn something useful. See you in the next one ✌️</p><h4>References</h4><ul><li><a href="https://example.cypress.io/">https://example.cypress.io/</a></li><li><a href="https://example.cypress.io/commands/actions">https://example.cypress.io/commands/actions</a></li><li><a href="https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/">https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/</a></li><li><a href="https://www.cypress.io/blog/2020/03/03/testing-edge-data-cases-with-network-stubbing-and-app-actions/">https://www.cypress.io/blog/2020/03/03/testing-edge-data-cases-with-network-stubbing-and-app-actions/</a></li><li><a href="https://docs.cypress.io/guides/references/best-practices.html">https://docs.cypress.io/guides/references/best-practices.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=addf46eac11a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding SonarQube coding rules using Java for the Test Automation Suite]]></title>
            <link>https://dilumpathi.medium.com/adding-sonarqube-coding-rules-using-java-for-the-test-automation-suite-c4083405278a?source=rss-1e78fd7783e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4083405278a</guid>
            <dc:creator><![CDATA[Dilum Pathiraja]]></dc:creator>
            <pubDate>Mon, 14 Dec 2020 12:45:59 GMT</pubDate>
            <atom:updated>2020-12-14T12:45:59.492Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*4O2TF4ElB_7NP05beQ0BQQ.png" /></figure><p>As QE’s or Developers we have to deal with hundreds or even thousands of code lines every day in our life. As humans dealing with such amount of code lines, there may be code smells, bugs, security vulnerabilities in our scripts. SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect such things in our scripts.</p><p>There are predefined set of rules in SonarQube and you can add them as an extension to your IDE or as a server to your pipeline. Apart from these predefined rules, anyone can write their own set of project-related rules and add that as a plugin to your project.</p><h3>Adding Custom Coding Rules</h3><p>There are two ways to extend coding rules:</p><ul><li>Writing coding rules using Java via a SonarQube plugin</li><li>Adding XPath rules directly through the SonarQube web interface</li></ul><p>But the preferred way is to write the rules via a SonarQube plugin. Here I’m only going to talk about writing the rules via a SonarQube plugin method using Java.</p><h3>Adding coding rules using Java</h3><p>Writing coding rules in Java is a six-step process:</p><ul><li>Create a SonarQube plugin.</li><li>Put a dependency on the API of the language plugin for which you are writing coding rules.</li><li>Create as many custom rules as required.</li><li>Generate the SonarQube plugin (jar file).</li><li>Place this jar file in the SONARQUBE_HOME/extensions/plugins directory.</li><li>Restart the SonarQube server.</li></ul><h3>Writing coding rules using Java</h3><p>Here I’m going to talk about how we can write the custom rules. For that, I’m using the Java example given by the <a href="https://github.com/SonarSource/sonar-custom-rules-examples">https://github.com/SonarSource/sonar-custom-rules-examples</a>.</p><p>In the sample project, the <strong>Checks </strong>package in the main is used to write the custom rules. In parallel to that, we can see four classes that help to execute the rules.</p><pre>RulesList</pre><p>As I mentioned above the custom rules that we created on the Checks package are added to a list and return that list in this class. So every time when you adding a new rule class to the Check package you have to add that class name to the list in this class.</p><pre>MyJavaRulesPlugin</pre><p>This class defines the server extensions and the batch extensions. In server extensions(MyJavaRulesDefinition), objects are instantiated during server startup, and in the batch extensions(MyJavaFileCheckRegistrar), objects are instantiated during code analysis.</p><pre>MyJavaRulesDefinition</pre><p>This is the server extension class that I explained above. In this class, we can define the attributes like REPOSITORY_KEY, REPOSITORY_NAME, RESOURCE_BASE_PATH &amp;, etc. Also, this will check whether the Rule annotation is there in custom rule classes as it’s a must.</p><h4>Writing a custom rule to verify whether the test method name is starting with the “test” or “verify” prefix.</h4><p>As an example, let&#39;s take we want a rule to verify that our test method names to start with “test” or “verify” prefix. So we can write a rule like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*E7aTRg3a6tZGkW5C9ENe2w.png" /><figcaption>Sample Custom Rule</figcaption></figure><p>Here you can see the <strong>Rule </strong>annotation is added before the class declaration as it is mandatory. I have overridden the <strong>visitMethod</strong> in the <strong>MethodTree</strong> class. If you need to check something in the class you can use the ClassTree, if you need to check something in the imports you can use the ImportTree. Likewise, there are so many classes that we can use to write the custom rules.</p><p>In the above example first I have added all the annotations in the test method to a list and check whether there is an annotation called “Test” to identify whether that method is a test method or not. If it&#39;s a test method I have checked that the “test” or “verify” prefix is available in the method name. If not I have thrown an error message.</p><p>After writing the rule we have to add the metadata files to the resource folder of the main package which consists of an HTML file and a JSON file. The file name should start with the key value that is given to Rule annotation in Rule class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*izTe7IJfVDe21zGmAFyG7Q.png" /><figcaption>TestMethodNames_java.html</figcaption></figure><p>In the HTML file, you have to mention the error message which is similar to the given in the Rule class, the noncompliant solution, and the compliant solution. When there is a sonar violation you can see these values in the sonar report.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*O0X9PtZCTwKLkU9eE3kvhw.png" /><figcaption>TestMethodNames_java.json</figcaption></figure><p>In the JSON file, you have to mention the issue title, type of the issue, issue status, time taken to fix the issue, and the severity of the issue. These values also can be found in the sonar report of a sonar violation.</p><p>Now the rule writing part is done. Now we have to test whether our rule is working fine or not. For that we can write a sample test as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*EGqW_Vu9UfVNru267I2kwQ.png" /><figcaption>TestClass</figcaption></figure><p>In this test I have verify the content in the TestMethodsNamesCheck.java file against our rule. So lets see what’s in the TestMethodsNamesCheck.java file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*Vh55Z4L7D7vrwvtfduUxiA.png" /><figcaption>TestMethodsNamesCheck.java</figcaption></figure><p>In TestMethodsNamesCheck.java file which is in the Test resources folder, I have included some sample Non-compliant and Compliant solutions. You can include any number of Noncompliant and Compliant solutions here. For the NonCompliant ones you have to mention it as a NonCompliant and the error message, which you gave in the Rule class. If the two messages are different the test will fail. Also there should be at least one Compliant and one Noncompliant solution in your test. For the Compliant ones you have to mention it as a Compliant and no message needed. Now you can execute the test and see whether your rule working fine.</p><p>The sample project can be found in <a href="https://github.com/SonarSource/sonar-custom-rules-examples">here</a> and more info <a href="https://docs.sonarqube.org/latest/extend/adding-coding-rules/">here</a>.</p><p>Hope you learn something useful. See you in the next one ✌️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4083405278a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>