<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Darmawan Hadiprasetyo on Medium]]></title>
        <description><![CDATA[Stories by Darmawan Hadiprasetyo on Medium]]></description>
        <link>https://medium.com/@dhadiprasetyo?source=rss-8a6bdcde59a5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2CWNOiSHmyJwE9um-E6EUw.jpeg</url>
            <title>Stories by Darmawan Hadiprasetyo on Medium</title>
            <link>https://medium.com/@dhadiprasetyo?source=rss-8a6bdcde59a5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 03:57:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dhadiprasetyo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[When employees leave, their AI tokens stay behind: closing the lag and risk of manual AI gateway…]]></title>
            <link>https://medium.com/@dhadiprasetyo/when-employees-leave-their-ai-tokens-stay-behind-closing-the-lag-and-risk-of-manual-ai-gateway-d7bbd4d65179?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7bbd4d65179</guid>
            <category><![CDATA[ai-platform]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[ai-infrastructure]]></category>
            <category><![CDATA[ai-gateway]]></category>
            <category><![CDATA[active-directory]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 00:00:55 GMT</pubDate>
            <atom:updated>2026-10-29T15:47:29.675Z</atom:updated>
            <content:encoded><![CDATA[<h3>When employees leave, their AI tokens stay behind: closing the lag and risk of manual AI gateway access</h3><blockquote>Why AI access does not end at Single Sign-On, the hidden risks and delays of manual provisioning, and how the litellm-directory-sync service bridges Active Directory and Okta to LiteLLM.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Xrq4VU_klFPigaD-TwtIg.png" /></figure><h3>1. The Friday afternoon incident</h3><p>At 16:45 on a Friday, human resources closes an offboarding ticket for a departing machine learning engineer. IT disables the engineer’s Active Directory account and revokes their Okta profile, cutting access to Slack, email, GitHub, and cloud consoles. From an identity standpoint, offboarding is complete.</p><p>Across the building, the platform team monitors model inference through the LiteLLM gateway, assuming corporate Single Sign-On protects gateway resources. Three hours later, an evaluation script on the former employee’s home server wakes up. It holds a bearer key issued by LiteLLM two months earlier, which the proxy accepts without challenge. Over the weekend, the script repeatedly queries Claude 3.5 Sonnet and runs up $800 a day in API charges.</p><p>The leak happens because an AI gateway handles inference traffic, not user identity. LiteLLM proxies model providers, balances upstream requests, caches responses, and records token counts. Directory state lives elsewhere. When an employee leaves or changes teams, Single Sign-On blocks their dashboard login, while their existing API keys remain valid at the proxy. Without an automated bridge between the directory and the gateway, those credentials keep running indefinitely.</p><h3>2. The operational risks and synchronization lag of manual access</h3><p>In many organizations, AI gateway access relies on manual processes. An engineer files a Jira or ServiceNow ticket, a platform admin logs into the LiteLLM dashboard, creates an API key by hand, sets an arbitrary budget limit, and pastes the secret into a private chat.</p><p>While manual workflows are often a natural starting point, they introduce operational risk and synchronization lag across four key areas:</p><p><strong>Offboarding lag and zombie credentials:</strong> Human resources notifies IT of employee departures, but AI gateways rarely appear on standard IT offboarding checklists. Even when a ticket reaches the platform team, it often sits in a backlog over the weekend. Meanwhile, unrevoked bearer tokens ( sk-...) stored in local terminal configurations, test harnesses, or personal devices continue to authenticate successfully against the gateway.</p><p><strong>Permission creep during internal transfers:</strong> When an engineer moves from research to customer operations, their account remains active. Because removing access requires someone to remember to file a downgrade ticket, that ticket rarely gets created. Over time, employees accumulate privileges for expensive frontier models and generous budgets that no longer match their job responsibilities.</p><p><strong>Operational bottleneck and ticket fatigue:</strong> Platform engineers end up acting as manual access brokers. Every new project, intern onboarding, and quota bump requires an administrative touch in the gateway dashboard. This administrative overhead pulls engineers away from building core platform capabilities and creates multi-day delays for developers waiting on credentials.</p><p><strong>Broken cost attribution and audit blindness:</strong> Under manual provisioning, engineers often share tokens across teams to bypass ticketing delays or use generic shared accounts. When the monthly invoice arrives, finance cannot attribute spending to specific departments or projects. In a security audit, the platform team cannot produce an authoritative record of who held access to specific models on a given date.</p><h3>3. The three things to sync from the directory to the gateway</h3><p>Closing these gaps requires synchronizing three specific dimensions of identity directly into the gateway:</p><ol><li><strong>User lifecycle and credential validity:</strong> The service must provision accounts the moment an employee joins an approved directory group and revoke keys immediately when their account is disabled. Tying accounts to immutable employee IDs rather than email addresses prevents impersonation and orphan accounts. When a departure occurs, the gateway must invalidate matching token caches so that all programmatic access terminates in minutes.</li><li><strong>Team membership and model access tiers:</strong> Departmental groups in Active Directory or Okta should dictate which upstream models an employee can query. Machine learning researchers may need unconstrained access to high-cost reasoning models such as gpt-4o and claude-3-5-sonnet, while general productivity users should route to lighter, cost-effective models like claude-3-5-haiku and gpt-4o-mini. When an employee changes departments in the directory, their model permissions must adjust automatically.</li><li><strong>Budgets, spend caps, and token quotas:</strong> Gateway spend limits must reflect organizational boundaries rather than static, one-time assignments. Synchronizing departmental budget policies ensures that users operate within defined monthly spending caps, such as $200 for core engineering and $50 for operations. Dynamically adjusting these caps during role transfers prevents runaway spending before unexpected costs appear on the provider bill.</li></ol><h3>4. The two clocks: why Single Sign-On and JIT leave programmatic gaps</h3><p>Identity providers and AI gateways manage access on fundamentally different timelines.</p><p>Interactive access runs through the identity provider using SAML assertions, OIDC tokens, browser sessions, and multi-factor prompts. These sessions expire every 8 to 12 hours, and an administrator can revoke them with a single click.</p><p>Programmatic access runs through the gateway. Developers generate long-lived bearer tokens ( sk-...) for terminal workflows, automated test suites, batch pipelines, and code editor extensions. These API keys often lack expiration dates and bypass the identity provider entirely, sending requests straight to the gateway reverse proxy.</p><p>Organizations often expect Just-In-Time (JIT) provisioning through Single Sign-On to handle access governance. JIT provisioning works well when a user opens the LiteLLM dashboard in a browser, reading incoming SAML attributes to create the user account on demand. But JIT handles only interactive logins. Departing employees never log in through the dashboard again, so JIT hooks never fire. As a result, existing API keys stay active, team budget allocations remain open, and old group permissions persist until someone intervenes by hand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/0*NRCEjiMzGYF1iSrQ.png" /></figure><h3>5. Architecture of an identity-to-gateway bridge</h3><p>Closing the gap requires a background synchronization service that connects enterprise directories directly to LiteLLM.</p><pre>[HR System] -&gt; [Active Directory / Okta]<br>                      │ (Read-Only LDAPS / System Log)<br>                      ▼<br>          [Directory Sync Service]<br>          ├── Change Ingestion (USN / Event Hooks)<br>          ├── Mapping Engine (Group -&gt; Team/Budget/Models)<br>          └── Transactional Outbox (Durable SQLite/PG)<br>                      │ (HTTP API / Outbox Worker)<br>                      ▼<br>           [LiteLLM AI Gateway] -&gt; Upstream LLMs (OpenAI, Anthropic, Bedrock)</pre><p>The bridge relies on three design patterns.</p><h4>Read-and-write asymmetry</h4><p>The synchronization service maintains strict privilege separation between identity providers and the gateway. It acts as a read-only client for Active Directory, connecting over LDAPS on port 636 with an unprivileged service account. For Okta, it requests only read scopes: okta.users.read, okta.groups.read, and okta.logs.read. The service holds no write permissions in either directory and cannot modify users, group memberships, or passwords.</p><p>All write operations target the LiteLLM gateway management API using a gateway master key. This separation restricts operational blast radius: a bug in the synchronization code cannot alter corporate directory records.</p><h4>Event capture without dual writes</h4><p>The service detects directory updates without requiring manual intervention from identity teams.</p><p>In Active Directory, it polls using the uSNChanged attribute. Update Sequence Numbers track object revisions sequentially on each domain controller. This avoids DirSync controls, which require directory replication rights ( DS-Replication-Read) that service accounts should never hold.</p><p>In Okta, the service pairs real-time Event Hooks with continuous polling. An HTTP listener accepts incoming webhooks for prompt updates, while a background worker polls the Okta System Log API using a high-water-mark cursor. If a network blip or firewall rule drops a webhook, the log poller catches the change on its next pass.</p><h4>Transactional outbox</h4><p>Making synchronous API calls directly from an ingestion worker to LiteLLM creates availability problems whenever the gateway restarts or hits rate limits.</p><p>Instead, the service stores changes in a PostgreSQL outbox table. When the ingestion engine detects an update, it normalizes the payload into a domain event, then writes both the updated state and the outbox event in a single database transaction. Both write together or roll back together.</p><p>A separate worker process reads pending records from the outbox table using FOR UPDATE SKIP LOCKED. It sends the updates to LiteLLM with exponential backoff, holding events in the queue for retry if the gateway returns HTTP 429 or 500.</p><h3>6. Automating join, move, and leave</h3><p>Lifecycle management centers on three state transitions: onboarding, internal transfers, and offboarding.</p><h4>Join: automated pre-provisioning</h4><p>When human resources adds a new hire to directory groups such as AI-Platform-Users or Engineering-Core, the sync service configures their gateway account before their start date.</p><p>The service calls LiteLLM’s /user/new endpoint, assigns the employee ID as the stable user identifier, links the user to their team, and sets the assigned budget policy. A new core engineer might receive a $200 rolling monthly budget with access restricted to gpt-4o, claude-3-5-sonnet, and text-embedding-3-small. The engineer can generate API keys through developer tooling on their first day without waiting for help desk tickets.</p><h4>Move: team transitions and privilege downgrades</h4><p>Unmanaged gateways accumulate stale permissions when employees change roles. If a machine learning researcher transfers to product operations, their account remains active, but their permissions must adjust.</p><p>The engine evaluates precedence across mapped groups to calculate the difference between directory memberships and existing gateway entitlements. It removes the user from the research team via /team/member_delete, assigns them to the operations team with /user/update, and switches the allowed model list from high-cost models to utility models like claude-3-5-haiku and gpt-4o-mini. The service lowers the monthly spend limit from $1,000 to $50 while preserving past usage history.</p><h4>Leave: immediate revocation mechanics</h4><p>When an employee departs, the service blocks keys immediately while deferring account deletion. Deleting a user immediately through /user/delete removes past cost attribution records, and proxy instances can continue serving requests if bearer tokens remain cached in Redis.</p><p>To shut off access safely, the worker runs a multi-step sequence:</p><ol><li>Remove the user from all teams with /team/member_delete.</li><li>Call /key/block on every active key belonging to the user.</li><li>Invalidate matching Redis token caches so that subsequent requests return HTTP 401 Unauthorized immediately.</li><li>Record an audit log entry with the revocation timestamp, actor, and target.</li><li>Schedule the account for deletion after a grace period, such as seven days, preserving billing history while blocking network traffic.</li></ol><pre>def reconcile_members(<br>    directory_members: set[str],<br>    gateway_members: set[str],<br>) -&gt; tuple[set[str], set[str]]:<br>    &quot;&quot;&quot;Compute member additions and removals using set difference.<br><br>    Args:<br>        directory_members: User IDs active in directory groups.<br>        gateway_members: User IDs active on the AI gateway.<br><br>    Returns:<br>        A tuple of (members_to_add, members_to_remove).<br>    &quot;&quot;&quot;<br>    members_to_add = directory_members - gateway_members<br>    members_to_remove = gateway_members - directory_members<br>    return members_to_add, members_to_remove</pre><p>Set difference determines which accounts need to be created or removed on each pass, adjusting gateway membership until it matches directory truth.</p><h3>7. Why event-driven sync still needs periodic reconciliation</h3><p>Webhooks provide fast updates, but they do not guarantee consistency on their own. Networks drop packets, receiving servers restart during deployments, and rotated webhook secrets temporarily break HMAC signature checks. An event triggers a check, but current state still lives in the directory.</p><p>The service runs a reconciliation sweep on a fixed schedule, such as every 60 minutes. The worker compares the full list of directory group members against active LiteLLM user accounts. Any discrepancy, whether an unrevoked key, a missing account, or an outdated team assignment, generates a corrective event in the transactional outbox.</p><h4>Safety circuits: the two-miss rule and blast-radius limits</h4><p>Automating reconciliation introduces the risk of accidental mass revocation. If an Active Directory query times out during a full sweep and returns zero records, a basic reconciliation loop would conclude that every employee was terminated and disable every active API key.</p><p>The engine uses two safeguards to prevent false purges.</p><p>The first safeguard is a two-miss rule. When an account exists in the gateway but disappears from directory search results, the service increments a miss counter in a tracking table. It flags the account as suspect rather than revoking access right away. The worker only issues revocation commands if the account remains missing across two consecutive reconciliation runs.</p><p>The second safeguard is a blast-radius cap. If a single reconciliation pass attempts to modify more than 10 percent of active accounts, the circuit breaker halts execution. The service logs an error, fires an alert, writes a dry-run report, and waits for administrator review before applying any changes.</p><h3>8. Solving the gap: building litellm-directory-sync with Claude</h3><p>To solve this lifecycle gap in production, I developed litellm-directory-sync, pairing with Claude as an architectural partner to design, build, and test an enterprise identity lifecycle service for LiteLLM.</p><p>Public repository: <a href="https://github.com/darmawanh/litellm-directory-sync">https://github.com/darmawanh/litellm-directory-sync</a></p><p>The project turns directory events into durable gateway actions through a clean, decoupled architecture.</p><h4>Key features</h4><p><strong>Read-only directory connectors:</strong> Connects to Active Directory over LDAPS on port 636 using sequence number polling ( uSNChanged), and to Okta through read-only OAuth scopes ( okta.users.read, okta.groups.read, okta.logs.read) combining real-time Event Hooks with System Log polling. The service holds zero write credentials in either directory.</p><p><strong>Dynamic group and role mapping:</strong> Maps directory groups directly to LiteLLM teams and roles ( proxy_admin, internal_user) with configurable precedence rules.</p><p><strong>Automated budget tiering:</strong> Binds departments or directory groups to rolling monthly spend limits and model allowlists, ensuring developers get the models they need while preventing runaway spending.</p><p><strong>Transactional outbox reliability:</strong> Stores every lifecycle event in a PostgreSQL outbox table within the same database transaction as the state change. A decoupled worker pulls events using FOR UPDATE SKIP LOCKED and calls LiteLLM endpoints with exponential backoff, surviving gateway restarts and network partitions.</p><p><strong>Safety-first reconciliation:</strong> Runs scheduled sweeps with a two-miss rule to protect against accidental mass revocation during directory timeouts, dry-run validation, and allowlists for non-human service accounts.</p><h4>How it is configured</h4><p>All settings are declared in config/config.yaml, using environment variable references ( os.environ/VAR) for credentials and host endpoints so that secrets never enter source control:</p><ul><li>identity_sources: Configures connections for Active Directory (server, search base, group DN, bind credentials, and USN polling intervals) and Okta (org URL, OAuth private key, group ID, and System Log cursor settings).</li><li>group_mappings: Maps directory groups to LiteLLM teams and roles with explicit precedence levels.</li><li>budget_policies and budget_mappings: Declares monthly spend caps and durations (such as $100 for engineering and $50 for standard users) and assigns them by group or department attribute.</li><li>gateway: Specifies the LiteLLM base URL and administrative master key.</li><li>reconciliation: Configures the cron schedule, dry-run flag, grace period, and two-miss rule state table.</li></ul><p>With this configuration in place, the service runs as a lightweight container alongside LiteLLM, continuously translating identity and organizational changes into real-time gateway access rules.</p><h3>9. Platform engineer access checklist</h3><p>Before putting an AI gateway into production, verify access governance against five baseline requirements:</p><ol><li><strong>Cap token lifespans:</strong> Disable non-expiring API keys and require developer credentials to rotate within 90 days.</li><li><strong>Automate offboarding:</strong> Ensure employee terminations revoke gateway bearer keys within 15 minutes.</li><li><strong>Restrict directory permissions:</strong> Keep directory connectors strictly read-only, limiting write credentials to gateway APIs.</li><li><strong>Buffer gateway mutations:</strong> Use a transactional outbox table so that gateway restarts and rate limits do not drop lifecycle events.</li><li><strong>Enforce safety circuits:</strong> Add consecutive-miss requirements and blast-radius caps to prevent network errors from triggering bulk key revocations.</li></ol><p>A gateway handles model routing, load balancing, and token accounting. Protecting corporate data and model budgets requires an automated control plane that tracks identity changes in the directory and updates gateway credentials the moment they occur.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7bbd4d65179" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Practical Guide to Load and Stress Testing: Fundamentals and k6 in Practice]]></title>
            <link>https://medium.com/@dhadiprasetyo/a-practical-guide-to-load-and-stress-testing-fundamentals-and-k6-in-practice-73e2eb85db7b?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/73e2eb85db7b</guid>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[load-testing]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[stress-testing]]></category>
            <category><![CDATA[k6]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sun, 11 Oct 2026 16:18:00 GMT</pubDate>
            <atom:updated>2026-10-11T16:18:00.954Z</atom:updated>
            <content:encoded><![CDATA[<p>Greetings, my name is Darmawan, and I am a Senior Software Development Engineer in Test (SDET). I’ve spent my career in testing, building test frameworks and test architecture from scratch. More recently that work has centered on <strong>AI test infrastructure</strong> — the platform layer AI-assisted testing runs on — and on putting AI-assisted testing into practice. One question comes up again and again, deceptively simple:</p><p><em>“Can you make sure this survives?”</em></p><p>That question is the start of every load test I run. And the answer is never a tool. It’s a number.</p><p>In this article, we’ll walk through how I do performance testing end to end: the core vocabulary every tester needs first, how I gather the requirement, and how I run load and stress tests in <strong>k6</strong>. The tool is the easy part — the concepts are the subject, and they stay the same whichever tool you pick. Let’s start with the ticket that lands on my desk most often.</p><h3>The Ticket That Starts It All</h3><p>A product manager drops me a message:</p><blockquote><em>“Black Friday is in three weeks, and checkout needs to survive it. Can you test it?”</em></blockquote><p>I’ve done this enough times to know the first thing I need isn’t a tool. It’s a number.</p><p>A flash sale isn’t steady traffic. It’s a <strong>spike</strong> — a wall of people arriving in the same minute, hammering the same path: browse, add to cart, check out, pay. “It works in staging” tells me nothing about that minute. So before I open a load-testing tool, before I write a single line of script, I go and find out what “survive” actually means in numbers.</p><h3>The Vocabulary You Need First</h3><p>Before we touch a tool, let me define the terms I’ll use for the rest of this article. These concepts are the same whether you use k6, JMeter, Gatling, or Locust — the tool changes, the vocabulary doesn’t. If you remember nothing else, remember these.</p><p><strong>Virtual User (VU).</strong> A VU is one simulated user. It’s a lightweight worker inside your load tool that behaves like a real person: it sends a request, waits for the reply, thinks for a moment, then sends the next one. When someone says “run 100 VUs,” they mean 100 simulated users hitting the system at the same time.</p><p><strong>Iteration.</strong> One full pass through what a VU does — one loop of the script. If a VU logs in, adds to cart, and checks out, that whole sequence is one iteration. A test’s total work is roughly <em>VUs × iterations</em>.</p><p><strong>Ramp-up.</strong> Instead of slamming the system with all VUs at once, I bring them up gradually. Ramp-up is the time it takes to go from zero (or a low number) to the target number of users. Ramping up mimics how real traffic arrives — people trickle in, they don’t all appear at once.</p><p><strong>Throughput.</strong> How much work the system handled per unit of time, usually measured in requests per second (req/s or RPS). This is the “how much did it do” number.</p><p><strong>Latency (response time).</strong> How long a single request took, from sending it to getting the full reply. Reported in milliseconds, and always as a distribution — p50, p95, p99 — never as a plain average (we’ll see why in a moment).</p><p><strong>Think time.</strong> The pause between one user action and the next — the seconds a real person spends reading the page or deciding what to click. Without think time, your VUs hammer the server like robots, and you test a scenario no real user creates.</p><p><strong>Concurrency.</strong> How many requests are in flight at the exact same moment. This is not the same as throughput: throughput is per second, concurrency is instantaneous. Ten users each waiting on a slow 2-second request is 10 concurrent requests, even if the throughput is only 5 req/s.</p><p><strong>Error rate.</strong> The fraction of requests that failed — timeouts, 5xx responses, connection drops. A test that hits its latency target but fails 20% of requests has not passed.</p><p>And the four <strong>types of test</strong> I’ll refer to by name, since they answer different questions:</p><ul><li><strong>Smoke test</strong> — one or two users, a minute. The question: <em>does it work at all?</em> I run this before anything else.</li><li><strong>Load test</strong> — expected production traffic, held steady. The question: <em>does it meet the target?</em></li><li><strong>Stress test</strong> — traffic ramped past capacity until it breaks. The question: <em>where’s the limit?</em></li><li><strong>Spike test</strong> — a sudden huge jump in traffic, then a drop. The question: <em>does it survive a surge?</em></li></ul><p>Those eight vocabulary terms and four test types are the foundation. Everything else in this article builds on them.</p><h3>What Performance Testing Actually Is</h3><p>Performance testing is the umbrella discipline: <strong>proving a system meets its performance requirements, and finding the limits where it stops meeting them.</strong> Under that umbrella sit many kinds of test — smoke, load, stress, spike, soak, volume, scalability, capacity, breakpoint, regression, and more. Each answers a different question, and a serious testing effort uses several of them.</p><p><strong>In this article, I’m focusing on two of them: load testing and stress testing.</strong> They’re the two I reach for most, and they cover the core questions — <em>does it meet the target?</em> and <em>where does it break?</em> Once you understand these two, the others become variations on the same theme.</p><p>I think about these two in terms of two halves:</p><ul><li><strong>Verification</strong> — does the system meet the target we agreed on? <em>(load testing)</em></li><li><strong>Exploration</strong> — where does it stop meeting it, and why? <em>(stress testing)</em></li></ul><p>Neither half is the whole of performance testing — but together they answer the question the ticket started with: <em>will this survive?</em></p><h3>The Four Questions Every Test Answers</h3><p>Whatever the type, a performance test answers exactly four questions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/0*zYsaSLFK44BP0UZl.png" /></figure><p>If a test doesn’t produce those four numbers, it isn’t finished.</p><h3>Why the Average Lies</h3><p>This is the single most important idea in performance testing, and it’s why I never report an average.</p><pre>Server A: everyone waits 100ms       average = 100ms <br>Server B: 99% wait 10ms, 1% wait 9s  average = 100ms</pre><p>Same average. Completely different systems. Server B is on fire — one in a hundred users is waiting nine seconds, and the average hides every one of them.</p><p>It’s like describing a city’s weather by its yearly average. Comfortable on paper — and it tells you nothing about the day the river flooded.</p><p>That’s what percentiles are for:</p><ul><li><strong>p50</strong> (median) — half the requests were faster than this.</li><li><strong>p95</strong>–95% were faster; the worst 5% are above it.</li><li><strong>p99</strong>–99% were faster; the worst 1% are above it.</li></ul><p><strong>My rule: p95 and p99 are what I report and what I alert on. The average is for dashboards nobody acts on.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/0*uIEUgv2hImXZ-O80.png" /><figcaption><em>What to look at: the gap between p95 and p99. A small slice of requests is far slower than the rest — and a plain average would have hidden every one of them.</em></figcaption></figure><h3>Open vs Closed Models</h3><p>This one is subtle, and it decides whether your test can find a breaking point at all. It comes down to one question: <strong>when the server slows down, does your test slow down with it — or does it keep pushing?</strong></p><ul><li><strong>Closed model:</strong> you set the number of <strong>users</strong>. Each user sends a request, waits for the reply, then sends the next. The request <em>rate</em> depends on how fast the server answers.</li><li><strong>Open model:</strong> you set the <strong>requests per second</strong>. The tool offers that rate regardless of what the server does.</li></ul><p>Let’s make that concrete with a small example. Say I want to load a checkout API that normally takes 200ms, and I want 50 requests per second. On a healthy server, 50 users each looping <em>send → wait 200ms → send</em> will naturally produce about 50 requests per second. So far, both models look identical.</p><p>Now the server slows down. Each checkout now takes <strong>2 seconds</strong> instead of 200ms. Watch what each model does:</p><p><strong>Closed model — 50 users, each waiting for the reply:</strong></p><pre>Each user: send → wait 2s → send → wait 2s → ... <br>50 users × (1 request every 2s) = 25 req/s</pre><p>The rate <strong>halved</strong>, from 50 req/s to 25 req/s — without anyone asking it to. The users are still there, but they’re stuck waiting, so they send fewer requests. My test just quietly reduced the load, right when the system was struggling. If the server got even slower, the load would drop even further. <strong>The test protects the server from the very overload I’m trying to create.</strong></p><p><strong>Open model — 50 requests per second, offered no matter what:</strong></p><pre>Every second: the tool fires 50 new requests, finished or not <br>Server handles ~25/s → the other 25 pile up in a queue</pre><p>The tool keeps offering 50 req/s. The server can only handle 25, so <strong>requests queue up</strong>. The queue grows, waiting time grows with it, and eventually requests time out or the system falls over. <strong>That’s the failure I’m trying to find</strong> — and only the open model exposes it.</p><p>A closed model reduces load exactly when you most want to see the pressure build. It’s a thermometer that cools down when you put it in hot water.</p><p>So: <strong>to find where a system breaks, use an open model.</strong> Remember this — it comes back when we pick an executor in k6.</p><h3>Little’s Law and the Knee</h3><p>There’s one equation behind capacity planning, and it’s worth memorising:</p><pre>concurrency = arrival rate × latency</pre><p>Concurrency is how many requests are in flight at once. It tells me how many requests must be in flight to sustain a rate — and therefore how much capacity I need. If I want 50 req/s and each request takes 0.2s, then 50 × 0.2 = 10 requests are in flight at any moment. I need to be able to hold 10.</p><p>Then there’s the <strong>knee</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*elOupMfWmyAyTehc.png" /><figcaption><em>What to look at: where the line bends. Left of the dashed mark, latency is flat — more load, same speed. Right of it, latency climbs steeply. That bend is the knee.</em></figcaption></figure><p>Think of a car engine. Up to a point, pressing the pedal gives you more speed. Then you hit the limit — the pedal does nothing more, and the engine just strains. Below the knee, latency is flat: the system absorbs more traffic without getting slower. At the knee it starts to climb. Past it, it explodes.</p><p><strong>Finding that bend is the entire point of a stress test</strong> — which is why I ramp the load rather than hold it constant. A constant load can only tell me whether one point is fine. A ramp finds the edge.</p><h3>Gathering the Requirement</h3><p><strong>I cannot test performance without a number.</strong> “It should be fast” is not testable. “p95 &lt; 300ms at 20 req/s” is.</p><p>Where the number comes from, and what I ask:</p><ul><li><strong>Product/business:</strong> “How many users at peak? Over how long?”</li><li><strong>The mix:</strong> “What will most people be doing?”</li><li><strong>Existing data:</strong> analytics or logs from a previous sale, if they exist.</li><li><strong>If nobody knows:</strong> I propose a number, get it agreed, and write it down.</li></ul><p>The requirement flows all the way through to production:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*paSxi-PBswlhMjoI.png" /></figure><p><strong>The trap I’ve hit more than once:</strong> a threshold with no requirement behind it is a <strong>guess</strong>. I once set a p95 threshold so loose that it passed against a result a hundred-odd times better than it — a green ✓ that proved nothing. <strong>A threshold is only useful if it can fail.</strong></p><h3>Part 2 — k6, Practice, and a Demo</h3><p>Now that the concepts are in place, let’s look at the tool. In this section, we’ll cover the k6 fundamentals you actually need, then run them against a small demo so you can see every step produce a real number.</p><h3>k6 in a Nutshell</h3><p>Every k6 script has two parts:</p><pre>export const options = { ... } // HOW to run: load shape, thresholds <br>export default function () { ... } // WHAT one user does, once</pre><p>options is <strong>declarative</strong> - k6 reads it and configures the run. default() is the <strong>work</strong> - the loop body. There&#39;s also setup(), which runs once before the load, and teardown(), which runs once after.</p><h3>Executors — The Six Load Shapes</h3><p>An <strong>executor</strong> is the kind of load pattern. k6 has six:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CIZATQyw0r6nwnHG.png" /></figure><p><strong>How I choose, in one question:</strong> <em>do I know the traffic, or do I know the users?</em> For an API, I know the traffic — requests arrive whether or not the server keeps up. So I use an arrival-rate executor.</p><p>Which executor for which test:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xn8OFocDuvOVvkRV.png" /></figure><h3>The Key Parameters</h3><pre>ramping-arrival-rate: { <br>  startRate: 5, <br>  timeUnit: &#39;1s&#39;, // rates are PER SECOND <br>  preAllocatedVUs: 50, <br>  maxVUs: 200, <br>  stages: [ <br>    { duration: &#39;15s&#39;, target: 20 }, <br>    { duration: &#39;30s&#39;, target: 60 }, <br>    { duration: &#39;30s&#39;, target: 100 }, <br>    { duration: &#39;15s&#39;, target: 0 }, <br>  ], <br>}</pre><ul><li><strong>timeUnit</strong> sets the scale for every number. startRate: 5, timeUnit: &#39;1s&#39; means 5 req/s; &#39;1m&#39; means 5 req/min - twelve times less.</li><li><strong>preAllocatedVUs</strong> - VUs created up front (allocating is expensive).</li><li><strong>maxVUs</strong> - the ceiling k6 may grow to if the rate needs more.</li><li><strong>stages</strong> - where the rate goes, over time.</li></ul><p>And the metric that tells me my test was lying: <strong>dropped_iterations</strong>. These are iterations that never began because no VU was available. <strong>If it&#39;s non-zero, the offered rate exceeded the delivered rate - my test isn&#39;t testing what it says it is.</strong> The fix is to raise maxVUs.</p><h3>Thresholds</h3><p>A <strong>threshold</strong> is a pass/fail rule for the whole run. k6 gives you two ways to assert something, and it’s worth knowing the difference:</p><ul><li><strong>check()</strong> - validates a single response during the run. It records whether the response passed and shows up in the summary as a pass rate, but <strong>it does not fail the run.</strong> A run where every check failed still exits successfully.</li><li><strong>A threshold</strong> — a rule applied to an aggregate metric across the whole run. If the rule is broken, <strong>the run fails</strong> with exit code <strong>99</strong>.</li></ul><p>That exit code is the whole point: CI reads it, so a threshold is what turns a performance test into something that can actually <strong>block a deploy</strong>. A check() that logs failures but exits 0 is invisible to a pipeline.</p><pre>thresholds: { <br>  http_req_failed: [&#39;rate&lt;0.01&#39;], // under 1% errors <br>  http_req_duration: [&#39;p(95)&lt;300&#39;, &#39;p(99)&lt;800&#39;], <br>  checks: [&#39;rate&gt;0.99&#39;], // 99% of checks passed <br>}</pre><p>Note that last line — you can even put a threshold on the checks rate, which is how you get check() failures to fail the run after all.</p><p>Aggregations depend on the metric type:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/0*3kUn5aNFCaaPFmKF.png" /></figure><p>p(N) is a percentile, 0-100, <strong>in milliseconds</strong>.</p><h3>The Metrics k6 Gives You Free</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aKBIw8Xvwss6XcXm.png" /></figure><p>Two things I always point out:</p><ul><li><strong>http_req_failed is a RATE (0-1), not a count.</strong> rate&lt;0.01 means &quot;under 1%&quot;, not &quot;under one request&quot;.</li><li><strong>http_req_duration EXCLUDES </strong><strong>blocked and </strong><strong>connecting.</strong> So a slow network or a saturated test rig hides in a metric I didn&#39;t look at. <strong>Always read the phases, not just the duration.</strong></li></ul><p>Reading the phases tells me which class of layer is at fault:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2Mk2OESM6EjEdNgv.png" /></figure><h3>Custom Metrics</h3><pre>import { Counter } from &#39;k6/metrics&#39;; <br>const status200 = new Counter(&#39;responses_200&#39;); <br><br>export default function () { <br>  if (res.status === 200) status200.add(1); <br>}</pre><p>Three kinds: Counter (only goes up), Gauge (up and down), Trend (a distribution - you can threshold its p(95)).</p><p>I split outcomes by class, so a run distinguishes “clean 200s” from “mostly 429s” from “mostly 5xx”. Collapsing them into “not 200” hides which happened — and those call for opposite responses.</p><h3>Meet the Demo</h3><p>To make all of this concrete, I built a small <strong>order service</strong> with a <strong>checkout</strong> endpoint.</p><p>It’s deliberately built to have three properties:</p><ul><li><strong>A precondition</strong> (login → cart), so a test can’t measure it without setting up state first.</li><li><strong>A variable cost</strong> (cart size), so the mix matters.</li><li><strong>A computable bottleneck</strong> (an internal worker pool), so I can <em>predict</em> the knee before I run anything.</li></ul><p>The numbers are demo-scale on purpose. <strong>The methodology is what scales, not the rate.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/0*Fi3GrALjFtobVFZM.png" /><figcaption><em>What to look at: two services up — the app and its datastore. Everything below runs against this stack.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*4aEphlo3ZsxxHUfP.png" /><figcaption><em>What to look at: which endpoints need a session token. Every one except health and products does — which is why a test that skips login gets a 401, not a number.</em></figcaption></figure><p>The bottleneck is the app’s own <strong>worker pool</strong>. Let me explain what that means, because it’s the key to the whole demo. When a checkout arrives, the app hands it to a <strong>worker</strong> — a fixed number of workers process checkouts, and each one handles a single checkout at a time. Think of them as <strong>cashiers at a checkout lane</strong>: there are only so many, and each serves one customer before taking the next.</p><p>My demo’s pool has <strong>5 workers</strong>, and each checkout occupies a worker for roughly <strong>100ms</strong> — that’s the app’s own processing time, not a call to anyone else. If all 5 workers are busy, the next request waits in line.</p><p>That gives me a computable ceiling. Each worker can serve 1 ÷ 0.1 = 10 checkouts per second, and there are 5 workers, so the app can handle <strong>5 × 10 = 50 req/s</strong>. Above 50 req/s, requests start queuing. Below it, they don’t. That’s the predicted knee — and I’ll test exactly this later.</p><p>Because the limit is the app’s own concurrency, it stays in scope: I’m measuring <em>my</em> service, and the external payment provider is still mocked (I’ll say so in the caveats).</p><p>And because every endpoint except health and products needs a valid session token — and checkout needs a cart owned by that session — a test that skips the setup gets a <strong>401/403</strong>, not a number. <strong>The test cannot accidentally work without its prerequisites.</strong> That’s the whole point: the setup phase is not optional.</p><h3>Step 1 — Smoke Test</h3><p>One user, thirty seconds. “Does it work at all?”</p><p>I always do this first. If the smoke test fails, nothing after it is worth running.</p><p>The whole script is almost nothing — one VU, a couple of GETs, loose thresholds:</p><pre>// 01-smoke.js <br>export const options = { <br>  vus: 1, <br>  duration: &#39;30s&#39;, <br>  thresholds: { <br>    http_req_failed: [&#39;rate==0&#39;], // zero errors, no exceptions <br>    http_req_duration: [&#39;p(95)&lt;1000&#39;], // loose on purpose -- just prove it works <br>  }, <br>}; <br><br>export default function () { <br>  const health = http.get(`${BASE_URL}/health`); <br>  check(health, { &#39;health is 200&#39;: (r) =&gt; r.status === 200 }); <br><br>  const products = http.get(`${BASE_URL}/products`); <br>  check(products, { &#39;products is 200&#39;: (r) =&gt; r.status === 200 }); <br><br>  sleep(1); <br>}</pre><p>Note constant-vus (the default when you just set vus) - a closed model is fine here. I&#39;m not looking for a breaking point, only &quot;does it work at all.&quot;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y7Ukub-mIaZdWv7-.png" /><figcaption><em>What to look at: </em><em>checks at 100% and </em><em>http_req_failed at 0%. That&#39;s the only question this run asks — does the endpoint answer at all? Latency is deliberately loose here.</em></figcaption></figure><h3>Step 2 — Baseline</h3><p>Checkout needs a session and a cart before it will answer. That state has to exist <strong>before</strong> the load starts — I can’t create it per request, or I’d be measuring the setup, not the checkout. So the demo has a <strong>seed script</strong> that creates users and carts directly, and k6’s setup() function loads them <strong>once</strong>, before the first request. Only then does the measured loop begin.</p><p>So this step shows three things:</p><ul><li>the seed script creating users and carts <strong>directly</strong>,</li><li>setup() fetching them <strong>once</strong>,</li><li>the measured loop hitting <strong>only</strong> checkout.</li></ul><p><strong>First, the seed — bulk, not one API call per user.</strong> The seed script asks the app to create everything in one request and writes the result to a file:</p><pre># app/seed.py <br>result = post( <br>  &quot;/admin/seed&quot;, <br>  {&quot;users&quot;: 200, &quot;cartsPerSize&quot;: 100, &quot;sizes&quot;: [1, 5, 10, 20, 100]}, <br>) <br><br>with open(&quot;/data/seed.json&quot;, &quot;w&quot;) as fh: <br>  json.dump(result, fh)</pre><p>That’s 200 users and <strong>100,000 carts in about a quarter of a second</strong>. <strong>Creating a million users through the API would take days and pollute the measurement</strong> — so I seed directly. That’s the point of this step.</p><p><strong>Then k6 loads that file once, in </strong><strong>setup().</strong> setup() runs a single time before any load, so the pool is built once, not per request:</p><pre>// lib.js <br>export function loadCarts() { <br>  http.post(`${BASE_URL}/admin/reset`); // start from a clean pool <br>  const bySize = {}; <br>  for (const user of raw) { <br>    for (const [size, ids] of Object.entries(user.carts)) { <br>      for (const cartId of ids) bySize[size].push({ token: user.token, cartId }); <br>    } <br>  } <br>  return { bySize }; <br>}</pre><pre>// 02-baseline.js<br>export function setup() {<br>  return loadCarts();   // runs ONCE, before the first request<br>}</pre><p><strong>Finally, the measured loop — and it touches only checkout:</strong></p><pre>export default function (data) {<br>  const size = pickWeighted(REALISTIC);        // realistic cart mix<br>  const cart = takeCart(data, size);           // a fresh, unconsumed cart<br><br>  const res = http.post(<br>    `${BASE_URL}/checkout`,<br>    JSON.stringify({ cartId: cart.cartId }),<br>    { headers: { Authorization: `Bearer ${cart.token}` } }<br>  );<br><br>  check(res, { &#39;checkout ok&#39;: (r) =&gt; r.status === 200 });<br>}</pre><p>That split — <strong>setup once, then measure only the thing you care about</strong> — is the whole discipline. If I created the cart inside the loop, I’d be measuring the setup.</p><p>And the load shape is an open model at a steady 20 req/s:</p><pre>scenarios: {<br>  baseline: {<br>    executor: &#39;constant-arrival-rate&#39;,<br>    rate: 20, timeUnit: &#39;1s&#39;, duration: &#39;60s&#39;,<br>    preAllocatedVUs: 30, maxVUs: 200,<br>  },<br>},</pre><p>I set the <strong>rate</strong>, not the users — k6 supplies whatever VUs it needs to hold that rate. That’s the open model from earlier, in practice.</p><p>I record the baseline p95 <strong>and the mix it was measured at</strong> — because a number without its mix isn’t a result.</p><p><strong>The baseline passed:</strong> throughput held at 19.8 req/s, p95 <strong>151.1ms</strong> (target 300ms), error rate 0%, and dropped_iterations at zero.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/0*EQ99e2I9rKUW0afY.png" /><figcaption><em>What to look at: 200 users and 100,000 carts created in one call — the pool the load will draw from.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Gg-Xpltr0NkWKatO.png" /><figcaption><em>What to look at: </em><em>http_req_duration p95 against the 300ms target, plus </em><em>dropped_iterations at zero - nothing was refused.</em></figcaption></figure><h3>Step 3 — Stress Test: Find the Knee</h3><p><strong>I predict the knee before I run.</strong> From the worker pool’s arithmetic above, I expect the limit at <strong>50 req/s</strong> — so that’s the number I write down <em>before</em> starting the test.</p><p>Then I ramp past it — 5 → 80 req/s — and show the result matching the prediction.</p><p>The executor is ramping-arrival-rate: an open model where I raise the <strong>offered rate</strong> on a schedule and let the queue build.</p><pre>// 03-stress.js<br>// PREDICTION: 5 workers / 0.1s service time = 50 req/s.<br>// Expect p95 to climb as the offered rate passes ~50/s.<br>scenarios: {<br>  stress: {<br>    executor: &#39;ramping-arrival-rate&#39;,<br>    startRate: 5,<br>    timeUnit: &#39;1s&#39;,<br>    preAllocatedVUs: 50,<br>    maxVUs: 600,          // past the knee, waiting requests need VUs<br>    gracefulStop: &#39;15s&#39;,  // bound the drain so the summary always prints<br>    stages: [<br>      { duration: &#39;20s&#39;, target: 20 },   // warm up, below the knee<br>      { duration: &#39;30s&#39;, target: 50 },   // approach the predicted knee<br>      { duration: &#39;30s&#39;, target: 80 },   // push past it<br>      { duration: &#39;10s&#39;, target: 0 },    // ramp down<br>    ],<br>  },<br>},</pre><p>Two details that matter. The cart size is held <strong>fixed at 10 items</strong> (~100ms) so the <em>only</em> variable is the rate. And the thresholds are the finding, not a safety net — I <em>want</em> this one to go red:</p><pre>thresholds: {<br>  http_req_duration: [&#39;p(95)&lt;300&#39;],   // this SHOULD fail past the knee<br>  http_req_failed: [&#39;rate&lt;0.05&#39;],     // guard: catch a broken run<br>  checks: [&#39;rate&gt;0.95&#39;],              // guard: catch a broken run<br>},</pre><p>That’s far more convincing than “the system got slow.” If the knee appears where I predicted, I understand the system. If it appears somewhere else, my assumption was wrong — and that’s a finding, not a failure.</p><p><strong>And it matched.</strong> The ramp kept asking for more — up to 80 req/s — but the <em>delivered</em> throughput stopped at around <strong>50 req/s</strong>, almost exactly the number I predicted, while p95 climbed from a flat <strong>102ms into the seconds</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/0*AOTxPHzsZ5ZsbsY4.png" /></figure><p>Look at what happens past the knee. The ramp asked for 57, then 67 — and the app didn’t just stop at 50, it <strong>delivered <em>less</em></strong>: 49.5, then 44.5. The queue got so backed up that requests took so long to finish that fewer completed at all. That’s the overload collapse: <strong>offered rate up, delivered rate down.</strong></p><p>And dropped_iterations went non-zero past the knee, which is exactly the signal I described earlier - the test was offering more than the app could take.</p><p>(One honesty note: those past-the-knee numbers move between runs. Across the runs I did, the delivered rate once past the knee sat somewhere in the <strong>44–50</strong> range, and the exact digits shift every time. The <em>shape</em> is stable — flat, then the bend, then the fall — even when the numbers move. That’s why I report the shape and the prediction, not a single heroic number.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eFpE-Id4UUYK9fnN.png" /><figcaption><em>What to look at: the VUs column. It sits at 1–5 while the app keeps up, then explodes — 16, then 87, then 175 — as requests start queuing. The </em><em>iters/s column on the right is the </em><strong><em>offered</em></strong><em> rate, still climbing to ~77 while the app delivers only ~50. That widening gap is the knee.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*VkCaP2itcl8Qax2x.png" /><figcaption><em>What to look at: the flat stretch, then the bend. Latency holds near 100ms below the knee and shoots into the seconds above it — the same story the VU column told, drawn as a curve.</em></figcaption></figure><h3>Step 4 — The Cart-Size Mix</h3><p>Now the payoff. I re-run the same endpoint with three mixes — <strong>all small</strong>, <strong>realistic</strong>, and <strong>all heavy</strong> — holding the rate <em>constant across the three</em> and the threshold the same, then put the results side by side.</p><p>One deliberate choice: I keep the rate <strong>low enough that every mix is sustainable</strong>. If I pushed the rate up, the heavy mix would stop measuring service time and start measuring queueing — and then I’d be testing the load, not the mix. The mix has to be the <em>only</em> variable.</p><p>The three mixes are just weight tables, and one env var picks which one runs:</p><pre>// 04-mix.js<br>const MIX  = __ENV.MIX  || &#39;realistic&#39;;<br>const RATE = Number(__ENV.MIX_RATE || 5);   // LOW, so every mix is sustainable<br><br>const MIXES = {<br>  &#39;all-small&#39;: [{ items: 1,   weight: 100 }],<br>  &#39;realistic&#39;: [{ items: 1,   weight: 60 },   // 60% tiny<br>                { items: 5,   weight: 30 },   // 30% small<br>                { items: 20,  weight: 9  },   //  9% medium<br>                { items: 100, weight: 1  }],  //  1% huge  &lt;- the tail<br>  &#39;all-heavy&#39;: [{ items: 100, weight: 100 }],<br>};<br><br>const choices = MIXES[MIX];</pre><p>Then I run it three times, changing only MIX:</p><pre>MIX=all-small k6 run 04-mix.js<br>MIX=realistic k6 run 04-mix.js<br>MIX=all-heavy  k6 run 04-mix.js</pre><p>The threshold is <strong>identical for all three</strong> — that’s the whole lesson:</p><pre>thresholds: {<br>  http_req_failed: [&#39;rate&lt;0.01&#39;],<br>  http_req_duration: [&#39;p(95)&lt;300&#39;, &#39;p(99)&lt;800&#39;],   // SAME threshold, every mix<br>},</pre><p>Same endpoint, same rate, same threshold. Only the mix moves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*V7Ay9x9VzJ49fG7I.png" /></figure><p><strong>Measured: p50 stays flat (56ms) while p99 moves roughly ten times (56.9ms → 551.1ms).</strong> The small fraction of heavy carts defines the tail.</p><p>Look at the realistic row: the <em>same endpoint</em>, at the <em>same rate</em>, passes the p95 requirement <strong>and breaches p99</strong> — because 1% of carts are huge and p99 looks at exactly the top 1%.</p><p>This is the most valuable lesson in the whole article, made concrete: <strong>the same endpoint passes or fails the same requirement depending on the mix.</strong> So a bare “p95 &lt; 300ms ✅” is meaningless without saying what mix produced it.</p><p>And here’s the instability I found. Same realistic mix, same code, three runs — only the random seed changed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/0*DpApI3CHxvDdsxjF.png" /></figure><p>p99 <strong>flipped between runs</strong>, because ~1% of requests are heavy and p99 sits right on that boundary — the 99th percentile either lands on a heavy request or it doesn’t. <strong>So I report several percentiles, and I say when a number is marginal.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/0*dvZsIKUVW9XFFN3V.png" /><figcaption><em>Mix 1 of 3 — </em><strong><em>all-small</em></strong><em> (every cart 1 item). p50/p95/p99 all sit at ≈56ms. No heavy carts, so no tail; passes the threshold with room to spare.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/0*2_RccjUCTMOjI4kt.png" /><figcaption><em>Mix 2 of 3 — </em><strong><em>realistic</em></strong><em> (60/30/9/1%). p50 stays flat at ≈56ms, but p99 jumps to ≈551ms. p95 passes the 300ms bar, p99 breaches it — the 1% of heavy carts lands exactly where p99 looks.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/0*NRhg4AjaI6w_-GoQ.png" /><figcaption><em>Mix 3 of 3 — </em><strong><em>all-heavy</em></strong><em> (every cart 100 items). Now even p50 is ≈551ms — the whole distribution shifts and it fails outright.</em></figcaption></figure><p><em>Read the three together: same endpoint, same rate, same threshold — only the mix changes, and the verdict flips from pass to marginal to fail.</em></p><h3>Step 5 — Spike Test</h3><p>Traffic jumps 10× instantly. Does it survive — and does it <em>recover</em> when the spike passes?</p><p>Recovery is the part people forget. A system that survives the spike but never comes back down is still down.</p><p>The stages <em>are</em> the test — a sharp jump up, a hold, then a drop back to prove recovery:</p><pre>// 05-spike.js -- cart size held at 1 item (~55ms) so the spike is about RATE<br>scenarios: {<br>  spike: {<br>    executor: &#39;ramping-arrival-rate&#39;,<br>    startRate: 10,<br>    timeUnit: &#39;1s&#39;,<br>    preAllocatedVUs: 50,<br>    maxVUs: 2000,<br>    stages: [<br>      { duration: &#39;15s&#39;, target: 10  },   // normal load<br>      { duration: &#39;5s&#39;,  target: 100 },   // the spike: jump 10x, fast<br>      { duration: &#39;20s&#39;, target: 100 },   // hold the spike<br>      { duration: &#39;5s&#39;,  target: 10  },   // drop back down<br>      { duration: &#39;15s&#39;, target: 10  },   // does it recover?<br>    ],<br>  },<br>},</pre><p>Two things to notice. The <strong>target: 10 after the spike</strong> is the recovery check - I hold at normal load for 15s and watch whether latency comes back down. And I only threshold the error rate here:</p><pre>thresholds: {<br>  http_req_failed: [&#39;rate&lt;0.05&#39;],   // did it survive? errors are the pass/fail<br>},</pre><p>I <em>don’t</em> threshold latency — a spike is <em>supposed</em> to be slow, so a latency threshold would fail by design and tell me nothing. Survival plus recovery is the whole question.</p><p><strong>It did both.</strong> The spike held at ~90 req/s with 0% errors and p95 at 1.56s — then dropped straight back to the pre-spike rate:</p><pre>10 req/s → 90 req/s (spike) → 10 req/s (recovered)</pre><p>(It plateaus at ~90, not 100, because 1-item carts take ~55ms — and 5 ÷ 0.055 ≈ 90. The same arithmetic again.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/0*1lmg59sfTmg59Xwj.png" /><figcaption><em>What to look at: two things — </em><em>http_req_failed staying at 0% (it survived the surge), and the rate falling back to the pre-spike level at the end (it recovered). A spike you survive but never come down from is still down.</em></figcaption></figure><h3>Reading the Results — The Traps I Check</h3><p>Every one of these comes from experience:</p><ul><li><strong>Vacuous thresholds.</strong> A threshold that can’t fail proves nothing. If the p95 target is 300ms and reality is 3ms, the ✓ is noise.</li><li><strong>The test rig as the bottleneck.</strong> Check blocked and connecting - my own client may be the limit, not the server.</li><li><strong>Test environment ≠ production.</strong> The number doesn’t transfer.</li><li><strong>Measuring the wrong component.</strong> If a rate limiter fires, I measured the limiter, not the app.</li><li><strong>dropped_iterations non-zero.</strong> The ramp wasn&#39;t delivered; the test is lying about its own load.</li><li><strong>A short test can’t see a rare tail.</strong> If 1% of traffic is heavy, I need thousands of samples for p99 to even be defined.</li><li><strong>A green latency number on a failed run.</strong> The one that bit me hardest, and it’s worth a paragraph.</li></ul><h3>The trap that bit me: a passing latency on a failing test</h3><p>Because checkout <strong>consumes</strong> the cart, every iteration needs a fresh one. On one of my runs the seeded carts ran out — so checkout started returning <strong>409 (already consumed)</strong>. Here’s what the summary said:</p><pre>✗ http_req_duration..............: p(95)=100.97ms        ← PASSES<br>  http_req_failed................: 93.34% (3407/3650)    ← 93% FAILED</pre><p><strong>The latency threshold passed while 93% of my requests failed.</strong> The 409s come back fast, so they drag the latency <em>down</em> — the test looked healthy at exactly the moment it was measuring nothing but errors.</p><p>The reason it slipped through: my stress script only thresholded http_req_duration. A run can fail a million requests and still print a green duration.</p><p><strong>The fix is three things:</strong></p><ul><li><strong>Reset the state before the run.</strong> The setup phase now clears the cart pool, so every run starts clean.</li><li><strong>Fail loudly when the data runs out</strong> — don’t wrap around and silently re-serve used carts.</li><li><strong>Threshold the error rate, not just the latency</strong> — http_req_failed: [&#39;rate&lt;0.05&#39;] and checks: [&#39;rate&gt;0.95&#39;].</li></ul><p><strong>The lesson: a latency number is only meaningful when the error rate is near zero.</strong> Before I trust a duration, I read http_req_failed and checks first. A green p95 on a 93%-failed run is worse than no number at all - it&#39;s a false pass.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Kl9xZgbI9aAyZWQX.png" /><figcaption><em>What to look at: the summary block. </em><em>checks at 100% and </em><em>http_req_failed at 0% say the run was valid; </em><em>✗ http_req_duration p(95)=3.64s is the knee going red on purpose; and </em><em>dropped_iterations: 362 is the proof - the ramp offered more than the app could accept. Read the first two lines before you trust the third.</em></figcaption></figure><h3>Pro-Tips for Your First Load Test</h3><p>A few things I wish someone had told me earlier:</p><ul><li><strong>Write the requirement down before you touch a tool.</strong> A test without a target is a demo, not a test.</li><li><strong>Do the smoke test first, always.</strong> One user for thirty seconds saves hours of chasing a broken script.</li><li><strong>Read the phases, not just the duration.</strong> waiting blames the server; blocked blames your rig.</li><li><strong>Predict the knee, then confirm it.</strong> When the number matches your arithmetic, you understand the system — not just the result.</li><li><strong>Report the caveats with the numbers.</strong> The conditions <em>are</em> part of the result.</li></ul><h3>Best Practices</h3><ul><li><strong>Model the mix, not just the endpoint.</strong> Real traffic is a distribution; so is your test.</li><li><strong>Use an open model to find a break.</strong> A closed model cools down when things get hot.</li><li><strong>Seed your data directly.</strong> Creating a million users through the API is slow and pollutes the measurement.</li><li><strong>Isolate state per VU.</strong> Shared carts produce phantom errors that don’t exist in production.</li><li><strong>Mock external dependencies — and say so.</strong> The real payment latency is not in your number.</li><li><strong>Let thresholds fail.</strong> A green ✓ you can’t trust is worse than a red one you can.</li></ul><h3>Conclusion</h3><p>Performance testing is a <strong>lifecycle</strong>, not a script. Get the vocabulary right, get the number first, seed the data, run it in the right model, and read the phases — not just the average. The tool is interchangeable; the concepts are not.</p><p>I hope this gives you a foundation you can carry into any tool you pick up next.</p><h3>Further Reading and Resources</h3><ul><li><a href="https://grafana.com/docs/k6/latest/">Official k6 Documentation</a></li><li>Grafana k6: <a href="https://grafana.com/docs/k6/latest/testing-guides/test-types/">Test Types</a> and <a href="https://grafana.com/docs/k6/latest/using-k6/scenarios/executors/">Executors</a></li><li><a href="https://en.wikipedia.org/wiki/Little%27s_law">Little’s Law (Wikipedia)</a> — the arithmetic behind capacity planning</li></ul><h3>Closing Note</h3><p>Thank you for learning with me. If you found this useful, feel free to share your experiences or ask questions in the comments below — I’m always happy to talk performance testing.</p><p>Happy testing!</p><p>Email: dhadiprasetyo@gmail.com<br>LinkedIn: <a href="https://www.linkedin.com/in/darmawanhadip">Darmawan Hadiprasetyo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73e2eb85db7b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI-Powered Browser Automation: Playwright MCP Server with LLMs]]></title>
            <link>https://medium.com/@dhadiprasetyo/ai-powered-browser-automation-playwright-mcp-server-with-llms-9911f0d513a8?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/9911f0d513a8</guid>
            <category><![CDATA[playwrights]]></category>
            <category><![CDATA[browser-automation]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[ai-testing]]></category>
            <category><![CDATA[automation-testing]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Wed, 02 Apr 2025 11:34:39 GMT</pubDate>
            <atom:updated>2025-04-02T11:34:39.202Z</atom:updated>
            <content:encoded><![CDATA[<p>​In the rapidly evolving landscape of software development, integrating Large Language Models (LLMs) with automation tools has opened new avenues for efficiency and innovation. One notable advancement is the Playwright Model Context Protocol (MCP) Server, which combines the capabilities of Playwright — a robust browser automation framework — with the intelligence of LLMs. This integration facilitates seamless browser interactions, enabling tasks such as web navigation, data extraction, and automated testing to be performed more intuitively.</p><h3><strong>Understanding Playwright MCP Server</strong></h3><p>The Playwright MCP Server acts as an intermediary between LLMs and web browsers. By leveraging Playwright’s automation features, it allows LLMs to interact with web pages in a real browser environment. This setup enables functionalities like executing JavaScript, taking screenshots, generating test code, and web scraping, all through structured data interactions without relying on pixel-based inputs.</p><p><strong>Key Features:</strong></p><ul><li><strong>LLM Integration:</strong> Facilitates direct communication between LLMs and web browsers for dynamic content interaction.​</li><li><strong>Comprehensive Automation:</strong> Supports tasks such as form filling, navigation, and data extraction.</li><li><strong>Test Code Generation:</strong> Automatically generates test scripts based on user interactions, streamlining the testing process.​</li></ul><h3>Setting Up Playwright MCP Server in Visual Studio Code Insiders</h3><p>Integrating the Playwright MCP Server with Visual Studio Code Insiders enhances the development workflow by embedding powerful automation capabilities directly into the editor. Follow these steps to set it up:</p><h3>Step 1: Install Visual Studio Code Insiders</h3><p>Before installing the Playwright MCP Server, ensure you have Visual Studio Code Insiders installed:</p><ol><li>Go to the <a href="https://code.visualstudio.com/insiders/">Visual Studio Code Insiders download page</a>.</li><li>Download the appropriate version for your operating system (Windows, macOS, or Linux).</li><li>Install the application following the provided instructions.</li><li>Launch <strong>VS Code Insiders</strong> and ensure it is running properly.</li></ol><h3>Step 2: Install GitHub Copilot Extension</h3><p>GitHub Copilot enhances your coding experience by providing AI-powered code suggestions. Follow these steps to install it:</p><ol><li>Open <strong>VS Code Insiders</strong>.</li><li>Go to the <strong>Extensions Marketplace</strong> (Ctrl+Shift+X on Windows/Linux, Cmd+Shift+X on macOS).</li><li>Search for <strong>GitHub Copilot</strong>.</li><li>Click <strong>Install</strong>.</li><li>Sign in with your GitHub account and activate Copilot.</li><li>Restart VS Code Insiders to apply changes.</li></ol><h3>Step 3: Install Playwright MCP Server in VS Code Insiders</h3><p>Now, you can install the Playwright MCP Server. Head to the <a href="https://github.com/executeautomation/mcp-playwright?tab=readme-ov-file">official Playwright MCP GitHub repository</a> and follow the installation steps in the <strong>Installation in VS Code</strong> section. This will guide you through the process of installing the MCP server in the VS Code Insiders.</p><ol><li>Click the <strong>VS Code Insiders</strong> button. After clicking the button, you’ll be redirected to VS Code Insiders, where the installation process will automatically begin.</li><li>Verify the Playwright MCP configuration:</li></ol><ul><li>Open the <strong>Command Palette</strong> in VS Code (use CMD+Shift+P).</li><li>Search for “MCP” and select <strong>MCP: List Servers</strong>.</li><li>In the list, you’ll find the <strong>Playwright</strong> option. Click on it, then select <strong>Show Configuration</strong> to view the MCP server settings.</li></ul><p>3. Start the MCP Server:</p><ul><li>Open the <strong>Command Palette</strong> again (use CMD+Shift+P).</li><li>Search for “MCP” and select <strong>MCP: List Servers</strong>.</li><li>Click on <strong>Playwright</strong>, then select <strong>Start Server</strong> to launch the MCP server.</li></ul><h3>Step 4: Experimenting Using GitHub Copilot</h3><p>In this step, you will explore how GitHub Copilot can assist in generating automation code for tasks. By interacting with Copilot, you will get real-time code suggestions based on your instructions. Follow these steps to experiment with GitHub Copilot in VS Code:</p><ol><li>Open <strong>GitHub Copilot</strong> in the secondary sidebar of VS Code.</li><li>Change the usage type to <strong>Agent</strong> and set the model to <strong>Claude 3.5 Sonnet</strong>.</li><li>Try the following sample command in GitHub Copilot:</li></ol><pre>Navigate to https://opensource-demo.orangehrmlive.com/. In the login page enter the username and password as &quot;Admin&quot; and &quot;admin123&quot; then perform login. Then on the dashboard, navigate to side menu and click on &quot;My Info&quot; menu, then edit the datas with this criterias:<br><br>- Random alphabetic employee full name<br>- Random 4 digits employee id<br>- Random 4 digits other id<br>- Random driver&#39;s license number following the Indonesia&#39;s regulation<br>- Expiry date change to 5 years from today<br>- Nationality change to Indonesia<br>- Marital status change to Married<br>- Gender change to female<br>- Then click on save button on Personal Details section.</pre><p>4. Enter this message in GitHub Copilot, and it will generate the corresponding code for you. Follow the generated instructions to run the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sLgPIpGSFWKflC4G3SmnfQ.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AuQSkg2XARRxMTNUdc11qw.gif" /></figure><h3>Conclusion</h3><p>By integrating the Playwright MCP Server with Visual Studio Code Insiders and GitHub Copilot, developers can harness the power of AI to automate complex browser tasks, enhancing productivity and streamlining workflows. This powerful combination simplifies web automation, making AI-driven testing and data extraction more accessible than ever.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9911f0d513a8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[API Mocking With WireMock as a Standalone Service: A Step-by-Step Guide]]></title>
            <link>https://medium.com/@dhadiprasetyo/api-mocking-with-wiremock-as-a-standalone-service-a-step-by-step-guide-12a9155f2061?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/12a9155f2061</guid>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[api-mocking]]></category>
            <category><![CDATA[api-development]]></category>
            <category><![CDATA[wiremock]]></category>
            <category><![CDATA[mocking]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sat, 29 Mar 2025 09:21:25 GMT</pubDate>
            <atom:updated>2025-03-29T09:21:25.835Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TycO3e-qiLDY9ieC.png" /><figcaption><a href="https://wiremock.org/">https://wiremock.org/</a></figcaption></figure><p>WireMock is a powerful tool for API mocking, allowing developers to simulate API responses without depending on external services. Running WireMock as a standalone service provides flexibility, making it easy to integrate into local development and testing environments. In this guide, we’ll walk through the process of setting up WireMock and configuring mock APIs, from basic to advanced examples.</p><h3>1. Setting Up WireMock as a Standalone Service</h3><h4>Step 1: Create a New Directory</h4><p>Before using WireMock, create a dedicated directory to store its configurations.</p><pre>mkdir wiremock-demo &amp;&amp; cd wiremock-demo</pre><h4>Step 2: Download and Install WireMock</h4><p>WireMock can be run as a standalone JAR file. Download the latest version using:</p><pre>curl -O https://repo1.maven.org/maven2/org/wiremock/wiremock-standalone/3.12.1/wiremock-standalone-3.12.1.jar</pre><p>Run WireMock:</p><pre>java -jar wiremock-standalone-3.12.1.jar</pre><p>By default, WireMock runs on port 8080. You can access the admin interface at <a href="http://localhost:8080/__admin/">http://localhost:8080/__admin/.</a></p><h4>Step 3: Verify the Installation</h4><p>Check if WireMock is running by sending a request:</p><pre>curl http://localhost:8080/__admin/mappings</pre><p>If WireMock is running correctly, you’ll receive a JSON response with the list of configured mocks.</p><h3>2. Creating Mock APIs</h3><p>When WireMock starts, it automatically creates two directories: mappings and __files. You can define stub mappings by placing JSON files in the mappings directory.</p><h3>Basic Mock API Example</h3><h4>Using API</h4><pre>curl -X POST http://localhost:8080/__admin/mappings \<br>    -H &quot;Content-Type: application/json&quot; \<br>    -d &#39;{<br>        &quot;request&quot;: {<br>            &quot;method&quot;: &quot;GET&quot;,<br>            &quot;url&quot;: &quot;/hello&quot;<br>        },<br>        &quot;response&quot;: {<br>            &quot;status&quot;: 200,<br>            &quot;body&quot;: &quot;Hello, WireMock!&quot;,<br>            &quot;headers&quot;: {<br>                &quot;Content-Type&quot;: &quot;text/plain&quot;<br>            }<br>        }<br>    }&#39;</pre><p>Now, test the endpoint:</p><pre>curl --location &#39;http://localhost:8080/hello&#39;</pre><p>Expected response:</p><pre>Hello, WireMock!</pre><h4>Using JSON File</h4><p>Create a new file inside mappings/hello.json:</p><pre>{<br>    &quot;request&quot;: {<br>        &quot;method&quot;: &quot;GET&quot;,<br>        &quot;url&quot;: &quot;/hello&quot;<br>    },<br>    &quot;response&quot;: {<br>        &quot;status&quot;: 200,<br>        &quot;body&quot;: &quot;Hello, WireMock!&quot;,<br>        &quot;headers&quot;: {<br>            &quot;Content-Type&quot;: &quot;text/plain&quot;<br>        }<br>    }<br>}</pre><p>Restart WireMock, and the API will be available.</p><h3>3. Advanced Mock API Configurations</h3><h4>Mock API with Query Parameters</h4><p>You can create mock APIs that respond differently based on query parameters:</p><pre>{<br>    &quot;request&quot;: {<br>        &quot;method&quot;: &quot;GET&quot;,<br>        &quot;urlPath&quot;: &quot;/greet&quot;,<br>        &quot;queryParameters&quot;: {<br>            &quot;name&quot;: {<br>                &quot;equalTo&quot;: &quot;Alice&quot;<br>            }<br>        }<br>    },<br>    &quot;response&quot;: {<br>        &quot;status&quot;: 200,<br>        &quot;body&quot;: &quot;Hello, Alice!&quot;<br>    }<br>}</pre><p>Now, test it:</p><pre>curl --location &#39;http://localhost:8080/greet?name=Alice&#39;</pre><p>Expected response:</p><pre>Hello, Alice!</pre><h4>Mock API with Delayed Responses</h4><p>Simulate slow responses to test timeout scenarios:</p><pre>{<br>    &quot;request&quot;: {<br>        &quot;method&quot;: &quot;GET&quot;,<br>        &quot;url&quot;: &quot;/slow-response&quot;<br>    },<br>    &quot;response&quot;: {<br>        &quot;status&quot;: 200,<br>        &quot;body&quot;: &quot;This response is delayed&quot;,<br>        &quot;fixedDelayMilliseconds&quot;: 3000<br>    }<br>}</pre><p>This will delay the response by 3 seconds.</p><h4>Mock API with Dynamic Responses (Templating)</h4><p>WireMock supports response templating using Handlebars:</p><pre>{<br>  &quot;request&quot;: {<br>    &quot;method&quot;: &quot;GET&quot;,<br>    &quot;urlPathPattern&quot;: &quot;/user/(.*)&quot;<br>  },<br>  &quot;response&quot;: {<br>    &quot;status&quot;: 200,<br>    &quot;body&quot;: &quot;Hello, {{request.pathSegments.1}}!&quot;,<br>    &quot;transformers&quot;: [<br>      &quot;response-template&quot;<br>    ],<br>    &quot;headers&quot;: {<br>      &quot;Content-Type&quot;: &quot;text/plain&quot;<br>    }<br>  }<br>}</pre><p>Now, testing GET /user/John will return:</p><pre>Hello, John!</pre><h3>4. Recording and Replaying Real API Traffic</h3><p>WireMock can capture and replay real API interactions, making it useful for testing.</p><h4>Step 1: Enable Recording Mode</h4><p>Start WireMock in proxy mode to capture traffic:</p><pre>java -jar wiremock-standalone-3.12.1.jar --proxy-all=&quot;https://jsonplaceholder.typicode.com&quot; --record-mappings</pre><h4>Step 2: Make a Request</h4><pre>curl http://localhost:8080/posts/1</pre><p>This will fetch real data from jsonplaceholder.typicode.com and store it as a stub in the mappings directory.</p><h4>Step 3: Replay Recorded Stubs</h4><p>Stop WireMock and restart it normally:</p><pre>java -jar wiremock-standalone-3.12.1.jar</pre><p>Now, calling http://localhost:8080/posts/1 will return the cached response from WireMock instead of hitting the real API.</p><h3>5. Learning More About WireMock</h3><p>To explore more about WireMock, check out these resources:</p><ul><li><a href="https://wiremock.org/docs/">Official WireMock Documentation</a></li><li><a href="https://github.com/wiremock/wiremock">GitHub Repository</a></li><li><a href="https://wiremock.org/docs/getting-started/">WireMock User Guide</a></li></ul><p>WireMock is an invaluable tool for API testing, allowing developers to simulate different scenarios easily. Whether you’re testing failure cases or building integration tests, WireMock can help streamline your development process.</p><p>At Kredivo, we utilize WireMock in a more advanced way by implementing a proxy-based mechanism. This allows developers to mock APIs without changing their application code. Additionally, we integrate WireMock with Docker, making it scalable and easy to manage in different environments.</p><p>Try it out and start mocking today!</p><p>Do you have any questions or want to see more advanced use cases? Let me know in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12a9155f2061" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Load Testing with JMeter and jp@gc Throughput Shaping Timer]]></title>
            <link>https://medium.com/@dhadiprasetyo/mastering-load-testing-with-jmeter-and-jp-gc-throughput-shaping-timer-1c97e758a0b5?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c97e758a0b5</guid>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[load-testing]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[jmeter]]></category>
            <category><![CDATA[jmeter-load-testing]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Mon, 12 Aug 2024 14:14:57 GMT</pubDate>
            <atom:updated>2024-08-12T14:14:57.963Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction:</h3><p>In today’s fast-paced digital world, ensuring that your applications can handle the expected load is crucial. This is where load testing comes into play, and Apache JMeter is one of the most popular tools for this purpose. However, managing throughput to simulate real-world scenarios can be tricky. That’s where the jp@gc Throughput Shaping Timer comes in. In this article, we’ll explore how to use JMeter along with the jp@gc Throughput Shaping Timer to precisely control your test throughput, ensuring your tests are as realistic as possible.</p><h3>Why Use jp@gc Throughput Shaping Timer?</h3><p>The jp@gc Throughput Shaping Timer is a plugin for JMeter that allows you to shape the throughput of your tests. This is particularly useful when you want to ensure that your tests hit a specific request per second (RPS) rate, which can be critical for load tests where consistency is key.</p><h3><strong>Install jp@gc Plugins</strong></h3><ul><li>Open JMeter.</li><li>Go to Options -&gt; Plugins Manager.</li><li>In the Available Plugins tab, find the &quot;jpgc - Throughput Shaping Timer&quot; and install it.</li><li>For the additional reporting, let’s also install plugin “3 Basic Graphs”.</li><li>Restart JMeter to apply the changes.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*iDxHayTk3NTj2cI6f5qMIQ.png" /><figcaption>Througput Shaping Timer</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*f1DckO04cAw_XYTEGEptGA.png" /><figcaption>3 Basic Graphs</figcaption></figure><h3>Setting Up Your First Test Plan with the JSONPlaceholder API</h3><p>Let’s create a simple test plan using the JSONPlaceholder API, specifically the /posts endpoint, to demonstrate how to use the Throughput Shaping Timer to control the request rate. Our goal is to test the following requirements:</p><ul><li><strong>Average QPS (Queries Per Second)</strong>: 5 QPS</li><li><strong>Peak QPS (Queries Per Second)</strong>: 10 QPS</li></ul><p>We’ll also demonstrate how to import a cURL command into JMeter.</p><h4>1. Create a New Test Plan:</h4><ul><li>Open JMeter and click on File -&gt; New.</li><li>Rename the test plan to “JSONPlaceholder Throughput Shaping Test”.</li></ul><h4>2. Add a Thread Group:</h4><ul><li>Right-click on the Test Plan.</li><li>Select Add -&gt; Threads (Users) -&gt; Thread Group.</li><li>Configure the number of threads (users) and the ramp-up period. For example:</li><li>Number of Threads (Users): 10 (this will achieve 10 QPS at peak).</li><li>Ramp-Up Period: 10 seconds</li><li>Loop Count: Infinite</li></ul><h4>3. Add a jp@gc Throughput Shaping Timer:</h4><ul><li>Right-click on the Thread Group.</li><li>Select Add -&gt; Timers -&gt; jp@gc - Throughput Shaping Timer.</li><li>Configure the desired throughput as below:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*Cxc-emFEq8c1wRrh8vBrZw.png" /></figure><ul><li>A little tip for setting the start and end RPS is to compare them with the number of threads and the ramp-up period. In our test, we set the number of threads to 10 to simulate 10 QPS at peak and the ramp-up period to 10 seconds, meaning all threads will start within 10 seconds. To achieve an average of 5 QPS, you can adjust the values in the first row and then check the graph to see if it meets your desired average QPS. The second row should be used to reach the peak of 10 QPS in our case, and the third row should gradually bring it back to the average of 5 QPS before finishing the test. The duration is entirely up to you, depending on how long you want to run the test. Just experiment with the numbers until the graph matches your expectations.</li></ul><h3>4. Importing HTTP Request from a cURL Command</h3><p>If you have a cURL command, you can easily import it into JMeter to create an HTTP Request Sampler. Let’s use the following cURL command as an example:</p><pre>curl - location &#39;https://jsonplaceholder.typicode.com/posts&#39;</pre><ul><li>Open JMeter.</li><li>Click on Tools -&gt; Import from cURL.</li><li>Paste your cURL command into the box.</li><li>Then click on Create Test Plan.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*GjAcIc5j-1_TzuvhtUfnTQ.png" /><figcaption>Importing cURL request.</figcaption></figure><h3>5. Adding Test Listeners Using the “3 Basic Graphs” Plugin</h3><p>After setting up your test plan, it’s essential to analyze the results effectively. One way to do this is by adding listeners that visually represent the test data. The “3 Basic Graphs” plugin is an excellent tool for this purpose, providing you with three key graphs: Response Times Over Time, Transactions Per Second, and Active Threads Over Time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*uNsJoyReCEs1cn-KXPdSmA.png" /><figcaption>jp@gc — 3 Basic Graphs</figcaption></figure><p>Understanding the Graphs:</p><ul><li><strong>Response Times Over Time</strong>: This graph shows how the response times of your requests vary over the duration of the test. It’s useful for spotting trends, such as increasing response times under load.</li><li><strong>Transactions Per Second</strong>: This graph indicates the number of transactions completed per second, helping you verify if your test meets the desired QPS targets.</li><li><strong>Active Threads Over Time</strong>: This graph displays the number of active threads during the test. It helps you understand how the load is distributed over time and how it correlates with response times and transaction rates.</li></ul><h3>6. Adding Test Listeners Using Summary Report</h3><p>In addition to the “3 Basic Graphs,” it’s also essential to add a <strong>Summary Report</strong> listener. This listener provides a detailed summary of your test results, enabling you to measure key performance metrics such as:</p><ul><li><strong>Average, Min, Max, and Standard Deviation of Response Times</strong>: These metrics help you understand the consistency of your application’s performance. The average response time gives you an overall picture, while the min, max, and standard deviation provide insights into the best, worst, and variability of the response times.</li><li><strong>Error Rate Percentage</strong>: This metric shows the percentage of requests that resulted in errors, which is crucial for assessing the reliability of your application under load.</li><li><strong>Total Average Throughput</strong>: Throughput measures the number of requests handled per unit of time, typically in requests per second (RPS). This helps you understand the capacity of your system to handle concurrent requests.</li></ul><p>Add the Summary Report Listener:</p><ul><li>Right-click on your Thread Group.</li><li>Select Add -&gt; Listener -&gt; Summary Report.</li><li>This will add the Summary Report listener to your test plan, providing a comprehensive overview of the test results.</li></ul><h3>7. Running The Test</h3><p>Now you can run the test. Once the test is complete, review the results in the Listener. You should observe that the throughput matches the parameters you set in the Throughput Shaping Timer. This alignment indicates that your load test accurately reflects the desired request rates, which is crucial for realistic performance testing.</p><p>If you find that the results don’t meet your expectations, you can always experiment with the configuration. Adjust the number of threads, the ramp-up period, and the jp@gc Throughput Shaping Timer settings. Fine-tuning these elements will help you achieve the precise load conditions required for your test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*WdAUxVXqfXYxcgHFm_8_Ig.png" /><figcaption><strong>Response Times Over Time</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/985/1*T0pF_2yqPlWl53h9ciYYEQ.png" /><figcaption><strong>Active Threads Over Time</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*Ms7UpFSQFTv0yaTHR7kApw.png" /><figcaption><strong>Transactions Per Second</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1009/1*G_rtp8cj1wy5kl8VZT-Xig.png" /><figcaption>Summary Report</figcaption></figure><h3>Conclusion</h3><p>The jp@gc Throughput Shaping Timer is an essential tool for anyone serious about load testing. By using this plugin with JMeter, you can create realistic test scenarios that ensure your application is ready to handle the load. Whether you’re a seasoned performance tester or just starting, mastering this tool will greatly enhance your testing capabilities.</p><h3>Further Reading and Resources</h3><ul><li><a href="https://jmeter.apache.org/usermanual/index.html">Official JMeter Documentation</a></li><li><a href="https://jmeter-plugins.org/wiki/ThroughputShapingTimer/">jp@gc Plugins for JMeter</a></li></ul><h3>Closing Note</h3><p>Feel free to share your experiences or ask questions in the comments below. Happy testing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c97e758a0b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling the Power of Think Time: Efficient Recordings in JMeter For API Performance Testing]]></title>
            <link>https://medium.com/@dhadiprasetyo/unveiling-the-power-of-think-time-efficient-recordings-in-jmeter-for-api-performance-testing-88770eaeb04d?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/88770eaeb04d</guid>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[jmeter]]></category>
            <category><![CDATA[api-testing]]></category>
            <category><![CDATA[stress-test]]></category>
            <category><![CDATA[load-testing]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sat, 30 Sep 2023 07:29:16 GMT</pubDate>
            <atom:updated>2023-09-30T07:29:16.666Z</atom:updated>
            <content:encoded><![CDATA[<p>Performance testing is a crucial aspect of ensuring that software applications meet the demands of real-world usage. Among the various tools available for performance testing, Apache JMeter stands out for its versatility and capabilities. One of the essential features in JMeter is the ability to record user interactions using a proxy server. However, for accurate performance testing, incorporating think time is paramount. In this article, we will explore the significance of think time and how to efficiently utilize it during recording in JMeter.</p><h3>Understanding Think Time</h3><p>Think time is the pause or delay between consecutive user actions in an application. It simulates how real users interact with an application, adding a level of realism to performance testing. When users interact with an application, they don’t instantly click on the next action; instead, they take a moment to process the information or perform an action.</p><p>In performance testing, replicating this behavior is crucial to generate accurate load scenarios. By incorporating think time during recording, we can emulate the natural rhythm of user interactions, ultimately leading to more realistic performance testing results.</p><h3>The Importance of Think Time in Performance Testing</h3><ol><li>Realism in Load Simulation: Think time replicates the actual behavior of users, ensuring that load testing closely resembles real-world scenarios. This helps in identifying potential performance issues that might occur in production.</li><li>Server Load Estimation: Accurate think time allows for a more precise estimation of the server load. It helps in simulating the load that the application will experience in the production environment, aiding in capacity planning.</li><li>Stress Testing Accuracy: During stress testing, simulating a sudden influx of users is more accurate with think time. It allows for a better understanding of how the application behaves under stressful conditions.</li></ol><h3>Incorporating Think Time in JMeter Recordings</h3><p>Let’s go through the steps to efficiently incorporate think time during the recording process in JMeter:</p><ol><li>Create a New Test Plan in JMeter:</li></ol><ul><li>Begin by initiating a new test plan in JMeter.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UWHdiDZOOKsQvqIgJCulKA.png" /></figure><p>2. Access the Template for Recording With Think Time:</p><ul><li>Navigate to the ‘File’ menu, then select ‘Templates,’ and opt for ‘Recording With Think Time.’ Click ‘Create’ to proceed.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*vaVEhWc2JCsEA9kIq8O_JA.gif" /></figure><p>3. Rename the Recording Controller:</p><ul><li>Expand the ‘Thread Group’ and select the ‘Recording Controller.’ Rename it accordingly by clicking on the controller.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*_dLp4QAbpUlgToGZJpAjRw.gif" /></figure><p>4. Configure the HTTP(S) Test Script Recorder:</p><ul><li>Access the ‘HTTP(S) Test Script Recorder.’ Adjust the ‘Target Controller’ within the ‘Test Plan Content’ section to point to the renamed ‘Login Test’ Recording Controller.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*UbSg5wB00mo9_MSnZCk01A.gif" /></figure><p>5. Apply URL Filters to Requests:</p><ul><li>Optionally, include or exclude specific URL patterns for your requests by navigating to the ‘Request Filtering’ section, located beside the ‘Test Plan Creation’ section.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aMBwuZ0oOq9GxY8y_EACjA.png" /></figure><p>6. Return to Test Plan Creation:</p><ul><li>Go back to the ‘Test Plan Creation’ section to continue configuring your test plan.</li></ul><h3>Recording HTTP(S) Request on Browser</h3><p>When it comes to recording HTTP(S) requests for performance testing, leveraging a web browser to mimic real user interactions is a popular approach. In this guide, we’ll demonstrate how to use Firefox for this purpose, given its ease of setup compared to Chrome.</p><ol><li>Launch the Firefox browser and access the settings.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*wANYGOuCk76k5hRwuDP5gw.gif" /></figure><p>2. Look for “Proxy settings” and select “Manual proxy configuration.” Set the HTTP proxy to “localhost” and the port to “8888” (aligning with the JMeter HTTP(S) Test Script Recorder port settings). Ensure to check “Also use this proxy for HTTPS” before clicking ‘OK’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UiM9y4pOSHwOYVQw4_W2mw.png" /><figcaption>Jmeter HTTP(S) Test Script Recorder</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LanFK18Gjk6hsTn9cvDxUw.png" /><figcaption>Firefox Proxy Setting</figcaption></figure><p>3. Setting up SSL Certificate:</p><ul><li>Navigate back to Firefox settings and search for “certificate.” Click on “View Certificates” and import the JMeter certificate found in “yourjmeterdirectorypath/bin/ApacheJMeterTemporaryRootCA.crt”.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*LBZYcTu_n9aIHyAuayz1zA.gif" /></figure><p>4. With the certificate imported, your Firefox browser is now ready to record network interactions within your web application.</p><p>5. Return to JMeter, navigate to the HTTP(S) Test Script Recorder, and click on the ‘Start’ button. This activates the JMeter proxy to record all network traffic from your web application, crucial for subsequent API performance testing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*q3pFUNOXzlExD_0Que-3tA.gif" /></figure><p>6. Open Firefox again and proceed to interact with your web application as you expect a user to. For instance, use a dummy login web application available at <a href="https://practicetestautomation.com/practice-test-login/">https://practicetestautomation.com/practice-test-login/</a> by Dmitry Shyshkin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*LyNVDCWMG6qrG38ZQvrxYQ.gif" /></figure><p>7. Once the necessary interactions are recorded, stop the recorder within JMeter by using the Transaction Control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*pfyHueIrfeYN9Klpm-9-MA.gif" /></figure><p>8. All HTTP(S) requests made in your web application are now recorded and available under your ‘Login Test Recording Controller’ in JMeter. Feel free to remove unnecessary requests and rename them to enhance clarity in your Listener.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*bOdNy3iUyQn0vk9iHDG89A.png" /></figure><p>9. Optionally, add assertions to each request, such as JSON assertion, HTML assertion, or Duration assertion, based on your testing requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2RkaBBZYWDemqrxukX3AWg.png" /></figure><p>10. Adjust the ‘Number of Threads (users),’ ‘Ramp-up period,’ ‘Loop count,’ and other settings within your Test Group to align with your performance testing objectives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*waMmwJWpkaRTZBhZBf5rgQ.png" /></figure><p>11. With all configurations in place, run your test and observe the results in the Result Tree Listener under your test plan.</p><p>By following these steps, you’ve effectively utilized Firefox in conjunction with JMeter to record HTTP(S) requests, paving the way for comprehensive API performance testing.</p><h3>Pro-Tips for Optimizing JMeter Performance Testing</h3><p>In the world of performance testing, every efficiency gained translates to a more accurate and insightful evaluation of your application’s capabilities. Here are some pro-tips to enhance your performance testing experience with JMeter:</p><ol><li>Utilize Terminal for Enhanced Efficiency: When executing performance tests, opt for running JMeter from the terminal instead of using the GUI. This approach maximizes efficiency and optimizes resource usage, making it the preferred method for serious performance testing.</li><li>Disable Listeners for Efficient Execution: When running JMeter tests from the terminal, ensure that any unnecessary Listeners are disabled. Listeners can consume significant resources and impact the accuracy of performance results. By disabling them, you allocate resources more efficiently for the actual testing processes.</li><li>Leverage Multiple Recording Controllers: To enhance the versatility and thoroughness of your test scenarios, feel free to add multiple Recording Controllers within your Thread Group. This allows you to record various user interactions efficiently.</li></ol><p>By implementing these pro-tips, you’ll maximize the potential of JMeter for performance testing, ensuring accurate results and efficient resource usage.</p><h3>Best Practices for Efficient Think Time Implementation</h3><ol><li>Realistic Think Time: Aim for think time values that closely match the actual behavior of users within the application.</li><li>Vary Think Time: Introduce variability in think time by using random timers. This mimics the diverse behavior of users and enhances the accuracy of load simulation.</li><li>Constant Monitoring and Adjustment: Continuously monitor and adjust think time values based on application changes, user feedback, or performance tuning efforts.</li></ol><h3>Conclusion</h3><p>Incorporating think time during recording in JMeter is essential for accurate performance testing that mirrors real-world user behavior. By understanding the significance of think time and following best practices, you can enhance the effectiveness of your performance testing efforts, identify potential performance bottlenecks, and ensure a seamless user experience for your application. Remember, precise think time integration is the key to unlocking the full potential of JMeter in your performance testing endeavors. Happy testing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88770eaeb04d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing a Robust Retry Mechanism for Failed Tests in Cucumber with Java]]></title>
            <link>https://medium.com/@dhadiprasetyo/implementing-a-robust-retry-mechanism-for-failed-tests-in-cucumber-with-java-23c8230e6488?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/23c8230e6488</guid>
            <category><![CDATA[quality-assurance]]></category>
            <category><![CDATA[bdd-testing]]></category>
            <category><![CDATA[testng]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sun, 24 Sep 2023 12:57:32 GMT</pubDate>
            <atom:updated>2023-09-24T12:57:32.347Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uWSHBsGiMZ582AwE" /></figure><p>In the realm of software testing, ensuring the stability and reliability of automated tests is paramount. One challenge testers often face is dealing with intermittent test failures. These failures may occur due to various factors such as network latency, system resource constraints, or occasional server issues. To mitigate the impact of such intermittent failures, implementing a robust retry mechanism can be a valuable approach. In this article, we will explore how to integrate a retry mechanism for failed tests using Cucumber with Java.</p><h3>Understanding the Need for a Retry Mechanism</h3><p>Automated tests may occasionally fail due to transient issues that are beyond the control of the test script or the application being tested. These failures can be frustrating, especially if the failures are intermittent and not reproducible.</p><p>By implementing a retry mechanism, we can automatically rerun the failed tests a predefined number of times, allowing for potentially successful subsequent attempts. This helps in achieving more reliable and stable test results, particularly in a continuous integration/continuous deployment (CI/CD) environment.</p><h3>Setting Up the Project</h3><p>To demonstrate the retry mechanism, let’s assume we have a Java project with Cucumber for behavior-driven development (BDD) and Maven/Gradle for dependency management. Ensure you have the necessary dependencies for Cucumber and any other testing libraries.</p><h3>Implementing the Retry Logic</h3><p>To utilize the rerun plugin, configure the Cucumber runner with the appropriate plugin option:</p><pre>package cucumber.runner;<br><br>import io.cucumber.testng.AbstractTestNGCucumberTests;<br>import io.cucumber.testng.CucumberOptions;<br><br>@CucumberOptions(<br>        features = &quot;src/test/java/cucumber/feature/Login.feature&quot;,<br>        glue = {&quot;cucumber.stepdefinitions&quot;},<br>        tags = &quot;@LoginTest&quot;,<br>        plugin = {&quot;rerun:target/failedrerun.txt&quot;}<br>)<br>public class LoginRunner extends AbstractTestNGCucumberTests {<br>}</pre><p>In this example, we’ve configured the plugin option with &quot;rerun:target/failedrerun.txt&quot;. This tells Cucumber to write information about failed scenarios to the failedrerun.txt file in the target directory.</p><p>Please note that you have the flexibility to customize the name of the rerun text file according to your specific requirements.</p><h3>Rerunning Failed Scenarios</h3><p>To rerun the failed scenarios, create another test runner specifically for reruns:</p><pre>package cucumber.runner;<br><br>import io.cucumber.testng.AbstractTestNGCucumberTests;<br>import io.cucumber.testng.CucumberOptions;<br><br>@CucumberOptions(<br>        features = &quot;@target/failedrerun.txt&quot;,<br>        glue = {&quot;cucumber.stepdefinitions&quot;}<br>)<br>public class FailedTestRerun extends AbstractTestNGCucumberTests {<br>}</pre><p>In this runner, we’ve specified features = &quot;@target/failedrerun.txt&quot;, which instructs Cucumber to run the scenarios listed in the failedrerun.txt file. Cucumber will now rerun only the failed scenarios from the previous test run.</p><p>You can utilize a TestNG XML suite to initiate the rerun of failed tests in the following manner:</p><pre>&lt;!DOCTYPE suite SYSTEM &quot;https://testng.org/testng-1.0.dtd&quot; &gt;<br><br>&lt;suite name=&quot;Our Suite&quot; verbose=&quot;1&quot; &gt;<br>    &lt;test name=&quot;YouTube test on Chrome&quot;&gt;<br>        &lt;classes&gt;<br>            &lt;class name=&quot;cucumber.runner.LoginRunner&quot; /&gt;<br>        &lt;/classes&gt;<br>    &lt;/test&gt;<br>    &lt;test name=&quot;Rerun failed tests&quot;&gt;<br>        &lt;classes&gt;<br>            &lt;class name=&quot;cucumber.runner.FailedTestRerun&quot; /&gt;<br>        &lt;/classes&gt;<br>    &lt;/test&gt;<br>&lt;/suite&gt;</pre><p>You can also trigger the test runner for failed tests using a Shell script in the Jenkins build steps, as illustrated below:</p><pre># Specify the Maven/Gradle command to execute your Cucumber test runner below<br># e.g: <br>gradle cleanTest test or mvn test<br><br># Then after all your tests are executed, Cucumber will create a txt file for failed tests<br><br># Check if the file exists<br>filename=&quot;/var/lib/jenkins/workspace/your-workspace-name/failedrerun.txt&quot;<br>if [ -e &quot;$filename&quot; ]; then<br>    # Check if the file size is greater than 0 bytes<br>    if [ -s &quot;$filename&quot; ]; then<br>        echo &quot;The file &#39;$filename&#39; exists and its size is greater than 0 bytes.&quot;<br>        # put your maven/gradle test command here to rerun failed tests<br>        # e.g: <br>        gradle cleanTest test -P reruntest or mvn test -Dtest=FailedTestRerun<br>    else<br>        echo &quot;The file &#39;$filename&#39; exists, but its size is 0 bytes.&quot;<br>    fi<br>else<br>    echo &quot;The file &#39;$filename&#39; does not exist.&quot;<br>fi</pre><h3>Conclusion</h3><p>Rerunning failed scenarios using the rerun plugin in Cucumber is a straightforward and efficient approach to improving the reliability of your automated tests. By giving failed scenarios a second chance to pass, you increase the robustness of your test suite and gain more confidence in the quality of your application. Integrate this approach into your testing process to achieve more stable and dependable automated tests.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23c8230e6488" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Comprehensive Guide to Serialization and Deserialization of POJO in Rest Assured]]></title>
            <link>https://medium.com/@dhadiprasetyo/a-comprehensive-guide-to-serialization-and-deserialization-of-pojo-in-rest-assured-c70993985a6?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/c70993985a6</guid>
            <category><![CDATA[testng]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[plain-old-java-object]]></category>
            <category><![CDATA[restassured]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sat, 16 Sep 2023 16:17:36 GMT</pubDate>
            <atom:updated>2023-09-16T16:17:36.614Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re unfamiliar with API testing using Rest Assured, I recommend starting with this introductory guide from my previous article:</p><p><a href="https://medium.com/@dhadiprasetyo/automate-api-testing-with-java-rest-assured-and-testng-ba48dd736e61">Automate API Testing with Java, Rest Assured, and TestNG</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZskF8WnoyV4Xfd38IVvTJA.jpeg" /></figure><p>Serialization and deserialization are crucial aspects of working with APIs, especially in the realm of automated testing. They involve converting Java objects into a format that can be easily transmitted over a network, typically as JSON or XML, and then converting the received response (JSON or XML) back into Java objects. This process is essential for efficiently handling data between the application and the API. In this guide, we’ll explore how to achieve serialization and deserialization of Plain Old Java Objects (POJOs) using Rest Assured, a popular Java-based library for testing RESTful APIs.</p><h3>Understanding Serialization and Deserialization</h3><p>Serialization is the process of converting a Java object into a format that can be easily transmitted over a network, such as JSON or XML. This allows the object’s state to be saved or sent across the network.</p><p>Deserialization is the reverse process, where the serialized JSON or XML data is converted back into a Java object, enabling you to work with the data in your Java application.</p><p>For effective testing using Rest Assured, it’s crucial to handle API responses as Java objects (POJOs) to make assertions and verifications easier.</p><h3>Creating a POJO (Plain Old Java Object)</h3><p>Let’s start by creating a simple POJO class that represents the structure of the data we expect to send or receive from the API.</p><pre>public class User {<br>    private String name;<br>    private String email;<br><br>    // Getters and setters<br>    // ...<br>}</pre><h3>Serialization — Converting POJO to JSON</h3><p>To serialize a POJO into JSON using Rest Assured, you can use the ObjectMapper class from the Jackson library, which Rest Assured utilizes internally.</p><pre>import com.fasterxml.jackson.databind.ObjectMapper;<br><br>// Create a User object<br>User user = new User();<br>user.setName(&quot;John Doe&quot;);<br>user.setEmail(&quot;john.doe@example.com&quot;);<br><br>// Use ObjectMapper to serialize the User object to JSON<br>ObjectMapper objectMapper = new ObjectMapper();<br>String json = objectMapper.writeValueAsString(user);<br><br>System.out.println(&quot;Serialized JSON: &quot; + json);</pre><p>In this example, the ObjectMapper is used to serialize the User object into a JSON string.</p><h3>Deserialization — Converting JSON to POJO</h3><p>Conversely, to deserialize a JSON response into a POJO, you’ll use the ObjectMapper to map the JSON data to your POJO&#39;s fields.</p><pre>// Sample JSON response<br>String jsonResponse = &quot;{\&quot;name\&quot;:\&quot;Alice\&quot;, \&quot;email\&quot;:\&quot;alice@example.com\&quot;}&quot;;<br><br>// Use ObjectMapper to deserialize the JSON response to a User object<br>User deserializedUser = objectMapper.readValue(jsonResponse, User.class);<br><br>System.out.println(&quot;Deserialized User: &quot; + deserializedUser.getName() + &quot;, &quot; + deserializedUser.getEmail());</pre><p>In this example, the ObjectMapper is used to deserialize the JSON response into a User object.</p><h3>Integrating Serialization and Deserialization with Rest Assured using Lombok</h3><p>When integrating serialization and deserialization with Rest Assured, using Lombok can streamline the creation of POJOs by reducing boilerplate code for getter, setter, and constructor methods. Lombok’s annotations can enhance code readability and maintainability. In this section, we’ll demonstrate how to use Lombok annotations with Rest Assured for serialization and deserialization.</p><p>To use Lombok in your project, you need to include the <a href="https://mvnrepository.com/artifact/org.projectlombok/lombok">Lombok dependency</a> in your Maven pom.xml or Gradle file.</p><p>Maven:</p><pre>&lt;!-- https://mvnrepository.com/artifact/org.projectlombok/lombok --&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.projectlombok&lt;/groupId&gt;<br>    &lt;artifactId&gt;lombok&lt;/artifactId&gt;<br>    &lt;version&gt;1.18.28&lt;/version&gt;<br>    &lt;scope&gt;provided&lt;/scope&gt;<br>&lt;/dependency&gt;</pre><p>Gradle:</p><pre>// https://mvnrepository.com/artifact/org.projectlombok/lombok<br>compileOnly &#39;org.projectlombok:lombok:1.18.28&#39;</pre><h3>Creating a Lombok-Enhanced POJO</h3><p>Let’s create a User class using Lombok annotations to reduce boilerplate code:</p><pre>import lombok.Data;<br><br>@Data<br>public class User {<br>    private String name;<br>    private String email;<br>}</pre><p>In this example, the @Data annotation from Lombok automatically generates the getter, setter, and toString methods for the User class.</p><h3>Integrating with Rest Assured</h3><p>In this section, we’ll demonstrate how to use Rest Assured along with Lombok-enhanced POJOs for making a POST request and asserting the expected response.</p><p>For instance, consider the following JSON body for our request:</p><pre>{<br>  &quot;name&quot;: &quot;Bob&quot;,<br>  &quot;email&quot;: &quot;bob@example.com&quot;<br>}</pre><p>Assuming that the request will yield a response echoing the structure of the sent request body, we can expect a response in the following format:</p><pre>{<br>  &quot;name&quot;: &quot;Bob&quot;,<br>  &quot;email&quot;: &quot;bob@example.com&quot;<br>}</pre><p>This illustrates a scenario where the API response mirrors the structure and content of the request body.</p><p>Now, let’s apply this in our Rest Assured test:</p><pre>package com.example.demo.api.ordertests.stepdefinitions;<br><br>import com.example.demo.model.User;<br>import com.fasterxml.jackson.core.JsonProcessingException;<br>import com.fasterxml.jackson.databind.ObjectMapper;<br>import io.restassured.http.ContentType;<br>import io.restassured.response.Response;<br>import org.testng.annotations.Test;<br><br>import static io.restassured.RestAssured.given;<br><br>public class TestPojo {<br>    @Test<br>    public void testLoginUser() throws JsonProcessingException {<br>        User user = new User();<br>        user.setEmail(&quot;bob@example.com&quot;);<br>        user.setName(&quot;Bob&quot;);<br><br>        ObjectMapper objectMapper = new ObjectMapper();<br><br>        // Let&#39;s serialize this User object into JSON<br>        String loginBody = objectMapper.writeValueAsString(user);<br>        <br>        Response responseLogin = given().baseUri(&quot;https://api.example.com&quot;)<br>                .contentType(ContentType.JSON)<br>                .body(loginBody).when().post(&quot;/users&quot;)<br>                .then().assertThat().statusCode(200)<br>                .extract().response();<br><br>        // Then let&#39;s deserialize the response body back into User object<br>        User createdUser = objectMapper.readValue(responseLogin.getBody().asString(), User.class);<br><br>        // Then do assertions<br>        Assert.assertEquals(user.getEmail(), createdUser.getEmail());<br>        Assert.assertEquals(user.getName(), createdUser.getName());<br>    }<br>}</pre><p>In this example, we first serialize a User object to JSON using the ObjectMapper. Then, we make a POST request with Rest Assured, sending the serialized JSON in the request body. Finally, we deserialize the response JSON into a User object for further assertions.</p><h3>Conclusion</h3><p>Serialization and deserialization of POJOs are essential processes in API testing. They allow for effective communication between the Java application and the API, enabling the seamless transfer of data. By utilizing Rest Assured in conjunction with serialization and deserialization techniques, you can efficiently automate testing and validation of RESTful APIs, ensuring the functionality and integrity of your application.</p><h3>Bonus Tips</h3><p>When working with fields that consist of more than one word separated by underscores, such as “error_code,” ensure proper mapping using @JsonProperty(&quot;error_code&quot;) for better readability and mapping in your POJO:</p><pre>@JsonProperty(&quot;error_code&quot;)<br>private Integer errorCode;</pre><p>Also, think about adding validation rules to your fields using annotations like @NotNull, @NotBlank, @Max, @Size, and similar ones. These annotations help enforce specific rules and constraints for the data stored in your Plain Old Java Objects (POJOs). I’m planning to include these validation checks in my next Medium article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c70993985a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Beginner’s Guide to MySQL Queries: From SELECT to Aggregations]]></title>
            <link>https://medium.com/@dhadiprasetyo/a-beginners-guide-to-mysql-queries-from-select-to-aggregations-913b03480ca3?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/913b03480ca3</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[query]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Mon, 04 Sep 2023 14:29:29 GMT</pubDate>
            <atom:updated>2023-09-04T14:29:57.487Z</atom:updated>
            <content:encoded><![CDATA[<p>MySQL is one of the most popular relational database management systems used by developers and businesses worldwide. Whether you’re building a website, a mobile app, or working on a data analytics project, understanding how to write MySQL queries is an essential skill. In this article, I will take you through the basics of MySQL queries, starting from the SELECT statement and progressing to more advanced topics like aggregations with SUM and COUNT. By the end, you’ll have a solid foundation to start querying databases with confidence.</p><h3>1. SELECT Statement</h3><p>The SELECT statement is the fundamental building block of MySQL queries, just like choosing balls from a box. It allows you to fetch data from a database table. Here’s a simple example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*segKjVG7cat-6vfbne9LMA.png" /></figure><ul><li>Imagine we have a container of balls which have different colors and size. Each ball will have their own ID which indicate the order of when they are put in the container. Now instead of calling it a container, let’s call it a table.</li><li>So here is what your table will look like with data of each ball:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JjG1pVELWFEmc-BczTYo1A.png" /></figure><ul><li>Using MySQL query you want to fetch all the ball from the table, here is where you need to use <strong><em>SELECT statement as you are selecting the balls from their table</em></strong>.</li></ul><pre>SELECT * FROM balls;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kEut0IH__a5gcn8c2D1vcQ.png" /></figure><ul><li>When writing query syntax always imagine or think about what field you want to see. Now instead of fetching all fields, let’s only fetch color and size.</li></ul><pre>SELECT color, size FROM balls;</pre><ul><li>Now let’s take a look at the result of our query. Notice that the id and volume field is now not showing up because we are not selecting it and we <strong>only want to see color and size</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKRoY1bA5J1Ha8JVNk0xTA.png" /></figure><h3>2. WHERE Clause</h3><ul><li>To filter the results based on specific conditions, you can use the WHERE clause, much like finding special balls with stickers. For instance, if you want to retrieve only the red balls:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OWZAgnm9oEuWYeISM1CVHQ.png" /></figure><pre>SELECT * FROM balls WHERE color = &#39;red&#39;;</pre><ul><li>Always put in mind that writing query syntax is just like saying words in english. For example: <strong><em>I want to select all balls from balls table where the color is red</em></strong>. Now you will see only Red color balls is displayed.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*116rA10B5lth9ZUInzdLxA.png" /></figure><ul><li>Now let’s try to filter your query where the volume is bigger than 10. Here the volume field has data type of decimal. For number data types such as integer and decimal you can use comparison operators such as less than (&lt;), bigger than (&gt;), equals to (=) and many more. Remember not to use single or double quotes in your where clause when referring to numerical data types.</li></ul><pre>SELECT * FROM balls WHERE volume &gt; 10;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JRYXX2wYykVo7mgkPiiX7A.png" /></figure><ul><li>Here you will see that all the results have volume bigger than 10.</li></ul><h3>3. ORDER BY Clause</h3><ul><li>The ORDER BY clause allows you to sort the results in ascending or descending order, just as you would organize the balls in the box. To sort balls by their volume in descending order:</li></ul><pre>SELECT * FROM balls ORDER BY volume DESC;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fs6Kwsp6Q5S3-MKip2zsnA.png" /></figure><ul><li>You will notice that now the datas in table are ordered by volume in descending (from largest volume to smallest volume). If you want to order it in ascending, simply replace the DESC keyword with ASC (or just remove the DESC, by default it will order your datas in ascending).</li></ul><pre>SELECT * FROM balls ORDER BY volume ASC; -- first ascending option<br>SELECT * FROM balls ORDER BY volume; -- second ascending option</pre><h3>4. LIMIT Clause</h3><ul><li>Sometimes, you may want to limit the number of rows returned, similar to taking only a few balls from the box. The LIMIT clause is handy for this purpose. To get the top 5 largest balls:</li></ul><pre>SELECT * FROM balls ORDER BY volume DESC LIMIT 5;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VajAVrJfqe8F4QMqMSwhbw.png" /></figure><ul><li>Using limit keyword is sometimes very important to optimize your query performance. Always use limit if you only want to take a look the preview of your datas or to take some sample.</li></ul><h3>5. Basic Aggregations</h3><p>Now, let’s introduce basic essential operations for analyzing data in MySQL: SUM, COUNT, AVG, MIN and MAX.</p><p><strong><em>SUM</em></strong> operation:</p><ul><li>is like asking someone to add up the volume of all the balls in the box. For example, to find the total volume of all the balls:</li></ul><pre>SELECT SUM(volume) FROM balls;</pre><ul><li>This query will add up the values of all the balls in the “balls” table and give you the total.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQuNrqAPBpuSlJWy_97yOA.png" /></figure><p><strong><em>COUNT operation:</em></strong></p><ul><li>on the other hand, is like counting how many balls are in the box. To find out how many balls you have in total:</li></ul><pre>SELECT COUNT(*) FROM balls;</pre><ul><li>This query will count all the rows (balls) in the “balls” table and give you the total count.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7aC8VE_aUPcWPLvh0xYkfw.png" /></figure><p>Before we continue learning about aggregations like AVG, MIN and MAX let’s take a look at GROUP BY clause explanation.</p><h3>6. GROUP BY Clause</h3><p>The GROUP BY clause is a powerful feature in SQL that allows you to group rows with similar values into summary rows. It’s commonly used with aggregation functions like SUM, COUNT, AVG, MAX, and MIN to perform calculations on grouped data. Imagine you have a box of colorful balls, and you want to group them by color to count how many you have of each color.</p><h4>Example — Grouping by Color</h4><p>Suppose you have a “balls” table with columns for “color,” “size,” and “volume.” You can use GROUP BY to count how many balls you have of each color:</p><pre>SELECT color, COUNT(*) as count<br>FROM balls<br>GROUP BY color;</pre><p>In this query:</p><ul><li>color is the column by which we want to group the rows.</li><li>COUNT(*) is an aggregation function that counts the number of rows in each group.</li><li>GROUP BY color instructs MySQL to group the rows based on the &quot;color&quot; column.</li></ul><p>The result will look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_1J5EgY7NOaCP67WMpXdBA.png" /></figure><h3>7. Basic Aggregations (Cont.)</h3><p><strong><em>AVG</em></strong> function:</p><ul><li>calculates the average of values within each group. For example, to find the average volume of balls in each color group:</li></ul><pre>SELECT color, AVG(volume)<br>FROM balls<br>GROUP BY color;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kFXHiYdcBg8ncmLAu_-ORg.png" /></figure><p><strong><em>MIN</em></strong><em> function:</em></p><ul><li>finds the minimum (smallest) value within each group. To find the smallest volume of balls in each color group:</li></ul><pre>SELECT color, MIN(volume)<br>FROM balls<br>GROUP BY color;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6qVbfrItnEo4mPbgfNhWmg.png" /></figure><p><strong><em>MAX</em></strong><em> function:</em></p><ul><li>finds the maximum (largest) volume within each group. To find the largest volume of balls in each color group:</li></ul><pre>SELECT color, MAX(volume)<br>FROM balls<br>GROUP BY color;</pre><p>By combining GROUP BY with aggregation functions, you can gain valuable insights into your dataset, such as the average, smallest, and largest values within each group. This is a valuable skill for anyone working with databases and data analysis.</p><h3>Conclusion</h3><p>MySQL queries are a crucial skill for anyone working with relational databases. Starting with the basics of SELECT, WHERE, ORDER BY, and LIMIT, you can retrieve and manipulate data effectively. As you progress, understanding aggregations like SUM and COUNT becomes essential for data analysis. By mastering these concepts, you’ll be well on your way to becoming proficient in MySQL and database management.</p><p>Remember that practice makes perfect, so don’t hesitate to experiment with these queries on your own databases. The more you work with MySQL, the more comfortable and proficient you’ll become in crafting queries and performing operations to retrieve and analyze the data you need. Happy querying!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=913b03480ca3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Asserting JSON Schema for API Testing with Java and RestAssured]]></title>
            <link>https://medium.com/@dhadiprasetyo/asserting-json-schema-for-api-testing-with-java-and-restassured-79b4a851f282?source=rss-8a6bdcde59a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/79b4a851f282</guid>
            <category><![CDATA[api-testing]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[restassured]]></category>
            <category><![CDATA[json-schema]]></category>
            <category><![CDATA[automation-testing]]></category>
            <dc:creator><![CDATA[Darmawan Hadiprasetyo]]></dc:creator>
            <pubDate>Sun, 03 Sep 2023 15:47:37 GMT</pubDate>
            <atom:updated>2023-09-03T15:47:37.755Z</atom:updated>
            <content:encoded><![CDATA[<p>API testing is a critical aspect of modern software development, ensuring that the application’s back-end functions correctly and consistently. When dealing with APIs, one of the essential tasks is to validate the structure of the data exchanged between the client and the server. JSON Schema, a powerful tool for defining the structure of JSON data, can be incredibly valuable in this regard. In this article, we will explore how to assert JSON Schema for API testing using Java and RestAssured, a popular testing library.</p><h3>What is JSON Schema?</h3><p>JSON Schema is a specification that defines the structure, format, and constraints of JSON documents. It provides a way to describe the expected structure of JSON data, making it a valuable tool for ensuring the integrity of data exchanged between API clients and servers.</p><p>JSON Schema uses a JSON object to define the rules and constraints that JSON data must adhere to. These rules can include data types, required fields, allowed values, and more. By validating JSON data against a JSON Schema, you can ensure that it conforms to the expected format and catch potential issues early in the development process.</p><h3>JSON Schema Structure</h3><ol><li>JSON Schema Version: The JSON Schema document should begin by specifying the version of the JSON Schema specification it adheres to. This is done using the &quot;$schema&quot; keyword, which indicates the URI of the schema version being used. For example:</li></ol><pre>{<br>    &quot;$schema&quot;: &quot;http://json-schema.org/draft-07/schema#&quot;,<br>    // Rest of the schema definition<br>}</pre><p>The URL may vary depending on the JSON Schema version you’re using.</p><p>2. Type Definitions: The core of a JSON Schema defines the data types and constraints for the JSON document. The most common data types include:</p><ul><li>&quot;object&quot;: Represents a JSON object (a collection of key-value pairs).</li><li>&quot;array&quot;: Represents a JSON array (an ordered list of values).</li><li>&quot;string&quot;: Represents a JSON string.</li><li>&quot;number&quot;: Represents a JSON number (integer or floating-point).</li><li>&quot;boolean&quot;: Represents a JSON boolean (true or false).</li><li>&quot;null&quot;: Represents a JSON null value.</li></ul><p>You can use the &quot;type&quot; keyword to specify the expected data type:</p><pre>{<br>    &quot;$schema&quot;: &quot;http://json-schema.org/draft-07/schema#&quot;,<br>    &quot;type&quot;: &quot;object&quot;,<br>    // Rest of the schema definition<br>}</pre><p>3. Properties and Items: For objects and arrays, you can define their properties and items. Properties describe the expected key-value pairs in an object, while items describe the expected elements in an array. You can use the &quot;properties&quot; and &quot;items&quot; keywords to define these, respectively:</p><p>For objects:</p><pre>{<br>    &quot;$schema&quot;: &quot;http://json-schema.org/draft-07/schema#&quot;,<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;properties&quot;: {<br>        &quot;name&quot;: {<br>            &quot;type&quot;: &quot;string&quot;<br>        },<br>        &quot;age&quot;: {<br>            &quot;type&quot;: &quot;integer&quot;<br>        }<br>    }<br>}</pre><p>For arrays:</p><pre>{<br>    &quot;$schema&quot;: &quot;http://json-schema.org/draft-07/schema#&quot;,<br>    &quot;type&quot;: &quot;array&quot;,<br>    &quot;items&quot;: {<br>        &quot;type&quot;: &quot;string&quot;<br>    }<br>}</pre><p>4. Required Properties: You can specify which properties within an object are required using the &quot;required&quot; keyword:</p><pre>{<br>    &quot;$schema&quot;: &quot;http://json-schema.org/draft-07/schema#&quot;,<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;properties&quot;: {<br>        &quot;name&quot;: {<br>            &quot;type&quot;: &quot;string&quot;<br>        },<br>        &quot;age&quot;: {<br>            &quot;type&quot;: &quot;integer&quot;<br>        }<br>    },<br>    &quot;required&quot;: [&quot;name&quot;]<br>}</pre><p>5. Additional Constraints: JSON Schemas allow you to specify additional constraints, such as minimum and maximum values for numbers, regular expressions for strings, and more, depending on the data type. These constraints are specified using various keywords specific to the data type. To understand more about this please refer to this doc: <a href="https://json-schema.org/understanding-json-schema/reference/">https://json-schema.org/understanding-json-schema/reference/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*5LiaSERWkSwZmcHqYjFJpw.png" /></figure><h3>How To Generate JSON Schema</h3><ol><li>Prepare your JSON data. For example we will be using this JSON data:</li></ol><pre>{<br>  &quot;order_id&quot;: &quot;1234567890&quot;,<br>  &quot;order_description&quot;: &quot;sample description&quot;,<br>  &quot;order_status&quot;: &quot;Processed&quot;,<br>  &quot;last_updated_timestamp&quot;: &quot;1671916800000&quot;,<br>  &quot;special_order&quot;: false<br>}</pre><p>2. To make it easier, we will be using online tools to generate our JSON Schema. Now open <a href="https://www.liquid-technologies.com/online-json-to-schema-converter">Liquid Technologies JSON to JSON Schema Converter</a> in your browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PbJBtXSz3zVvzATaA4Wy9g.png" /></figure><p>3. Paste your JSON into Sample JSON Document field and click Generate Schema.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l4IUomvnJ_2ZZDw7y_rDoA.png" /></figure><p>4. Copy the generated JSON Schema and store it in your Java project under src/test/resources/<strong><em>yourjsonschema</em></strong>.json using json file type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*341oXNj4L0sCIb8A0GkA9A.png" /></figure><pre>├── src<br>│   ├── main<br>│   │   ├── java<br>│   │   └── resources<br>│   └── test<br>│       ├── java<br>│       └── resources<br>│           └── yourjsonschema.json // here is your json schema located</pre><p>5. Review the generated JSON Schema carefully. It will include data types, properties, and constraints based on the data you provided. Depending on the tool, you may have options to modify or customize the generated schema. This can include adding additional constraints, changing property names, or specifying default values.</p><h3>Asserting JSON Schema with RestAssured</h3><ol><li>Add JSON Schema Validator dependency to your Java project. <a href="https://mvnrepository.com/artifact/io.rest-assured/json-schema-validator/5.3.1">https://mvnrepository.com/artifact/io.rest-assured/json-schema-validator/5.3.1</a></li><li>In this tutorial I have provided you with mock JSON server using this url <a href="https://my-json-server.typicode.com/darmawanh/processOrder/success">https://my-json-server.typicode.com/darmawanh/processOrder/success</a>.</li><li>You can try perform GET request to that url.</li><li>Now let’s implement it all into a rest assured test:</li></ol><pre>@Test<br>public void testJsonSchema() {<br>    String baseUri = &quot;https://my-json-server.typicode.com&quot;;<br><br>    given().baseUri(baseUri)<br>            .when().get(&quot;/darmawanh/processOrder/success&quot;)<br>            .then().assertThat().statusCode(200)<br>            .body(JsonSchemaValidator.matchesJsonSchemaInClasspath(&quot;yourjsonschema.json&quot;))<br>            .extract().response();<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*jRj3KW8Gt2pESCOEtzmVOA.gif" /></figure><p>Code explanation:</p><ul><li>In the above example, we’re using JsonSchemaValidator.matchesJsonSchemaInClasspath to validate the API response against a JSON Schema located in your classpath. Make sure to replace &quot;yourjsonschema.json&quot; with the actual path to your JSON Schema file.</li></ul><p>5. Another approach to validate our JSON schema is by using org.hamcrest.MatcherAssert. Using this method allow us to add message to our assertion for a better and readable error in our reporting tool.</p><pre>@Test<br>public void testJsonSchema() {<br>    String baseUri = &quot;https://my-json-server.typicode.com&quot;;<br><br>    Response response = given().baseUri(baseUri)<br>            .when().get(&quot;/darmawanh/processOrder/success&quot;)<br>            .then().assertThat().statusCode(200)<br>            .extract().response();<br><br>    MatcherAssert.assertThat(<br>            &quot;Validate json schema&quot;,<br>            response.getBody().asString(),<br>            JsonSchemaValidator.matchesJsonSchemaInClasspath(&quot;yourjsonschema.json&quot;)<br>    );<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Ld1FyGqyKaSqf_ZAcX_Vig.gif" /></figure><h3>Conclusion</h3><p>Asserting JSON Schema for API testing with Java and RestAssured is a valuable practice that helps ensure data consistency and integrity in your API interactions. By adding JSON Schema validation to your API tests, you can catch potential issues early in the development process and provide more robust and reliable APIs to your users.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79b4a851f282" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>