<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bilgehan Aygün on Medium]]></title>
        <description><![CDATA[Stories by Bilgehan Aygün on Medium]]></description>
        <link>https://medium.com/@bilgehanaygun?source=rss-947355e812ee------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*l49u2lKx4UXu9t3lHgxfEg.jpeg</url>
            <title>Stories by Bilgehan Aygün on Medium</title>
            <link>https://medium.com/@bilgehanaygun?source=rss-947355e812ee------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:51:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bilgehanaygun/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Http Connection Pool TTL Trap: Debugging Intermittent API Failures]]></title>
            <link>https://levelup.gitconnected.com/http-connection-pool-ttl-trap-debugging-intermittent-api-failures-3b48ead33694?source=rss-947355e812ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b48ead33694</guid>
            <category><![CDATA[pod-lifecycle]]></category>
            <category><![CDATA[graceful-shutdown]]></category>
            <category><![CDATA[connection-pool]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[nosuchhttpexception]]></category>
            <dc:creator><![CDATA[Bilgehan Aygün]]></dc:creator>
            <pubDate>Tue, 31 Mar 2026 15:46:52 GMT</pubDate>
            <atom:updated>2026-09-27T12:44:04.021Z</atom:updated>
            <content:encoded><![CDATA[<h3>Http ConnectionPooling TTL Trap: Intermitten Api Failures</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_BRwVB9NZunsCG-pnZhU-g.png" /></figure><h3>Problem Definition</h3><p>We had been experiencing an unique kind of problem between two services: The one performing API call was throwing <strong>NoHttpResponseException.<br></strong> Interestingly: the api calls that are triggered right after the erroneous one are succeeding: Meaning, somehow; a one time error was happening periodically which is not related to the application level but more like infrastructure level.</p><h3>Root Cause Detection</h3><p>The exception itself also strictly indicates that the request is not handled by any http server, in other terms the underlying TCP connection was already closed!</p><p>As I dug deeper: I have noticed a common pattern between the target api’s pod lifecycle and the exception’s occurring times.</p><p>The target-api is a high-scale service handling millions of requests and has ~15 deployments accross two kubernetes clusters. As I query the pod’s lifecycle on NewRelic, I realized a pattern: every single exception is occured at times where at least one pod of target-api was dying and a new one was being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dq88SR2R0stjZj51s5OkXA.png" /></figure><p>This immediately made me think of that there was a problem with the graceful shutdown of the target-api. So I checked the pod’s logs and it’s lifecycle more closely but I’ve seen nothing suspicious. The logs were obvious:</p><p>1- Pod receives SIGTERM, sets Readiness=DOWN.<br>2- It sleeps for configured sleep period (15s).<br>3- As the sleep period is over, it waits for active requests to complete.<br>4- It starts closing Tomcat (Spring’s Http Server), Kafka Producer etc.</p><pre>{&quot;message&quot;:&quot;Health check is entering unhealthy state sleeping for 15000 ms&quot;,&quot;service_name&quot;:&quot;target-api&quot;,&quot;thread.name&quot;:&quot;SpringApplicationShutdownHook&quot;,&quot;timestamp&quot;:&quot;2026-03-28T15:39:49.546254912Z&quot;, &quot;level&quot;:&quot;INFO&quot;}<br>{&quot;message&quot;:&quot;Health check sleep is done&quot;,&quot;service_name&quot;:&quot;target-api&quot;,&quot;thread.name&quot;:&quot;SpringApplicationShutdownHook&quot;,&quot;timestamp&quot;:&quot;2026-03-28T15:40:04.546565862Z&quot;, &quot;level&quot;:&quot;INFO&quot;}<br>{&quot;message&quot;:&quot;Commencing graceful shutdown. Waiting for active requests to complete&quot;,&quot;service_name&quot;:&quot;target-api&quot;,&quot;thread.name&quot;:&quot;SpringApplicationShutdownHook&quot;,&quot;timestamp&quot;:&quot;2026-03-28T15:40:04.548526775Z&quot;, &quot;level&quot;:&quot;INFO&quot;}<br>{&quot;message&quot;:&quot;Graceful shutdown complete&quot;,&quot;service_name&quot;:&quot;target-api&quot;,&quot;thread.name&quot;:&quot;tomcat-shutdown&quot;,&quot;timestamp&quot;:&quot;2026-03-28T15:40:04.551124375Z&quot;, &quot;level&quot;:&quot;INFO&quot;}</pre><p>So everything regarding to the target-api’s graceful shutdown seemed complete. This directed me to check the configurations of the caller-api.</p><h4>More Root Cause Detection!</h4><p>Even though there were multiple services that are calling the target-api , only a specific caller service was experiencing the error, and only it had <strong>connection-pooling enabled </strong>with below<strong> </strong>configurations that none of the other services had:</p><pre>      httpclient:<br>        hc5:<br>          enabled: true<br>        max-connections: 50<br>        max-connections-per-route: 25<br>        time-to-live: 30<br>        time-to-live-unit: SECONDS</pre><p>Somehow; this had to be the cause of the problem. I first thought it was somehow resolving the “dying” pod’s IP because of the connection-pooling since:</p><p><strong>If there would be no connection-pooling:</strong></p><p>❗ <em>Every single HTTP call would resolve the “living” pod’s IP dynamically as the request is made. This introduces an overhead per HTTP request but it’s the safe and default way mostly.</em></p><p><strong>But what connection-pooling does instead:</strong></p><p>❗ <em>It resolves the target pod’s IP once the connection is established, and holds it until either the first request over that TCP connection fails, or the configured time-to-live period is over. It has no capability of figuring-out if the IP it is holding statically is still there or not!</em></p><p>In order to better understand the “IP lifecycle” I drawed below diagram by inspecting kubernetes events, application logs etc:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*97bu9adW9Cc8Wlozr8iCNg.png" /></figure><blockquote>Note: The application’s Readiness state switches run completely async from the kubernetes state switches, for this reason they are shown inmseparate timelines.</blockquote><p>The diagram clearly shows: It is not possible for a new connection to be established after t=~1s since the “dying” pod’s IP gets removed from IP tables at that time.</p><p>But what if the connection is established right before the pod enters TERMINATING state, and connection is kept alive beyond over the termination period?</p><h3>The Solution</h3><p>So the problem was about the configuration: <strong>If any connection established while the pod was healthy, it will be kept until the pod dies, since connection’s 30 seconds time-to-live is much larger than the target pod’s overall shutdown process of 15 seconds. The first request after the Tomcat is shutdown was immediately failing. After the failing request, the caller recognizes the target connection are no longer healthy and drops the dead connection but this causes 1 time exceptions occasionally.</strong></p><p>The solution was obvious: Decreasing the time-to-live of the connections in the connection-pool to around 15 seconds.</p><pre>httpclient:<br>        hc5:<br>          enabled: true<br>        max-connections: 50<br>        max-connections-per-route: 25<br>        time-to-live: 10<br>        time-to-live-unit: SECONDS</pre><p>Even if it is theoretically possible to set the time-to-live to 15 seconds, to exactly match the time period of shutdown process (15 seconds + microseconds), it shouldn’t be done practically.</p><p>As I observed that in production environment; it takes up to 1 to 5 seconds for “dying” pod’s IP to get removed from IP tables. So as a safe configuration I decreased the TTL to 10 seconds, and as I deploy to production, the Sentry errors of NoHttpResponseException stopped being received :).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b48ead33694" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/http-connection-pool-ttl-trap-debugging-intermittent-api-failures-3b48ead33694">Http Connection Pool TTL Trap: Debugging Intermittent API Failures</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing CI Pipeline in Monorepos]]></title>
            <link>https://medium.com/@bilgehanaygun/optimizing-ci-pipeline-in-monorepos-6745203509e0?source=rss-947355e812ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/6745203509e0</guid>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[monorepo]]></category>
            <category><![CDATA[github-actions]]></category>
            <dc:creator><![CDATA[Bilgehan Aygün]]></dc:creator>
            <pubDate>Tue, 10 Jun 2025 11:57:17 GMT</pubDate>
            <atom:updated>2025-06-10T11:57:17.906Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c71CoiYwb3KfNKc8tCnIKw.png" /></figure><h3>CI issues in monorepos</h3><p>A monorepo contains multiple projects and libraries, and usually CI pipelines are triggered per commit push, even if the commit contains changes on a single project of that monorepo, CI pipeline of every single app/lib on that monorepo is triggered. This causes inefficiency, build queue and of course costs extra money.</p><p>This blog covers how to resolve this problem by leveraging git tags along with an approach called “affected” which basically means our commits will only trigger the CI pipelines of the apps/libs that we made some change on.</p><h3>Problem Definition</h3><p>CI Pipelines usually configured to be triggered per commit push. They watch a repo for new commits to be pushed.</p><p>If a user is working on a single app in a monorepo, a push still would trigger CI pipeline of every single app/lib on that monorepo. Causes long build times, long build queues, and costs extra money.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fEdVWTdNGaxMTCu0WhrjAQ.png" /></figure><p>If the tests are consisting of “small” tests like unit tests since the CI pipeline is going to execute quick, this situation might be ignored.</p><p>But if the CI pipeline executes slow, especially if it is executing large tests like e2e tests, which can take really long depending on the size of the project, this situation is unacceptable at some point.</p><h3>Typical CI Pipeline Of a Front-end Application</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hNKb5nDx22zaF9vNwaYydQ.png" /></figure><h3>What we need</h3><p>If a commit is not including any changes in a app/lib in the monorepo, the CI stage must be skipped. This requires an initial stage in the pipeline named “Check if affected”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vqDPREVHWDGRaeLa7MWqPA.png" /></figure><h3>But this is already offered by many CI provider, and not enough in many cases. What is missing?</h3><p>Most CI providers like jenkins, gitlab allows you to define a sourceset, so the pipeline will only trigger if a change is made in this sourceset. But this is not enough in most cases. Let’s say you pushed some changes on App A in commit 1 and the pipeline for App A is failed. On your next push, you pushed changes on App B, but this time App A will not retrigger due to the CI provider’s affected check. You will need to manually retry the pipeline of App A.</p><p>We will see how to resolve this kind of issues and will have a fully optimized CI pipeline by the approach “Tagging the last successful commit”</p><h3>Tagging the Last Successful Commit</h3><p><strong>1.</strong> A new feature branch is created from master. In the first commit, our pipeline should check for the differences between. the branching point and Commit 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aPc64xUcH4UouGWITALuqA.png" /></figure><p>In this case only the CI of Application A will be triggered. And Let’s assume the pipeline is successfully completed.</p><p><strong>2.</strong> When we push another commit that contains changes on App B, the pipeline of App A should only check the difference between Commit 2 and Commit 1 <strong>Not the branching point and Commit 2</strong>. Because when Commit 1 is pushed the pipeline was already successfully executed. And no changes has been made to it in Commit 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OZq-MTkf1vJ42W017fnMww.png" /></figure><h3>But how can we know at which commit our pipeline is successfully executed for a specific app?</h3><p>The answer lies in the git tags. We will tag the commit for application A. But the git tags are global across the repo, they are not branch specific. So the tag name also needs to contain the name of the feature branch.</p><p>This leads us to use the following format for a git tag: <strong>tag-lsc-${branch_name}-${app_ name}</strong></p><h3>The Flow with Last Successful Commit</h3><p>Each time a pipeline is successfully completed, it will tag that commit as the last successful commit for that application and for that branch.</p><p>The pipeline at its first stage will check for the diff between the freshly pushed commit and the last successful tagged commit and will only trigger a full pipeline if there is a change between these two commits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RShq1OEsXhyOxFR7UC7Wgg.png" /></figure><p>Pipeline for microservice B will check for diff between Commit 3 and Commit 2 (Last Successful Commit for App B) and will skip the pipeline and directly mark as successfull.</p><p>Our Pipeline In The End:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iKH7EjFNEJFm6O2WGZeLHg.png" /></figure><h3>Implementation with Jenkins CI and Nx Monorepo</h3><h3>Tagging the last successful commit</h3><ol><li>Define the variables</li></ol><pre> environment {<br>   SERVICE_NAME = &quot;Payment&quot;<br>   IS_SERVICE_AFFECTED = false<br>   TAG_PREFIX = &quot;tag-lsc-payment-&quot;<br> }</pre><p>2. Very last stage of the pipeline is the one tagging the commit (Note: if the tag is already pointing to another commit, deleting the tag and and tagging the new commit is the only way. There is no tag carrying option.)</p><pre>stage(&#39;Tag last-successful-commit&#39;) {<br>  GIT_BRANCH_NAME = getGitBranchName()<br>  def currentCommitSha = sh(script: &#39;git rev-parse HEAD&#39;, returnStdout: true).trim()<br>  echo &quot;${env.TAG_PREFIX_SERVICE}$GIT_BRANCH_NAME will be deleted from current point and will be set to ${currentCommitSha}&quot;<br>  sh &quot;git tag -d ${env.TAG_PREFIX}$GIT_BRANCH_NAME || true&quot;<br>  sh &quot;git tag ${env.TAG_PREFIX}$GIT_BRANCH_NAME&quot;<br>  sh &quot;git push origin -d ${env.TAG_PREFIX}$GIT_BRANCH_NAME || true&quot;<br>  sh &quot;git push origin ${env.TAG_PREFIX}$GIT_BRANCH_NAME&quot;<br>}</pre><h3>Affected Check Stage</h3><p>This will be the very first stage of the pipeline as we’ve already stated. But rather than relying on a manual source set providing for a diff check, we can rely on the mono-repo managers affected check.</p><p>Most of the monorepo managers and the NX For example, creates a dependency graph between your projects and libs, and it provides a command to check if an application is affected between two commits.</p><p><strong>In the following example, if the common-components is changed that means every app/lib depending on it is also changed so marked as affected.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oQC_i73SxuHiNK3xJ3bwvw.png" /></figure><p>NX affected command also runs a “git diff” on the background. Then checks if any filename matched with its defined source sets (apps and libs in this case)</p><p>Code Example for Affected Check Stage:</p><pre>stage(&#39;Check if affected&#39;) {<br>  LAST_SUCCESSFUL_COMMIT = getLastSuccessfulCommit()<br>  def affectedProjects = sh(script: &quot;pnpm nx show projects --affected --base=$LAST_SUCCESSFUL_COMMIT --head=HEAD&quot;)<br>  if (affectedProjects.contains(&quot;$SERVICE_NAME&quot;)) {<br>  environment IS_SERVICE_AFFECTED = true<br>  echo &quot;$SERVICE_NAME is affected&quot;<br>  }<br>}</pre><p>the affected command checks between current position and LAST_SUCCESSFUL_COMMIT as the base, if there is no any, it fallbacks to the branching point as the base as we can see in the following code snippet:</p><pre>def getLastSuccessfulCommit() {<br>  GIT_BRANCH_NAME = getGitBranchName()<br>  TAG_NAME = &quot;${env.TAG_PREFIX}$GIT_BRANCH_NAME&quot;<br>  def tagExists = sh(script: &quot;git rev-parse -q --verify ${TAG_NAME}^{commit}&quot;, returnStatus: true) == 0<br>  <br>  if (tagExists) {<br>    return TAG_NAME<br>  }<br><br>  def branchingPoint = sh(script: &#39;git merge-base master HEAD&#39;, returnStdout: true).trim()<br>  return branchingPoint<br>}</pre><p>Full Pipeline:</p><pre>pipeline {<br>  environment {<br>    SERVICE_NAME = &quot;Payment&quot;<br>    IS_SERVICE_AFFECTED = false<br>    TAG_PREFIX_SERVICE = &quot;tag-lsc-payment-&quot;<br>  }<br>    stages {<br>        stages {<br>            stage(&#39;Git Clone&#39;) // FETCH REPO<br>            stage(&#39;Install&#39;) // NPM INSTALL, YARN INSTALL, PNPM INSTALL<br>            stage(&#39;Check if affected&#39;) {<br>                LAST_SUCCESSFUL_COMMIT = getLastSuccessfulCommit()<br>                def affectedProjects = sh(script: &quot;pnpm nx show projects --affected --base=$LAST_SUCCESSFUL_COMMIT --head=HEAD&quot;)<br>                if (affectedProjects.contains(&quot;$SERVICE_NAME&quot;)) {<br>                environment IS_SERVICE_AFFECTED = true<br>                echo &quot;$SERVICE_NAME is affected&quot;<br>                }<br>            }<br>            stage(&quot;Build, Test, Lint&quot;) {<br>                when { expression { return IS_SERVICE_AFFECTED.toBoolean() } }<br>                //Typescript compile, bundle, run unit tests, lint and formating check<br>            }<br>            stage(&#39;Create Containers&#39;) {<br>                when { expression { return IS_SERVICE_AFFECTED.toBoolean() } }<br>                // RUN MICROSERVICES, BACK-END APPLICATIONS, REDIS, POSTGRES, ELASTIC etc.<br>            }<br>            stage(&#39;Run E2E tests&#39;) {<br>                when { expression { return IS_SERVICE_AFFECTED.toBoolean() } }<br>                // RUN E2E TESTS<br>            }<br>            stage(&#39;Tag last-successful-commit&#39;) {<br>                GIT_BRANCH_NAME = getGitBranchName()<br>                def currentCommitSha = sh(script: &#39;git rev-parse HEAD&#39;, returnStdout: true).trim()<br>                echo &quot;${env.TAG_PREFIX_SERVICE}$GIT_BRANCH_NAME will be deleted from current point and will be set to ${currentCommitSha}&quot;<br>                sh &quot;git tag -d ${env.TAG_PREFIX}$GIT_BRANCH_NAME || true&quot;<br>                sh &quot;git tag ${env.TAG_PREFIX}$GIT_BRANCH_NAME&quot;<br>                sh &quot;git push origin -d ${env.TAG_PREFIX}$GIT_BRANCH_NAME || true&quot;<br>                sh &quot;git push origin ${env.TAG_PREFIX}$GIT_BRANCH_NAME&quot;<br>            }<br>        }<br>    }<br>  }<br>}</pre><h3>Conclusion</h3><p>Optimizing the CI for monorepos, running only the CI of apps you’ve worked on in a specific branch is crucial if your CI contains long running tasks like e2e’s.</p><p>Reduces your CI costs dramatically.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6745203509e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Contract Testing and Kafka Contract Tests with Spring Cloud Contract]]></title>
            <link>https://medium.com/@bilgehanaygun/contract-testing-and-kafka-contract-tests-in-spring-applications-a34753e80caf?source=rss-947355e812ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/a34753e80caf</guid>
            <category><![CDATA[consumer-driven-testing]]></category>
            <category><![CDATA[spring-cloud-contract]]></category>
            <category><![CDATA[consumer-driven-contract]]></category>
            <category><![CDATA[contract-testing]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Bilgehan Aygün]]></dc:creator>
            <pubDate>Sun, 08 Jun 2025 04:45:52 GMT</pubDate>
            <atom:updated>2025-08-28T14:45:49.852Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ImOdTqzG36BSlhZk_yeA4Q.jpeg" /></figure><h3>What is Contract Testing?</h3><p>Contract testing consists of microservice-level tests that verify whether the Data Transfer Object (DTO) used during communication between any two microservices is valid in both services.</p><p>A contract is a pact that the microservices must obey; it contains the DTOs they use to communicate with each other, and contains additional communication details.</p><p>This communication may occur via HTTP requests, Kafka, Redis, or other mechanisms, and contract tests can be written for any of these. Although contract tests are implemented differently for each of these communication methods, the core idea remains the same. In this article, after understanding the general structure of contract tests, we will focus on how to implement contract tests for Kafka (since its slightly more complex then others) within Spring applications.</p><h4>Why contract tests are crucial and why is it hard to cover microservice communication with other testing types?</h4><p>The DTO used during communication between two microservices must be defined in <strong>both</strong> services.</p><p>If that DTO changes in one microservice without the other knowing, the second service will fail. This can’t be catched with microservice level tests (integration or unit tests)</p><p>Without contract tests, the only way to detect such issues is running <strong>end-to-end(E2E)</strong> tests which requires to spin up the full environment where both microservices and many other services that your microservices need are running such as postgres, redis etc.</p><p><strong>Why we don’t rely on E2E tests for this purpose:</strong></p><ol><li><strong>Costly Setup: </strong>It requires to spin up a full environment.</li><li><strong>Flakiness &amp; coverage gaps: </strong>We can’t make sure if the test case that contains our microservice communication exists in our e2e tests, and e2e’s are flaky most of the time.</li><li><strong>Difficult debugging: S</strong>ince many components take part in these tests a simple deserialization error might stem from: amisconfiguration in Kafka, a bug in the receiver application, a bug in the sender application…</li></ol><p>Contract tests, on the other hand, verify at the <strong>microservice level</strong> that both sides adhere to the agreed-upon DTO and related communication details (the <strong>contract</strong>).</p><p>Next, we’ll walk through an example project and see how to write <strong>Kafka contract tests</strong> in Spring applications using <strong>Spring Cloud Contract</strong>.</p><p><strong>A simple project setup</strong>: <br>- ReturnOrder microservice (is the event producer)<br>- Payment microservice (is the event consumer)<br>- Kafka as communication channel</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ADzsCTObPeniUrsch57bVw.png" /></figure><p>All consumer classes should extend our <strong>AbstractProducer</strong> allowing us to get all the consumers by a single method (More on the later chapter.).</p><pre>public abstract class AbstractProducer {  public abstract String getTopic();}</pre><p>A ReturnOrder event:</p><pre>@Data<br>@NoArgsConstructor<br>@AllArgsConstructor<br>public class ReturnOrderEvent implements Serializable {<br><br>  @Serial<br>  private static final long serialVersionUID = 1587253L;<br><br>  private UUID orderId;<br>  private UUID productId;<br>  private String productName;<br>  private BigDecimal price;<br>}</pre><h3>Testing The Producer Microservice</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZmoXeltX3lm241DiE-U-zg.png" /></figure><p>The producer microservice needs following dependencies:</p><pre>plugins { id &quot;org.springframework.cloud.contract&quot; }<br><br>dependencies {<br>    implementation &#39;org.springframework.cloud:spring-cloud-starter-contract-verifier&#39;<br>}</pre><p>With this dependency a new source set will be automatically defined under <strong>src/contractTest/java.</strong></p><h3>Defining The Contract</h3><p>The contract is defined in the producer microservice and deployed to a central maven repo as a jar. This jar is really small in terms of size, so during the tests contract is fetched by gradle or maven.</p><p>To defined the contract we need the following dependency:</p><pre>dependencies {<br>    testImplementation &#39;org.springframework.cloud:spring-cloud-contract-spec-groovy:4.0.0&#39;<br>}</pre><p>A contract for the ReturnOrderEvent can be seen below:</p><pre>package contracts<br><br>import org.springframework.cloud.contract.spec.Contract;<br><br>Contract.make {<br>    description &quot;Should publish ReturnOrderEvent into &#39;return-order&#39; topic&quot;<br><br>    label &#39;returnOrderEvent&#39;<br><br>    input {<br>        triggeredBy(&#39;publishReturnOrderEvent()&#39;)<br>    }<br><br>    outputMessage {<br>        sentTo(&#39;return-order&#39;)<br>        body([<br>                orderId     : $(consumer(regex(uuid())), producer(anyUuid())),<br>                productId   : $(consumer(regex(uuid())), producer(anyUuid())),<br>                productName : $(consumer(regex(&#39;[A-Za-z0-9]{10,20}&#39;)), producer(regex(&#39;[A-Za-z0-9]{10,20}&#39;))),<br>                price       : $(consumer(regex(&#39;[0-9]+(\\.[0-9]{1,2})?&#39;)), producer(anyNumber()))<br>        ])<br>    }<br>}</pre><ul><li><strong>description</strong>: is a note to ourselves (devs)</li><li><strong>triggeredBy:</strong> is the name of the method that produces the event in the producer microservice that is supposed to obey the details that are defined here.</li><li><strong>label</strong>: during the consumer tests the contract itself will generate the events using this label (more detailly in the related chapter).</li><li><strong>sentTo</strong>: is the topic name.</li><li><strong>body</strong>: contains the details of the event. The event must contain of 4 fields.</li><li><strong>consumer()</strong>: Defines the rules that the events produced by the producer microservice must obey.</li><li><strong>producer()</strong>: Defines the data type of the fields of the event that will be sent to the consumer microservice during consumer tests.</li></ul><h4>Generating producer tests</h4><p>We are not handwriting the producer tests but they are <strong>auto-generated</strong> by Spring Cloud Contract accordingly to the contract we defined.</p><pre>gradle apps:returnorder:contractTest</pre><p>And the generated tests can be found under &quot;$projectRoot/apps/returnorder/build/generated-test-sources&quot;. Below we can see the generated tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s5E9fVASvt_qi3VI_C-5gQ.png" /></figure><pre>public class ContractVerifierTest extends BaseKafkaProducerContractTest {<br>    @Inject ContractVerifierMessaging contractVerifierMessaging;<br>    @Inject ContractVerifierObjectMapper contractVerifierObjectMapper;<br><br>    @Test<br>    public void validate_returnOrderKafkaContract() throws Exception {<br>        // when:<br>            publishReturnOrderEvent();<br><br>        // then:<br>            ContractVerifierMessage response = contractVerifierMessaging.receive(&quot;return-order&quot;,<br>                    contract(this, &quot;returnOrderKafkaContract.yml&quot;));<br>            assertThat(response).isNotNull();<br><br>        // and:<br>            DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload()));<br>            assertThatJson(parsedJson).field(&quot;[&#39;orderId&#39;]&quot;).matches(&quot;[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}&quot;);<br>            assertThatJson(parsedJson).field(&quot;[&#39;productId&#39;]&quot;).matches(&quot;[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}&quot;);<br>            assertThatJson(parsedJson).field(&quot;[&#39;productName&#39;]&quot;).matches(&quot;[A-Za-z0-9]{10,20}&quot;);<br>            assertThatJson(parsedJson).field(&quot;[&#39;price&#39;]&quot;).matches(&quot;-?(\\d*\\.\\d+|\\d+)&quot;);<br>    }<br>}</pre><p>We can see that auto-generated test calls <strong>publishReturnOrderEvent() </strong>w in order to publish a message to kafka. Then it receives the message from kafka and compares the content with the rules defined in the contract.</p><p>So now we need to define the test class which contains the <strong>publishReturnOrderEvent()</strong> method and it should be globally accessible to auto-generated test classes.</p><h4>Defining the BaseProducer test class</h4><p>Below is a simple impelementation of the producer test class. Fires up a kafka container as a test container and produces a message as the same way our microservice would do since it uses the producer class that is defined in the microservice .</p><pre>@ExtendWith(SpringExtension.class)<br>@AutoConfigureMessageVerifier<br>@SpringBootTest(<br>  webEnvironment = SpringBootTest.WebEnvironment.NONE,<br>  classes = {KafkaProducerTestConfig.class}<br>)<br>@Testcontainers<br>@Import(ReturnOrderProducer.class)<br>public abstract class BaseKafkaProducerContractTest {<br><br>  @Autowired<br>  private ReturnOrderProducer returnOrderProducer;<br><br>  @ServiceConnection<br>  static KafkaContainer kafka = new KafkaContainer(DockerImageName<br>    .parse(&quot;confluentinc/cp-kafka:latest&quot;))<br>    .withKraft();<br><br>  static {<br>    kafka.start();<br>    System.setProperty(&quot;spring.kafka.bootstrap-servers&quot;, kafka.getBootstrapServers());<br>  }<br><br>  public void publishReturnOrderEvent() {<br>    var returnOrderEvent = new ReturnOrderEvent(<br>      UUID.randomUUID(), UUID.randomUUID(), &quot;randomproduct1&quot;, BigDecimal.TEN<br>    );<br>    returnOrderProducer.publishReturnOrderEvent(returnOrderEvent);<br>  }<br>}</pre><p>This base producer test class has to be extended by our auto-generated producer tests so only that way kafka container and other configurations will actually be used by our producer tests and the methods we defined that are producing the messages will be accessible. For this purpose we should add the below lines to <strong>build.gradle</strong> which basically tells the spring cloud contract that this class is our base test class for producer tests.</p><pre>contracts {<br>    baseClassForTests = &quot;com.bilge.returnorder.configuration.test.BaseKafkaProducerContractTest&quot;<br>    contractsDslDir = file(&quot;src/integration/resources/contracts&quot;)<br>}<br><br>tasks.named(&#39;contractTest&#39;) {<br>    useJUnitPlatform()<br>}</pre><p>Calling back the flow diagram for producer tests, the last thing we are supposed to do is to implement the MessageVerifierReceiver Interface that will help the spring cloud contract to receive the messages from Kafka.</p><h4>Implementing MessageVerifierReceiver Interface</h4><pre>public class KafkaMessageVerifier implements MessageVerifierReceiver&lt;Message&lt;?&gt;&gt; {<br><br>  private final Map&lt;String, BlockingQueue&lt;Message&lt;?&gt;&gt;&gt; broker = new ConcurrentHashMap&lt;&gt;();<br><br>  @Override<br>  public Message&lt;?&gt; receive(String destination, YamlContract contract) {<br>    return receive(destination, 15, TimeUnit.SECONDS, contract);<br>  }<br><br>  @Override<br>  @SneakyThrows<br>  public Message&lt;?&gt; receive(String destination, long timeout, TimeUnit timeUnit, YamlContract contract) {<br>    broker.putIfAbsent(destination, new ArrayBlockingQueue&lt;&gt;(10));<br>    BlockingQueue&lt;Message&lt;?&gt;&gt; messageQueue = broker.get(destination);<br><br>    return messageQueue.poll(timeout, timeUnit);<br>  }<br><br>  @KafkaListener(topics = &quot;#{@allTopics}&quot;, groupId = &quot;random&quot;)<br>  public void listen(Message&lt;?&gt; payload, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {<br>    broker.putIfAbsent(topic, new ArrayBlockingQueue&lt;&gt;(10));<br>    BlockingQueue&lt;Message&lt;?&gt;&gt; messageQueue = broker.get(topic);<br>    messageQueue.add(payload);<br>  }<br>}</pre><p>Here our MessageVerifierReceiver listens to all topics by the help of a bean named allTopics (which is defined below). Puts the messages in an in-memory queue and receive methods reads the messages from queue in every 15 secs.</p><pre>@TestConfiguration<br>public class KafkaProducerTestConfig {<br><br>  private final ApplicationContext applicationContext;<br><br>  public KafkaProducerTestConfig(ApplicationContext applicationContext) {<br>    this.applicationContext = applicationContext;<br>  }<br><br>  @Bean<br>  public String[] allTopics() {<br>    Map&lt;String, AbstractProducer&gt; producerBeans = applicationContext.getBeansOfType(AbstractProducer.class);<br><br>    return producerBeans.values().stream()<br>      .map(AbstractProducer::getTopic).distinct().toArray(String[]::new);<br>  }<br><br>  @Bean<br>  KafkaMessageVerifier kafkaTemplateMessageVerifier() {<br>    return new KafkaMessageVerifier();<br>  }<br>}</pre><p>Remember that every producer class in our application was supposed to extend our AbstractProducer abstract class. By the help of this approach we can now collect all the producers that are defined in our application. And with a single @KafkaListener annotated method we can listen to all topics across our application.</p><p>The producer tests are completed with this. Running the &quot;gradle apps:returnorder:contractTest” command now will generate the producer tests extending our BaseTest and properly accessing the producer methods we defined.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aKwuxBn3iUMMneTZtm4cCA.png" /></figure><p>Running the tests now will fire up the Kafka container, will publish messages, and will validate the messages on the contract we defined. Anytime we make a change on the event we publish, that is a violation to the rules of the contract, test will fail.</p><h3>Testing The Consumer Microservice</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qo2mDOZtRa859I8MfoICUw.png" /><figcaption>The flow for consumer contrcat tests.</figcaption></figure><p>Of course the exact same contract we defined in the producer side will be used during the consumer tests. By the help of the contract, <strong>Stub Jar will be generated on the producer microservice</strong> and will be deployed to a maven repo so the consumer microservice pull and use during the tests.</p><p>The Stub Jar is responsible for publishing the messages to the kafka during consumer tests so the consumer microservice can read the messages. The Stub Jar also contains the contract so the messages it publishes will be adhere to the rules of the contract. Actually not only the Stub Jar itself but with the <strong>MessageVerifierSender</strong> they will be together responsible for publishing the messages. <strong>MessageVerifierSender</strong> is an interface from Spring Cloud Contract that should be implemented for this purpose.</p><p>The consumer microservice than will read the message as it would do in the production and will check if it is able to consume properly. Anytime a change is made on the consumer logic without the producer side knowing that, the test will fail since the contract will remain the same.</p><h4>Generating the stub jars</h4><p>On the producer microservice running the “gradle apps:returnorder:verifierStubsJar” command generates the stub jars. The generated jars can be seen under “$appRoot/build/libs”. These jars have to be published to a maven repo than can will be accessible from consumer microservice during the tests.</p><h4>Adding the dependencies to the consumer microservice</h4><pre>dependencies {<br>    implementation &#39;org.springframework.cloud:spring-cloud-starter-contract-stub-runner:4.0.0&#39;<br>    implementation &#39;org.springframework.cloud:spring-cloud-starter-contract-verifier:4.0.0&#39;<br>}</pre><h4>Implementing the MessageVerifierSender Interface</h4><pre>public class KafkaContractConsumerTestConfig {<br><br>  @Lazy<br>  @Autowired<br>  private KafkaTemplate&lt;String, Object&gt; kafkaTemplate;<br><br>  @Autowired<br>  @Lazy<br>  private KafkaAdmin kafkaAdmin;<br><br>  @Bean<br>  MessageVerifierSender&lt;?&gt; kafkaMessageVerifierSender() {<br>    return new MessageVerifierSender&lt;&gt;() {<br>      @Override<br>      public void send(final Object message, final String destination, @Nullable final YamlContract contract) {<br>        send(message, Map.of(), destination, contract);<br>      }<br><br>      @SneakyThrows<br>      @Override<br>      public void send(Object payload, Map headers, String destination, @Nullable YamlContract contract) {<br>        kafkaAdmin.createOrModifyTopics(TopicBuilder.name(destination).build());<br>        ProducerRecord&lt;String, Object&gt; producerRecord = new ProducerRecord&lt;&gt;(destination, payload);<br>        kafkaTemplate.send(producerRecord).get();<br>      }<br>    };<br>  }<br>}</pre><p>A simple class that creates the topic if it is not present, we may configure the headers according to our needs, and send the messages to kafka.</p><p>We need a BaseConsumerTest class just like we had in producer tests. This will be responsible for firing up a kafka container and injecting the StubTrigger to the TestContext. StubTrigger is the mechanism that uses Stub Jars and MessageVerifierSender to send the messages to the kafka. We will be manually calling the trigger method on it during the tests.</p><pre>@ExtendWith(SpringExtension.class)<br>@BootstrapWith(SpringBootTestContextBootstrapper.class)<br>@ContextConfiguration(classes = {KafkaAutoConfiguration.class, KafkaContractConsumerTestConfig.class})<br>public abstract class BaseKafkaContractConsumerTest {<br><br>  @Autowired<br>  protected StubTrigger stubTrigger;<br><br>  @ServiceConnection<br>  static KafkaContainer kafka = new KafkaContainer(<br>    DockerImageName.parse(&quot;confluentinc/cp-kafka:latest&quot;))<br>    .withKraft();<br><br>  static {<br>    kafka.start();<br>    System.setProperty(&quot;spring.kafka.bootstrap-servers&quot;, kafka.getBootstrapServers());<br>  }<br>}</pre><h4>Adding the Consumer Tests</h4><pre>@AutoConfigureStubRunner(<br>  ids = &quot;com.bilge:returnorder:+:stubs:stubs&quot;,<br>  repositoryRoot = &quot;file:/path/to/local/m2/repository&quot;,<br>  stubsMode = StubRunnerProperties.StubsMode.LOCAL<br>)<br>@SpringBootTest(classes = ReturnOrderListener.class)<br>public class ReturnOrderEventConsumerTest extends BaseKafkaContractConsumerTest {<br><br>  @MockBean<br>  RefundService refundService;<br><br>  @Test<br>  void returnOrderEventConsumesSuccessfully() {<br>    stubTrigger.trigger(&quot;returnOrderEvent&quot;);<br>    verify(refundService, timeout(10000)<br>      .times(1))<br>      .processReturnOrderAndRefund(any());<br>  }<br>}</pre><p>Here @AutoConfigureStubRunner is responsuble for making the stub jar accessible, local maven repo can be used for development purposes.</p><p>We simply send a message to kafka by calling the <strong>stubTrigger.trigger(“eventName”)</strong> the eventName is the name of the event (defined in the label field) from the contract, and we ensure that it is consumed by our application.</p><blockquote>With that we have completed the consumer tests. By these tests we can now ensure no microservice will make a change without the other knowing because if so the contract tests will fail, thus we won’t see any unexpected errors in the production :)</blockquote><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a34753e80caf" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>