<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Wenbo Zong on Medium]]></title>
        <description><![CDATA[Stories by Wenbo Zong on Medium]]></description>
        <link>https://medium.com/@zongwb?source=rss-76d580d9531e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*TxBSW_3tMQR0L-FX.</url>
            <title>Stories by Wenbo Zong on Medium</title>
            <link>https://medium.com/@zongwb?source=rss-76d580d9531e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 09:33:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@zongwb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Two Game Changers for Better Software Development: API to Code Generation and Automated Testing]]></title>
            <link>https://zongwb.medium.com/two-game-changers-for-better-software-development-api-to-code-generation-and-automated-testing-bcbfd3b39ced?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/bcbfd3b39ced</guid>
            <category><![CDATA[software-methodology]]></category>
            <category><![CDATA[automated-testing]]></category>
            <category><![CDATA[software-best-practices]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Sat, 20 Apr 2024 13:08:25 GMT</pubDate>
            <atom:updated>2024-04-20T13:08:25.228Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m3yJF83Rr8XZ8J4L" /></figure><p>Recently I tried to upgrade SQLAlchemy from 1.4 to 2.0 and sqlacodegen from v1 to v2, and I was forced to change my code to resolve some incompatibility issues introduced by these libraries. Luckily, I already had automated testing in place which made this experience not too bad — still quite a tedious process but I was confident that my changes would be well tested without putting in much extra effort. Hence this post.</p><p>First off, I think I’m writing about the obvious, again. However, not every team is practising these two important methodologies yet, namely <em>API specs to code</em>, and <em>automated testing</em>. I covered API to Code in a recent <a href="https://zongwb.medium.com/api-spec-to-code-or-code-to-api-spec-935ceba5bf6a">post</a>. Let’s talk about automated testing.</p><p>Everyone knows testing is important and yet most developers hate testing (my personal experience). I can think of a few common reasons:</p><ul><li>Testing is less satisfactory than coding</li><li>Not enough time is allocated to testing</li><li>Difficult to set up dependencies for testing</li><li>Testing is time consuming and slows down delivery</li></ul><p>We will not argue against the first two reasons as they are quite personal and depend on how the software project is managed. I’d like to focus on addressing the last two arguments in this piece.</p><h4>Setting up dependencies for testing</h4><p>End-to-end testing with external dependencies is not easy, whether it’s a database or third-party service. That’s true. However, developer tools have advanced so much that nowadays it’s already quite easy to spin up a local Docker image for <em>almost</em> everything. Most dependencies can be emulated using local Docker images. Of course, they need to be created in the right order and prepared with the right data/state, which is again doable with <em>code</em> (rather than manually setting up). The only dependency that is difficult to emulate locally is third-party services. I don’t have a good solution, but you may try to mock.</p><h4>Slow down delivery?</h4><p>After practising automated testing myself, I have a strong view on this argument. Yes, it slows down delivery initially, because writing automated testing takes time. However, there will <em>definitely</em> be gains in speed for future deliveries:</p><ul><li>Lots (and lots) of time can be saved when refactoring, if automated testing already covers the code. Otherwise, you’d need to re-test the code with many test cases. In fact, I’d argue that because there’s extra testing needed, the refactoring may never happen.</li><li>With automated testing, it’s a lot easier and faster to fix common code, for similar reasons to refactoring.</li><li>Both of the above arguments become more pronounced as the codebase grows.</li><li>New team members would not normally dare to change code unless there’s an easy way to do regression test.</li></ul><p>I found a picture that very well summarises the productivity curve of practising automated testing (<a href="https://www.simform.com/blog/test-automation-strategy/">source</a>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*1cgjCpBmBb3FDYAg.png" /><figcaption>Productivity curve of practising automated testing</figcaption></figure><h4>There is more!</h4><p>In addition to delivery speed, there is another obvious benefit and another less obvious benefit:</p><ul><li>The obvious: Generally speaking, code quality is proportional to the amount of automated testing.</li><li>The not-so-obvious: Test cases can serve as <em>documentation</em>. This is often overlooked but it is very important! In fact, testing code can be sample client code.</li></ul><h4>Wrap up</h4><p>There are many best practices that can boost code quality and delivery speed. Adopting API to code generation and automated testing are two game changers in my own experience. You don’t have to agree but if you haven’t tried any of them, please do give it a try and then decide if they work for you. I hope they do.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bcbfd3b39ced" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[API Spec to Code or Code to API Spec?]]></title>
            <link>https://zongwb.medium.com/api-spec-to-code-or-code-to-api-spec-935ceba5bf6a?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/935ceba5bf6a</guid>
            <category><![CDATA[api-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-best-practices]]></category>
            <category><![CDATA[open-api]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Thu, 11 Jan 2024 12:40:36 GMT</pubDate>
            <atom:updated>2024-01-11T12:40:36.888Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2nd8shwCUwtMtyNuPEDRTg.jpeg" /><figcaption>Source: <a href="https://blog.openapihub.com/en-us/openapi-swagger-automate-your-api-development-process/">OpenAPIHub</a></figcaption></figure><p>Few software engineers don’t like writing documentation. Some never did. Many tools have been created to create documentation from code, such as Doxygen, Godoc. It’s probably useful for implementation documentation that explains how a function should be used, e.g. Golang has the built-in document generator. However, for API services, I think the API-first approach should be preferred, for a few reasons:</p><ol><li>It liberally forces us to think of API first — This is very important!</li><li>Once the API is ready, the API consumer and API provider can work concurrently.</li><li>Boilerplate code can be (re-)generated easily, taking away the tedious work from developers.</li><li>The implementation and API doc are in sync, which can be enforced by CI/CD.</li></ol><p>I think the barrier to adopt this approach is quite low given the many tools available. If you haven’t tired it, I think you should and it can be a game changer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=935ceba5bf6a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Applications on GKE]]></title>
            <link>https://zongwb.medium.com/setting-up-applications-on-gke-7003aa91a3ad?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/7003aa91a3ad</guid>
            <category><![CDATA[ingress]]></category>
            <category><![CDATA[google-kubernetes-engine]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Sun, 15 Jan 2023 12:52:20 GMT</pubDate>
            <atom:updated>2023-01-15T12:52:20.616Z</atom:updated>
            <content:encoded><![CDATA[<p>Ingress, service mesh, workload identity, Cloud SQL Auth proxy and more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/465/1*fYj_yh83WMG2PvauImqjBg.png" /></figure><p><em>Disclaimer</em>: Kubernetes (K8S) is a complex topic and I’d not claim to be an expert in it. This article specifically focuses on how to set up applications on Google Kubernetes Engine (GKE). Needless to say, there are many ways to achieve the same goal and this is just my approach.</p><p>GKE, being Google’s K8S implementation, supports the standard K8S API and also adds Google-specific API to make it easier to integrate with other Google services. From an application developer’s standpoint, the interactions with the standard K8S API are relatively easier to handle. In contrast, it is the interactions with the Google-specific K8S API that often gives me a hard time. Specifically, I find the ingress (load balancer and beyond) and the access to Google-managed resources the most tricky topics. Hence, in this article I’ll only highlight a few key learnings on these topics and for the curious, please check out the sample code in my <a href="https://github.com/zongwb/gke-samples">Github repository</a>.</p><p>This article is structured as follows:</p><ul><li>Platform lock-in?</li><li>Ingress and communication within the GKE cluster</li><li>Access (and be accessed by) Google-managed resources</li></ul><h3>Avoid Platform Lock-in. Or really?</h3><p>Developers hate platform lock-in and there’s no exception with K8S: We always try to use the standard K8S API and shun the platform-specific ones. However, my experience with GKE is such that it’s practically impossible to avoid using platform-specific features. Even if it’s possible, it would mean a lot of extra efforts (sometimes hackish) and much more complex configurations, often end up in some unchartered territory. The gain (being platform neutral) vs. the cost (complexity, efforts) is quite unjustified to my liking.</p><h3>Ingress and communication within the GKE cluster</h3><p>Let’s start with a wishlist for the ingress and the internal traffic management:</p><ul><li>HTTP to HTTPS redirect: Must-have</li><li>Domain redirect: Must-have</li><li>Fully managed SSL certificates: Must-have</li><li>Web application firewall: Strongly desired</li><li>Session affinity: Must-have</li><li>Custom http headers: Nice to have</li><li>mutual TLS (mTLS) between pods/workloads: Nice to have</li></ul><p>The list of course could be longer, but these are what I am most concerned with. Now let’s map these items to what is available on GKE and from the opensource community, as shown in the following table:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/557803d68ce963b104ebf30998e3a03c/href">https://medium.com/media/557803d68ce963b104ebf30998e3a03c/href</a></iframe><p>Apparently, if you really want to avoid platform lock-in, you can use Nginx. Regarding internal traffic management, I had trouble enabling Istio on GKE back in 2021, and later found out that Google actually deprecated Istio on GKE starting 2022 and replaced it with Anthos Service Mesh (see <a href="https://cloud.google.com/istio/docs/istio-on-gke/overview">here</a>). If you really want the service mesh features on GKE, then ASM should be the way to go.</p><p><em>PS: I really hope that the K8S Gateway API will unify ingress and service mesh and Google will implement it with native integrations with other Google-managed resources, such as Cloud Armor.</em></p><h3>Access (and be accessed by) Google-managed resources</h3><p>It is almost inevitable that our GKE applications need to access other Google-managed resources, e.g. Cloud Storage, Cloud SQL. The recommended way is <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity">Workload Identity</a>, which basically involves creating a GCP (IAM) service account with the appropriate permissions, creating a K8S service account and binding the two together.</p><h4>Access Cloud SQL</h4><p>For connecting to a Cloud SQL database, the recommended way is to use <a href="https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine">Cloud SQL Auth proxy</a> with the sidecar pattern.</p><h4>Access the GKE application</h4><p>Another issue is that you may need to access your GKE applications from outside of the GKE cluster. For example, you may have a Cloud Function (in a different region) that needs to call your GKE application. In this case, you can create an internal LoadBalancer service and annotate it to enable access from other Google Cloud regions.</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: ${INTERNAL_LB_NAME}<br>  annotations:<br>    networking.gke.io/load-balancer-type: &quot;Internal&quot;<br>    networking.gke.io/internal-load-balancer-allow-global-access: &quot;true&quot;<br>  labels:<br>    app: ${SVC_NAME}<br>spec:<br>  type: LoadBalancer<br>  selector:<br>    app: ${SVC_NAME}<br>  ports:<br>  - port: 80<br>    targetPort: 8080<br>    protocol: TCP</pre><h3>Wrap up</h3><p>The above is a quick recap of the few key learnings from setting up applications on GKE. Nothing beats code and practice. For those curious, please check out two complete (although simple) samples on my <a href="https://github.com/zongwb/gke-samples">Github repo</a>.</p><h3>References</h3><ul><li><a href="https://cloud.google.com/architecture/exposing-service-mesh-apps-through-gke-ingress">https://cloud.google.com/architecture/exposing-service-mesh-apps-through-gke-ingress</a></li><li><a href="https://medium.com/contino-engineering/configuring-ddos-protection-with-google-cloud-armor-for-your-gke-provisioned-istio-ingressgateway-a9e862dc1683">https://medium.com/contino-engineering/configuring-ddos-protection-with-google-cloud-armor-for-your-gke-provisioned-istio-ingressgateway-a9e862dc1683</a></li><li><a href="https://gateway-api.sigs.k8s.io/">https://gateway-api.sigs.k8s.io/</a></li><li><a href="https://thenewstack.io/the-gateway-api-is-in-the-firing-line-of-the-service-mesh-wars/">https://thenewstack.io/the-gateway-api-is-in-the-firing-line-of-the-service-mesh-wars/</a></li><li><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity">https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity</a></li><li><a href="https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine">https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine</a></li><li><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#global_access">https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#global_access</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7003aa91a3ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[This Is a Sign That You Should Start Documenting Your System]]></title>
            <link>https://zongwb.medium.com/this-is-a-sign-that-you-should-start-documenting-your-system-e30f70a1de17?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e30f70a1de17</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[agile-software-teams]]></category>
            <category><![CDATA[software-documentation]]></category>
            <category><![CDATA[software-culture]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Sun, 28 Feb 2021 15:13:24 GMT</pubDate>
            <atom:updated>2024-08-02T00:09:21.487Z</atom:updated>
            <content:encoded><![CDATA[<p>This post is probably a better fit for a tweet, but I don’t play twitter that much :)</p><p><em>When should you write a document for your system?</em></p><p>When someone asks you questions about your application and it takes you more than five seconds to give an answer with <em>certainty</em>, you probably should start documenting (in whatever format that you see fit, e.g. even as simple as just a diagram).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e30f70a1de17" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Does Golang’s Garbage Collection Affect Latency? An Informal Analysis]]></title>
            <link>https://zongwb.medium.com/how-does-golangs-garbage-collection-affect-latency-an-informal-analysis-9a44cdbe487d?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a44cdbe487d</guid>
            <category><![CDATA[garbage-collection]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[latency]]></category>
            <category><![CDATA[performance-tuning]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Sun, 21 Jun 2020 15:58:19 GMT</pubDate>
            <atom:updated>2024-01-07T02:41:26.468Z</atom:updated>
            <content:encoded><![CDATA[<p>In this post, I will share an <em>informal</em> analysis on how the Golang’s garbage collection (GC) affects the latency. The analysis is informal as it is <em>not</em> a rigorous mathematical formulation and only serves as a ballpark estimate of the latency’s lower bound.</p><blockquote>Before we start, it is important to emphasise that the analysis in this post is only applicable to the high-throughput applications. On the other hand, if the application has low throughput, GC rarely plays a significant role in latency.</blockquote><h3>Garbage Collection Metrics</h3><p>In order to understand how Golang garbage collection works and interpret the GC traces, <a href="https://www.ardanlabs.com/blog/2018/12/garbage-collection-in-go-part1-semantics.html">here’s</a> an excellent article if you are not familiar with it.</p><p>I’ve written a simple Jupyter notebook to process the GC traces (<a href="https://github.com/zongwb/golang-gc-analysis">https://gn jbithub.com/zongwb/golang-gc-analysis</a>). For this analysis, we will be looking at three metrics:</p><ul><li>Average GC cycle: the average duration of a GC cycle, measured from the start of one GC to the start of the next GC. This is the wall clock time.</li><li>Average GC CPU time (Normalized): the average CPU time spent on GC in one cycle, normalized/converted to wall clock time.</li><li>% of GC CPU time: percentage of CPU time spent on GC.</li></ul><p>To illustrate what these metrics mean, let’s consider a contrived example. Assume the throughput is 10,000 QPS (Note: It has to be high throughput for our analysis to make sense), and more importantly assume the requests arrive at <em>evenly spaced intervals</em>, i.e. constant arrival rate. This means there are 10 requests every millisecond, as visualised in the following diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*X-TP6K0HngZHP6WjiZquqQ.png" /><figcaption>Visualising the requests arrival at evenly space time intervals</figcaption></figure><p>Let’s further assume the average GC cycle is 1 second, and average GC CPU time is 10 milliseconds. It follows that 1% of CPU time (10ms / 1000ms=1%) is spent on GC. When GC is taking place, the requests that arrive during this window will be <em>not</em> be processed until the GC is completed. That means the GC will introduce <em>extra</em> latency for these requests. With the uniform distribution assumption, the requests that arrive during the active GC window will suffer a varying amount of latency, as illustrated in the following diagram:</p><figure><img alt="Latency of requests arriving at different time instance (relative to GC start)" src="https://cdn-images-1.medium.com/max/652/1*zs2UuPbMqDpBd0hL28pe3Q.png" /><figcaption>Latency of requests arriving at different time instance (relative to GC start)</figcaption></figure><p><em>Therefore, we can reason that half of the requests arriving in this 10-millisecond GC window, which is equivalent to 0.5%, will suffer an extra 5ms latency at the minimum. In other words, the 99.5-percentile latency will be at least 5ms.</em></p><h3>Analysis of an IO-Bound Application</h3><p>I ran profiling on a simple application that was designed to benchmark the redis GET throughput. The latency is calculated by the Prometheus library. The results with one and two CPUs are summarised as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dssAqVUtYpNpfOo0w2z-Dg.png" /></figure><p>We can draw two observations from this experiment:</p><ul><li>There is no significant difference in performance between one- and two-CPU cases. This roughly shows that most of the time is spent on network IO, instead of computation.</li><li>We’d expect about 0.27% (average of 0.53% and 0.55%, divided by 2)of requests would suffer at least 4ms (average 8.36ms and 7.79ms, divided by 2) latency due to GC (taking the average of the two runs). The actual latency is of course higher, with 99.5-percentile latency at 25ms.</li></ul><h3>CPU-Bound Applications</h3><p>It would be incomplete if we don’t consider CPU-bound applications. Generally speaking, the latency of CPU-bound applications is less affected by GC because computation is generally quicker to complete than a network RPC, and GC may not be able to interrupt the computation (I’m not too sure about this, though).</p><p>I ran profiling on a sample application that computes the signature using the ecsda package. To avoid contention between goroutines, I forced the number of computing goroutines to be the same as CPU cores. The result of running with 1-, 2-, 4-CPU is shown below (Note the latency is expressed in <em>microseconds</em>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*00khptDigvzU3gMj7soTCA.png" /></figure><p>Had we reasoned the latency using the same argument as before, we’d expect the 99th-percentile latency to be around 1.32ms for the 1-CPU case. However, the actual figure is only 127 microseconds, an order of magnitude lower. My explanation is that the computations are not interrupted by GC (at least most of the time), hence the GC does not affect the latency in this application .</p><h3>Wrap up</h3><p>Latency is probably related to garbage collection in very complicated ways. While it’s difficult to establish a direct relationship between them, we may estimate the latency’s lower bound for an application under certain assumptions: (1) high throughput; (2) requests at constant arrival rate.</p><p>Keep in mind that this analysis is more applicable to IO-bound applications than to CPU-bound applications. Even for IO-bound applications, the estimate should only be taken as a lower bound, and is usually way below the actual latency.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a44cdbe487d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some Thoughts on Aligning Event Handlers with Domain Boundary in EDA Design]]></title>
            <link>https://zongwb.medium.com/some-thoughts-on-aligning-event-handlers-with-domain-boundary-in-eda-design-dd95d91e428f?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd95d91e428f</guid>
            <category><![CDATA[anti-corruption-layer]]></category>
            <category><![CDATA[event-driven-architecture]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Sun, 07 Jun 2020 14:56:13 GMT</pubDate>
            <atom:updated>2020-06-07T14:56:13.531Z</atom:updated>
            <content:encoded><![CDATA[<h3>Some Thoughts on Aligning Event Handlers with Domain Boundaries in Event-Driven Architecture</h3><p>Recently while refactoring the order system (of a B2B e-commerce platform) to an <em>event-driven architecture</em> (EDA), we ran into a situation where we needed to decide whether or not an event handler should be part of the Order service domain. Even though it did not cause us too much trouble to decide what to do (given the constraints we were facing), I found it a really interesting problem that would arise frequently and was closely related to the DDD and microservices concepts.</p><p>In this essay I will try to summarise the thought processes we came through when deciding the alignment of the event handlers to domains, and hope to demonstrate how the DDD and/or microservices guidelines can be applied in this process.</p><p>Note that this essay is <em>not</em> about how to map a bounded context (domain) to microservices; There are plenty of excellent writings on that topic, e.g. <a href="https://medium.com/@justinmichaelholmes/microservices-and-the-bounded-context-part-1-5a3337b6b9de">here</a>, <a href="https://docs.microsoft.com/en-us/azure/architecture/microservices/model/microservice-boundaries">here</a>, and <a href="https://vladikk.com/2018/01/21/bounded-contexts-vs-microservices/">here</a>.</p><h3>Background</h3><p>I’ve been reading about Domain-Drive Design (DDD), on and off. Strictly speaking I’m not a DDD practitioner, but I do find some of the DDD concepts very relevant and useful when designing a microservice architecture, especially the following ones:</p><ul><li>High cohesion within a service and loose coupling between services</li><li>Bounded context is a domain model that is often realised by one or more microservices</li><li>An aggregate is a good candidate for a microservice</li></ul><p>However, for any specific system we still need to decide what is the exact <em>boundary</em> of a bounded context (aka domain). Only with well defined boundaries, the services can achieve high cohesion and loose coupling.</p><p>Note that I will use the terms event handler and worker almost interchangeably. However, when there is a need to distinguish, I use <em>handler</em> as a more generic term as it can be implemented as an independent process or as function in a process, whereas I refer to a <em>worker</em> as an independent process.</p><h3>The Problem</h3><p>The system in question is a B2B e-commerce platform that involves multiple services. There is also a separate ERP system that requires orders be synchronised from the e-commerce platform in real time. For the purpose of illustration, suffice to consider three services in the e-commerce platform, namely the Order, Inventory and Notification services. A conceptual overview of these services and their interactions is shown in the following diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cGeEKviWdrV3vsu-Yt36Sw.png" /><figcaption>Conceptual overview of the services involved and their interactions</figcaption></figure><p>The legacy system was all RPC based, and the motivations to move to an event-drive architecture include (1) better performance and (2) more importantly better handling of the order lifecycle when failures occur. Long story short, the new EDA design is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*uEhCVf3AMpJoV7IiEkenkQ.png" /><figcaption>Event-driven architecture (simplified for clarity), domain boundary not considered</figcaption></figure><p>Specifically, the handling of the event_reserve_stock depends on the Inventory service (or we may call it a <em>command</em>). The handling of the event_order_created is open-ended, because the Order service does not know exactly what services are interested in this event and new handlers may be added in the future. In our case, the Notification service and the ERP system are interested in the event_order_created.</p><p>Note that the event handlers in grey colour are at the centre of this discussion: Which domain do they belong to? The decision has practical implications, and we’ll see why next.</p><h3>Implications of Aligning the Event Handlers with Domains</h3><p>Before we talk about possible assignments, it is important to understand the decision’s implications, both technically and team-wise.</p><ol><li>Resource allocation: Assigning a handler to one service domain means the team responsible for that service will implement the handler, hence it has an impact on resource allocation.</li><li>Domain expertise and efficiency: Centralizing the handlers in one domain means the pieces of event handling logic are grouped closer to one another, and it’s relatively easier for the team (responsible for the domain) to build up domain knowledge. It also means that the team can work more efficiently (less communication overhead, familiar codebase, etc.).</li><li>Intrusion to existing services: If a service already exists with an established API, putting the handler into the service domain means we need to retrofit the message subscription mechanism into the service.</li><li>Technology choice: Each domain may be implemented with a different technology stack, which in turn affects how we implement the handler. This is obvious as that’s what the microservices architecture promises to enable in the first place.</li></ol><h3>What Options Do We Have?</h3><p>Okay, enough background info. In our case, we identified three options to assign each handler, and let’s call them <em>external handler</em>, <em>internal handler</em> and <em>integration layer handler</em>, respectively. As an example, the different options for the event_reserve_stock handler is illustrated below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cAfo0TKhdbowKXtfwVuzVg.png" /><figcaption>Options of assigning handlers. Dotted arrows indicate messaging, and solid arrows indicate RPC</figcaption></figure><p>Now let’s look at each of the options in a bit more detail.</p><h4><strong>External Handler</strong></h4><p>In this design, the handlers are implemented within the <em>depended</em> services, except for those in their own layer (i.e. the integration layer, more on this in a bit). The system diagram would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ig-zA1gjxK4wtS6-5gXR5A.png" /><figcaption>Event driven architecture with external handlers</figcaption></figure><p>A few salient points can be observed from this diagram:</p><ul><li>The message queue (MQ) is either a common component that sits between the Order and other services, or owned by the Order service but exposed to external event subscribers.</li><li>The handler for the event_reserve_stock becomes part of the Inventory service.</li><li>The Notification service implements one handler for event_order_created.</li><li>Another handler for event_order_created<em> </em>stands in the integration layer, dealing with the ERP system.</li></ul><p>In practice, if the depended services are already developed, we would need to consider if it’s easy or even possible to retrofit the handlers. When it’s possible to retrofit, we will need to update a few services at the same time and the communication and coordination effort is a function of the number of teams involved.</p><h4><strong>Internal Handler</strong></h4><p>Another alternative is to have the handlers in the Order service domain, and the handlers will invoke the Inventory and Notification services’ API to get their job done, respectively, as illustrated below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rjnJWbDcTDw6GiXHy4lwiw.png" /><figcaption>Event driven architecture with internal handlers (workers)</figcaption></figure><p>A few important points to highlight:</p><ul><li>The MQ is now owned by the Order service for the most part, but it needs to allow access by the worker (i.e. handler) in the integration layer.</li><li>The handlers can be implemented as standalone processes (may be grouped) independent of the core Order service, hence they are labelled workers. These workers actually form an <strong><em>anti-corruption</em></strong> layer (see <a href="https://www.amazon.com/Patterns-Principles-Practices-Domain-Driven-Design/dp/1118714709/ref=sr_1_1?crid=1TFV5JE705MRL&amp;keywords=patterns%2C+principles%2C+and+practices+of+domain-driven+design&amp;qid=1579075240&amp;sprefix=patterns%2C+princi%2Caps%2C348&amp;sr=8-1">DDD</a>), as they will shield the core Order service from any changes in the depended services (Inventory and Notification services in this case).</li></ul><p>Using internal handlers allows more flexibility in the message payload. If the message payload contains the order_id, the handler would need to query the Order database to retrieve the order details before calling the Inventory service. This is only possible if the handler belongs to the Order domain. In contrast, if the handler belongs to the Inventory service, i.e. external handlers, we would need to provide the necessary order details in the message payload so that the handler does not need to invoke the Order service API to fetch those details (which leads a circular dependency).</p><h4>Integration Layer Worker</h4><p>As the e-commerce and ERP are two separate systems, the order synchronisation should be treated as an integration concern. We cannot change the ERP as it is a purchased solution, but the Order domain relies on the ERP system to provide order fulfilment data (e.g. order shipping status). Typically, we create a worker to synchronise orders data from the e-commerce platform to the ERP and vice versa.</p><p>Apparently, the workers in the integration layer have to know both systems. Having an integration layer helps keep the Order domain (1) focused on its core responsibilities (e.g. managing the order life cycle) and (2) isolated from the specific details of the ERP system (especially if we need to replace the ERP system).</p><h3>Making a Decision</h3><p>We considered the following factors when deciding which option to go with.</p><ol><li>For request-response style inter-service communications, prefer synchronous RPC as it is easier to understand and reason about the system behaviour. On the other hand, asynchronous messaging should be contained within a single domain. This can ensure that a single domain is easier to understand and debug. In addition, synchronous API helps keep cross-team communication to a minimum.</li><li>Isolate changes. When the depended service changes its API, there will be a ripple effect on the dependent service. Having the event handler inside the Order service means the handler can act as an <em>anti-corruption</em> layer that shields the changes in the depended services (e.g. Inventory service). This is actually a very nice property as it helps keep the core Order service stable.</li><li>If an event handler carries out a non-essential task to the core domain, the worker is better kept outside the core domain. Doing so adheres better to the single responsibility principle and also increases the cohesion of the core domain. What is a non-essential handler? It usually means what the handler does does not affect the core domain. For example, email notification for order status change is purely for-information-only and does not affect the order lifecycle, therefore the corresponding handler is non-essential. Whether the worker should go to the integration layer or become part of the other domain, it is a case by case decision.</li><li>Try to avoid the integration layer when possible, as it can quickly become a dumping ground for event handlers. Maintenance may eventually become a headache.</li><li>Keep the teams aligned with their domains (bounded contexts), so that it’s easier to build up domain expertise.</li></ol><h4>The Verdict</h4><p>With the above considerations and taking into account our team’s realities, we decided to go with the internal handlers for the most part, <strong><em>and</em></strong> move the worker in the (conceptual) integration layer into the Order domain. There are a few reasons for this decision:</p><ul><li>The other services are already developed and we want to minimise retrofitting effort.</li><li>It keeps the changes within one domain, so it’s more efficient in terms of testing, debugging and communication. Also good for domain knowledge.</li><li>It allows for more flexible message payload, as we may not be able to foresee future changes. Worst comes to worst, the internal handlers can query the database to get the data needed (but missing in the payload).</li><li>In the long term, the workers will function as an anti-corruption layer so that the core Order service will be more stable.</li></ul><h3>Wrap Up</h3><p>The above is the thought processes we had gone through while refactoring the B2B e-commerce system to an event-driven architecture. It certainly has specifics that only apply to our scenario, however, I think there are some principles that can guide us when it comes to determine which domain an event handler should belong to.</p><ul><li>Minimise the cross-domain dependencies in the form of asynchronous messaging (very subjective, just my preference).</li><li>If the result of an API call or processing an event is not essential to the domain’s core logic, it can be done through asynchronous message in a <em>fire-and-forget</em> manner and the event handler should not be part of the domain.</li><li>If the core domain logic needs to handle failure scenarios and/or has scalability requirements, it can be implemented using asynchronous messaging. But I think it’s better to contain the asynchronous messaging within the domain so that it’s easier for the team to understand the system behaviour.</li><li>Keep an eye on the domain and team alignment.</li></ul><p><em>PS: This is a tough topic to write, and I was struggling to decide what details to include and what to leave out so that there is enough context and yet the main points are still clear. Hope I’ve achieved my goal. Please leave a message if you have any comment, thanks!</em></p><h3>References</h3><p>Scott Millett, Nick Tune. <a href="https://www.amazon.com/Patterns-Principles-Practices-Domain-Driven-Design/dp/1118714709/ref=sr_1_1?crid=1TFV5JE705MRL&amp;keywords=patterns%2C+principles%2C+and+practices+of+domain-driven+design&amp;qid=1579075240&amp;sprefix=patterns%2C+princi%2Caps%2C348&amp;sr=8-1">Patterns, Principles, and Practices of Domain-Driven Design</a></p><ul><li><a href="https://www.freecodecamp.org/news/follow-these-practical-principles-and-get-well-designed-microservices-boundaries-ef2deffd69e3/">Follow these practical principles to get well-designed microservices boundaries</a></li><li><a href="https://medium.com/transferwise-engineering/from-monolith-to-distributed-monolith-fd53d8dbbeba">From monolith to distributed monolith</a></li><li><a href="https://www.simplethread.com/youre-not-actually-building-microservices/">You&#39;re Not Actually Building Microservices - Simple Thread</a></li><li><a href="https://jimmybogard.com/the-false-dichotomy-of-monoliths-and-microservices/">The False Dichotomy of Monoliths and Microservices</a></li><li><a href="https://medium.com/@justinmichaelholmes/microservices-and-the-bounded-context-part-1-5a3337b6b9de">Microservices and The Bounded Context: Part 1</a></li><li><a href="https://docs.microsoft.com/en-us/azure/architecture/microservices/model/microservice-boundaries">Identifying microservice boundaries - Azure Architecture Center</a></li><li><a href="https://vladikk.com/2018/01/21/bounded-contexts-vs-microservices/">Bounded Contexts are NOT Microservices</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd95d91e428f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Idempotency: A Three-Step Approach]]></title>
            <link>https://zongwb.medium.com/idempotency-a-three-step-approach-d607895c2b93?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d607895c2b93</guid>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[idempotency]]></category>
            <category><![CDATA[optimistic-locking]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Thu, 28 May 2020 09:21:42 GMT</pubDate>
            <atom:updated>2020-05-28T09:46:03.556Z</atom:updated>
            <content:encoded><![CDATA[<p>Idempotency is a much desired property when designing applications, especially distributed applications because they are more prone to failures and error handling (e.g. retry) is often essential. In this essay I try to present a somewhat systematic approach to implementing idempotency.</p><h3>What is idempotency?</h3><p>Let’s begin with refreshing ourselves with the definition of idempotency (from <a href="https://en.wikipedia.org/wiki/Idempotence">wikipedia</a>):</p><pre><em>Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application.</em></pre><p>Before we proceed, consider two trivial questions.</p><p>(1) Is a <em>read</em> operation idempotent? For example,</p><ul><li>First read, X=10</li><li>Second read, X=20</li></ul><p>(2) How about HTTP <em>Delete</em>? For example,</p><ul><li>First delete, status=200</li><li>Second delete, status=404</li></ul><blockquote>The exact <em>response</em> of the second operation in each example is different, but these operations are actually deemed idempotent because they do not change the <em>state</em> of the resources. Hence, in the context of software, we may define idempotency as <strong><em>no side effects</em></strong>.</blockquote><h4>Idempotency != Safety</h4><p>It is also worth noting that idempotency is different from <em>safety</em>. In the context of HTTP APIs, a HTTP method is idempotent if it guarantees that repeating a request multiple times has the same effect as issuing the request once. A quick summary of the HTTP methods is shown in the following table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cWsZCnJWs47PqwTEuVh7AQ.png" /><figcaption>Safety and idempotency of HTTP methods</figcaption></figure><h3>Why idempotency is important?</h3><p>To illustrate why idempotency is a desired property for applications, let’s look at a request-response style RPC service and a messaging-based service, respectively.</p><h4>Why Is Idempotency Important for RPC?</h4><p>Consider the following API call to deduct money from an account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*todOCR1g9oKR-OKlYNPmxg.png" /><figcaption>A failure scenario in a RPC-based system</figcaption></figure><p>If the caller does not receive a response, the caller cannot know the status of the request, because there are a few possibilities:</p><pre>+------------------+----------------------------------------+<br>| <strong>Request status   |  Reason for not receiving the response </strong>|<br>+------------------+----------------------------------------+<br>| Processed        |  Network error or server crashes       |<br>| Half-processed   |  Server crashes                        |<br>| Being processed  |  Timeout                               |<br>| Not processed    |  Server crashes                        |<br>+------------------+----------------------------------------+</pre><p>The most common strategy to handle this type of error is <strong><em>retry</em></strong>. Some RPC frameworks may automatically retry on timeout, without the application code knowing it. (This might be a dangerous thing if the API is not idempotent.)</p><h4>Why Is Idempotency Important for Messaging?</h4><p>For messaging-based systems, there are two distinct failure scenarios, <em>duplicate publishing</em> and <em>duplicate fetching</em>, which are illustrated in the following two diagrams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*ccyKN6m16rOqD5DkH1FwXw.png" /><figcaption>Duplicate publishing in a messaging-based system</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*HCmSRlpYMqR8x_v0rI6nxA.png" /><figcaption>Duplicate fetching in a messaging-based system</figcaption></figure><blockquote>If the API is idempotent, the API callers can handle errors easily by simply retrying. Similarly, if the message handler is idempotent, it will allow <em>at-least-once</em> delivery (i.e. duplicate message publishing or fetching)</blockquote><h3>A Three-Step Approach</h3><p>Before we present a solution, let’s consider the relationship between idempotency and concurrency.</p><p>Would idempotency issues arise in <strong>concurrent</strong> applications?</p><ul><li><strong>Yes</strong>. Reasons include duplicate messages, duplicate requests.</li></ul><p>Would idempotency issues arise in <strong>single-threaded</strong> applications?</p><ul><li><strong>Yes</strong>. Ditto.</li></ul><blockquote>The only difference is that it’s slightly easier to handle in a single-threaded application, because not sharing data between requests &amp; not updating the same data means no data race.</blockquote><p>Hence, as a general solution, we need to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*iThOxbfz-QtUlDVjRRSWNQ.png" /><figcaption>A three-step approach to idempotency</figcaption></figure><ol><li>Serialize concurrent requests by locking the resources;</li><li>Check for duplicate request(s), return immediately if duplicate and release the resources;</li><li>Execute the application logic if not duplicate and release the resources.</li></ol><h4><strong>How to uniquely identify a request/operation?</strong></h4><p>To be able to distinguish a duplicate operation (API call, or event), we need an <strong><em>idempotency key</em></strong>. Typically, the idempotency key is a combination of request_id and resource_id. The exact identifier to be used depends on the application. For example, when reserving the inventory for an order, the order_id and sku_id may be used as the idempotency key.</p><p>The actual implementation would need to store the resource modification history (i.e. operation log) in some kind of database so that the next attempt can be checked for duplication. It also requires the ability to update the resource and the operation log <strong><em>atomically, </em></strong>in order to guard against failures. The processing flow is illustrated as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rd7p9A9d40Dsd3At0qrRHQ.png" /><figcaption>Flowchart of the lock-check-execute approach</figcaption></figure><h4>Example</h4><p>Let’s demonstrate how to apply the <em>lock-check-execute</em> approach to a familiar example: inventory reservation. As mentioned earlier, the order_id and sku_id can be used as the idempotency key.</p><p>Assume the MySQL tables are defined as follows:</p><pre>CREATE TABLE inventory_tab (<br>    `sku_id` INT NOT NULL PRIMARY KEY,<br>    `count` INT NOT NULL DEFAULT 0,<br>    `price` DOUBLE NOT NULL DEFAULT 0,<br>    `updated_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP<br>);<br>CREATE TABLE inventory_dedup_tab (<br>    `id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,<br>    `order_id` BIGINT NOT NULL,<br>    `sku_id` INT NOT NULL,<br>    `quantity` INT NOT NULL DEFAULT 0,<br>    `updated_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,<br>    UNIQUE KEY (`order_id`, `sku_id`)<br>);</pre><p>And the pseudo code of the processing logic is shown below:</p><pre>START TRANSACTION;</pre><pre>INSERT INTO inventory_dedup_tab(order_id, sku_id, `quantity`)<br>    VALUES(<a href="http://twitter.com/OrderNumber"><strong>@OrderNumber</strong></a><strong>, </strong><a href="http://twitter.com/Sku"><strong>@Sku</strong></a>, <a href="http://twitter.com/Quantity">@Quantity</a>);</pre><pre>UPDATE inventory_tab set count=count+<a href="http://twitter.com/Quantity">@Quantity</a> <br>    WHERE <strong>sku_id=</strong><a href="http://twitter.com/Sku"><strong>@Sku</strong></a><strong> AND count+</strong><a href="http://twitter.com/Quantity"><strong>@Quantity</strong></a><strong>&gt;=0</strong>; <br>SELECT ROW_COUNT() INTO <a href="http://twitter.com/affected_rows">@affected_rows</a>;</pre><pre>IF (<a href="http://twitter.com/affected_rows">@affected_rows</a> &gt; 0) THEN<br>    COMMIT; <br>ELSE<br>    ROLLBACK;<br>END IF;</pre><p>The processing flow for this particular example would then look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*3I0uziladmNAcqluBV6aag.png" /><figcaption>Processing flow for reserving inventory idempotently</figcaption></figure><p>Of course, locking is expensive and distributed locking is even error prone, we should try to avoid it if possible. In this example, we can actually skip checking for duplication altogether, because the operation log is able to guarantee only one attempt will succeed. The simplified flowchart is as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*RRXNPmYrTH9ZJXFFgBkUsA.png" /><figcaption>Simplified processing flow for reserving inventory idempotently</figcaption></figure><h3>Idempotency vs Optmistic Locking</h3><p>Again, I’d like to highlight that optimistic locking does not guarantee idempotency. Please refer to my earlier post for a bit more details: <a href="https://medium.com/@zongwb/optimistic-locking-idempotency-936b87d1adc0">https://medium.com/@zongwb/optimistic-locking-idempotency-936b87d1adc0</a></p><h3>Wrap up</h3><p>Idempotency is a desired property for any API to have, as it greatly simplifies the calling clients. I would urge all API developers to keep this in mind and strive for idempotency whenever possible. It is also a good practice to document if an API is idempotent and the side effects if not.</p><p>Again, thanks for reading :-)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d607895c2b93" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimistic Locking != Idempotency]]></title>
            <link>https://zongwb.medium.com/optimistic-locking-idempotency-936b87d1adc0?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/936b87d1adc0</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[idempotency]]></category>
            <category><![CDATA[optimistic-locking]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Wed, 15 Jan 2020 06:40:41 GMT</pubDate>
            <atom:updated>2020-01-15T06:47:42.160Z</atom:updated>
            <content:encoded><![CDATA[<h3>Optimistic Locking != Idempotency</h3><p>The purpose of this essay is to highlight that optimistic locking and idempotency are two <strong><em>distinct</em></strong> concepts. I know this is <em>not</em> new, but I’ve seen enough online writings that confuse the two concepts and even claim that one can simply use optimistic locking to achieve idempotency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*PivdwYuqLle2-UNTuDDwog.jpeg" /></figure><h4>Optimistic Locking</h4><p>Optimistic locking is based on <em>atomic</em> <a href="https://en.wikipedia.org/wiki/Compare-and-swap">Compare and Swap (CAS)</a> primitives, which means to check if the state has changed since the beginning of the operation when committing.</p><p>Let’s use an example to quickly illustrate the idea of optimistic locking. In particular, we focus on the optimistic locking use case involving a database.</p><p>Image a use case of deducting payment. If a user has $100 balance and the item costs $80, then the final balance is $20. However, if the item costs $105, the deduction should fail.</p><p>The most common approach to optimistic locking, to my knowledge, is versioning. A simple implementation (pseudocode) is illustrated below:</p><pre>SELECT balance, version FROM balance_tab WHERE uid=$uid;</pre><pre>If ($balance &gt; $price) {<br>    $new_balance = $balance - $price <br>    $version_new = $version + 1<br>} else {<br>    return “not enough balance”;<br>}</pre><pre>UPDATE balance_tab SET balance=$new_balance, version=$version_new WHERE uid=$uid AND version=$version</pre><p>Note: The above implementation is free of the <a href="https://en.wikipedia.org/wiki/ABA_problem">ABA</a> problem.</p><h4>Idempotency</h4><p>Definition from <a href="https://en.wikipedia.org/wiki/Idempotence">wiki</a>: <em>Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application.</em></p><p>Basically, in the context of software, a read operation is idempotent but an update is generally not. I will not go into details of how to implement idempotency, which is a big topic and fully deserves a separate article. (Of course, you can find plenty of materials online.)</p><h4>Optimistic Locking != Idempotency</h4><p>Even if your API utilises optimistic locking, it is not automatically idempotent. To prove this, we only need a single counter example. So let’s illustrate with the above example.</p><p>Imagine the first deduction request is processed but the client does not receive the response due to network error. It’s common that the client would retry the <em>same</em> request. The following shows how the data changes over time.</p><pre>Task: deduct 10 from the account<br>Initial: balance = 100, version=10<br>Expect: balance = 90, version = 11</pre><pre>Request #1:<br>version=10  ⇒ version=11<br>balance=100 ⇒ balance=90</pre><pre>Request #2 (duplicate due to client retrying):<br>version=11 ⇒ version=12<br>balance=90 ⇒ balance=80</pre><p>You can see that the result is semantically wrong, i.e. a double deduction problem.</p><p>It is clear in this example that even if we implement an API using optimistic locking, we must still make extra effort to make the API idempotent.</p><h4>Conclusion</h4><p>Optimistic locking and idempotency are two distinct concepts.</p><p>Optimistic locking is basically a locking mechanism that avoids data corruptions. If applied twice, it can possibly lead to semantically wrong data. Idempotency, on the other hand, is more like a property of a system and it can must be implemented with care.</p><p>Depending on the application, it could be the case that one can utilise optimistic locking to achieve idempotency, but I think that is more like an exception than a rule in general.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=936b87d1adc0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Eventual Consistency in Cache Update with Read/Write Split]]></title>
            <link>https://zongwb.medium.com/eventual-consistency-in-cache-update-with-read-write-split-9861a54e618a?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9861a54e618a</guid>
            <category><![CDATA[read-write-split]]></category>
            <category><![CDATA[binlog-parser]]></category>
            <category><![CDATA[cache-invalidation]]></category>
            <category><![CDATA[eventual-consistency]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Wed, 13 Nov 2019 08:57:06 GMT</pubDate>
            <atom:updated>2020-05-28T09:48:37.918Z</atom:updated>
            <content:encoded><![CDATA[<h3>Eventual Consistency for Cache Update with Read-Write Splitting</h3><p>We all know that cache update/invalidation is hard to do correctly. Things can become particularly interesting when you set up the DB for read-write splitting. In this post, we will talk about how to achieve eventual consistency for cache update with read-write splitting.</p><h3><strong>Background</strong></h3><p>First, let’s review the Cache Aside Pattern. A more detailed description can be found in my previous post <a href="https://medium.com/@zongwb/how-to-invalidate-or-update-cache-correctly-5dce2db9dde5">here</a>.</p><ul><li>The read path</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*SHebjDMFy2UzsZW051tCrQ.png" /><figcaption>Cache aside pattern: read path</figcaption></figure><ul><li>The write path (cache update/invalidation)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*ltC6vp0xa3v09Fd3-JgRWw.png" /><figcaption>Cache update</figcaption></figure><p>When we set up read-write splitting for our MySQL database, it looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/1*q5xBljrdc5XucjJaDIL8Tg.jpeg" /></figure><h3><strong>The problem</strong></h3><p>Now, when caching is combined with read-write splitting, something interesting happens, as illustrated below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*1G641m0fucYaSZMcockXoQ.png" /><figcaption>Problem: Cache update with read-write split</figcaption></figure><p>The sequence of actions are:</p><ol><li>Client A updates the data in the <strong><em>master</em></strong> DB</li><li>Client B tries to read the data, but it’s not found in the cache</li><li>Client B loads the data from the <strong><em>slave</em></strong> DB</li><li>Client A deletes the data from cache (not present anyway)</li><li>Data is synced from the master to the slave DB (background job)</li><li>Client B writes the data to the cache (as it was a cache miss previously), and the data is <em>stale</em> (different from the master copy).</li></ol><p>As you can see, if things happen in the above sequence, we end up having inconsistent data in the DB and cache!</p><h3><strong>The solution</strong></h3><p>The root cause for the data inconsistency is the delay in syncing the data from the master to the slave DB. To fix it, we should delete (or update) the cache <strong><em>after the slave DB is updated</em></strong>. We can parse the binlog of the slave DB(not master!) to capture the update event, then delete/update the cache accordingly, as illustrated below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*EOrd902pffwK5_JWEbndZg.png" /><figcaption>Solution: Cache update with read-write split</figcaption></figure><p>The sequence of actions are:</p><ol><li>Client A updates the data in the <strong><em>master</em></strong> DB</li><li>Client B tries to read the data, but it’s not found in the cache</li><li>Client B loads the data from the <strong><em>slave</em></strong> DB</li><li>Client B writes the data to the cache (as it was a cache miss previously), and the data is <em>stale</em> (different from the master copy)</li><li>Data is synced from the master to the slave DB (background job)</li><li>The binlog parser captures the update event of the <strong><em>slave</em></strong> DB and sends the event to an MQ; The update logic, which subscribes to the MQ, picks up the event and deletes the (stale) data in the cache. The cached data is <em>eventually</em> consistent!</li></ol><p>There is one edge case, though: If the time lag between step 4 and 3 is very long, step 6 may happen before step 4. Arguably, it takes much longer for the update event to go through the cache update pipeline and the probability of step 6 happening before step 4 should be sufficiently low in practice.</p><p>PS: For the binlog parsers, there are a few opensource implementations such as <a href="https://github.com/alibaba/canal">Canal</a> (Alibaba) and <a href="https://debezium.io/documentation/reference/0.10/connectors/mysql.html">Debezium</a> (Redhat).</p><p>As a side note, the cache update pipeline in the above diagram can be actually generalised to a powerful data event framework, which is able to capture all sorts of update events and process them for different purposes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*0WIBmOpKCxOPUIjOg0mrvQ.png" /><figcaption>Generic data event processing framework</figcaption></figure><h3><strong>Wrap up</strong></h3><p>In this post, I tried to describe the tricky situation of updating/invalidating cache with MySQL set up for read-write splitting, and how to achieve eventual consistency by parsing the binlog (of the slave DB) to capture the update event.</p><p>This is yet another example of designing for eventual consistency. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9861a54e618a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Task Queue and Worker Pool: How to Configure Them?]]></title>
            <link>https://zongwb.medium.com/task-queue-and-worker-pool-how-to-configure-them-68079cbbff74?source=rss-76d580d9531e------2</link>
            <guid isPermaLink="false">https://medium.com/p/68079cbbff74</guid>
            <category><![CDATA[task-queues]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[worker-pool]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Wenbo Zong]]></dc:creator>
            <pubDate>Wed, 02 Oct 2019 08:50:07 GMT</pubDate>
            <atom:updated>2019-10-02T08:50:07.558Z</atom:updated>
            <content:encoded><![CDATA[<p>This essay discusses some practical tips for configuring the task queue and worker pool.</p><h3><strong>Background</strong></h3><p>Task queue is a common tool for traffic shaping. Basically, a task queue acts as a buffer between the requests (tasks) and processors to smooth the intermittent heavy (bursty) loads. Such traffic shaping has a few important benefits:</p><ul><li>Maximise the system throughput by reducing unnecessary overhead (e.g. context switching).</li><li>Protect the backend resources, as burst loads can cause (1) processes/services to fail or even crash (e.g. OOM), or (2) requests to time out due to slow response. This is often an overlooked aspect!</li><li>Make systems behaviour more predictable and easier to reason about, and hence allow optimal configuration (instead of guess work).</li></ul><p>Typically, a task queue is used together with a worker pool, in a producer-consumer pattern, as illustrated below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*NWvh7ZRtnuclPCmRSu2i4w.jpeg" /><figcaption>Typical structure of a task queue with a worker pool</figcaption></figure><p>With a task queue, it’s easy to apply the “fail fast, fail early” principle.</p><ul><li><em>Enqueue logic</em>: If queue is full, drop the request and return an error to client immediately. <strong>Fail early!</strong></li><li><em>Dequeue logic</em>: Check if there is enough time to process the request; if not, discard and return error to client. <strong>Fail fast!</strong></li></ul><h3><strong>Queue Occupancy</strong></h3><p>Let’s look at the various state the queue may be in depending on the traffic volume.</p><ul><li><em>Normal traffic</em>: Queue should be empty, as every incoming request should be immediately picked up by a worker. In this case, the client only experiences processing delay.</li><li><em>Heavy traffic</em>: Queue should be occupied but not fully and all workers are busy. In this case, the client experiences both waiting and processing delay.</li><li><em>Bursty traffic</em>: queue is quickly filled up then clears up slowly. All workers are busy, then some become idle. The client experiences both waiting and processing delay.</li><li><em>Overwhelming traffic</em>: queue is fully occupied, and all workers are busy. The client experiences the maximum waiting delay and processing delay.</li></ul><h3><strong>Design Parameters</strong></h3><p>Now, we turn to the main topic of this essay: How to configure the task queue and worker pool? The following parameters need to be considered carefully:</p><ul><li>Number of workers</li><li>Length of the queue</li><li>Client timeout (only for synchronous request-response type)</li></ul><p>Here, is it true that the more, the merrier? Meaning, should we always aim to have longer queues, more workers?</p><p><strong>Number of workers</strong></p><p>It depends on the nature of the application. For <em>computation-bound</em> applications, the number of workers should be directly proportional to the number of CPU cores; I’ve seen suggestions to set to 2x the number of CPU cores. For example, if you are running on a 16-core computer, you can set to 32 workers.</p><p>For <em>IO-bound</em> applications, it depends on the bandwidth of the hard disk. My own experience shows that 10~20 concurrent IO threads can maximise the hard disk bandwidth.</p><p>For <em>network-bound</em> applications, it depends on the external service’s capacity. Normally, the workers should not saturate the external service’s capacity (especially if there are other applications using that same service).</p><p><strong>Queue length and client timeout</strong></p><p>These two parameters are closely related and need to satisfy some constraints.</p><ul><li><em>avg_proc_delay</em> (aka 1/<em>throughput_per_worker</em>): Average time needed to process a task (after dequeuing from the queue)</li><li><em>max_wait_in_queue</em>: When dequeuing a task, discard the task if waiting time &gt; <em>max_wait_in_queue</em></li><li><em>client_timeout</em>: ~= <em>max_wait_in_queue</em> + <em>avg_proc_delay</em> + <em>network_delay</em></li><li><em>queue_length</em>: Satisfies <em>queue_length</em> / (<em>num_workers</em> * <em>throughput_per_worker</em>) &lt; <em>client_timeout</em></li></ul><p>Let’s use one example to illustrate how to arrive at the right values. Imagine a computation bound application running on a 24-core server. Assume that</p><ul><li><em>throughput_per_worker</em> = 100 QPS (aka <em>avg_proc_delay</em> = 10ms)</li><li><em>num_workers</em> = 24 (i.e. same as the number of cores)</li><li><em>client_timeout</em> = 5sec (arbitrary, but sensible)</li></ul><p>Then, we would have</p><p><em>max_wait_in_queue</em> ~= 5sec-10ms = 4.99sec (not accounting for network_delay!), and</p><p><em>queue_length</em>~= 4.99 * (100*24) = 11,976.</p><h3>Wrap up</h3><p>It must be emphasised that the above descriptions assume the tasks are uniform in nature, meaning they take roughly the same time to process. If the tasks are diverse, it becomes almost impossible to estimate the optimal settings as the processing delay for a worker changes dynamically. Then it would be advisable to consider to use a different worker pool for each type of tasks, something like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*k5ZXVJ4dmqT1amC6lRvsrA.jpeg" /></figure><p>That’s it. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68079cbbff74" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>