<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Felipe Ascari on Medium]]></title>
        <description><![CDATA[Stories by Felipe Ascari on Medium]]></description>
        <link>https://medium.com/@felipe.ascari_49171?source=rss-97fb4100a5de------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xV2mfTn7xnCcAya5HFUudQ.png</url>
            <title>Stories by Felipe Ascari on Medium</title>
            <link>https://medium.com/@felipe.ascari_49171?source=rss-97fb4100a5de------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:14:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@felipe.ascari_49171/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Green Tea: Understanding Go’s Garbage Collector]]></title>
            <link>https://medium.com/@felipe.ascari_49171/green-tea-understanding-gos-garbage-collector-21cc1bc08725?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/21cc1bc08725</guid>
            <category><![CDATA[green-tea]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[new-feature-releases]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[garbage-collector]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Tue, 24 Feb 2026 21:51:50 GMT</pubDate>
            <atom:updated>2026-02-24T21:51:50.441Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How Go 1.26’s GC improvements work and how to observe them in practice</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAbDzzBW5C3YcMbtoPgpCQ.png" /></figure><h3>Introduction</h3><p>The Go team has been quietly doing great work on the runtime for years. Every release brings improvements that most developers never notice, and that is exactly the point. Go 1.26 is no different. It ships with a set of GC improvements called “Green Tea,” and the best part is that you get them for free. There is nothing to configure, nothing to opt into. Just upgrade to Go 1.26 and the new GC behavior is active by default.</p><p>Garbage collection is one of those things Go developers rarely think about until they have to. The runtime handles memory automatically, and for most services that is enough. But when latency matters, when you are running a high-throughput API or operating under tight memory constraints, understanding how the GC behaves becomes important.</p><p>Green Tea is not a rewrite. It refines the existing concurrent collector with better pacing, lower CPU overhead, and smarter heap sizing. This article explains what those improvements mean in practice and shows them using a demo application with a blockchain workload and a set of monitoring endpoints.</p><h3>A Brief History of Go’s Garbage Collector</h3><p>Understanding where Go’s GC is today requires knowing where it came from.</p><ul><li><strong>Go 1.0–1.3</strong>: A stop-the-world mark-sweep collector. The entire program paused while the GC ran. Simple to implement, painful for latency.</li><li><strong>Go 1.5</strong>: The GC became concurrent. Most of the marking work now happens alongside the running application, which dramatically reduced pause times.</li><li><strong>Go 1.8</strong>: Stop-the-world pauses dropped below 100 microseconds in typical workloads.</li><li><strong>Go 1.19</strong>: Introduced GOMEMLIMIT, a soft memory limit that lets the runtime trade CPU time for memory headroom.</li><li><strong>Go 1.26 (“Green Tea”)</strong>: Improved GC pacing, reduced write barrier overhead, and more accurate heap goal calculations.</li></ul><p>Each release moved the GC further from something you need to manage and closer to something that just works.</p><h3>What “Green Tea” Actually Changes</h3><p>The name is informal, it comes from an internal proposal, but the changes are concrete. Green Tea focuses on four areas.</p><p><strong>Pacing</strong>: The GC scheduler now makes better decisions about when to start a collection cycle. The goal is to start early enough to finish before the heap limit is reached, but not so early that CPU time is wasted.</p><p><strong>Write barrier overhead</strong>: During concurrent marking, the runtime tracks pointer writes made by the application. Green Tea reduces the cost of those write barriers, which shows up as lower GCCPUFraction in production workloads.</p><p><strong>Heap goal accuracy</strong>: The formula heap_goal = live_heap * (1 + GOGC/100) has always been the baseline, but the actual trigger lagged behind the ideal in bursty allocation scenarios. Green Tea tightens that gap.</p><p><strong>CPU fraction</strong>: The combined effect is that GC activities consume a smaller share of total CPU time for the same heap size and allocation rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*1wUS1Rx8zs5uUX0ppzDp7Q.png" /></figure><ul><li><strong>White</strong>: Not yet visited. These are candidates for collection.</li><li><strong>Gray</strong>: Visited, but their outgoing references have not been scanned yet.</li><li><strong>Black</strong>: Fully scanned and confirmed reachable.</li></ul><p>The GC begins by marking all root objects (globals, goroutine stacks) gray. It then processes gray objects one by one, turning them black and graying their references, until no gray objects remain. Everything still white at the end is garbage.</p><h3>Write Barriers</h3><p>The tricky part of concurrent marking is that the application keeps running while the GC marks. A goroutine can overwrite a pointer while the GC is mid-scan, which breaks the invariant that black objects never point to white ones. Write barriers solve this by intercepting pointer writes and ensuring the referenced object is shaded gray before the write completes.</p><p>Green Tea reduces the cost of these barriers, which is why the CPU fraction improvements are most visible in allocation-heavy code paths.</p><h3>GC Pacing</h3><p>The runtime uses GOGC to decide when to trigger a new collection:</p><pre>heap_goal = live_heap * (1 + GOGC/100)</pre><p>With the default GOGC=100, a new GC cycle begins when the heap reaches twice the size of live data after the previous collection. Lower values mean more frequent collections with less memory usage. Higher values do the opposite. GOMEMLIMIT adds a ceiling that prevents the heap from growing past a fixed size, regardless of GOGC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*2W4Ykl2TrxSlVDoisavZTw.png" /></figure><h3>Practical Examples</h3><p>The following examples use the demo application. All endpoints run on http://localhost:8080 after starting the server with go run cmd/api/main.go.</p><h3>1. Reading Runtime Metrics</h3><p>The runtime/metrics package, introduced in Go 1.16, provides structured and named access to GC internals. Unlike runtime.ReadMemStats, which requires stopping all goroutines briefly to collect a consistent snapshot, runtime/metrics reads are cheaper and more granular. The /gc/metrics endpoint collects a curated set of these metrics and returns them in a single response:</p><pre>curl http://localhost:8080/gc/metrics | jq</pre><p>A typical response:</p><pre>{<br>  &quot;/gc/heap/allocs:bytes&quot;: 2457600,<br>  &quot;/gc/heap/allocs:objects&quot;: 4521,<br>  &quot;/gc/heap/frees:bytes&quot;: 1843200,<br>  &quot;/gc/heap/frees:objects&quot;: 3890,<br>  &quot;/gc/heap/goal:bytes&quot;: 4194304,<br>  &quot;/gc/heap/objects:objects&quot;: 631,<br>  &quot;/gc/heap/tiny/allocs:objects&quot;: 210,<br>  &quot;/gc/cycles/total:gc-cycles&quot;: 15,<br>  &quot;/gc/cycles/forced-gc:gc-cycles&quot;: 2,<br>  &quot;/gc/pause/total:seconds&quot;: 0.001234,<br>  &quot;/memory/classes/total:bytes&quot;: 8388608,<br>  &quot;/memory/classes/heap/released:bytes&quot;: 1048576,<br>  &quot;/sched/goroutines:goroutines&quot;: 12,<br>  &quot;alloc_rate_mb_per_sec&quot;: 2.34,<br>  &quot;go_version&quot;: &quot;go1.26&quot;<br>}</pre><p>The implementation reads exactly the metrics it needs using metrics.Read:</p><pre>sample := []metrics.Sample{<br>    {Name: &quot;/gc/heap/allocs:bytes&quot;},<br>    {Name: &quot;/gc/heap/allocs:objects&quot;},<br>    {Name: &quot;/gc/heap/frees:bytes&quot;},<br>    {Name: &quot;/gc/heap/frees:objects&quot;},<br>    {Name: &quot;/gc/heap/goal:bytes&quot;},<br>    {Name: &quot;/gc/heap/objects:objects&quot;},<br>    {Name: &quot;/gc/heap/tiny/allocs:objects&quot;},<br>    {Name: &quot;/gc/cycles/total:gc-cycles&quot;},<br>    {Name: &quot;/gc/cycles/forced-gc:gc-cycles&quot;},<br>    {Name: &quot;/gc/pause/total:seconds&quot;},<br>    {Name: &quot;/memory/classes/total:bytes&quot;},<br>    {Name: &quot;/memory/classes/heap/released:bytes&quot;},<br>    {Name: &quot;/sched/goroutines:goroutines&quot;},<br>}<br>metrics.Read(sample)</pre><p>/gc/heap/goal:bytes is the most actionable field. It is the heap size the GC is targeting before triggering the next cycle. If this value is consistently close to /memory/classes/total:bytes, the heap is running hot and you should consider a lower GOGC or a GOMEMLIMIT boundary. /gc/heap/tiny/allocs:objects tracks small allocations under 16 bytes that the allocator batches together. A high count there is normal and not a concern.</p><h3>2. Allocation Patterns</h3><p>Not all allocations behave the same way under the GC. Objects that are created and immediately discarded (short-lived) are cheap: they are never marked during the concurrent phase and collected in bulk. Objects that survive multiple GC cycles (long-lived) increase the live heap size, raise the heap goal, and leave less room for new allocations before the next cycle triggers.</p><p>The /stress endpoint supports three patterns: short-lived, long-lived, and mixed. It runs the workload across multiple goroutines concurrently. Each allocation is 1 MB:</p><pre># Short-lived: allocate 1MB, hash it with SHA-256, discard<br>curl -X POST http://localhost:8080/stress \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 100, &quot;goroutines&quot;: 4, &quot;pattern&quot;: &quot;short-lived&quot;}&#39;<br><br># Long-lived: keep all 1MB allocations referenced until the goroutine exits<br>curl -X POST http://localhost:8080/stress \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 100, &quot;goroutines&quot;: 4, &quot;pattern&quot;: &quot;long-lived&quot;}&#39;<br><br># Mixed: 50% short-lived, 50% long-lived<br>curl -X POST http://localhost:8080/stress \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 100, &quot;goroutines&quot;: 4, &quot;pattern&quot;: &quot;mixed&quot;}&#39;</pre><p>Response:</p><pre>{<br>  &quot;duration&quot;: &quot;2.345s&quot;,<br>  &quot;goroutines&quot;: 4,<br>  &quot;allocations&quot;: 100,<br>  &quot;pattern&quot;: &quot;short-lived&quot;,<br>  &quot;gc_collections&quot;: 12,<br>  &quot;gc_pause_total_ms&quot;: 1.23,<br>  &quot;gc_cpu_fraction&quot;: 0.015,<br>  &quot;heap_objects_delta&quot;: 88,<br>  &quot;memory_delta_mb&quot;: -1.4,<br>  &quot;final_alloc_mb&quot;: 3.2,<br>  &quot;num_goroutines&quot;: 6,<br>  &quot;throughput_mb_per_sec&quot;: 170.5<br>}</pre><p>gc_cpu_fraction is the number to watch. With short-lived allocations it will be higher because the GC runs frequently, but heap_objects_delta will be close to zero because most objects were collected. With long-lived allocations gc_cpu_fraction drops because fewer cycles trigger, but final_alloc_mb stays elevated because the live set grew. <br>The mixed pattern sits between the two and is the most representative of real-world services that cache some data while processing transient requests.</p><h3>3. Controlled Benchmark</h3><p>The /gc/benchmark endpoint isolates the GC behavior for a specific allocation size and count. It allocates the requested objects, retains them or not based on the pattern, then calls runtime.GC() explicitly before reading the final MemStats. That forced collection ensures the after-snapshot reflects a settled heap, not one that is mid-cycle:</p><pre>curl -X POST http://localhost:8080/gc/benchmark \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 10000, &quot;size_kb&quot;: 1, &quot;pattern&quot;: &quot;short-lived&quot;}&#39;</pre><p>Response:</p><pre>{<br>  &quot;duration_ms&quot;: 45.2,<br>  &quot;alloc_before_mb&quot;: 12.5,<br>  &quot;alloc_after_mb&quot;: 4.1,<br>  &quot;heap_objects_delta&quot;: -4389,<br>  &quot;gc_runs&quot;: 4,<br>  &quot;gc_pause_total_ms&quot;: 1.2,<br>  &quot;gc_cpu_fraction&quot;: 0.012,<br>  &quot;throughput_mb_per_sec&quot;: 234.5<br>}</pre><p>heap_objects_delta is the net change in live heap objects (HeapObjects after minus before). For short-lived patterns it should be negative after the forced GC, meaning more objects were collected than remained live. For long-lived patterns it will be a large positive number equal to the allocation count. throughput_mb_per_sec is computed as (allocations * size_kb / 1024) / duration_seconds, so it reflects raw allocation throughput, not network throughput.</p><h3>4. Finalizers</h3><p>runtime.SetFinalizer registers a function that the GC calls before reclaiming an object. It is sometimes used for cleanup logic, but it has real costs: the GC does not guarantee when the finalizer runs. Finalizers delay the collection of their objects by at least one GC cycle. And objects with finalizers cannot be part of a reference cycle.</p><pre>curl -X POST http://localhost:8080/gc/finalizers \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;count&quot;: 100, &quot;trigger_gc&quot;: true}&#39;</pre><p>Response:</p><pre>{<br>  &quot;objects_created&quot;: 100,<br>  &quot;finalizers_executed&quot;: 100,<br>  &quot;duration_ms&quot;: 12.3,<br>  &quot;gc_runs&quot;: 2,<br>  &quot;finalizer_latency_ms&quot;: 0.123,<br>  &quot;warning&quot;: &quot;Finalizers are not guaranteed to run immediately; some may execute on future GC cycles&quot;<br>}</pre><p>The demo calls runtime.GC() twice. The first cycle marks the objects unreachable and queues their finalizers. The second cycle actually reclaims the memory. This two-cycle delay is inherent to how finalizers work and is one reason they should not be used for anything time-sensitive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*MFVMJ_RRVUPOlCB-FZE3iA.png" /></figure><h3>5. Profiling</h3><p>For deeper analysis, the /gc/profile endpoint generates pprof profiles and writes them to the OS temp directory. It supports four profile types: heap, cpu, goroutine, and allocs:</p><pre># Heap profile: forces a GC first, then writes live heap data<br>curl -X POST http://localhost:8080/gc/profile \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;profile_type&quot;: &quot;heap&quot;}&#39;<br><br># CPU profile, collected over 5 seconds<br>curl -X POST http://localhost:8080/gc/profile \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;profile_type&quot;: &quot;cpu&quot;, &quot;duration_seconds&quot;: 5}&#39;<br><br># Goroutine profile: snapshot of all current goroutine stacks<br>curl -X POST http://localhost:8080/gc/profile \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;profile_type&quot;: &quot;goroutine&quot;}&#39;</pre><p>The response includes the file path and the exact command to open the profile:</p><pre>{<br>  &quot;profile_path&quot;: &quot;/tmp/heap.pprof&quot;,<br>  &quot;view_command&quot;: &quot;go tool pprof -http=:8080 /tmp/heap.pprof&quot;,<br>  &quot;file_size_kb&quot;: 42,<br>  &quot;duration_ms&quot;: 3.1,<br>  &quot;gc_runs&quot;: 1,<br>  &quot;heap_alloc_mb&quot;: 4.2<br>}</pre><p>Run the command from view_command to open the profile in the browser. In the flame graph, the inuse_space view shows what is live on the heap right now. The alloc_objects view shows cumulative allocations over the process lifetime. Comparing the two tells you whether heap pressure comes from objects that are truly long-lived or from a high allocation rate that the GC is keeping up with.</p><h3>6. Parallel Mining with GC Metrics</h3><p>The /mine endpoint runs a CPU-intensive proof-of-work operation. Each goroutine mines its own block concurrently. With goroutines: 4, four blocks are appended to the chain in parallel, each identified by the data string suffixed with the worker ID, for example test-block-worker-0. The use case snapshots MemStats before and after via runtime.ReadMemStats to isolate the GC impact of the operation:</p><pre>curl -X POST http://localhost:8080/mine \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;data&quot;: &quot;test-block&quot;, &quot;goroutines&quot;: 4}&#39;</pre><p>Response:</p><pre>{<br>  &quot;blocks&quot;: [...],<br>  &quot;duration&quot;: &quot;1.234s&quot;,<br>  &quot;goroutines&quot;: 4,<br>  &quot;total_blocks&quot;: 5,<br>  &quot;gc_runs&quot;: 0,<br>  &quot;gc_pause_ms&quot;: 0.0,<br>  &quot;heap_delta_mb&quot;: 0.3,<br>  &quot;gc_cpu_fraction&quot;: 0.002<br>}</pre><p>total_blocks is the full chain length after mining, including the genesis block. A low gc_cpu_fraction here is expected. Proof-of-work is CPU-bound with minimal allocations. The inner loop increments a nonce and hashes it repeatedly with no heap growth, so the GC has almost nothing to do. Try running the same call with GOGC=50 and GOGC=200 and you will see that gc_runs stays near zero in both cases, which confirms the workload is not allocation-driven.</p><h3>Benchmarks and Analysis</h3><p>To see how GOGC affects GC behavior in practice, start the server with different values and run the benchmark endpoint against each. Kill the previous instance before starting the next to avoid port conflicts:</p><pre># Default (GOGC=100)<br>GOGC=100 go run cmd/api/main.go &amp;<br>curl -X POST http://localhost:8080/gc/benchmark \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 50000, &quot;size_kb&quot;: 1, &quot;pattern&quot;: &quot;short-lived&quot;}&#39;<br>kill %1<br><br># Aggressive (GOGC=50) - more frequent GC, lower peak memory<br>GOGC=50 go run cmd/api/main.go &amp;<br>curl -X POST http://localhost:8080/gc/benchmark \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 50000, &quot;size_kb&quot;: 1, &quot;pattern&quot;: &quot;short-lived&quot;}&#39;<br>kill %1<br><br># Conservative (GOGC=200) - less frequent GC, higher peak memory<br>GOGC=200 go run cmd/api/main.go &amp;<br>curl -X POST http://localhost:8080/gc/benchmark \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;allocations&quot;: 50000, &quot;size_kb&quot;: 1, &quot;pattern&quot;: &quot;short-lived&quot;}&#39;<br>kill %1</pre><p>Expected behavior:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*9xY19QQ9ddNNa4cBZH3F5g.png" /></figure><p>Run each configuration at least five times and look at the median gc_pause_total_ms and gc_cpu_fraction. Single runs can be misleading because GC cycles do not align perfectly with the benchmark window.</p><h3>Practical Recommendations</h3><p>I am not a GC expert, and I will not pretend to have precise thresholds for every situation. What I can share is what I actually look at when something feels off.</p><p><strong>Start with </strong>gc_cpu_fraction<strong>.</strong> If the GC is spending a noticeably large share of CPU time, that is worth investigating. What &quot;noticeably large&quot; means depends on your service, but if you run the benchmark endpoint and the number feels high compared to what the application is doing, that is a signal to look deeper before changing anything.</p><p><strong>Profile before tuning.</strong> Changing GOGC without profiling first is guessing. Use the /gc/profile endpoint to get a heap profile, open it with go tool pprof, and find out where the allocations are actually coming from. The answer is usually not where you expect.</p><p>GOMEMLIMIT<strong> is worth knowing about if you run in containers.</strong> Without it, the runtime does not know there is a memory ceiling. Setting it to something close to your container limit gives the GC a boundary to work with. I would not pick a specific number here because it depends entirely on your workload.</p><p><strong>Finalizers are a last resort.</strong> If you find yourself reaching for runtime.SetFinalizer, try defer or an explicit Close method first. Finalizers work, but the two-cycle delay and the lack of timing guarantees make them hard to reason about.</p><p>These are not rules. They are the things I check first. The right answer always depends on what the profiler shows.</p><h3>Conclusion</h3><p>Go’s garbage collector has improved steadily over the years. Green Tea in Go 1.26 continues that work with tighter pacing, less write barrier overhead, and more accurate heap sizing. For most applications the improvement is transparent, latency gets slightly better and gc_cpu_fraction drops a bit. For allocation-heavy services, the difference is measurable.</p><p>The best way to understand it is to run the examples. Clone the repository, start the server, and compare the numbers across different patterns and GOGC values. The metrics the runtime exposes through runtime/metrics and runtime.ReadMemStats give you a complete picture of what the GC is doing at any point in time.</p><p>The full source code for all examples in this article is available at <a href="https://github.com/fascari/go-runtime-demo">github.com/fascari/go-runtime-demo</a>. Clone it, run the server, and experiment with the endpoints directly.</p><p><strong>References:</strong></p><ul><li><a href="https://tip.golang.org/doc/gc-guide">A Guide to the Go Garbage Collector</a></li><li><a href="https://pkg.go.dev/runtime/metrics">runtime/metrics package documentation</a></li><li><a href="https://pkg.go.dev/runtime">runtime package documentation</a></li><li><a href="https://go.dev/doc/go1.26">Go 1.26 Release Notes</a></li><li><a href="https://github.com/golang/go/issues/73581">Proposal: Green Tea GC improvements</a></li><li><a href="https://go.dev/ref/mem">Go memory model</a></li><li><a href="https://pkg.go.dev/runtime/pprof">pprof documentation</a></li></ul><p><em>This article is part of a series on Go runtime internals. Check out the previous piece on </em><a href="https://medium.com/@felipe.ascari_49171/understanding-gos-scheduler-how-goroutine-management-works-65131986ee2c"><em>Understanding Go’s Scheduler</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21cc1bc08725" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Distributed Locks and Fencing Tokens]]></title>
            <link>https://medium.com/@felipe.ascari_49171/distributed-locks-and-fencing-tokens-31904c71f61b?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/31904c71f61b</guid>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[distributed-locks]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[redis]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Thu, 22 Jan 2026 14:02:00 GMT</pubDate>
            <atom:updated>2026-01-22T14:02:00.229Z</atom:updated>
            <content:encoded><![CDATA[<p>A implementation in Go showing how fencing tokens prevent race conditions in distributed systems</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*sAGPyhVV5zxiCFbQLsQ8HA.png" /></figure><p>I recently came across a <a href="https://www.linkedin.com/posts/sukhad007_imagine-there-is-1-concert-ticket-left-two-activity-7416568411121532928-lcZH?utm_source=share&amp;utm_medium=member_desktop">LinkedIn post by Sukhad Anand</a> that explained a common concurrency problem. Imagine two users trying to buy the last concert ticket at the exact same millisecond. On a single server, you’d use a mutex. But with 10 servers? Both might check availability, both see “available,” and suddenly you’ve sold one seat twice.</p><p>The post introduced me to fencing tokens as a solution. I decided to build a proof of concept to really understand how they work. This article walks through that implementation.</p><h3>What Are Distributed Locks?</h3><p>When you have multiple servers that don’t share memory, they need a “source of truth” to coordinate access to shared resources. Redis is commonly used for this with the SETNX command (SET if Not eXists).</p><p>Here’s the basic idea.</p><pre>// Server A tries to acquire lock<br>SET ticket_45 &quot;server_A&quot; NX PX 5000<br>// NX: only set if key doesn&#39;t exist<br>// PX 5000: expire after 5 seconds</pre><p>If Redis returns OK, Server A got the lock. If Server B tries the same command while A holds the lock, Redis returns nil and B knows to back off.</p><h3>The Zombie Process Problem</h3><p>But there’s a subtle bug. What if Server A gets the lock, then freezes for 6 seconds due to garbage collection? Here’s what happens.</p><ol><li>Server A acquires lock (token 100)</li><li>Server A pauses for 6 seconds</li><li>Lock expires at 5 seconds</li><li>Server B acquires lock (token 101)</li><li>Server A wakes up and writes data (thinks it still has the lock)</li></ol><p>Now both servers are modifying the same resource. The lock didn’t protect anything.</p><h3>Enter Fencing Tokens</h3><p>The solution is simple but clever. Every time a lock is granted, Redis gives it a monotonically increasing number. The storage layer checks this number and rejects operations with old tokens.</p><p>Here’s how I implemented the token generation.</p><pre>func (rl *RedisLock) Acquire(ctx context.Context, key string) (int64, error) {<br>    lockKey := rl.lockKeyPrefix + key<br>    <br>    // Generate monotonically increasing token<br>    token, err := rl.client.Incr(ctx, rl.tokenKey).Result()<br>    if err != nil {<br>        return 0, fmt.Errorf(&quot;failed to generate token: %w&quot;, err)<br>    }<br>    <br>    entry := Entry{<br>        Token:      token,<br>        ServerID:   &quot;&quot;,<br>        AcquiredAt: time.Now(),<br>    }<br>    <br>    data, _ := json.Marshal(entry)<br>    <br>    // Try to acquire lock<br>    result, err := rl.client.SetNX(ctx, lockKey, data, rl.ttl).Result()<br>    if err != nil {<br>        return 0, fmt.Errorf(&quot;failed to acquire lock: %w&quot;, err)<br>    }<br>    <br>    if !result {<br>        return 0, ErrLockAcquisition<br>    }<br>    <br>    return token, nil<br>}</pre><p>The INCR command in Redis is atomic and guarantees that each token is unique and increasing.</p><h3>Validating Tokens at the Storage Layer</h3><p>The storage layer validates tokens before allowing any operation.</p><pre>func (ls *LockedStore) Save(key string, value any) error {<br>    return ls.lockManager.ExecuteWithLock(context.Background(), key, func(token int64) error {<br>        if !ls.validateToken(key, token) {<br>            return fmt.Errorf(&quot;token %d rejected: newer token already processed&quot;, token)<br>        }<br>        <br>        if err := ls.store.Save(key, value); err != nil {<br>            return err<br>        }<br>        <br>        ls.recordToken(key, token)<br>        return nil<br>    })<br>}<br><br>func (ls *LockedStore) validateToken(key string, token int64) bool {<br>    ls.mu.RLock()<br>    defer ls.mu.RUnlock()<br>    lastToken, exists := ls.lastProcessedToken[key]<br>    if !exists {<br>        return true<br>    }<br>    return token &gt; lastToken<br>}</pre><p>Going back to zombie example.</p><ol><li>Server A gets token 100</li><li>Server A freezes</li><li>Lock expires</li><li>Server B gets token 101 and writes successfully</li><li>Server A wakes up with token 100</li><li>Storage sees token 100 &lt; 101 and rejects the operation</li></ol><p>The zombie process can’t cause data corruption because its token is outdated.</p><h3>Why Lua Scripts Matter</h3><p>One tricky part is releasing locks. You can’t just do a GET followed by a DEL because another process might acquire the lock in between.</p><pre>// This has a race condition!<br>data := redis.Get(lockKey)<br>if data.Token == myToken {<br>    // Another process could acquire here<br>    redis.Del(lockKey)  // Might delete someone else&#39;s lock!<br>}</pre><p>The solution is a Lua script, which Redis executes atomically.</p><pre>if redis.call(&#39;get&#39;, KEYS[1]) == ARGV[1] then<br>    return redis.call(&#39;del&#39;, KEYS[1])<br>else<br>    return 0<br>end</pre><p>No other Redis command can execute while the script runs, making the check-and-delete operation safe.</p><h3>Why Redis Instead of ZooKeeper or Etcd?</h3><p>The original post mentions ZooKeeper and Etcd as options for distributed locking. These are consensus-based systems designed specifically for coordination. So why did I choose Redis?</p><p>For this proof of concept, Redis offered a simpler path to understanding the core concept of fencing tokens. It’s lightweight, easy to set up with Docker Compose, and most developers already know it. The focus was on demonstrating how fencing tokens solve the zombie process problem, not on building production-grade distributed consensus.</p><p>That said, there are important trade-offs. ZooKeeper and Etcd provide stronger guarantees through consensus algorithms like Raft or ZAB. They’re designed for coordination and handle network partitions more gracefully. If you’re building financial systems or managing critical inventory, those guarantees matter.</p><p>Redis with fencing tokens gets you pretty far. The token validation at the storage layer provides the safety you need for many use cases. But if you need absolute consistency across network partitions, ZooKeeper or Etcd are worth the additional complexity.</p><p>For learning and for many real-world applications, Redis strikes a good balance between simplicity and correctness.</p><h3>Testing the Implementation</h3><p>I built this as a simple key-value store with three operations: Save, Retrieve, and Delete. Each operation acquires a lock with a fencing token. You can run it with either in-memory storage or Redis.</p><p>To test it yourself, run these commands.</p><pre># Start Redis<br>docker compose up -d<br><br># Run with Redis storage<br>make run<br><br># Test concurrent writes<br>curl -X POST http://localhost:8080/save \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;key&quot;:&quot;test&quot;,&quot;value&quot;:&quot;hello&quot;}&#39;</pre><h3>When Do You Need This?</h3><p>Fencing tokens are necessary when you need strong consistency guarantees. Think financial transactions, inventory management, or any case where duplicate operations would be catastrophic.</p><p>For less critical scenarios, simple Redis locks without fencing tokens might be enough. The trade-off is between implementation complexity and correctness guarantees.</p><h3>Conclusion</h3><p>Distributed systems are hard because of these subtle timing issues. Fencing tokens give you a simple way to reason about correctness. No matter what delays or failures happen, operations with outdated tokens will always be rejected.</p><p>Building this proof of concept revealed something important. The devil is in the details. A basic distributed lock seems simple until you consider garbage collection pauses, network delays, or process crashes. Each of these can create windows where your assumptions break down.</p><p>Fencing tokens don’t eliminate these problems. They give you a mechanism to detect and reject stale operations. The storage layer becomes the final arbiter of correctness, not the lock itself. This shift in thinking is what makes the pattern powerful.</p><p>If you’re building systems where consistency matters, fencing tokens are worth the extra complexity. The alternative is discovering race conditions in production, which is always more expensive than getting it right upfront.</p><p>The code is straightforward enough that you can adapt it to your specific needs. Start with the lock manager, add token validation to your storage layer, and test thoroughly. Distributed systems don’t forgive shortcuts.</p><h3>References</h3><ol><li><a href="https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html">Martin Kleppmann — “How to do distributed locking”</a></li><li><a href="https://redis.io/docs/manual/patterns/distributed-locks/">Redis Documentation — Distributed Locks</a></li><li><a href="https://www.linkedin.com/posts/sukhad007_imagine-there-is-1-concert-ticket-left-two-activity-7416568411121532928-lcZH">Original LinkedIn Post by Sukhad Anand</a></li><li><a href="https://github.com/fascari/kv-store">Project Source Code</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31904c71f61b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Go’s Scheduler: How Goroutine Management Works]]></title>
            <link>https://medium.com/@felipe.ascari_49171/understanding-gos-scheduler-how-goroutine-management-works-65131986ee2c?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/65131986ee2c</guid>
            <category><![CDATA[memory-management]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[goroutines]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Thu, 18 Dec 2025 14:02:42 GMT</pubDate>
            <atom:updated>2025-12-18T14:02:42.311Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How Go efficiently manages thousands of concurrent tasks using the G-M-P model</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*f9kCroGIEZ6KPHRXHTCkSA.png" /></figure><h3>Introduction</h3><p>Understanding how the Go scheduler works has always been something I wanted to dive deeper into. The decisions we make when using goroutines and asynchronous structures have real implications on application performance and behavior, yet the scheduler often feels like a black box. This article serves to solidify this knowledge and compile information that can help others gain a better understanding of these internals.</p><p>Go manages <strong>thousands of goroutines</strong> using just a handful of CPU cores through its sophisticated <strong>scheduler</strong>. This scheduler multiplexes lightweight goroutines onto operating system threads with impressive efficiency, making Go’s concurrency model feel natural and performant.</p><p>The scheduler operates on the <strong>G-M-P model</strong>, implementing a work-stealing algorithm that keeps all CPU cores busy. Understanding how goroutines are scheduled, their state transitions, and how the runtime handles blocking operations reveals why Go excels at concurrent programming.</p><h3>The Problem with Traditional Threading</h3><p>Traditional threading models create a 1:1 mapping between application threads and OS threads. Each OS thread consumes about 2MB of stack memory and requires expensive kernel-level context switching. Creating 10,000 threads would require 20GB just for stacks, making this approach impractical for highly concurrent applications.</p><p>Go solves this with <strong>goroutines</strong>, which start with only 2KB of stack and grow dynamically. Context switching happens in user space, making it significantly faster. This allows applications to run 100,000+ goroutines efficiently.</p><p>The challenge becomes mapping thousands of goroutines onto a small number of OS threads while keeping all CPU cores busy. The Go scheduler solves this through the G-M-P model.</p><h3>The G-M-P Model</h3><p>The Go scheduler uses three components that work together to execute goroutines efficiently.</p><h4>Architecture Overview</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*mjRyOMYGOlu231H7AK4BAQ.png" /></figure><h4>G (Goroutine)</h4><p>A goroutine is the basic unit of execution in Go.</p><p>Each goroutine starts with just 2KB of stack that grows dynamically as needed. It contains saved registers for context switching and tracks what it’s waiting on, whether that’s a channel, timer, or other synchronization primitive. Goroutines are ultra-lightweight compared to OS threads, which is what makes Go’s concurrency model so powerful.</p><p><strong>Code example:</strong></p><pre>// internal/domain/blockchain/blockchain.go<br>func (bc *Blockchain) MineParallel(data string, numGoroutines int) ([]Block, time.Duration) {<br>    // ...<br>    for i := 0; i &lt; numGoroutines; i++ {<br>        wg.Add(1)<br>        go func(id int) {  // Creates a new Goroutine (G)<br>            defer wg.Done()<br>            <br>            // Each goroutine enters _Grunnable state<br>            // Scheduler assigns to an available P<br>            // Transitions to _Grunning when executing<br>            blockData := data + &quot;-worker-&quot; + strconv.Itoa(id)<br>            block := bc.AddBlock(blockData)<br>            blocksChan &lt;- block<br>        }(i)<br>    }<br>    // ...<br>}</pre><p>When you call go func(), the runtime allocates a new G structure, initializes it with 2KB of stack, places G in an execution queue, and then G waits to be scheduled.</p><h4>M (Machine/OS Thread)</h4><p>M represents a real operating system thread that executes goroutines.</p><p>Each M has a 1:1 mapping with an OS thread. It can be blocked by syscalls, but this won’t stop other goroutines from running. The number of Ms is limited by GOMAXPROCS, which defaults to the number of CPU cores. An M rapidly switches between goroutines, providing the illusion of parallel execution even when there are more goroutines than threads.</p><p><strong>Runtime configuration:</strong></p><pre>// cmd/api/main.go<br>func main() {<br>    // Sets number of OS threads that can execute Go code simultaneously<br>    runtime.GOMAXPROCS(runtime.NumCPU())<br>    // ...<br>}</pre><h4>P (Processor/Logical Processor)</h4><p>P represents a resource required to execute Go code. Think of it as a “context” or “logical CPU”.</p><p>The number of Ps is defined by GOMAXPROCS, which defaults to the number of CPU cores. Each P maintains a local queue of runnable goroutines and has its own memory cache (mcache) to reduce contention. When a P runs out of work, it can steal goroutines from other Ps, ensuring all CPU cores stay busy.</p><h3>The Relationship</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*CPOG7dg7WdQ_cbY0WifaDw.png" /></figure><h4><strong>Why this design?</strong></h4><p>The main insight is <strong>decoupling</strong>. Gs (work) can be moved between Ps, Ms (threads) can be blocked without blocking Ps, and Ps (contexts) can be transferred between Ms. This design allows the scheduler to keep Ms busy even when some are blocked, balance load across all available CPUs, and minimize lock contention with local queues.</p><h3>The Work-Stealing Algorithm</h3><p><strong>Work-stealing</strong> is a scheduling strategy where idle processors (Ps) actively “steal” work from busy processors instead of waiting for centralized task distribution. Each P maintains its own local queue of goroutines. When a P runs out of work, it looks for goroutines in other Ps’ queues and takes half of them. This decentralized approach eliminates bottlenecks and keeps all CPU cores productive.</p><h4>The Architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*sjPA4NZHSsvNKu05_1YVwQ.png" /></figure><h4>The Scheduling Loop</h4><p><strong>1. Check P’s local queue (61 times)</strong></p><pre>// Try to get G from local queue - fast path<br>if gp := p.runqget(); gp != nil {<br>    return gp<br>}</pre><p><strong>2. Check global queue (1 in 61)</strong></p><pre>// Every 61 ticks, check global queue for fairness<br>if schedtick % 61 == 0 {<br>    if gp := globrunqget(p, 1); gp != nil {<br>        return gp<br>    }<br>}</pre><p><strong>3. Steal from other Ps</strong></p><pre>// Try to steal from other Ps&#39; local queues<br>for i := 0; i &lt; 4; i++ {<br>    for enum := 0; enum &lt; gomaxprocs; enum++ {<br>        p2 := allp[...]<br>        if gp := runqsteal(p, p2); gp != nil {<br>            return gp<br>        }<br>    }<br>}</pre><p><strong>4. Check network poller</strong></p><pre>// Check if any goroutine is waiting for network I/O<br>if gp := netpoll(false); gp != nil {<br>    return gp<br>}</pre><h4>Work Stealing in Action</h4><p>Here’s how work-stealing keeps cores busy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*9KPEyW1svmU_wWjHaP018w.png" /></figure><p>Work-stealing provides significant benefits: all CPUs stay busy, load balancing happens automatically without manual intervention, and there’s no centralized coordinator that could become a bottleneck.</p><h3>Goroutine States</h3><p>Goroutines transition through different states as they execute, wait, and complete.</p><h4>The State Machine</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*6RNCUqig-zk0PwWN8J73XA.png" /></figure><h4>State Descriptions</h4><p>_Gidle Newly allocated goroutine structure that has not yet been initialized.</p><p>_Grunnable Ready to execute and waiting to be scheduled. The goroutine is in a run queue (either local or global).</p><p>_Grunning Currently executing on an M with access to a P, actively running user code.</p><p>_Gwaiting The goroutine is blocked waiting for something like a channel send/receive, lock acquisition, timer, or network I/O. It&#39;s not in any run queue during this time.</p><p>_Gsyscall (special) The goroutine is executing a blocking syscall. While the M is blocked, the P can be stolen by another M to keep running other goroutines. When the syscall completes, the goroutine returns to the _Grunnable state.</p><h4>Practical Example: Synchronization</h4><pre>// internal/domain/blockchain/blockchain.go<br>func (bc *Blockchain) AddBlock(data string) Block {<br>    bc.mu.Lock()          // May transition to _Gwaiting<br>    defer bc.mu.Unlock()  // if lock is held<br>    <br>    // ... mining logic ...<br>    <br>    bc.mineBlock(&amp;newBlock)  // _Grunning (CPU-intensive)<br>    bc.chain = append(bc.chain, newBlock)<br>    <br>    return newBlock<br>}</pre><p><strong>What happens:</strong> The goroutine tries to acquire mu.Lock(). If the lock is available, it continues in the _Grunning state. If the lock is held by another goroutine, it enters the _Gwaiting state. The scheduler removes G from P and executes another G. When the lock is released, G returns to the _Grunnable state. Eventually it gets scheduled again, transitioning from _Grunnable back to _Grunning.</p><h3>Cooperative Scheduling</h3><p>CPU-intensive goroutines can monopolize a processor without yielding. The runtime.Gosched() function allows goroutines to voluntarily give up execution time.</p><h4>The Problem</h4><pre>// Without yields - can monopolize a P<br>for {<br>    // Intensive computation<br>    hash = sha256.Sum256(data)<br>    nonce++<br>}</pre><p>A goroutine doing pure computation can run for milliseconds without yielding, starving other goroutines on the same P.</p><h4>The Solution</h4><pre>// internal/domain/blockchain/blockchain.go<br>func (bc *Blockchain) mineBlock(block *Block) {<br>    target := &quot;&quot;<br>    for i := 0; i &lt; bc.difficulty; i++ {<br>        target += &quot;0&quot;<br>    }<br>    for {<br>        block.Hash = calculateHash(*block)<br>        if block.Hash[:bc.difficulty] == target {<br>            break<br>        }<br>        block.Nonce++<br>        // Yield every 100k iterations to allow other goroutines to run<br>        if block.Nonce%100000 == 0 {<br>            runtime.Gosched()<br>        }<br>    }<br>}</pre><p>Calling runtime.Gosched() transitions the goroutine from _Grunning to _Grunnable and places it back in the P&#39;s local queue. The scheduler picks the next goroutine to execute, preventing monopolization and ensuring fair execution across all goroutines.</p><h3>Preemption (Go 1.14+)</h3><p>Since Go 1.14, the scheduler can preempt goroutines even without function calls or explicit yields.</p><p><strong>Before Go 1.14:</strong></p><pre>// Would never yield!<br>for {<br>    i++  // No function calls = no preemption<br>}</pre><p><strong>Go 1.14+:</strong></p><pre>// Gets preempted after approximately 10ms<br>for {<br>    i++  // Scheduler can interrupt via signals<br>}</pre><h4>How It Works</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*64N7Qvy6nueqq6E1MpjO4Q.png" /></figure><p>This mechanism ensures fairness even when goroutines run tight loops without function calls. Developers don’t need to add explicit yields everywhere, which is especially important for library code where users shouldn’t have to worry about scheduler cooperation.</p><h3>Syscall and Network I/O Handling</h3><h4>The Syscall Problem</h4><p>When a goroutine makes a blocking syscall, what happens to M and P?</p><pre>// Example of blocking syscall<br>data, _ := os.ReadFile(&quot;large_file.txt&quot;)</pre><h4>The Solution: P Handoff</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*5FSB5gJToH6IPerjS12Vtw.png" /></figure><p>When a syscall blocks an M, the P detaches and finds another available M to continue executing other goroutines. This design ensures that blocking syscalls only affect the thread making the call, not the processor’s ability to schedule other work. The blocked M remains in the syscall state until the operation completes, at which point it tries to reacquire a P or places its goroutine in the global queue.</p><h3>Network Poller</h3><p>Go includes a specialized component called the <strong>netpoller</strong> that handles network I/O operations efficiently. Instead of blocking entire threads while waiting for network data, the netpoller uses non-blocking I/O primitives provided by the operating system (epoll on Linux, kqueue on BSD/macOS, IOCP on Windows).</p><p>When a goroutine performs a network operation that would block, the netpoller registers the file descriptor with the OS and transitions the goroutine to the _Gwaiting state. The M is then free to execute other goroutines. When the OS signals that the network operation is ready (data available to read or socket ready to write), the netpoller wakes the waiting goroutine by moving it back to _Grunnable.</p><pre>// Appears blocking in code, but non-blocking under the hood<br>conn, _ := net.Dial(&quot;tcp&quot;, &quot;example.com:80&quot;)<br>conn.Write([]byte(&quot;GET / HTTP/1.1\r\n\r\n&quot;))<br>data, _ := conn.Read(buf)</pre><h4><strong>How it works:</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*CEjoO7bQqnkdYqdII-OlsQ.png" /></figure><p>The netpoller design is amazing because thousands of goroutines can wait for I/O without requiring thousands of threads. You only need a few Ms to handle massive numbers of concurrent network connections, completely avoiding the traditional thread-per-connection overhead.</p><h3>Practical Experiment</h3><p>The accompanying code demonstrates scheduler behavior through a stress test:</p><pre>func (s *BlockchainService) StressTest(allocations, goroutines int) StressTestResult {<br>    var wg sync.WaitGroup<br>    for i := 0; i &lt; goroutines; i++ {<br>        wg.Add(1)<br>        go func(id int) {<br>            defer wg.Done()<br>            for j := 0; j &lt; allocations; j++ {<br>                data := make([]byte, 1024*1024)<br>                hash := sha256.Sum256(data)<br>                _ = hex.EncodeToString(hash[:])<br>                <br>                if j%10 == 0 {<br>                    runtime.Gosched()<br>                }<br>            }<br>        }(i)<br>    }<br>    wg.Wait()<br>}</pre><p>Running this test shows how the scheduler distributes work across all Ps, with the work-stealing algorithm automatically balancing load across cores.</p><h3>Best Practices</h3><p><strong>Don’t spawn unlimited goroutines.</strong> It’s tempting to throw go in front of everything, but creating a goroutine for every item in a million-item dataset will overwhelm the scheduler. Instead, use a worker pool pattern with a fixed number of goroutines pulling work from a channel. The scheduler is efficient, but it&#39;s not magic.</p><p><strong>Give CPU-bound work some breathing room.</strong> If you’re running tight loops doing heavy computation (cryptography, image processing, etc.), throw in a runtime.Gosched() every so often. This lets other goroutines on the same P get some CPU time instead of being starved. Yes, Go 1.14+ has preemption, but cooperative yielding is still more predictable.</p><p><strong>Don’t hold locks while doing expensive stuff.</strong> This one bites people all the time. If you need to do some heavy calculation and then update shared state, do the calculation first, then grab the lock just to write the result. Holding a lock during a 100ms operation means every other goroutine waiting on that lock is stuck in _Gwaiting unnecessarily.</p><p><strong>Leave GOMAXPROCS alone (usually).</strong> The runtime sets it to the number of CPU cores by default, which is what you want 99% of the time. Only mess with it if you have a very specific reason and you’ve profiled your application to prove it helps.</p><h3>Conclusion</h3><p>Go’s scheduler enables efficient concurrent programming through the G-M-P model. Goroutines (G) are multiplexed onto OS threads (M) via logical processors (P), with each component playing a specific role. The work-stealing algorithm automatically balances load across all CPU cores without centralized coordination.</p><p>Key characteristics make this design powerful, goroutines start at 2KB versus 2MB for threads, blocking syscalls don’t block Ps (only Ms), the network poller handles thousands of connections without thread-per-connection overhead, and preemption (since Go 1.14) ensures fair scheduling even in tight loops.</p><p>Understanding these internals helps write more efficient concurrent code and make informed architectural decisions.</p><p>To dive deeper into these topics, check out the <a href="https://docs.google.com/document/d/1TTj4T2JO42uD5ID9e89oa0sLKhJYD0Y_kqxDv3I3XMw">Go Scheduler Design Doc</a> for the original design decisions. The <a href="https://golang.org/s/go11sched">Scalable Go Scheduler Design</a> document covers the improvements made for better scalability. For details on the preemption feature, see the <a href="https://golang.org/doc/go1.14#runtime">Go 1.14 Release Notes</a>.</p><h3>References</h3><ol><li><a href="https://docs.google.com/document/d/1TTj4T2JO42uD5ID9e89oa0sLKhJYD0Y_kqxDv3I3XMw">Scalable Go Scheduler Design Doc</a> — Dmitry Vyukov’s original design document</li><li><a href="https://golang.org/doc/go1.14#runtime">Go 1.14 Release Notes — Runtime</a> — Asynchronous preemption implementation</li><li><a href="https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part1.html">Scheduling In Go Series</a> — William Kennedy’s comprehensive three-part series on the Go scheduler</li><li><a href="https://golang.org/doc/effective_go#concurrency">Effective Go — Concurrency</a> — Official Go documentation on concurrency patterns</li></ol><p><em>All code examples are available on </em><a href="https://github.com/fascari/go-runtime-demo"><em>GitHub</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=65131986ee2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Performance Profiling in Go]]></title>
            <link>https://medium.com/@felipe.ascari_49171/understanding-performance-profiling-in-go-4176de1e4abd?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/4176de1e4abd</guid>
            <category><![CDATA[pprof]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[profiling]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Thu, 04 Dec 2025 13:02:34 GMT</pubDate>
            <atom:updated>2025-12-04T13:02:34.462Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*AsUbfCLd3ARBAiZsa56DpA.png" /></figure><h3>Introduction</h3><p>Performance analysis in Go has been a knowledge gap in my development journey. While I could write functional code, understanding where bottlenecks hide and how to systematically identify them in Go applications felt like uncharted territory. After diving deep into Go’s profiling ecosystem, I decided to share what I learned along the way.</p><p>Go includes built-in tools for understanding where your application spends time and memory. These tools help you identify performance bottlenecks without relying on external services or complex monitoring infrastructure. Learning to use pprof effectively means you can optimize any Go application, from simple CLI tools to complex APIs handling thousands of requests per second.</p><p>Performance optimization starts with measurement. Without data, you’re just guessing which parts of your code are slow. Go’s profiling tools collect this data automatically from your running application, showing exactly where CPU cycles go and where memory gets allocated.</p><h3>What is pprof?</h3><p>pprof is Go’s profiler. It collects runtime data about your program and presents it in various formats. Think of it as an X-ray machine for your application, letting you see inside what’s happening while the code runs.</p><p>When you enable pprof, it exposes HTTP endpoints that provide different views into your program’s behavior:</p><p><strong>CPU profiling</strong> records where the program spends processing time. It samples the call stack at regular intervals and builds a picture of which functions consume the most CPU cycles.</p><p><strong>Memory profiling</strong> tracks heap allocations. It shows which parts of your code allocate memory, how much, and whether that memory gets freed or accumulates over time.</p><p><strong>Goroutine profiling</strong> displays all running goroutines and what they’re doing. This helps identify goroutine leaks where goroutines are created but never cleaned up.</p><p><strong>Block profiling</strong> shows where goroutines wait on synchronization primitives like channels and mutexes. High blocking times suggest concurrency bottlenecks.</p><p><strong>Mutex profiling</strong> reveals contention on locks. When multiple goroutines compete for the same mutex, performance suffers.</p><h3>Adding pprof to Your Application</h3><p>Enabling pprof requires importing one package and starting an HTTP server. The net/http/pprof package automatically registers handlers when imported.</p><p>Here’s a simple example:</p><pre>package main<br><br>import (<br>    &quot;log&quot;<br>    &quot;net/http&quot;<br>    _ &quot;net/http/pprof&quot;<br>    &quot;os&quot;<br>)<br>func main() {<br>    // Start pprof server if enabled<br>    if os.Getenv(&quot;ENABLE_PPROF&quot;) == &quot;true&quot; {<br>        pprofAddr := os.Getenv(&quot;PPROF_ADDR&quot;)<br>        if pprofAddr == &quot;&quot; {<br>            pprofAddr = &quot;localhost:6060&quot;<br>        }<br>        <br>        go func() {<br>            log.Printf(&quot;pprof server starting on %s&quot;, pprofAddr)<br>            if err := http.ListenAndServe(pprofAddr, nil); err != nil {<br>                log.Printf(&quot;pprof server error: %v&quot;, err)<br>            }<br>        }()<br>    }<br>    // Your application code here<br>    // ...<br>}</pre><p>Start your application with profiling enabled:</p><pre>ENABLE_PPROF=true go run main.go</pre><p>Navigate to http://localhost:6060/debug/pprof/ to see available profiles. Using an environment variable keeps profiling disabled by default, which is important because pprof endpoints expose internal details you don&#39;t want public.</p><p>The project analyzed in this article uses Uber/FX for dependency injection, which requires a slightly different setup. If you’re curious about integrating pprof with FX lifecycle management, check the repository linked at the end.</p><h3>Understanding Profiling Concepts</h3><p>Before diving into examples, understanding how profiling works helps interpret the data correctly.</p><p><strong>Sampling vs Tracing</strong></p><p>pprof uses sampling for CPU profiles. Instead of tracking every function call (which would be prohibitively expensive), it takes snapshots of the call stack at regular intervals (100 times per second by default). These samples build a statistical picture of where time is spent.</p><p>Memory profiling also samples allocations. By default, it records one allocation per 512KB allocated. You can adjust this rate, but higher rates add more overhead to your program.</p><p><strong>Flat vs Cumulative Time</strong></p><p>When analyzing CPU profiles, you’ll see two key metrics:</p><p>Flat time shows how long a function itself ran, excluding functions it called. High flat time means the function does significant work directly.</p><p>Cumulative time includes both the function’s own time and time spent in functions it calls. High cumulative time with low flat time means the function delegates most work to other functions.</p><p>For example, a REST handler might have high cumulative time (it handles the entire request) but low flat time (it mostly calls other functions).</p><p><strong>In-Use vs Allocated</strong></p><p>Memory profiles show two different views:</p><p>In-use memory represents allocations currently held by the program. This reveals actual memory consumption.</p><p>Allocated memory includes everything the program has ever allocated, even if already freed. This helps find allocation hot spots that might not show up in in-use stats if the allocations are short-lived.</p><h3>Collecting and Analyzing Profiles</h3><p>With pprof enabled, collecting a profile means making an HTTP request to the appropriate endpoint.</p><h4>CPU Profile</h4><p>Capture 30 seconds of CPU activity:</p><pre>curl http://localhost:6060/debug/pprof/profile?seconds=30 &gt; cpu.prof</pre><p>While the profile collects, your application should be doing representative work. If you’re profiling an API, generate realistic traffic during this window.</p><p>Analyze the profile using pprof’s web interface:</p><pre>go tool pprof -http=:8081 cpu.prof</pre><p>This opens a browser with several views:</p><p>The <strong>flame graph</strong> shows the call stack as nested rectangles. Wider rectangles represent more time. It makes hot paths visually obvious.</p><p>The <strong>top</strong> view lists functions consuming the most CPU, sorted by flat or cumulative time.</p><p>The <strong>graph</strong> view displays function calls as a directed graph with time annotations.</p><p>The <strong>source</strong> view shows your code with time spent per line, making it easy to spot expensive operations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*NGWuVLZXVeKf_1zuaGsOJg.png" /><figcaption>CPU Profile Flame Graph showing hot paths and function call hierarchy</figcaption></figure><p>Let’s look at an example. In a transaction processing system, CPU profiling revealed significant time in type conversions:</p><pre>func fromDomain(transaction domain.Transaction) transactionModel {<br>    return transactionModel{<br>        ID:              transaction.ID,<br>        AccountID:       transaction.AccountID,<br>        Amount:          transaction.Amount,<br>        Type:            string(transaction.Type),  // Allocates memory<br>        PreviousBalance: transaction.PreviousBalance,<br>        NewBalance:      transaction.NewBalance,<br>        TransactionID:   transaction.TransactionID,<br>        IdempotencyKey:  transaction.IdempotencyKey,<br>    }<br>}</pre><p>The string(transaction.Type) conversion allocates memory each time. Since TransactionType already is a string type underneath, this allocation is unnecessary:</p><pre>type TransactionType string<br><br>const (<br>    Deposit    TransactionType = &quot;DEPOSIT&quot;<br>    Withdrawal TransactionType = &quot;WITHDRAWAL&quot;<br>)<br><br>func (t TransactionType) String() string {<br>    return string(t)  // No allocation needed<br>}</pre><p>This change seems tiny, but when the function runs thousands of times per second, the allocations add up. Profiling showed this was worth fixing.</p><h4>Memory Profile</h4><p>Heap profiling works differently than CPU profiling. Instead of timed sampling, it captures a snapshot of current allocations:</p><pre>curl http://localhost:6060/debug/pprof/heap &gt; heap.prof<br>go tool pprof -http=:8081 heap.prof</pre><p>Memory profiles help answer questions like:</p><ul><li>Which parts of code allocate the most memory?</li><li>Are allocations being freed or accumulating?</li><li>Where are potential memory leaks?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kOHQLOxbFAJ_8oLRVs546Q.png" /><figcaption>Memory Profile showing allocation hotspots and memory consumption</figcaption></figure><p>To detect memory leaks, compare two heap profiles taken at different times:</p><pre>curl http://localhost:6060/debug/pprof/heap &gt; heap_start.prof<br># Run the application for a while<br>curl http://localhost:6060/debug/pprof/heap &gt; heap_end.prof<br>go tool pprof -base=heap_start.prof heap_end.prof</pre><p>The comparison shows allocations that grew between the two snapshots. If memory usage is climbing steadily, this reveals where new allocations are coming from.</p><p>In one case, heap profiling revealed that JSON encoding was allocating new buffers for every request:</p><pre>func (h Handler) CreateTransaction(w http.ResponseWriter, r *http.Request) {<br>    // Process request...<br>    <br>    encoder := json.NewEncoder(w)<br>    encoder.Encode(response)<br>}</pre><p>Under high load, these allocations accumulated faster than garbage collection could reclaim them. The solution uses sync.Pool to reuse buffers:</p><pre>var bufferPool = sync.Pool{<br>    New: func() interface{} {<br>        return &amp;bytes.Buffer{}<br>    },<br>}<br><br>func (h Handler) CreateTransaction(w http.ResponseWriter, r *http.Request) {<br>    // Process request...<br>    <br>    buf := bufferPool.Get().(*bytes.Buffer)<br>    buf.Reset()<br>    defer bufferPool.Put(buf)<br>    <br>    encoder := json.NewEncoder(buf)<br>    encoder.Encode(response)<br>    w.Write(buf.Bytes())<br>}</pre><p>sync.Pool maintains a cache of reusable objects. When you get a buffer from the pool, you might get a freshly created one or a previously used one. When you put it back, it’s available for reuse. This reduces allocation pressure and helps the garbage collector.</p><h4>Goroutine Profile</h4><p>Goroutine profiles show all running goroutines and their current state:</p><pre>curl http://localhost:6060/debug/pprof/goroutine &gt; goroutine.prof<br>go tool pprof goroutine.prof</pre><p>Inside pprof, use top to see how many goroutines are doing what:</p><pre>(pprof) top<br>      flat  flat%   sum%        cum   cum%<br>       245 32.40% 32.40%        245 32.40%  runtime.gopark<br>       180 23.81% 56.21%        180 23.81%  database/sql.(*DB).conn</pre><p>This output shows 245 goroutines parked (sleeping) and 180 waiting for database connections. If you see hundreds of goroutines blocked on the same operation, you’ve found a bottleneck.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jHSANoJ0om28Zy97lCfB7g.png" /><figcaption>Goroutine Profile showing goroutines waiting for database connections</figcaption></figure><p>In this example, the database connection pool was too small. Many goroutines were waiting for available connections:</p><pre>func Connect(cfg config.Database) (*gorm.DB, error) {<br>    sqlDB, err := sql.Open(&quot;postgres&quot;, cfg.DSN())<br>    if err != nil {<br>        return nil, err<br>    }<br><br>    sqlDB.SetMaxOpenConns(100)      // Increased from 25<br>    sqlDB.SetMaxIdleConns(25)       // Increased from 10<br>    sqlDB.SetConnMaxLifetime(time.Hour)<br>    sqlDB.SetConnMaxIdleTime(10 * time.Minute)<br><br>    return gorm.Open(postgres.New(postgres.Config{<br>        Conn: sqlDB,<br>    }), &amp;gorm.Config{<br>        Logger: logger.Default.LogMode(logger.Silent),<br>    })<br>}</pre><p>Increasing the pool size let more goroutines work concurrently instead of waiting.</p><h3>Common Performance Patterns</h3><p>Profiling various applications reveals patterns that commonly cause performance issues.</p><h4>Database N+1 Queries</h4><p>One frequent bottleneck is the N+1 query pattern. Instead of fetching all needed data in one query, code fetches a list then queries for related data in a loop:</p><pre>func (r Repository) FindDailyTransactions(ctx context.Context, startDate, endDate time.Time) ([]domain.DailyTransaction, error) {<br>    var results []domain.DailyTransaction<br>    <br>    exchanges, _ := r.db.Find(&amp;Exchange{})<br>    <br>    for _, exchange := range exchanges {  // N queries<br>        var txs []Transaction<br>        r.db.Where(&quot;exchange_id = ? AND created_at BETWEEN ? AND ?&quot;, <br>            exchange.ID, startDate, endDate).Find(&amp;txs)<br>        // Aggregate...<br>    }<br>    <br>    return results, nil<br>}</pre><p>For 10 exchanges, this executes 11 queries (1 + 10). Each query has network overhead, parsing overhead, and round-trip latency.</p><p>Consolidate into a single query with joins and database-level aggregation:</p><pre>func (r Repository) FindDailyTransactions(ctx context.Context, startDate, endDate time.Time) ([]domain.DailyTransaction, error) {<br>    var transactions []domain.DailyTransaction<br>    <br>    err := r.db.WithContext(ctx).<br>        Table(&quot;transactions t&quot;).<br>        Select(`e.name as exchange, <br>            DATE(t.created_at) as date, <br>            SUM(CASE WHEN t.type = &#39;DEPOSIT&#39; THEN t.amount ELSE -t.amount END) as total_amount`).<br>        Joins(&quot;JOIN accounts a ON t.account_id = a.id&quot;).<br>        Joins(&quot;JOIN exchanges e ON a.exchange_id = e.id&quot;).<br>        Where(&quot;t.created_at BETWEEN ? AND ?&quot;, startDate, endDate).<br>        Where(&quot;t.deleted_at IS NULL&quot;).<br>        Group(&quot;e.name, DATE(t.created_at)&quot;).<br>        Order(&quot;DATE(t.created_at), e.name&quot;).<br>        Scan(&amp;transactions).Error<br>    <br>    return transactions, err<br>}</pre><p>This change reduced response time from 2.3 seconds to 340 milliseconds. The database can aggregate data much more efficiently than application code fetching and combining results.</p><h4>Unnecessary Allocations</h4><p>Many performance issues come from unnecessary memory allocations in frequently called code.</p><p>Pre-allocating slices when you know the size avoids repeated growth and copying:</p><pre>// Before: repeated allocations as slice grows<br>func ProcessItems(items []string) []string {<br>    var results []string<br>    for _, item := range items {<br>        results = append(results, process(item))<br>    }<br>    return results<br>}<br><br>// After: single allocation<br>func ProcessItems(items []string) []string {<br>    results := make([]string, 0, len(items))<br>    for _, item := range items {<br>        results = append(results, process(item))<br>    }<br>    return results<br>}</pre><p>String concatenation in loops creates many intermediate strings:</p><pre>// Before: allocates on each concatenation<br>func BuildMessage(parts []string) string {<br>    msg := &quot;&quot;<br>    for _, part := range parts {<br>        msg += part<br>    }<br>    return msg<br>}<br><br>// After: single final allocation<br>func BuildMessage(parts []string) string {<br>    var builder strings.Builder<br>    for _, part := range parts {<br>        builder.WriteString(part)<br>    }<br>    return builder.String()<br>}</pre><p>strings.Builder pre-allocates a buffer and builds the final string efficiently.</p><h4>Connection Pool Sizing</h4><p>Applications using databases or other network resources need appropriately sized connection pools. Too small and goroutines wait for connections. Too large and you waste resources or overwhelm the database.</p><p>Goroutine profiling reveals if goroutines are blocked waiting:</p><pre>(pprof) top<br>      flat  flat%   sum%        cum   cum%<br>       180 23.81% 56.21%        180 23.81%  database/sql.(*DB).conn</pre><p>If many goroutines block on (*DB).conn, increase the pool:</p><pre>sqlDB.SetMaxOpenConns(100)  // Total connections<br>sqlDB.SetMaxIdleConns(25)   // Kept open when idle</pre><p>Monitor your database’s connection limits. If you set MaxOpenConns higher than the database allows, you’ll get connection errors.</p><h3>Performance Optimization Workflow</h3><p>Effective optimization follows a systematic process:</p><ol><li><strong>Establish baseline metrics.</strong> Note current response times, memory usage, and throughput. Without a baseline, you can’t tell if changes help.</li><li><strong>Profile under realistic load.</strong> Generate representative traffic. Use load testing tools if needed. Collect CPU, memory, and goroutine profiles.</li><li><strong>Identify the biggest bottleneck.</strong> Analyze profiles to find functions with high cumulative time or excessive allocations. Focus on code that runs frequently (hot paths).</li><li><strong>Make one targeted change.</strong> Don’t change multiple things simultaneously. If you do and performance improves, you won’t know which change helped.</li><li><strong>Profile again and compare.</strong> Collect new profiles and verify the bottleneck is reduced. Monitor your metrics to confirm improvement.</li><li><strong>Repeat for the next bottleneck.</strong> Optimization is iterative. Fix the biggest issue, measure again, find the next issue.</li></ol><h3>Tools</h3><p>Go includes everything needed for profiling:</p><p><strong>pprof</strong> collects and displays CPU, memory, and goroutine profiles</p><p><strong>go tool pprof</strong> analyzes profile data with interactive commands and visualizations</p><p><strong>go tool trace</strong> provides detailed execution traces showing goroutine scheduling and blocking</p><p>For load generation during profiling, tools like hey or wrk simulate realistic traffic.</p><p>The complete code examples are available at: <a href="https://github.com/felipeascari/exchange-crypto-service-api">https://github.com/felipeascari/exchange-crypto-service-api</a></p><h3>Final Thoughts</h3><p>Performance optimization is measurement and targeted fixes. Profile, find bottlenecks, fix them, repeat.</p><p>Common problems: unnecessary allocations, inefficient database queries, undersized connection pools. pprof makes them visible.</p><p>Don’t guess where time goes. Collect data, analyze it, make changes, profile again. One change at a time so you know what worked. This applies to any Go app.</p><p>The tools are built into Go. Import net/http/pprof, start your server, start profiling.</p><h3>References</h3><ul><li><a href="https://pkg.go.dev/net/http/pprof">pprof Package Documentation</a></li><li><a href="https://go.dev/blog/pprof">Profiling Go Programs</a></li><li><a href="https://go.dev/doc/diagnostics">Diagnostics Documentation</a></li><li><a href="https://dave.cheney.net/high-performance-go-workshop">Dave Cheney’s Performance Workshop</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4176de1e4abd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GORM in Go: My Experience and Trade-offs]]></title>
            <link>https://medium.com/@felipe.ascari_49171/gorm-in-go-my-experience-and-trade-offs-9eb89408ee34?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/9eb89408ee34</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[orm]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Sun, 23 Nov 2025 14:36:32 GMT</pubDate>
            <atom:updated>2025-11-23T14:36:32.750Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*V94peyEsTUvmECG9FzfJAA.png" /></figure><h3>Introduction</h3><p>GORM is the most popular ORM in Go, but its use sparks ongoing debate. Some developers value its productivity, while others argue it violates Go’s simplicity principles.</p><p>The choice between ORM and raw SQL depends on your specific context. This article shares what I discovered works well, what doesn’t, and why I chose traditional GORM over code generation alternatives.</p><h3>The ORM Debate</h3><p>The Go community leans skeptical on ORMs. Common criticisms include leaky abstractions, hidden performance costs, and added complexity that conflicts with Go’s explicit nature.</p><p>One frequently cited issue is the N+1 query problem. This occurs when fetching a collection of records and then accessing a relationship for each record individually. For example, fetching 100 users and then making a separate query for each user’s account results in 101 queries instead of 2. What appears as simple object navigation in code can generate hundreds of database queries, severely impacting performance. While GORM provides Preload() to address this, developers must remember to use it, and the problem isn’t obvious from the code.</p><p>However, ORMs do reduce CRUD boilerplate, provide query building interfaces, and simplify relationship handling.</p><p>Experienced Go developers often recommend starting with database/sql or libraries like sqlx, adding complexity only when needed.</p><h3>GORM vs Raw SQL</h3><p>The choice between GORM and raw SQL isn’t binary. Each has clear strengths.</p><h4>Simple Operations</h4><p>GORM excels at basic CRUD operations:</p><pre>func (r Repository) Create(ctx context.Context, account Account) error {<br>    return r.db.WithContext(ctx).Create(&amp;account).Error<br>}</pre><p>The equivalent raw SQL requires more code:</p><pre>func (r Repository) Create(ctx context.Context, account Account) error {<br>    query := `INSERT INTO accounts (user_id, exchange_id, balance, created_at, updated_at)<br>              VALUES ($1, $2, $3, NOW(), NOW()) RETURNING id, created_at, updated_at`<br>    <br>    return r.db.QueryRowContext(ctx, query, account.UserID, account.ExchangeID, account.Balance).<br>        Scan(&amp;account.ID, &amp;account.CreatedAt, &amp;account.UpdatedAt)<br>}</pre><p>GORM is more concise. Raw SQL is more explicit.</p><h4>Complex Queries</h4><p>For complex queries, raw SQL becomes preferable:</p><pre>func (r Repository) GetDailyStats(ctx context.Context, startDate, endDate time.Time) ([]Stats, error) {<br>    query := `<br>        SELECT e.name, DATE(t.created_at) as date,<br>               SUM(CASE WHEN t.type = &#39;DEPOSIT&#39; THEN t.amount ELSE -t.amount END) as total<br>        FROM transactions t<br>        JOIN accounts a ON t.account_id = a.id<br>        JOIN exchanges e ON a.exchange_id = e.id<br>        WHERE t.created_at BETWEEN $1 AND $2<br>        GROUP BY e.name, DATE(t.created_at)<br>        ORDER BY date, e.name`<br>    <br>    var stats []Stats<br>    return stats, r.db.Raw(query, startDate, endDate).Scan(&amp;stats).Error<br>}</pre><p>Trying to build this with GORM’s query builder becomes awkward and harder to read than the SQL itself.</p><h3>Transaction Management with GORM</h3><p>One of GORM’s strongest features is transaction handling. It provides a clean abstraction that handles commit/rollback automatically.</p><h4>Basic Transaction</h4><pre>func (r Repository) TransferFunds(ctx context.Context, fromID, toID uint, amount float64) error {<br>    return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {<br>        // Debit from source account<br>        if err := tx.Model(&amp;Account{}).Where(&quot;id = ?&quot;, fromID).<br>            Update(&quot;balance&quot;, gorm.Expr(&quot;balance - ?&quot;, amount)).Error; err != nil {<br>            return err<br>        }<br>        <br>        // Credit to destination account<br>        if err := tx.Model(&amp;Account{}).Where(&quot;id = ?&quot;, toID).<br>            Update(&quot;balance&quot;, gorm.Expr(&quot;balance + ?&quot;, amount)).Error; err != nil {<br>            return err<br>        }<br>        <br>        // If we return nil, transaction commits<br>        // If we return error, transaction rolls back automatically<br>        return nil<br>    })<br>}</pre><p>The transaction automatically commits if the function returns nil, or rolls back if it returns an error. This prevents common bugs where developers forget to call commit or rollback.</p><h4>Manual Transaction Control</h4><p>For more complex scenarios, GORM supports manual control:</p><pre>func (r Repository) ProcessBatch(ctx context.Context, items []Item) error {<br>    tx := r.db.WithContext(ctx).Begin()<br>    defer func() {<br>        if r := recover(); r != nil {<br>            tx.Rollback()<br>        }<br>    }()<br><br>    for _, item := range items {<br>        if err := tx.Create(&amp;item).Error; err != nil {<br>            tx.Rollback()<br>            return err<br>        }<br>    }<br><br>    return tx.Commit().Error<br>}</pre><h4>Nested Transactions with SavePoints</h4><p>GORM supports savepoints for complex transaction scenarios:</p><pre>func (r Repository) ComplexOperation(ctx context.Context) error {<br>    return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {<br>        // First operation<br>        if err := tx.Create(&amp;record1).Error; err != nil {<br>            return err<br>        }<br>        <br>        // Nested transaction with savepoint<br>        err := tx.Transaction(func(tx2 *gorm.DB) error {<br>            return tx2.Create(&amp;record2).Error<br>        })<br>        <br>        if err != nil {<br>            // Inner transaction rolled back, outer continues<br>            log.Println(&quot;Inner transaction failed, continuing...&quot;)<br>        }<br>        <br>        // Final operation<br>        return tx.Create(&amp;record3).Error<br>    })<br>}</pre><p>Implementing equivalent functionality with raw SQL requires manually managing BEGIN, COMMIT, ROLLBACK, and SAVEPOINT commands, which is error-prone.</p><h4>Transaction Context Propagation</h4><p>GORM makes it easy to pass transactions through your application layers:</p><pre>type contextKey string<br><br>const txKey contextKey = &quot;tx&quot;<br><br>func TXFromContext(ctx context.Context) *gorm.DB {<br>    if tx := ctx.Value(txKey); tx != nil {<br>        return tx.(*gorm.DB)<br>    }<br>    return nil<br>}<br><br>func (r Repository) Create(ctx context.Context, account Account) error {<br>    db := r.db<br>    if tx := TXFromContext(ctx); tx != nil {<br>        db = tx<br>    }<br>    return db.WithContext(ctx).Create(&amp;account).Error<br>}</pre><p>This pattern allows transactions to propagate through application layers, ensuring all operations participate in the same transaction when needed.</p><h3>About GORM Gen</h3><p>GORM Gen is a code generation tool that creates type-safe query builders from database schemas. It works similarly to type-safe query builders in other languages.</p><p>Basic concept:</p><pre>// Generate code from database<br>g := gen.NewGenerator(gen.Config{OutPath: &quot;./query&quot;})<br>g.UseDB(db)<br>user := g.GenerateModel(&quot;users&quot;)<br>g.ApplyBasic(user)<br>g.Execute()<br><br>// Use generated code<br>u := query.Use(db).User<br>results, _ := u.Where(u.Name.Like(&quot;%john%&quot;)).Find()</pre><p>The generated code provides compile-time type safety and IDE autocomplete for database queries, eliminating string-based column references.</p><p>However, I chose not to use GORM Gen in this project. The main reasons were the required code generation step in the development workflow and model duplication issues. GORM Gen generates its own model structs, which would conflict with the repository models already in the project’s architecture. For my use case, traditional GORM with selective raw SQL provided better simplicity without sacrificing functionality.</p><h3>My Approach: Hybrid GORM and Raw SQL</h3><p>In practice, I use both GORM and raw SQL where each excels.</p><p>GORM for standard operations:</p><pre>type Repository struct {<br>    db *gorm.DB<br>}<br><br>func (r Repository) Create(ctx context.Context, account Account) error {<br>    return r.db.WithContext(ctx).Create(&amp;account).Error<br>}<br><br>func (r Repository) FindByID(ctx context.Context, id uint) (Account, error) {<br>    var account Account<br>    err := r.db.WithContext(ctx).First(&amp;account, id).Error<br>    return account, err<br>}</pre><p>Raw SQL for complex queries:</p><pre>func (r Repository) GetDailyStats(ctx context.Context) ([]DailyStat, error) {<br>    query := `<br>        SELECT DATE(created_at) as date, <br>               SUM(amount) as total,<br>               COUNT(*) as count<br>        FROM transactions<br>        WHERE type = &#39;DEPOSIT&#39;<br>        GROUP BY DATE(created_at)<br>        ORDER BY date DESC`<br>    <br>    var stats []DailyStat<br>    return stats, r.db.Raw(query).Scan(&amp;stats).Error<br>}</pre><p>GORM transactions for business logic:</p><pre>func (uc UseCase) processTransaction(<br>    ctx context.Context,<br>    accountID uint,<br>    amount float64,<br>    transactionType domain.TransactionType,<br>    idempotencyKey string,<br>    opts ...TransactionOption,<br>) (string, error) {<br>    // Validations before starting transaction<br>    if err := validateAmount(amount); err != nil {<br>        return &quot;&quot;, err<br>    }<br><br>    if err := uc.checkIdempotency(ctx, accountID, idempotencyKey); err != nil {<br>        return &quot;&quot;, err<br>    }<br><br>    transactionID := generateTransactionID()<br><br>    // Execute all database operations in a single transaction<br>    err := uc.accountRepository.ExecuteInTransaction(ctx, func(ctx context.Context) error {<br>        return uc.executeTransaction(ctx, accountID, amount, transactionType, transactionID, idempotencyKey, cfg)<br>    })<br><br>    return transactionID, err<br>}<br><br>func (uc UseCase) executeTransaction(<br>    ctx context.Context,<br>    accountID uint,<br>    amount float64,<br>    transactionType domain.TransactionType,<br>    transactionID string,<br>    idempotencyKey string,<br>    cfg transactionConfig,<br>) error {<br>    // Read account with transaction lock<br>    account, err := uc.accountRepository.FindByID(ctx, accountID)<br>    if err != nil {<br>        return err<br>    }<br><br>    // Validate business rules<br>    if err := uc.validateTransaction(ctx, account, amount, cfg); err != nil {<br>        return err<br>    }<br><br>    previousBalance := account.Balance<br><br>    // Update account balance<br>    if err := uc.updateAccountBalance(ctx, &amp;account, amount, transactionType); err != nil {<br>        return err<br>    }<br><br>    // Create transaction record<br>    return uc.transactionRepository.Create(ctx, domain.Transaction{<br>        AccountID:       accountID,<br>        Type:            transactionType,<br>        Amount:          amount,<br>        PreviousBalance: previousBalance,<br>        NewBalance:      account.Balance,<br>        TransactionID:   transactionID,<br>        IdempotencyKey:  idempotencyKey,<br>    })<br>}</pre><p>This example demonstrates a financial transaction that must be atomic. The ExecuteInTransaction wrapper ensures that all operations succeed or fail together. If any step fails, all changes are rolled back automatically. This is critical for financial operations where partial updates would leave the system in an inconsistent state.</p><p>The pattern separates validations that can happen outside the transaction from operations that must be atomic. Idempotency checks and basic validations run first, avoiding unnecessary transaction overhead. Only the actual data modifications happen inside the transaction boundary, keeping the transaction scope minimal and reducing lock contention.</p><p>This hybrid approach provides clarity about what each tool does best.</p><h3>Conclusion</h3><p>GORM works well for standard database operations and transaction management. Its automatic commit/rollback handling prevents common transaction bugs, and the query builder is sufficient for most filtering needs.</p><p>For complex queries, raw SQL remains clearer and more maintainable than ORM query builders. The ability to use GORM’s Raw() method means you can stay within the same framework while writing explicit SQL.</p><p>I skipped GORM Gen because the code generation workflow and model duplication didn’t align with the project’s existing clean architecture. Traditional GORM provided the needed functionality without additional complexity.</p><p>The choice between ORM and raw SQL isn’t binary. Use GORM where it simplifies your code, and use raw SQL where clarity matters more. Both can coexist in the same codebase effectively.</p><h3>References</h3><ul><li>GORM Documentation: <a href="https://gorm.io/docs/">https://gorm.io/docs/</a></li><li>GORM Gen: <a href="https://gorm.io/gen/">https://gorm.io/gen/</a></li><li>Go database/sql package: <a href="https://pkg.go.dev/database/sql">https://pkg.go.dev/database/sql</a></li><li>sqlx — Extensions to database/sql: <a href="https://github.com/jmoiron/sqlx">https://github.com/jmoiron/sqlx</a></li><li>sqlc — Generate type-safe code from SQL: <a href="https://sqlc.dev/">https://sqlc.dev/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9eb89408ee34" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rethinking Version Control: More Than Just a Place to Store Code]]></title>
            <link>https://medium.com/@felipe.ascari_49171/rethinking-version-control-more-than-just-a-place-to-store-code-9470feda8f86?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/9470feda8f86</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Tue, 11 Nov 2025 13:22:55 GMT</pubDate>
            <atom:updated>2025-11-11T13:22:55.210Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*Th8NzVpsTb7pXTR0gYtKdQ.png" /></figure><p>I wanted to step away from my usual Go content and talk about something more general, something that still makes me reflect a lot as a software developer whenever I see how different teams use version control.</p><p>I started coding around 2005. Back then, I worked with <strong>no version control at all</strong>. CVS was rarely mentioned. Everything relied on file locks, comments to avoid overwriting each other’s changes, and a jungle of files like file.old, file.bkp, and many others. Different times.</p><p>Eventually I got into CVS, then SVN. I struggled a lot with both. And then Git showed up. That is when things started to click for me.</p><h3>From Code Bucket to Communication Tool</h3><p>For a long time, I treated version control as nothing more than a <strong>code bucket</strong>, a place to store files. That was my own lack of understanding. Many people already used it properly, but I did not. For me, it was basically a smarter ZIP file.</p><p>That changed over time. I slowly began to understand version control as a <strong>communication tool</strong>, a channel between developers, between teams, even between areas of a company.</p><p>But it really clicked in 2022 when I joined Mercado Livre (the largest e-commerce platform in Latin America). The company is huge and every team works differently, but the team I joined had a very mature development process, and <strong>Git was at the center of it</strong>.</p><p>We had a Pull Request template we had to follow closely: a checklist, a detailed description, links to issues, screenshots when needed. We also had internal guidelines on how to write good commits, how to structure branches, how to open clean PRs. It all felt natural and made the process easier, not heavier.</p><p>This is where version control stopped being just Git and became a <strong>fundamental part of how we communicated, documented, and organized our work</strong>.</p><p>Knowing how to do a proper squash, a fixup, a clean rebase, or how to write a commit message that actually helps someone is not extra. <strong>It is part of being an effective developer</strong>.</p><p>Today, whenever I see a messy PR with unclear messages, no description, and a pile of unrelated commits, I cannot help thinking about how much friction that creates. It slows down code review, breaks communication, and often leads to rework.</p><blockquote><em>So here is an invitation, think about how you are using version control. Learn more about it. Treat it as the powerful tool it actually is, not just a storage system.</em></blockquote><h3>Taking Care of Your Commit History</h3><p>Your commit history is the project’s timeline. It tells the story of decisions, fails, fixes, improvements, and how the system grew. A clean history helps future contributors, <strong>including you six months from now</strong>.</p><p>Isn’t it great when you look at your project and realize the commits actually tell a story? When you can review PRs commit by commit, no matter how large they are? (Although PRs should be kept focused and have a reduced scope, of course.) Each commit becomes a chapter that makes sense on its own, helping reviewers understand not just what changed, but the reasoning behind each step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c-WVOLTKe3E4BB7d_tXSTg.png" /><figcaption>Example from one of my repositories showing how commits tell a clear story</figcaption></figure><p>Here are some simple practices that make a big difference:</p><h3>1. Write clear commit messages</h3><p>A commit message should explain <strong>what</strong> changed and <strong>why</strong>.</p><p><strong>Bad:</strong></p><pre>git commit -m &quot;adjustments&quot;</pre><p><strong>Better:</strong></p><pre>git commit -m &quot;Fix race condition when updating user profile data&quot;</pre><h3>2. Use a consistent format</h3><p>Some teams adopt conventions like <a href="https://www.conventionalcommits.org/">Conventional Commits</a>.</p><pre>git commit -m &quot;feat: add payment retry logic&quot;<br>git commit -m &quot;fix: correct null pointer when loading cart&quot;</pre><h3>3. Make small, focused commits</h3><p>Avoid commits that mix unrelated changes. Smaller commits make reviews easier and history cleaner.</p><pre>git add user_service.go<br>git commit -m &quot;refactor: extract email validation logic&quot;</pre><h3>4. Use rebase and squash when needed</h3><p>Before merging, clean up your branch.</p><pre>git rebase -i main</pre><p>To squash commits, mark them as s during the interactive rebase.</p><h3>5. Review your history before merging</h3><p>A quick look at git log can save someone a lot of confusion.</p><pre>git log --oneline</pre><h3>6. Tag important releases</h3><p>Tags make it easy to track meaningful points in the project.</p><pre>git tag -a v1.2.0 -m &quot;Release 1.2.0&quot;<br>git push origin v1.2.0</pre><h3>7. Document your conventions</h3><p>Keep your commit rules, branching strategy, and PR expectations in your repo. It helps new developers ramp up quickly.</p><h3>Final Thoughts</h3><p>Good version control is about more than code. <strong>It is about communication, clarity, and efficiency</strong>. A clean history and thoughtful use of Git can drastically improve how teams work together.</p><p>If this made you reflect a bit on your own workflow, that is already a win.</p><h3>Resources</h3><p>Below are a few resources that helped shape how I think about commits and version control:</p><ul><li><a href="https://chris.beams.io/posts/git-commit/">How to Write a Git Commit Message</a></li><li><a href="https://chris.beams.io/git-commit#seven-rules">The Seven Rules of a Great Git Commit Message</a></li><li><a href="https://www.conventionalcommits.org/en/v1.0.0/">Conventional Commits</a></li><li><a href="https://nvie.com/posts/a-successful-git-branching-model/">A Successful Git Branching Model</a></li><li><a href="https://www.atlassian.com/git/tutorials/rewriting-history/git-rebase">Git Rebase Tutorial</a></li><li><a href="https://www.git-scm.com/book/en/v2/Git-Tools-Rewriting-History">Git Tools — Rewriting History</a></li></ul><p>I wanted to add one last note. I felt like writing this because adopting this mindset genuinely helped me become a better developer. It made my work clearer, my communication stronger, and my contributions easier for others to follow. Sharing these experiences feels important, and I hope they help others the same way they helped me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9470feda8f86" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenTelemetry: Vendor-Agnostic Observability in Go Applications]]></title>
            <link>https://medium.com/@felipe.ascari_49171/opentelemetry-vendor-agnostic-observability-in-go-applications-2aa9c06c764f?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/2aa9c06c764f</guid>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Sat, 08 Nov 2025 15:03:53 GMT</pubDate>
            <atom:updated>2025-11-08T15:03:53.352Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XtLlsbngoS0cKsLAl85edQ.png" /></figure><h3>Introduction</h3><p>In an increasingly distributed world, where microservices communicate across multiple networks and dependencies, observability has become a fundamental pillar of modern software engineering. But how can we ensure complete visibility into our application’s behavior without being locked into a single vendor? The answer lies in <strong>OpenTelemetry (OTEL)</strong>.</p><p>In this article, we’ll explore the fundamental concepts of OpenTelemetry, analyze its practical implementation in a didactic Go project, compare it with proprietary solutions like New Relic, and discuss the advantages and disadvantages of this approach.</p><h3>What is OpenTelemetry?</h3><p>OpenTelemetry is an open-source observability framework that provides a standardized set of APIs, libraries, agents, and instrumentation to collect, process, and export telemetry data (traces, metrics, and logs) from applications.</p><h4>The Three Pillars of Observability</h4><p><strong>1. Traces (Distributed Tracing)</strong></p><ul><li>Allows tracking requests across multiple services</li><li>Each operation creates a “span” containing timing information, metadata, and context</li><li>Spans are organized into “traces” representing the complete path of a request</li></ul><p><strong>2. Metrics</strong></p><ul><li>Numerical aggregations over time (counters, gauges, histograms)</li><li>Useful for understanding trends and alerts</li><li>Examples: request rate, average latency, memory usage</li></ul><p><strong>3. Logs</strong></p><ul><li>Discrete records of events occurring in the application</li><li>Contextualized with trace information for correlation</li></ul><h4>OpenTelemetry Architecture</h4><p>The OTEL architecture consists of:</p><ul><li><strong>SDK</strong>: API implementations in different languages</li><li><strong>Instrumentation</strong>: Automatic (via libraries) or manual (via code)</li><li><strong>Exporters</strong>: Send data to observability backends</li><li><strong>Collector</strong>: Optional component that receives, processes, and exports telemetry</li></ul><h3>Practical Implementation</h3><p>Let’s analyze the OpenTelemetry implementation in a cryptocurrency transaction management API developed in Go, demonstrating observability concepts and implementation patterns.</p><h4>1. Tracer Initialization</h4><p>Tracer configuration is the first step. In the project, this is done using the dependency injection pattern with Uber FX:</p><pre>// pkg/telemetry/tracer.go<br>func InitTracer(serviceName string) (*trace.TracerProvider, error) {<br>    ctx := context.Background()<br>    // Configure OTLP exporter via HTTP<br>    exporter, err := otlptracehttp.New(ctx,<br>        otlptracehttp.WithInsecure(),<br>        otlptracehttp.WithEndpoint(&quot;localhost:4318&quot;),<br>    )<br>    if err != nil {<br>        return nil, err<br>    }<br>    // Define service resources (metadata)<br>    res, err := resource.New(ctx,<br>        resource.WithAttributes(<br>            semconv.ServiceName(serviceName),<br>            semconv.ServiceVersion(&quot;1.0.0&quot;),<br>        ),<br>    )<br>    if err != nil {<br>        return nil, err<br>    }<br>    // Create TracerProvider<br>    tp := trace.NewTracerProvider(<br>        trace.WithBatcher(exporter),<br>        trace.WithResource(res),<br>    )<br>    // Set global provider<br>    otel.SetTracerProvider(tp)<br>    otel.SetTextMapPropagator(propagation.TraceContext{})<br>    return tp, nil<br>}</pre><p><strong>Key Points:</strong></p><ul><li>Uses OTLP (OpenTelemetry Protocol) via HTTP</li><li>Configures semantic metadata (service name, version)</li><li>Span batching for performance optimization</li><li>Context propagation via TraceContext for distributed tracing</li></ul><h4>2. HTTP Middleware</h4><p>One of OpenTelemetry’s major advantages is the availability of ready-made middlewares for major web frameworks. In the project, we use <strong>Gorilla Mux</strong>, one of the most popular HTTP routers in the Go ecosystem.</p><pre>// internal/middleware/otel.go<br>func SetupOTEL(r *mux.Router, serviceName string) {<br>    r.Use(otelmux.Middleware(serviceName))<br>}</pre><p>The otelmux middleware (from package go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux) automatically:</p><ul><li>Creates spans for each HTTP request</li><li>Captures method, path, status code</li><li>Propagates tracing context between services</li><li>Records errors and exceptions</li></ul><h4>Available Middlewares for Other Frameworks</h4><p>OpenTelemetry maintains a rich instrumentation ecosystem for various Go frameworks:</p><ul><li><strong>Gorilla Mux</strong>: go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux</li><li><strong>Gin</strong>: go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin</li><li><strong>Echo</strong>: go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho</li><li><strong>Chi</strong>: go.opentelemetry.io/contrib/instrumentation/github.com/go-chi/chi/otelchi</li><li><strong>Fiber</strong>: go.opentelemetry.io/contrib/instrumentation/github.com/gofiber/fiber/otelfiber</li><li><strong>net/http</strong>: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp</li></ul><h3>3. Custom Repository Instrumentation</h3><p>Beyond automatic instrumentation, the project implements manual tracing for database operations:</p><pre>// pkg/telemetry/repository.go<br>func TraceRepository(ctx context.Context, operation string, <br>    fn func(ctx context.Context) ([]attribute.KeyValue, error), <br>    attrs ...attribute.KeyValue) error {<br>    <br>    ctx, span := otel.Tracer(&quot;repository&quot;).Start(ctx, operation)<br>    defer span.End()<br><br>    span.SetAttributes(attrs...)<br><br>    additionalAttrs, err := fn(ctx)<br>    if len(additionalAttrs) &gt; 0 {<br>        span.SetAttributes(additionalAttrs...)<br>    }<br><br>    if err != nil {<br>        span.RecordError(err)<br>        span.SetStatus(codes.Error, err.Error())<br>        return err<br>    }<br><br>    return nil<br>}</pre><p><strong>Practical Usage:</strong></p><pre>// internal/app/transaction/repository/repository.go<br>func (r Repository) FindDailyTransactions(ctx context.Context, <br>    startDate, endDate time.Time) ([]domain.DailyTransaction, error) {<br>    <br>    transactions := make([]domain.DailyTransaction, 0)<br><br>    err := telemetry.TraceRepository(ctx, &quot;repository.find_daily_transactions&quot;,<br>        func(ctx context.Context) ([]attribute.KeyValue, error) {<br>            endDate = endDate.Add(24*time.Hour - time.Nanosecond)<br><br>            err := r.db.WithContext(ctx).<br>                Table(&quot;transactions t&quot;).<br>                Select(`e.name as exchange, <br>                    DATE(t.created_at) as date, <br>                    SUM(CASE WHEN t.type = &#39;DEPOSIT&#39; THEN t.amount ELSE -t.amount END) as total_amount`).<br>                Joins(&quot;JOIN accounts a ON t.account_id = a.id&quot;).<br>                Joins(&quot;JOIN exchanges e ON a.exchange_id = e.id&quot;).<br>                Where(&quot;t.created_at BETWEEN ? AND ?&quot;, startDate, endDate).<br>                Group(&quot;e.name, DATE(t.created_at)&quot;).<br>                Order(&quot;DATE(t.created_at), e.name&quot;).<br>                Scan(&amp;transactions).Error<br><br>            if err != nil {<br>                return nil, err<br>            }<br><br>            return []attribute.KeyValue{<br>                attribute.Int(&quot;result_count&quot;, len(transactions)),<br>            }, nil<br>        },<br>        attribute.String(&quot;start_date&quot;, startDate.Format(time.DateOnly)),<br>        attribute.String(&quot;end_date&quot;, endDate.Format(time.DateOnly)),<br>    )<br><br>    return transactions, err<br>}</pre><p>This approach provides:</p><ul><li><strong>Granularity</strong>: Visibility into specific database operations</li><li><strong>Rich Context</strong>: Custom attributes (dates, counts)</li><li><strong>Error Tracking</strong>: Automatic error capture with stack traces</li><li><strong>Performance Insights</strong>: Duration of complex queries</li></ul><h3>4. OpenTelemetry Collector</h3><p>The project uses the OTEL Collector as an intermediary between the application and the observability backend. The Collector is a vendor-agnostic proxy that can receive, process, and export telemetry data, acting as a central hub in the observability pipeline.</p><h4>What is the OTEL Collector?</h4><p>The OpenTelemetry Collector is a standalone service that sits between your instrumented applications and your observability backends. Think of it as a smart middleware that provides several key capabilities:</p><p><strong>Architecture Components:</strong></p><ol><li><strong>Receivers</strong>: Accept telemetry data in various formats (OTLP, Jaeger, Zipkin, Prometheus, etc.)</li><li><strong>Processors</strong>: Transform, filter, batch, or enrich the telemetry data</li><li><strong>Exporters</strong>: Send processed data to one or more backends</li></ol><p><strong>Why Use a Collector?</strong></p><p>Instead of sending traces directly from your application to Jaeger (or any backend), the Collector provides:</p><ul><li><strong>Decoupling</strong>: Your application doesn’t need to know about backend details. Change backends without touching application code.</li><li><strong>Performance</strong>: Batching and buffering reduce network overhead and improve application performance.</li><li><strong>Resilience</strong>: If a backend goes down, the Collector can queue data temporarily.</li><li><strong>Data Processing</strong>: Filter sensitive information, sample high-volume traces, or add custom attributes before exporting.</li><li><strong>Multi-Backend Support</strong>: Send the same telemetry to multiple destinations (e.g., Jaeger for traces and Prometheus for metrics).</li><li><strong>Protocol Translation</strong>: Receive data in one format (e.g., Zipkin) and export it in another (e.g., OTLP).</li></ul><h4>Configuration Example</h4><pre># otel-collector-config.yml<br>receivers:<br>  otlp:<br>    protocols:<br>      grpc:<br>        endpoint: 0.0.0.0:4317<br>      http:<br>        endpoint: 0.0.0.0:4318<br><br>processors:<br>  batch:<br>    timeout: 10s<br>    send_batch_size: 1024<br><br>exporters:<br>  otlp:<br>    endpoint: jaeger:4317<br>    tls:<br>      insecure: true<br><br>service:<br>  pipelines:<br>    traces:<br>      receivers: [otlp]<br>      processors: [batch]<br>      exporters: [otlp]</pre><p>In this configuration:</p><ul><li>The Collector listens on ports 4317 (gRPC) and 4318 (HTTP) for incoming traces</li><li>The batch processor groups spans together to reduce network calls</li><li>Traces are then exported to Jaeger using the OTLP protocol</li></ul><h3>Jaeger: Trace Visualization and Analysis</h3><p>The project uses <strong>Jaeger</strong> as the observability backend for trace storage and visualization.</p><h4>What is Jaeger?</h4><p>Jaeger is an open-source distributed tracing platform created by Uber and donated to the Cloud Native Computing Foundation (CNCF). It’s one of the most popular tools for analyzing OpenTelemetry traces.</p><blockquote><strong><em>What is CNCF?</em></strong><em> The Cloud Native Computing Foundation is a Linux Foundation project founded in 2015 that hosts critical cloud-native infrastructure projects like Kubernetes, Prometheus, and OpenTelemetry. CNCF projects are vendor-neutral, open-source, and backed by major tech companies including Google, Microsoft, Amazon, and Red Hat. CNCF certification ensures maturity, production-readiness, and broad industry support.</em></blockquote><p><strong>1. Distributed Trace Visualization</strong></p><ul><li>Intuitive web interface for trace exploration</li><li>Waterfall visualization showing duration of each span</li><li>Correlation between services and dependencies</li></ul><p><strong>2. Performance Analysis</strong></p><ul><li>Identification of bottlenecks in requests</li><li>Latency comparison between different paths</li><li>Service Dependency Graph to visualize architecture</li></ul><p><strong>3. Root Cause Analysis</strong></p><ul><li>Drill-down into specific spans to investigate errors</li><li>Visualization of custom attributes and tags</li><li>Exception stack traces</li></ul><h4>Project Configuration</h4><pre># docker-compose.yml<br>services:<br>  jaeger:<br>    image: jaegertracing/all-in-one:latest<br>    container_name: jaeger<br>    ports:<br>      - &quot;16686:16686&quot; # Jaeger UI<br>      - &quot;14250:14250&quot; # Jaeger gRPC (for OTLP)<br>    environment:<br>      - COLLECTOR_OTLP_ENABLED=true<br>    networks:<br>      - app_network<br><br>  otel-collector:<br>    image: otel/opentelemetry-collector-contrib:latest<br>    container_name: otel-collector<br>    command: [&quot;--config=/etc/otel-collector-config.yml&quot;]<br>    volumes:<br>      - ./otel-collector-config.yml:/etc/otel-collector-config.yml<br>    ports:<br>      - &quot;4317:4317&quot;   # OTLP gRPC receiver<br>      - &quot;4318:4318&quot;   # OTLP HTTP receiver<br>    depends_on:<br>      - jaeger</pre><p>The demo application sends traces to the OTEL Collector (port 4318), which processes and forwards them to Jaeger. The UI is available at <a href="http://localhost:16686.">http://localhost:16686.</a></p><h4>Alternatives to Jaeger</h4><p>While Jaeger is excellent, there are other OpenTelemetry-compatible tools worth considering:</p><p><strong>Zipkin</strong> — Simple and lightweight tracing system, ideal for small to medium projects with straightforward requirements.</p><p><strong>Tempo (Grafana)</strong> — Highly scalable tracing backend with native Grafana integration and cost-effective object storage support (S3, GCS).</p><p><strong>Elastic APM</strong> — Comprehensive observability solution with deep Elasticsearch/Kibana integration, excellent for log correlation.</p><p><strong>SigNoz</strong> — Modern open-source platform combining metrics, traces, and logs in a single interface, positioning itself as a Datadog alternative.</p><p><strong>Honeycomb</strong> — SaaS-only solution with powerful querying capabilities and high cardinality support, great for complex debugging scenarios.</p><p><strong>Lightstep (ServiceNow Cloud Observability)</strong> — Enterprise-grade platform with ML-powered analysis and anomaly detection, focused on large-scale deployments.</p><h3>OpenTelemetry vs New Relic: A Comparison</h3><p>Throughout my career, I’ve extensively used observability tools like New Relic and Datadog across multiple companies. Based on this hands-on experience, I believe comparing OpenTelemetry with New Relic provides valuable insights, as New Relic is one of the most established APM (Application Performance Monitoring) solutions on the market. This comparison reflects real-world usage patterns and trade-offs I’ve encountered in production environments.</p><p><strong>OpenTelemetry (used in the project):</strong></p><pre>// Initialization<br>tp, _ := telemetry.InitTracer(&quot;exchange-crypto-api&quot;)<br>defer tp.Shutdown(context.Background())<br><br>// Middleware<br>router.Use(otelmux.Middleware(&quot;exchange-crypto-api&quot;))<br><br>// Manual tracing<br>ctx, span := otel.Tracer(&quot;repository&quot;).Start(ctx, &quot;find_transactions&quot;)<br>defer span.End()<br>span.SetAttributes(attribute.String(&quot;user_id&quot;, userID))</pre><p><strong>New Relic:</strong></p><pre>// Initialization<br>app, _ := newrelic.NewApplication(<br>    newrelic.ConfigAppName(&quot;exchange-crypto-api&quot;),<br>    newrelic.ConfigLicense(&quot;YOUR_LICENSE_KEY&quot;),<br>)<br><br>// Middleware<br>router.Use(nrgorilla.Middleware(app))<br><br>// Manual tracing<br>txn := newrelic.FromContext(ctx)<br>segment := txn.StartSegment(&quot;find_transactions&quot;)<br>defer segment.End()<br>segment.AddAttribute(&quot;user_id&quot;, userID)</pre><p><strong>Observations:</strong></p><ul><li>Similar APIs, but New Relic requires license key</li><li>Switching from New Relic to another backend requires complete refactoring</li><li>With OTEL, only the exporter needs to change</li></ul><h3><strong>OpenTelemetry: Pros and Cons</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y8gtQyWaY-3M0vrVNHG9NA.png" /></figure><h4>Key Points</h4><p><strong>Why Choose OpenTelemetry:</strong></p><ul><li>You’re not locked into a single vendor and can start with Jaeger (free), migrate to Tempo, or use commercial SaaS without code changes</li><li>It’s a CNCF standard supported by Google, Microsoft, Amazon, and Splunk — the future of observability</li><li>Cost-effective with free SDK/Collector and choice of free (Jaeger, Zipkin) or paid backends</li><li>Sensitive data stays in your infrastructure for compliance (GDPR, LGPD, HIPAA)</li><li>Choose the best backend for each need: Jaeger for traces, Prometheus for metrics, Loki for logs</li><li>Hundreds of official libraries for frameworks, databases, and message brokers</li></ul><p><strong>Challenges to Consider:</strong></p><ul><li>Requires understanding of SDK configuration, Collector pipelines, and backend deployment</li><li>You’re responsible for high availability, scalability, backup, and security if self-hosting</li><li>No out-of-the-box dashboards or advanced ML/AI analysis</li><li>Teams need to learn distributed tracing concepts and backend-specific tools</li><li>Debugging can be complex across multiple components (app, collector, backend)</li><li>Many backend options require research and testing</li></ul><h3>Conclusion</h3><p>OpenTelemetry represents a paradigm shift in observability: <strong>standardization without vendor lock-in</strong>. As we’ve seen through the Exchange Crypto Service demo implementation, it’s possible to instrument a Go application elegantly and efficiently, ensuring complete visibility into distributed behavior.</p><h4>Key Takeaways</h4><ol><li><strong>OTEL is the future</strong>: Adopted by the CNCF and supported by industry giants, it’s the natural path for cloud-native observability.</li><li><strong>Flexibility has a cost</strong>: Freedom of choice comes with the responsibility of configuration, operation, and maintenance.</li><li><strong>Ideal for microservices</strong>: Context propagation and distributed tracing shine in distributed architectures.</li><li><strong>Trade-off with SaaS</strong>: New Relic and similar solutions offer faster time-to-value but lock you into their ecosystem.</li><li><strong>Jaeger + OTEL = Sweet Spot</strong>: For many projects, the self-hosted combination offers the best cost-benefit.</li></ol><h4>Final Recommendation</h4><p><strong>Start with OpenTelemetry if:</strong></p><ul><li>Building from scratch or can refactor</li><li>Have a competent technical team</li><li>Want total control and avoid lock-in</li><li>Seek long-term cost optimization</li></ul><p><strong>Consider proprietary solutions if:</strong></p><ul><li>Need immediate results</li><li>Small team or no observability expertise</li><li>Budget allows</li><li>Prioritize advanced features (ML, AIOps)</li></ul><p>In the end, observability isn’t a luxury, it’s a necessity. OpenTelemetry democratizes access to enterprise-level distributed tracing, allowing any project, regardless of size or budget, to have deep visibility into its behavior.</p><p>As Charity Majors, observability pioneer, said: <em>“Observability is about asking questions you didn’t know you needed to ask.”</em> With OpenTelemetry, you have the tools to ask those questions — and get the answers.</p><h3>Demo Project Repository</h3><p><strong>Exchange Crypto Service</strong>: <a href="https://github.com/fascari/exchange-crypto-service-api">https://github.com/fascari/exchange-crypto-service-api</a></p><p>The project demonstrates:</p><ul><li>Complete OTEL + Jaeger setup</li><li>Middleware for Gorilla Mux</li><li>Custom repository instrumentation</li><li>Configured OTEL Collector</li><li>Docker Compose for local environment</li></ul><h3>References</h3><p>[1] OpenTelemetry. (2024). <em>OpenTelemetry Documentation</em>. Cloud Native Computing Foundation. <a href="https://opentelemetry.io/docs/">https://opentelemetry.io/docs/</a></p><p>[2] Jaeger Project. (2024). <em>Jaeger: open source, distributed tracing platform</em>. <a href="https://www.jaegertracing.io/">https://www.jaegertracing.io/</a></p><p>[3] Sigelman, B. H., et al. (2010). <em>Dapper, a Large-Scale Distributed Systems Tracing Infrastructure</em>. Google Technical Report.</p><p>[4] Majors, C., Fong-Jones, L., &amp; Miranda, G. (2022). <em>Observability Engineering: Achieving Production Excellence</em>. O’Reilly Media.</p><p>[5] Beyer, B., Jones, C., Petoff, J., &amp; Murphy, N. R. (2016). <em>Site Reliability Engineering: How Google Runs Production Systems</em>. O’Reilly Media.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2aa9c06c764f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Concurrent Transactions in Go: Saga, Queues and DDD Aggregates (Part 2/2)]]></title>
            <link>https://medium.com/@felipe.ascari_49171/concurrent-transactions-in-go-saga-queues-and-ddd-aggregates-part-2-2-067961e3bae2?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/067961e3bae2</guid>
            <category><![CDATA[go-programming]]></category>
            <category><![CDATA[apache-kafka]]></category>
            <category><![CDATA[distributed-system-design]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Fri, 31 Oct 2025 12:33:05 GMT</pubDate>
            <atom:updated>2025-10-31T12:33:05.016Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKZyvf47INy-_oRX5DOW9A.png" /></figure><p>Welcome back! In <a href="https://medium.com/@felipe.ascari_49171/concurrent-transactions-in-go-pessimistic-vs-optimistic-locks-part-1-2-33f866561feb">Part 1</a>, we explored database-level strategies for handling concurrent transactions: Pessimistic Locks, Optimistic Locks, and Idempotency. We saw how these techniques work at the database layer to prevent race conditions.</p><p>In this second part, we’ll shift our focus to <strong>architectural patterns</strong> that complement database strategies. These patterns are essential when dealing with very high volume, multiple services, or complex distributed transactions.</p><p>We’ll cover:</p><ul><li><strong>Message Queue Serialization</strong>: Kafka-based external coordination</li><li><strong>Saga Pattern</strong>: Distributed transactions across services</li><li><strong>DDD Aggregates</strong>: Defining consistency boundaries</li></ul><p>Let’s continue!</p><h3>Strategy 4: Serialization via Message Queue</h3><p>For <strong>very high volume</strong> scenarios, database locks become the bottleneck. The solution is to move serialization to <strong>before</strong> the database.</p><h4>Architecture Flow</h4><p>The flow works in 4 steps:</p><ol><li><strong>Client → API Handler</strong>: Sends POST /transactions request</li><li><strong>API → Kafka Producer</strong>: Validates and generates event using accountID as partition key</li><li><strong>Kafka Topic → Consumers</strong>: Routes to partitions (Partition 0: accounts 1,4,7… / Partition 1: accounts 2,5,8… / Partition 2: accounts 3,6,9…)</li><li><strong>Consumers → PostgreSQL</strong>: Each consumer processes its partition sequentially</li></ol><p>The API returns <strong>202 Accepted</strong> immediately without waiting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*Rs6YyWEAipFrDBvfmWj7hA.png" /><figcaption>Message queue architecture with partition-based routing</figcaption></figure><h4>Conceptual Implementation</h4><pre>// Producer: API Handler<br>func (h Handler) CreateTransaction(w http.ResponseWriter, r *http.Request) {<br>    event := TransactionEvent{<br>        AccountID:      req.AccountID,<br>        Amount:         req.Amount,<br>        Type:           req.Type,<br>        IdempotencyKey: req.IdempotencyKey,<br>    }<br>    <br>    // Using accountID as partition key ensures order<br>    err := h.kafkaProducer.Send(ctx, event, PartitionKey(req.AccountID))<br>    <br>    // Returns 202 Accepted immediately<br>    w.WriteHeader(http.StatusAccepted)<br>}<br><br>// Consumer: Background Processor<br>func (c Consumer) ProcessEvents() {<br>    for event := range c.eventChannel {<br>        // Processes sequentially per partition<br>        // No locks needed in the database!<br>        c.transactionUseCase.Execute(ctx, event)<br>    }<br>}</pre><h4>Benefits</h4><p>✅ <strong>Massive Throughput</strong>: Can scale horizontally with partitions.<br>✅ <strong>No Database Locks</strong>: Each partition processes sequentially.<br>✅ <strong>Natural Backpressure</strong>: Queue absorbs load spikes.<br>✅ <strong>Audit Trail</strong>: Events are recorded in the queue.<br>✅ <strong>Retry and DLQ</strong>: Easy to implement retry and dead letter queue.</p><h4>Trade-offs</h4><p>❌ <strong>Operational Complexity</strong>: Kafka/RabbitMQ adds infrastructure.<br>❌ <strong>Asynchronous API</strong>: Client doesn’t receive immediate response (needs polling/webhook).<br>❌ <strong>Eventual Consistency</strong>: Latency between queue write and processing.<br>❌ <strong>Cost</strong>: Additional messaging infrastructure.</p><h4>When to Use</h4><ul><li>Very high volume (&gt; 10k TPS — <em>Transactions Per Second</em> - per account)</li><li>You can tolerate asynchronous processing</li><li>Need complete event audit trail</li><li>Multiple services need to react to transactions</li></ul><h3>Strategy 5: Saga Pattern for Distributed Transactions</h3><p>The <strong>Saga Pattern</strong> is useful when the transaction involves <strong>multiple services</strong> or aggregates that cannot be in the same database transaction.</p><h4>Scenario</h4><pre>Withdrawal of $500:<br>1. Debit from account (Account Service)<br>2. Create audit record (Audit Service)<br>3. Send notification (Notification Service)<br>4. Update daily limit (Limit Service)</pre><h4>Implementation: Choreography-based Saga</h4><pre>// Initial event<br>type WithdrawalInitiated struct {<br>    TransactionID string<br>    AccountID     uint<br>    Amount        float64<br>}<br><br>// Account Service<br>func (s AccountService) OnWithdrawalInitiated(event WithdrawalInitiated) {<br>    err := s.debitAccount(event.AccountID, event.Amount)<br>    if err != nil {<br>        s.eventBus.Publish(WithdrawalFailed{...})<br>        return<br>    }<br>    s.eventBus.Publish(AccountDebited{...})<br>}<br><br>// Audit Service<br>func (s AuditService) OnAccountDebited(event AccountDebited) {<br>    s.createAuditLog(event)<br>    s.eventBus.Publish(AuditLogged{...})<br>}<br><br>// Compensation in case of error<br>func (s AccountService) OnWithdrawalFailed(event WithdrawalFailed) {<br>    s.creditAccount(event.AccountID, event.Amount) // Rollback<br>}</pre><h4>Benefits</h4><p>✅ <strong>Decoupling</strong>: Services don’t need to know each other.<br>✅ <strong>Resilient</strong>: Failures in one service don’t block others.<br>✅ <strong>Scalable</strong>: Each service scales independently.<br>✅ <strong>Auditable</strong>: All events are recorded.</p><h4>Trade-offs</h4><p>❌ <strong>High Complexity</strong>: Many events and handlers to manage.<br>❌ <strong>Eventual Consistency</strong>: No ACID between services.<br>❌ <strong>Compensations</strong>: Rollback logic can be complex.<br>❌ <strong>Difficult Debugging</strong>: Flow distributed across multiple services.</p><h4>Does Saga Minimize Concurrency Problems?</h4><p><strong>Yes and no</strong>. Saga solves concurrency <strong>between services</strong>, but <strong>not within an aggregate</strong>. In our case, we still need pessimistic/optimistic locks for the account balance. Saga is complementary, not a substitute.</p><h3>Strategy 6: DDD and the Aggregate Concept</h3><p>Domain-Driven Design teaches us that <strong>the aggregate is the boundary of strong consistency</strong>.</p><h4>Applying to Our Case</h4><pre>// Account is our AGGREGATE ROOT<br>type Account struct {<br>    ID         uint<br>    Balance    float64<br>}<br><br>// Aggregate guarantees invariants<br>func (a *Account) Debit(amount float64) error {<br>    if a.Balance &lt; amount {<br>        return errors.New(&quot;insufficient balance&quot;)<br>    }<br>    a.Balance -= amount<br>    return nil<br>}</pre><h4>Principles</h4><ol><li><strong>One transaction = one aggregate</strong>: Lock/optimism applied only to the Account aggregate</li><li><strong>Between aggregates = eventual consistency</strong>: Use events/saga</li><li><strong>Protected invariants</strong>: Aggregate root guarantees business rules</li></ol><h4>In Our Code</h4><pre>func (uc UseCase) processTransaction(...) error {<br>    // Database transaction involves ONLY the Account aggregate<br>    err := uc.accountRepository.ExecuteInTransaction(ctx, func(ctx context.Context) error {<br>        // Pessimistic lock on the AGGREGATE<br>        account, err := uc.accountRepository.FindByID(ctx, accountID)<br>        <br>        // Invariant guaranteed by the aggregate<br>        if account.Balance &lt; amount {<br>            return ErrInsufficientBalance<br>        }<br>        <br>        account.Balance -= amount<br>        return uc.accountRepository.Update(ctx, account)<br>    })<br>    <br>    // Events for other aggregates/services are sent OUTSIDE the transaction<br>    uc.eventBus.Publish(TransactionCreated{...})<br>}</pre><h4>Benefits</h4><p>✅ <strong>Clear Boundaries</strong>: We know exactly where to apply locks.<br>✅ <strong>Less Contention</strong>: Locks only on what is critical.<br>✅ <strong>Scalable</strong>: Different aggregates = zero contention.<br>✅ <strong>Clear Modeling</strong>: Code reflects business domain.</p><h3>Conclusion</h3><p>There is no <strong>silver bullet</strong> for concurrency in distributed systems. The choice depends on your specific context:</p><ul><li><strong>Volume</strong>: How many transactions per second?</li><li><strong>Latency</strong>: Can it be asynchronous?</li><li><strong>Consistency</strong>: Eventual or strong?</li><li><strong>Complexity</strong>: Does the team have expertise?</li><li><strong>Cost</strong>: Available infrastructure?</li></ul><h3>Key Takeaways</h3><ol><li><strong>Start Simple</strong>: Pessimistic lock + idempotency covers 80% of cases</li><li><strong>Scale Gradually</strong>: Add message queues only when database locks become the bottleneck</li><li><strong>Define Boundaries</strong>: Use DDD aggregates to know where to apply strong consistency</li><li><strong>Test Under Load</strong>: Always validate with tools like JMeter or Gatling</li><li><strong>Defense in Depth</strong>: Combine strategies (locks + idempotency + queues)</li></ol><p>For critical financial operations like our cryptocurrency exchange, <strong>pessimistic lock + idempotency + DDD aggregates</strong> offers the best balance between security and simplicity. As volume grows, add message queues for external serialization, always keeping the database lock as the last line of defense.</p><p>The important thing is to <strong>understand the trade-offs</strong> and choose consciously based on your context. What works for Netflix might not work for your startup, and that’s okay.</p><p><strong>About the code</strong>: The examples are based on a didactic cryptocurrency exchange system in Go, using GORM, PostgreSQL and hexagonal architecture. The complete code demonstrates pessimistic lock with SELECT FOR UPDATE, idempotency with unique constraints, and ACID transactions.</p><p><strong>GitHub Repository</strong>: <a href="https://github.com/fascari/exchange-crypto-service-api">github.com/fascari/exchange-crypto-service-api</a></p><p><strong>References:</strong></p><ul><li><a href="https://martinfowler.com/eaaCatalog/">Martin Fowler — Patterns of Enterprise Application Architecture</a></li><li><a href="https://microservices.io/patterns/data/saga.html">Saga Pattern — Chris Richardson (Microservices.io)</a></li><li><a href="https://www.domainlanguage.com/ddd/">Domain-Driven Design — Eric Evans</a></li><li><a href="https://www.postgresql.org/docs/current/mvcc.html">PostgreSQL Documentation — Concurrency Control</a></li></ul><p>👈 <strong>Read</strong>: <a href="https://medium.com/@felipe.ascari_49171/concurrent-transactions-in-go-pessimistic-vs-optimistic-locks-part-1-2-33f866561feb">Concurrent Transactions in Microservices: Pessimistic, Optimistic Locks and Idempotency (Part 1/2)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=067961e3bae2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Concurrent Transactions in Go: Pessimistic vs Optimistic Locks (Part 1/2)]]></title>
            <link>https://medium.com/@felipe.ascari_49171/concurrent-transactions-in-go-pessimistic-vs-optimistic-locks-part-1-2-33f866561feb?source=rss-97fb4100a5de------2</link>
            <guid isPermaLink="false">https://medium.com/p/33f866561feb</guid>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[go-programming]]></category>
            <category><![CDATA[distributed-system-design]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Felipe Ascari]]></dc:creator>
            <pubDate>Fri, 24 Oct 2025 18:11:36 GMT</pubDate>
            <atom:updated>2025-10-24T18:11:36.670Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bvaS7EECQUQr6BSVFZoXeQ.png" /></figure><p>As a software engineer working with microservices architectures, I&#39;ve encountered numerous scenarios where theoretical knowledge meets production reality—and concurrency control is one of the most critical.</p><p>In this two-part article, I&#39;ll explore patterns, trade-offs, and solutions that I&#39;ve implemented, always backing concepts with actual code examples. This tackles one of the most fundamental challenges in distributed systems: <strong>how to handle concurrent transactions safely and efficiently</strong>.</p><p><strong>Part 1</strong> covers database-level strategies: Pessimistic Locks, Optimistic Locks, and Idempotency.<br><strong>Part 2</strong> will explore architectural patterns: Message Queues, Saga Pattern, and DDD Aggregates.</p><p>Let&#39;s dive in!</p><h4>The Real Problem of Concurrency</h4><p>Imagine a cryptocurrency exchange system where multiple users perform simultaneous withdrawals from the same account. Without proper concurrency control, two withdrawals of $500 could be processed at the same time on an account with a balance of $600, resulting in a negative balance of -$400. This is one of the classic problems in distributed systems: <strong>race conditions</strong>.</p><p>In this article, I’ll explore different strategies for handling concurrent transactions in microservices that share a single database, using real examples from a cryptocurrency exchange system built in Go.</p><h4>The Scenario: Financial Transaction System</h4><p>Let’s analyze a real use case: creating transactions (deposits and withdrawals) in exchange accounts. The flow is apparently simple:</p><pre>func (uc UseCase) executeTransaction(<br>    ctx context.Context,<br>    accountID uint,<br>    amount float64,<br>    transactionType domain.TransactionType,<br>    transactionID string,<br>    idempotencyKey string,<br>) error {<br>    // 1. Fetch the account<br>    account, err := uc.accountRepository.FindByID(ctx, accountID)<br>    if err != nil {<br>        return err<br>    }<br><br>    // 2. Validate balance and limits<br>    if err := uc.validateTransaction(ctx, account, amount, cfg); err != nil {<br>        return err<br>    }<br><br>    previousBalance := account.Balance<br><br>    // 3. Update the balance<br>    if err := uc.updateAccountBalance(ctx, &amp;account, amount, transactionType); err != nil {<br>        return err<br>    }<br><br>    // 4. Record the transaction<br>    return uc.transactionRepository.Create(ctx, domain.Transaction{<br>        AccountID:       accountID,<br>        Type:            transactionType,<br>        Amount:          amount,<br>        PreviousBalance: previousBalance,<br>        NewBalance:      account.Balance,<br>        TransactionID:   transactionID,<br>        IdempotencyKey:  idempotencyKey,<br>    })<br>}</pre><p>The problem? Between steps 1 and 3, another transaction might have modified the account balance. This is the <strong>critical moment</strong> where race conditions happen.</p><h3>Strategy 1: Pessimistic Lock (SELECT FOR UPDATE)</h3><h4>How It Works</h4><p>Pessimistic locking assumes that conflicts <strong>will happen</strong> and preventively blocks the resource. In our code, we implement this using SELECT FOR UPDATE:</p><pre>func (r Repository) FindByID(ctx context.Context, id uint) (domain.Account, error) {<br>    var model Account<br><br>    db := r.db<br>    if tx := database.TXFromContext(ctx); tx != nil {<br>        // Within a transaction, use SELECT FOR UPDATE<br>        db = tx.Clauses(clause.Locking{Strength: clause.LockingStrengthUpdate})<br>    }<br><br>    if err := db.WithContext(ctx).First(&amp;model, id).Error; err != nil {<br>        return domain.Account{}, err<br>    }<br><br>    return model.toDomain(), nil<br>}</pre><p>When executed within a transaction, the database:</p><ol><li>Locks the account record</li><li>Prevents other transactions from reading or modifying that record</li><li>Releases the lock only when the transaction is committed or rolled back.</li></ol><h4>Generated SQL</h4><pre>BEGIN;<br>SELECT * FROM accounts WHERE id = 123 FOR UPDATE;<br>UPDATE accounts SET balance = balance - 500 WHERE id = 123;<br>INSERT INTO transactions (...) VALUES (...);<br>COMMIT;</pre><h4>Benefits</h4><p>✅ <strong>Total Safety</strong>: Impossible to have race conditions. The database guarantees serialization.<br>✅ <strong>Simplicity</strong>: Clean business logic, no additional retry code or version validation.<br>✅ <strong>Guaranteed Consistency</strong>: Ideal for critical operations like financial transactions.<br>✅ <strong>Full ACID</strong>: Leverages the full power of relational database transactions.</p><h4>Trade-offs</h4><p>❌ <strong>Contention</strong>: Other transactions are <strong>blocked</strong> waiting for the lock to be released.<br>❌ <strong>Deadlocks</strong>: If transaction A locks account 1 and waits for account 2, while transaction B locks account 2 and waits for account 1, we have a deadlock.<br>❌ <strong>Reduced Throughput</strong>: In high-volume scenarios, forced serialization reduces throughput.<br>❌ <strong>Increased Latency</strong>: Transactions can wait for seconds in high-contention scenarios.<br>❌ <strong>Limited Scalability</strong>: Makes sharding and load distribution difficult.</p><h4>When to Use</h4><ul><li>Critical financial operations (like our case)</li><li>Moderate transaction volume (&lt; 1000 TPS — <em>Transactions Per Second -</em> per account)</li><li>When consistency is more important than performance</li><li>When the cost of an inconsistency is high</li></ul><h3>Strategy 2: Optimistic Lock (Optimistic Locking)</h3><h4>How It Works</h4><p>Optimistic locking assumes that conflicts are <strong>rare</strong> and only validates at commit time. Uses a version field:</p><pre>type Account struct {<br>    ID         uint<br>    Balance    float64<br>    Version    int  // Version control field<br>}<br><br>func (r Repository) UpdateWithOptimisticLock(ctx context.Context, account domain.Account) error {<br>    result := r.db.WithContext(ctx).<br>        Model(&amp;Account{}).<br>        Where(&quot;id = ? AND version = ?&quot;, account.ID, account.Version).<br>        Updates(map[string]interface{}{<br>            &quot;balance&quot;: account.Balance,<br>            &quot;version&quot;: account.Version + 1,<br>        })<br>    <br>    if result.RowsAffected == 0 {<br>        return ErrOptimisticLockFailed // Retry required<br>    }<br>    <br>    return result.Error<br>}</pre><h4>Generated SQL</h4><pre>UPDATE accounts <br>SET balance = 1500, version = 43 <br>WHERE id = 123 AND version = 42;<br>-- If version changed, RowsAffected = 0</pre><h4>Benefits</h4><p>✅ <strong>No Blocking</strong>: Multiple simultaneous reads without contention.<br>✅ <strong>Better Throughput</strong>: Ideal for read-heavy scenarios.<br>✅ <strong>No Deadlocks</strong>: No locks to cause deadlocks.<br>✅ <strong>Scalability</strong>: Works well with read replicas.</p><h4>Trade-offs</h4><p>❌ <strong>Retry Logic</strong>: Application needs to implement retry when versions conflict.<br>❌ <strong>Starvation</strong>: In high contention, some transactions may never achieve commit.<br>❌ <strong>Complexity</strong>: More complex code to manage retries and backoff.<br>❌ <strong>Wasted Work</strong>: Transactions can process everything and fail at the end.</p><h4>Implementation Example with Retry</h4><pre>func (uc UseCase) executeWithRetry(ctx context.Context, accountID uint, amount float64) error {<br>    maxRetries := 3<br>    <br>    for attempt := 0; attempt &lt; maxRetries; attempt++ {<br>        account, err := uc.accountRepository.FindByID(ctx, accountID)<br>        if err != nil {<br>            return err<br>        }<br>        <br>        // Process transaction...<br>        account.Balance -= amount<br>        <br>        err = uc.accountRepository.UpdateWithOptimisticLock(ctx, account)<br>        if err == nil {<br>            return nil // Success!<br>        }<br>        <br>        if errors.Is(err, ErrOptimisticLockFailed) {<br>            time.Sleep(time.Millisecond * time.Duration(10 * (1 &lt;&lt; attempt))) // Exponential backoff<br>            continue<br>        }<br>        <br>        return err<br>    }<br>    <br>    return errors.New(&quot;max retries exceeded&quot;)<br>}</pre><h4>When to Use</h4><ul><li>Conflicts are rare (e.g., different users access different accounts)</li><li>Reads are much more frequent than writes</li><li>Read latency is critical</li><li>You can tolerate retries in the application</li></ul><h3>Strategy 3: Idempotency with Unique Constraints</h3><p>A complementary approach (and present in our code) is to use <strong>idempotency keys</strong> with unique constraints in the database:</p><pre>CREATE UNIQUE INDEX idx_account_idempotency <br>ON transactions(account_id, idempotency_key);</pre><pre>func (r Repository) CheckIdempotency(ctx context.Context, accountID uint, idempotencyKey string) error {<br>    var existing transactionModel<br>    err := db.Where(&quot;idempotency_key = ? AND account_id = ?&quot;, idempotencyKey, accountID).<br>        First(&amp;existing).Error<br><br>    if err == nil {<br>        return ErrDuplicateIdempotencyKey{TransactionID: existing.TransactionID}<br>    }<br>    <br>    return nil<br>}</pre><h4>How It Works</h4><ol><li>Client generates a unique UUID as idempotencyKey</li><li>System checks if a transaction with the same key already exists</li><li>Unique constraint ensures that even in a race condition, only one transaction will be recorded</li><li>Client can resend requests with the same key without duplication</li></ol><h4>Benefits</h4><p>✅ <strong>Protection Against Duplication</strong>: Duplicate requests don’t cause duplicate transactions.<br>✅ <strong>Safe Retry</strong>: Client can retry without fear.<br>✅ <strong>Database Guarantee</strong>: Unique constraint is the last line of defense.<br>✅ <strong>Complements Locks</strong>: Works together with pessimistic or optimistic locks.</p><h4>Trade-offs</h4><p>❌ <strong>Client Must Generate Key</strong>: Additional responsibility on the client.<br>❌ <strong>Doesn’t Solve Balance Race Condition</strong>: Still needs lock for the balance.<br>❌ <strong>Extra Storage</strong>: Additional unique index.</p><blockquote><em>💡 </em><strong><em>Note</em></strong><em>: These strategies can be combined. For example, use pessimistic lock for balance updates AND idempotency for duplicate prevention.</em></blockquote><h3>What’s Next?</h3><p>In <strong>Part 2</strong>, we’ll explore architectural patterns that complement these database strategies:</p><ul><li><strong>Message Queue Serialization</strong>: How to handle very high volume (&gt;10k TPS) with Kafka</li><li><strong>Saga Pattern</strong>: Managing distributed transactions across multiple services</li><li><strong>DDD Aggregates</strong>: Defining consistency boundaries in your domain model</li></ul><p>We’ll also provide practical recommendations on how to combine these strategies for different scenarios.</p><p><strong>About the code</strong>: The examples are based on a real cryptocurrency exchange system in Go, using GORM, PostgreSQL and hexagonal architecture. The complete code demonstrates pessimistic lock with SELECT FOR UPDATE, idempotency with unique constraints and ACID transactions.</p><p><strong>GitHub Repository</strong>: <a href="https://github.com/fascari/exchange-crypto-service-api">github.com/fascari/exchange-crypto-service-api</a></p><p><strong>References:</strong></p><ul><li><a href="https://martinfowler.com/eaaCatalog/">Martin Fowler — Patterns of Enterprise Application Architecture</a></li><li><a href="https://www.postgresql.org/docs/current/mvcc.html">PostgreSQL Documentation — Concurrency Control</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33f866561feb" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>