<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lomig Mégard on Medium]]></title>
        <description><![CDATA[Stories by Lomig Mégard on Medium]]></description>
        <link>https://medium.com/@lomigmegard?source=rss-a63638eaef02------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4hPRBXVirU6wSB1BoluI-Q.jpeg</url>
            <title>Stories by Lomig Mégard on Medium</title>
            <link>https://medium.com/@lomigmegard?source=rss-a63638eaef02------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:55:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lomigmegard/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Java 25 for Scala Services: Production Impact Report]]></title>
            <link>https://medium.com/swissborg-engineering/java-25-for-scala-services-production-impact-report-3b7a23234b9f?source=rss-a63638eaef02------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b7a23234b9f</guid>
            <category><![CDATA[jvm]]></category>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Lomig Mégard]]></dc:creator>
            <pubDate>Fri, 19 Dec 2025 10:41:32 GMT</pubDate>
            <atom:updated>2025-12-19T10:42:11.397Z</atom:updated>
            <content:encoded><![CDATA[<p>This October, we migrated our Scala services to Java 25. We rolled out the upgrade incrementally, enabling one feature at a time to measure impact. Here’s what we observed.</p><h3>The Starting Point</h3><p>Our backend services run Scala 3 on the JVM, packaged in containers using <a href="https://adoptium.net/temurin">Eclipse Temurin JRE 24</a>. The metrics in this post come from one of our main backend services, a high-traffic system that spends a lot of its time parsing and decoding JSON and Protobuf payloads. This makes it a good candidate for measuring memory optimisations: lots of objects, lots of strings.</p><p>All graphs show aggregated data across all replicas of this service.</p><h3>What Java 25 Brings to the Table</h3><p>Java’s six-month release cadence delivers regular performance wins. For Scala services, these often come “for free” with no code changes required. Two features caught our attention:</p><p><a href="https://openjdk.org/jeps/450">Compact Object Headers</a> shrinks object headers from 96–128 bits down to 64 bits. Introduced as experimental in Java 24, it <a href="https://openjdk.org/jeps/519">became a standard feature in Java 25</a>. Services with millions of small objects in memory (most long-running Scala applications) benefit directly from reduced heap usage. A <a href="https://www.reddit.com/r/scala/comments/1jptiv3/xxusecompactobjectheaders_is_your_new_turbo/">post by Andriy Plokhotnyuk on r/scala</a> first caught our attention on this one.</p><p><a href="https://openjdk.org/jeps/192">String Deduplication</a> with G1 identifies duplicate strings and collapses them to share the same underlying char[] array. Backend services typically hold a lot of repetitive string data: configuration values, JSON keys, or any identifier which an underlying string representation. The <a href="https://tschatzl.github.io/2025/08/12/jdk25-g1-serial-parallel-gc-changes.html">recent G1 improvements in JDK 25</a> make this feature more effective than before.</p><h3>Our Approach: One Change at a Time</h3><p>We rolled out each change separately over several days. This isolates the impact of each feature and lets us attribute anomalies to specific changes.</p><h4>Phase 1: Java 25 Baseline (October 9th)</h4><p>Upgrade from Java 24 to Java 25, no additional flags. We monitored for a few days and observed no significant changes in memory, CPU, or GC behaviour.</p><h4>Phase 2: Compact Object Headers (October 13th)</h4><p>Enabled with -XX:+UseCompactObjectHeaders. Results:</p><ul><li>~6% drop in heap memory</li><li>Slight increase in non-heap memory</li><li>Slight decrease in CPU, within normal variance</li><li>No significant GC changes</li></ul><h4>Phase 3: String Deduplication (October 15th)</h4><p>Enabled with -XX:+UseStringDeduplication. Observed impact:</p><ul><li>~8% additional drop in heap memory</li><li>~3% increase in non-heap memory</li><li>~3% increase in average GC duration, but GC frequency dropped proportionally; net time spent on GC per second stayed roughly constant</li></ul><h3>Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*922qkVd7uAMBnUzONIeolQ.png" /><figcaption>Heap (average)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xpLbgTmlEA3q7HGvIouQFw.png" /><figcaption>Non-heap (average)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gby2KISoo4mNWqjlo2vQDg.png" /><figcaption>vCPU (sum)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ndq6D0_su0NWq-lauwp_4g.png" /><figcaption>GC ops (average)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n_Nh0yEcVEmeFPS3dA4F8Q.png" /><figcaption>GD duration (average)</figcaption></figure><h3>Takeaways</h3><p><strong>Heap savings are real.</strong> Compact object headers plus string deduplication delivered ~14% reduction in heap usage.</p><p><strong>Non-heap memory increased.</strong> The heap reductions came with higher non-heap usage. If your container memory limits assume a specific heap-to-total-memory ratio, you’ll need to adjust. We now allocate proportionally less to the heap to accommodate the off-heap increase.</p><p><strong>String deduplication should be the default with G1.</strong> There’s no significant drawback, and if your services handle repetitive string data the memory savings are substantial.</p><h3>What’s Next</h3><p>We’re keeping an eye on <a href="https://bugs.openjdk.org/browse/JDK-8359211">Automatic Heap Sizing for G1</a>. This would let G1 dynamically adjust heap size based on container memory limits, simplifying configuration significantly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b7a23234b9f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swissborg-engineering/java-25-for-scala-services-production-impact-report-3b7a23234b9f">Java 25 for Scala Services: Production Impact Report</a> was originally published in <a href="https://medium.com/swissborg-engineering">SwissBorg Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging AWS Aurora for Event Sourcing]]></title>
            <link>https://medium.com/swissborg-engineering/leveraging-aws-aurora-for-event-sourcing-e8323dce58b6?source=rss-a63638eaef02------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8323dce58b6</guid>
            <category><![CDATA[aurora]]></category>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[event-sourcing]]></category>
            <category><![CDATA[akka]]></category>
            <category><![CDATA[cqrs]]></category>
            <dc:creator><![CDATA[Lomig Mégard]]></dc:creator>
            <pubDate>Tue, 22 Feb 2022 12:51:21 GMT</pubDate>
            <atom:updated>2022-02-22T12:51:21.932Z</atom:updated>
            <content:encoded><![CDATA[<p>At SwissBorg, we extensively use Event Sourcing, particularly for our core services. To us, having a reliable source of truth that can be easily replayed and audited is a must, and going this way made a lot of sense when we designed our architecture years ago. It also allows horizontal scalability, which we hoped we would need soon enough — and we do now. And since we are on JVM lands, <a href="https://doc.akka.io/docs/akka/current/typed/index-persistence.html">akka-persistence</a> stood out from the crowd as a mature library that would help us implement such an architecture.</p><h4>First approach</h4><p>Back in 2018, we chose Cassandra as the backend for akka-persistence. Event journals match the way append-only NoSQL databases like Cassandra work, and its scalability is outstanding. The cherry on top, there was a <a href="https://github.com/akka/akka-persistence-cassandra">production-ready akka-persistence plugin</a> awaiting us!</p><p>We put together a forecast gaming application (at that time called Community App) to battle test our architecture and had it running for two years. We built it on top of this Akka+Cassandra event-store architecture, and it worked like a charm. However, when we started our flagship product (the SwissBorg App), we quickly realized that the benefits offered by Cassandra were not worth its infrastructure and operational cost, at least not at our scale. The time had come to look for an alternative.</p><h4>Research</h4><p>At the time, some of our services already needed a relational database, and we had a lot of in-house knowledge of this type of database. Therefore, it made a lot of sense to start by exploring what options existed to use akka-persistence with a relational DB backend. It led us to look in-depth into the <a href="https://github.com/akka/akka-persistence-jdbc/">akka-persistence-jdbc</a> project.</p><p>This plugin is generic enough to connect to any JDBC compliant database, so we could in practice use the same one we were already using: PostgreSQL. This ability to only use one database technology would inevitably result in lower maintenance and operational costs, while reducing the cognitive load of our engineers.</p><p>On top of this, our infrastructure provider being AWS, we could use<a href="https://aws.amazon.com/rds/aurora/postgresql-features/"> Aurora</a> — a managed PostgreSQL-Compatible database — to have better performance and scalability than just a plain instance of PostgreSQL that we would have to operate.</p><p>However, akka-persistence-jdbc limited us because it was made to support only JDBC-compliant database technologies. This constraint makes the implementation generic and seldom allows the user to leverage the more specific features of the underlying database like we wanted to do with PostgreSQL.</p><p>We knew that we would grow well past 100 million events in one journal table and that it wouldn’t take too long to reach these volumes. Since akka-persistence-jdbc has no notion of partitioning, we would have had to push a single PostgreSQL table to its limits without a clear way of scaling it further. At this point, we started wondering how we could use PostgreSQL table partitioning features while leveraging the qualities of akka-persistence-jdbc.</p><p>At the time, we were working with our good friends from <a href="https://softwaremill.com/">SoftwareMill</a>, and this turned out to be an excellent opportunity to collaborate on something that we could give back to the community. It led to the creation of <a href="https://github.com/SwissBorg/akka-persistence-postgres">akka-persistence-postgres</a>, a fork of akka-persistence-jdbc that focuses on expanding the functionalities provided by the original plugin while making use of PostgreSQL’s specific features.</p><p>SoftwareMill already wrote a <a href="https://blog.softwaremill.com/why-we-need-yet-another-akka-persistence-plugin-4e08993eb04e">good blog post</a> about it that dwells on what it brings over the base plugin and its ideas.</p><h4>How are we leveraging akka-persistence-postgres?</h4><p>Some of our core services use the CQRS pattern, implemented with akka-cluster and akka-persistence-postgres, and powered by AWS Aurora. In the scope of our architecture, we make use of the low-latency read replicas, the auto-scaling storage and the high availability capabilities. The following diagram showcases how CQRS is typically implemented in our services:</p><figure><img alt="CQRS pattern leveraging read replicas" src="https://cdn-images-1.medium.com/max/1024/1*KNYKUABObNM1D2vvn1LKKQ.png" /><figcaption>CQRS pattern leveraging read replicas</figcaption></figure><ol><li>The journey starts with a call to the service’s gRPC server, where the Protobuf is decoded and basic validation is performed.</li><li>When the call is about a specific entity, a command (C of <strong>C</strong>QRS) is created and sent to the entity. The routing to the correct shard and actual persistent actor is handled by akka-cluster. The sharding allows the handling of commands to scale horizontally, adding more nodes to the cluster as required.</li><li>The persistent entity rebuilds its state by replaying all its past events. In order to have consistent reads, the actor queries the Aurora writer (or <em>primary</em>) node.</li><li>If a command leads to a change of state, new events are created and persisted in the journal. That’s where horizontal scaling stops, PostgreSQL only having one writer node at any given time.</li><li>To support queries (Q of C<strong>Q</strong>RS), one or more projections create dedicated views of the events’ journal. A projection process continuously listens and reads events from the journal. In theory, the read replicas could be used by the projection process, however, the current implementation of the persistence plugin forbids it. This non-obvious limitation will be covered in a future blog post.</li><li>Arbitrary updates are made to the projection table(s).</li><li>Aurora takes care of the replication, eventually propagating the changes to the read replicas.</li><li>The read-only calls to the gRPC server can leverage the projections. For cases where consistent reads are needed, we can directly query the persistent entity. That ends up being very performant because it reads the in-memory state of the sharded entities.</li><li>Since projections are eventually consistent, we don’t mind having the reader nodes also ultimately replicated. It allows us to horizontally scale the reader nodes we need — up to 15, which is Aurora’s maximum.</li></ol><h4>In numbers</h4><p>To give an idea of the scale we are currently facing, we have these numbers for our most prominent journal:</p><ul><li>Journal size of 182Gb, 21 partitions for a total of 420 million events;</li><li>900 events inserted per minute on a 24h average, with spikes up to 6,000;</li><li>0.14 milliseconds per insert on average;</li><li>Around 40,000 eventual reads per minute, spread across two read replicas.</li></ul><figure><img alt="Average startup time of a single persistent entity (events replay) is between 15 and 45ms" src="https://cdn-images-1.medium.com/max/1024/0*t0kB-D1SHMD0616d" /><figcaption>Average startup time of a single persistent entity (events replay)</figcaption></figure><figure><img alt="Aurora average replication lag of 2 read replicas is between 2 and 25ms" src="https://cdn-images-1.medium.com/max/1024/0*Zu3eVpPkF4ycHuNC" /><figcaption>Aurora average replication lag of 2 read replicas</figcaption></figure><p>As we saw in the previous paragraphs, our current architecture is not fully horizontally scalable. However, we are far from reaching its hard limits. The writer node can still be scaled vertically. Currently, we use a db.r6g.2xlarge instance with 64Gb of memory and 8 vCPUs, giving us a comfortable margin until we reach the most powerful instance available, the db.r6g.16xlarge with 512Gb of memory and 64 vCPUs. In terms of storage space, we are pretty far from Aurora’s limit of 128TB. If either the RAM, CPU or storage becomes an issue, we can always create new Aurora clusters, spreading our different journals on them.</p><h4>Future and next steps</h4><p>As explained in the previous section, we are far from reaching the hard limits of the database, however, some queries from the akka-persistence plugin could still be optimized. Namely, the ones that do not scale alongside the number of partitions, inevitably leading to slower execution as the journals grow. An example of such a query is the one that retrieves the highest sequence number per persistence_id, the query planner being forced to go over all partitions to answer it. We are currently exploring some ideas for<a href="https://github.com/SwissBorg/akka-persistence-postgres/issues/155"> supporting tables</a> that keep track of key metadata per persistence_id.</p><p>As we grow, we will both improve our current solution based on Aurora and keep looking-out for new technologies or alternative approaches.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8323dce58b6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swissborg-engineering/leveraging-aws-aurora-for-event-sourcing-e8323dce58b6">Leveraging AWS Aurora for Event Sourcing</a> was originally published in <a href="https://medium.com/swissborg-engineering">SwissBorg Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>