<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mitsunori Komatsu on Medium]]></title>
        <description><![CDATA[Stories by Mitsunori Komatsu on Medium]]></description>
        <link>https://medium.com/@komamitsu?source=rss-880d4ea6ce25------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*aTlePLHTaLMC67AKxY_msA.png</url>
            <title>Stories by Mitsunori Komatsu on Medium</title>
            <link>https://medium.com/@komamitsu?source=rss-880d4ea6ce25------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:34:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@komamitsu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Wormhole4j v0.3.0: Supports Multi-threaded Concurrency]]></title>
            <link>https://komamitsu.medium.com/wormhole4j-v0-3-0-supports-multi-threaded-concurrency-b9da79879c8c?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9da79879c8c</guid>
            <category><![CDATA[multithreading]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[data-structures]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Wed, 06 May 2026 01:16:33 GMT</pubDate>
            <atom:updated>2026-05-06T01:16:33.118Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Wormhole4j is a Java implementation of the <strong>Wormhole</strong> index, an ordered in-memory data structure from the EuroSys ’19 paper, <em>“Wormhole: A Fast Ordered Index for In-memory Data.”</em> By using the strengths of hash tables, prefix trees, and B+ trees, it achieves a worst-case lookup complexity of <strong>O(log L)</strong>, where <strong>L</strong> is the length of the key. This makes it very fast for both point lookups and range scans.</p><p>While earlier versions of the library were fast, they only worked in single-threaded environments. Most high-performance systems today need thread safety and concurrency. Version 0.3.0 fixes this. The concurrency model follows the design described in the original paper. This post describes how we implemented it in Java, what challenges we encountered, and what the benchmarks show.</p><h3>Why Concurrent Ordered Indexes Are Hard</h3><p>Designing a concurrent ordered index is much harder than a concurrent hash map. In a hash map, keys are independent. In an ordered index, keys are linked in a specific sequence to allow range scans.</p><p>Using one “big lock” is the easiest way to make code thread-safe, but it stops multiple threads from working at the same time, which ruins performance. To avoid this, Wormhole4j uses the strategy from the original paper, which splits operations into three groups:</p><ul><li><strong>Read-only (GET, SCAN):</strong> These should be lock-free and not block each other.</li><li><strong>Leaf-only writes (PUT/DELETE that stay within one leaf node):</strong> These only affect one leaf node and only need a lock on that specific node.</li><li><strong>Structural writes (PUT/DELETE that cause a split/merge):</strong> These are the most complex because they change both the leaf list and the internal metadata.</li></ul><p>The goal is to make sure that a structural change — like a node split — doesn’t cause a reader to see incorrect data.</p><h3>The Design: Lock-Free Reads via QSBR RCU</h3><p>To keep reads fast, Wormhole4j uses <strong>QSBR RCU</strong> (Quiescent State-Based Reclamation Read-Copy-Update) for metadata. The index keeps two copies of the metadata: an active table and an inactive table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/0*98jPqNZ3JSZlD689.png" /></figure><p>When a structural change happens, the writer updates the inactive table and then flips a pointer to make it the active one. To safely update the “old” table later, the system needs to know when all readers are finished. Instead of using expensive reference counting, Wormhole4j uses QSBR RCU, where threads signal a “quiescent state” (a quiet moment) when they aren’t using the index.</p><p>Because the QSBR mechanism needs to know which threads are active to work correctly, you must register and unregister each thread:</p><pre>// Enrollment is required for the QSBR RCU mechanism<br>wormhole.registerThread();<br>try {<br>    String value = wormhole.get(&quot;some_key&quot;);<br>    wormhole.put(&quot;another_key&quot;, 42);<br>    ...<br>} finally {<br>    // Signaling a quiescent state upon completion<br>    wormhole.unregisterThread();<br>}</pre><p>We chose to do it this way on purpose. While it requires an extra step for the user, it removes the heavy overhead usually needed to track concurrent readers, which makes the library much faster.</p><h3>Fine-Grained Locking on Leaf Nodes</h3><p>For local changes, Wormhole4j uses fine-grained locking. Each leaf node has its own lock, so writers only compete with other threads trying to access the exact same data.</p><p>To keep readers from being blocked by these locks, we use version numbers. Readers check a version timestamp before and after reading a node. If the versions don’t match, it means a change happened, and the reader simply tries again.</p><h3>Testing Correctness with Lincheck</h3><p>Concurrency bugs are very hard to find with standard tests because they only happen when threads interact in specific, rare ways. To make sure v0.3.0 is truly thread-safe, we used <a href="https://github.com/JetBrains/lincheck">Lincheck</a>, a tool for testing concurrent data structures on the JVM.</p><p>Lincheck checks different thread orders to verify <strong>linearizability</strong>: the rule that every concurrent execution must act like a valid sequential one. By using a standard TreeMap as our guide, we verified that the concurrent code works correctly.</p><p>During testing, we used a small key range and a tiny leaf node size (set to 4). This forced the index to perform splits and merges constantly, which helped us find several real bugs that would have stayed hidden in normal tests. One example: a reader could follow stale leaf node metadata immediately after a split, landing on the wrong node and returning an incorrect result. Without Lincheck systematically exploring thread interleavings, this kind of bug is nearly impossible to reproduce reliably.</p><h3>Benchmark Results</h3><p>We compared Wormhole4j v0.3.0 against ConcurrentSkipListMap (CSLM) using Java 21 with different thread counts (from 1+1 up to 8+8).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*rYSvmurxt7mDLWHA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*XilbSQLLgIy6JIuT.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*LjmwVWB6OSc4T_Ir.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*SPGDeZYWAraFzRfc.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*xZ558gg-SSn4wbik.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*po3HIZRCGGbyH2Zp.png" /></figure><ul><li><strong>PUT + GET:</strong> Wormhole4j was faster than CSLM across all thread counts. It performed best with String keys, where its lookup logic is much more efficient than the string comparisons used by a skip list.</li><li><strong>PUT + SCAN:</strong> Wormhole4j has a strong advantage in PUT speed. However, for numeric key scans under heavy write pressure, the frequent node splits can make performance closer to CSLM.</li><li><strong>Scalability:</strong> Both structures scaled well up to about 4+4 threads. Wormhole4j kept its performance lead as more threads were added.</li></ul><h3>Lessons Learned</h3><p>Developing v0.3.0 showed that testing concurrent code is often more difficult than implementing it. While the plan for QSBR RCU and dual tables was important, strict linearizability testing was what actually made the implementation reliable. Lincheck is now part of the automated tests to make sure future updates do not break thread safety.</p><h3>Closing</h3><p>Wormhole4j v0.3.0 brings high-performance, verified concurrent indexing to Java. By combining QSBR RCU, dual-table metadata, and leaf-level locking, it offers a fast and reliable alternative to standard concurrent collections.</p><ul><li><strong>GitHub:</strong> <a href="https://github.com/komamitsu/wormhole4j">komamitsu/wormhole4j</a></li><li><strong>Original Paper:</strong> <a href="https://dl.acm.org/doi/10.1145/3302424.3303955">Wormhole: A Fast Ordered Index for In-memory Data</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9da79879c8c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering Performance in Wormhole4j: Lessons from Version 0.2]]></title>
            <link>https://komamitsu.medium.com/engineering-performance-in-wormhole4j-lessons-from-version-0-2-7e4424aa9000?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e4424aa9000</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[performance-improvement]]></category>
            <category><![CDATA[key-value-store]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[index]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Sun, 21 Dec 2025 06:50:36 GMT</pubDate>
            <atom:updated>2025-12-21T06:50:36.922Z</atom:updated>
            <content:encoded><![CDATA[<h3>Engineering Performance in Wormhole4j: Lessons from Version 0.2</h3><p>In <a href="https://komamitsu.medium.com/wormhole4j-a-fast-ordered-map-for-java-724dc7f88e6e">the previous post</a>, I introduced <a href="https://github.com/komamitsu/wormhole4j">Wormhole4j</a> v0.1, a Java implementation of the Wormhole index (i.e., sorted map). Developing version v0.2 involved adding support for numeric keys and pushing performance to its absolute limit by selectively breaking abstractions to reduce Java instance generation costs.</p><h3>When Abstraction Costs Matter</h3><p>In most Java applications, I/O dominates performance, and clean abstractions are the right choice. However, Wormhole4j is:</p><ul><li>Fully in-memory (no I/O)</li><li>Computation-bound</li><li>Called millions of times in hot paths</li></ul><p>In such cases, abstraction costs become visible and worth addressing.</p><h3>The Evolution of Key Logic</h3><p>Wormhole logically treats keys as Tries, meaning it processes them token-by-token.</p><ul><li>The logical data structure</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xPj-NYrCZROAfJ91" /></figure><ul><li>The physical data structure</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f22zsS25_PIvK6RT" /></figure><p>Because of this, raw numeric types cannot be used directly and must be encoded into a lexicographically ordered format.</p><h3>1. Hexadecimal Strings (The Slow Starting Point)</h3><p>My first attempt involved converting integers into fixed-width hexadecimal strings. By flipping the sign bit via an XOR operation with 0x80000000, I ensured that numeric order matched lexicographical order.</p><pre>public class WormholeForIntKey&lt;V&gt; extends Wormhole&lt;Integer, V&gt; {<br>  ...<br>  <br>  @Override<br>  String encodeKey(Integer key) {<br>    // Encode an int into a lexicographically ordered hexadecimal string.<br>    return String.format(&quot;%08x&quot;, 0x80000000 ^ key); // 42 -&gt; &quot;8000002a&quot;<br>                                    // Integer.MIN_VALUE  -&gt; &quot;00000000&quot;<br>                                    // Integer.MAX_VALUE  -&gt; &quot;ffffffff&quot;<br>  }<br>}</pre><p>This approach was simple to implement but resulted in performance only about 1/3 of java.util.TreeMap due to the high costs of string formatting and allocation.</p><h3>2. The byte[] Wrapper (Performance regression of String key)</h3><p>To avoid string overhead, I transitioned to encoding keys into byte[]. However, byte[] is unsuitable for a HashMap key because its equals() and hashCode() methods use identity rather than content. I wrapped the array in an EncodedKey class to handle this correctly:</p><pre>class EncodedKey implements Comparable&lt;EncodedKey&gt; {<br>  private final byte[] content;<br><br>  @Override<br>  public boolean equals(Object o) {<br>    if (this == o) return true;<br>    if (o == null || getClass() != o.getClass()) return false;<br>    EncodedKey that = (EncodedKey) o;<br>    return Arrays.equals(content, that.content);<br>  }<br><br>  @Override<br>  public int hashCode() {<br>    // return Arrays.hashCode(content);<br>    return (short) (0x7FFF &amp; LongHashFunction.xx3().hashBytes(content));<br>  }<br>  ...<br>}</pre><p>While faster than hexadecimal strings, the constant creation of EncodedKey and ByteBuffer instances created allocation pressure:</p><pre>@Override<br>EncodedKey encodeKey(Integer key) {<br>  ByteBuffer buf = ByteBuffer.allocate(4);<br>  buf.putInt(0x80000000 ^ key);<br>  return new EncodedKey(buf.array());<br>}</pre><p>Furthermore, unifying all keys into this wrapper caused a performance regression for string keys, which were faster when handled directly in version 0.1.</p><h3>3. The Hybrid Key Approach (Removing the intermediate EncodedKey)</h3><p>To restore performance for strings, I stopped using the EncodedKey wrapper. I adopted a design using Object as a generic key type and an enum to track the type, utilizing a utility class for all operations:</p><pre>final class EncodedKeyUtils {<br>  private EncodedKeyUtils() {}<br><br>  static Object createEmpty(EncodedKeyType encodedKeyType) {<br>    switch (encodedKeyType) {<br>      case STRING:<br>        return &quot;&quot;;<br>      case BYTE_ARRAY:<br>        return ByteArray.EMPTY_INSTANCE;<br>      default:<br>        throw new AssertionError();<br>    }<br>  }<br><br>  static int length(EncodedKeyType encodedKeyType, Object obj) {<br>    switch (encodedKeyType) {<br>      case STRING:<br>        assert obj instanceof String;<br>        return ((String) obj).length();<br>      case BYTE_ARRAY:<br>        assert obj instanceof ByteArray;<br>        return ((ByteArray) obj).length();<br>      default:<br>        throw new AssertionError();<br>    }<br>  }<br><br>  static Object slice(EncodedKeyType encodedKeyType, Object obj, int length) {<br>    switch (encodedKeyType) {<br>      case STRING:<br>        assert obj instanceof String;<br>        return ((String) obj).substring(0, length);<br>      case BYTE_ARRAY:<br>        assert obj instanceof ByteArray;<br>        return ((ByteArray) obj).slice(length);<br>      default:<br>        throw new AssertionError();<br>    }<br>  }<br><br>  static int get(EncodedKeyType encodedKeyType, Object obj, int pos) {<br>    switch (encodedKeyType) {<br>      case STRING:<br>        assert obj instanceof String;<br>        return ((String) obj).charAt(pos);<br>      case BYTE_ARRAY:<br>        assert obj instanceof ByteArray;<br>        return ((ByteArray) obj).get(pos);<br>      default:<br>        throw new AssertionError();<br>    }<br>  }<br>  ...<br>}</pre><p>This allowed string keys to avoid extra object wrapping, restoring their performance to original levels.</p><h3>4. The IntWrapper Refinement (Avoiding creating byte[])</h3><p>For numeric keys, I created a specialized IntWrapper to manage logical key lengths for the Trie without converting to byte[]. This improved efficiency for hashCode() and equals() compared to the version using byte[].</p><pre>final class IntWrapper implements Comparable&lt;IntWrapper&gt; {<br>  private static final int SIZE = 4;<br>  private static final int[] BITMASKS =<br>      new int[] {<br>        0x00000000, // Length: 0<br>        0xFF000000, // Length: 1<br>        0xFFFF0000, // Length: 2<br>        0xFFFFFF00, // Length: 3<br>        0xFFFFFFFF, // Length: 4<br>      };<br>  static final IntWrapper EMPTY_INSTANCE = new IntWrapper(0, 0);<br>  private static final char[] HEX_CHARS = &quot;0123456789ABCDEF&quot;.toCharArray();<br>  // The value is already masked based on the length.<br>  private final int value;<br>  private final int length;<br><br>  IntWrapper(int value) {<br>    this.length = SIZE;<br>    this.value = value;<br>  }<br><br>  private IntWrapper(int value, int length) {<br>    assert length &lt;= SIZE;<br>    this.length = length;<br>    this.value = value &amp; BITMASKS[length];<br>  }<br><br>  @Override<br>  public int compareTo(IntWrapper other) {<br>    int minLen = Math.min(length, other.length);<br>    int mask = BITMASKS[minLen];<br>    int x1 = value &amp; mask;<br>    int x2 = other.value &amp; mask;<br>    if (x1 != x2) {<br>      return Integer.compareUnsigned(x1, x2);<br>    }<br>    return length - other.length;<br>  }<br><br>  int get(int pos) {<br>    assert pos &lt; length;<br>    return (value &gt;&gt;&gt; ((SIZE - 1 - pos) &lt;&lt; 3)) &amp; 0xFF;<br>  }<br><br>  @Override<br>  public boolean equals(Object o) {<br>    if (this == o) {<br>      return true;<br>    }<br>    if (!(o instanceof IntWrapper)) {<br>      return false;<br>    }<br>    IntWrapper other = (IntWrapper) o;<br>    return length == other.length &amp;&amp; value == other.value;<br>  }<br><br>  @Override<br>  public int hashCode() {<br>    return value * 31 + length;<br>  }<br>  ...<br>}</pre><h3>Another Optimization: Flattening the LeafNode</h3><p>LeafNode instance consists of KeyValues, Tags and KeyReferences instances. Tags contains sorted the hash values of the encoded keys, which is used for point lookup operation. KeyReferences contains fully or partially sorted key index, which is used for scan operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/0*a_QLIoBMnH1CqAOP" /></figure><p>To eliminate the instance generation cost of managing thousands of small objects, I flattened the LeafNode structure. Based on experience, I suspected that breaking abstractions to direct lists of key-values, tags, and references would be &quot;dirtier&quot; but faster.</p><p>Original Abstracted Structure:</p><pre>final class LeafNode&lt;K, V&gt; {<br>  private final EncodedKeyType encodedKeyType;<br>  final Object anchorKey;<br>  private final int maxSize;<br>  private final KeyValues&lt;K, V&gt; keyValues;<br>  // All references are always sorted by hash.<br>  private final Tags&lt;K, V&gt; tags;<br>  // Some references are sorted by key.<br>  private final KeyReferences&lt;K, V&gt; keyReferences;<br><br>  private static class KeyValues&lt;K, V&gt; {<br>    private int count;<br>    private final KeyValue&lt;K, V&gt;[] entries;<br>    ...<br>  }<br>  private static class Tags&lt;K, V&gt; {<br>    private int count;<br>    private final KeyValues&lt;K, V&gt; keyValues;<br>    private final int[] entries;<br>    ...<br>  }<br>  private static class KeyReferences&lt;K, V&gt; {<br>    private final EncodedKeyType encodedKeyType;<br>    private final KeyValues&lt;K, V&gt; keyValues;<br>    private int count;<br>    private final int[] entries;<br>    private int numOfSortedEntries;<br>    ...<br>  }<br>  ...<br>}</pre><p>Optimized Flattened Structure:</p><pre>final class LeafNode&lt;K, V&gt; {<br>  private static final int TUPLE_SIZE = 3;<br>  private static final int ENCODED_KEY_OFFSET = 0;<br>  private static final int KEY_OFFSET = 1;<br>  private static final int VALUE_OFFSET = 2;<br>  private final EncodedKeyType encodedKeyType;<br>  final Object anchorKey;<br>  private final int maxSize;<br><br>  ...<br><br>  // For KeyValues<br>  private int keyValuesCount;<br>  private final Object[] keyValues;<br><br>  ...<br><br>  // For Tags<br>  private int tagsCount;<br>  private final int[] tags;<br><br>  ...<br><br>  // For KeyRefs<br>  private int keyRefsCount;<br>  private final int[] keyRefs;<br>  private int numOfSortedKeyRefs;<br><br>  ...<br>}</pre><p>In the flattened version, I avoided KeyValue instantiation by storing the encoded key, original key, and value sequentially within a single Object[].</p><p>Memory Layout:</p><pre>keyValues array (Object[])<br>┌───────────────────────────────────────────────────────────────────────┐<br>│                                                                       │<br>├─────────────┬─────────────┬─────────────┬─────────────┬───────────────┤<br>│ EncodedKey₀ │    Key₀     │   Value₀    │ EncodedKey₁ │    Key₁ ...   │<br>├─────────────┼─────────────┼─────────────┼─────────────┼───────────────┤<br>│   [0]       │   [1]       │   [2]       │   [3]       │   [4]   ...   │<br>└─────────────┴─────────────┴─────────────┴─────────────┴───────────────┘<br>      ↑             ↑             ↑             ↑             ↑<br>      │             │             │             │             │<br>   ENCODED_      KEY_         VALUE_        ENCODED_       KEY_<br>   KEY_OFFSET    OFFSET       OFFSET        KEY_OFFSET     OFFSET<br>   (0)           (1)          (2)           (0)            (1)<br><br>Index calculation for n-th entry:<br>┌────────────────────────────────────────┐<br>│ base_index = n * TUPLE_SIZE (3)        │<br>│ encoded_key_index = base + 0           │<br>│ key_index         = base + 1           │<br>│ value_index       = base + 2           │<br>└────────────────────────────────────────┘<br>Example for 3 key-value pairs:<br>┌──────┬──────┬──────┬──────┬──────┬──────┬──────┬──────┬──────┐<br>│ EK₀  │  K₀  │  V₀  │ EK₁  │  K₁  │  V₁  │ EK₂  │  K₂  │  V₂  │<br>├──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤<br>│  [0] │  [1] │  [2] │  [3] │  [4] │  [5] │  [6] │  [7] │  [8] │<br>└──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┘<br>   └──── KV entry 0 ────┘ └──── KV entry 1 ────┘ └──── KV entry 2 ────┘</pre><p>This change made the code more complex, but it reduced the Java instance generation costs and improved overall speed.</p><h3>Final Results</h3><p>With these optimizations, Wormhole4j v0.2 is now faster or competitive with TreeMap:</p><ul><li>Significantly faster scan() API for full scans, prefix scans, and range scans (inclusive/exclusive) - 2-10x faster than tree-based alternatives</li><li>Excellent performance for String keys — 25–63% faster get/put operations than tree-based structures</li><li>Competitive get() and put() performance with optimized tree structures for numeric keys</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*61veQZ-5ScEEhPad" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*ZkGFyjVJGq4lIkuy" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*9bxlWoTgub7Ol6h9" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*a4TiVTXvZgv8yyjl" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*eoxNrCtAlguafpq1.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*tpc1p-P30SMvSM3K" /></figure><h3>Technical Lessons for Java Developers</h3><h3>Reduce Instantiation in Hot Paths</h3><p>Although Java instance generation cost is low, the cumulative effect of millions of small objects has a significant impact in non-I/O bound workloads.</p><h3>Wrong hashCode() implementation causes performance regressions</h3><p>When implementing hashCode() for long values, do not simply cast to int, as it discards the top 32 bits and causes hash collisions.</p><pre>final class LongWrapper implements Comparable&lt;LongWrapper&gt; {<br>  private final long value;<br>  private final int length;<br>  ...<br>  @Override<br>  public int hashCode() {<br>    return (int)(value * 31 + length);<br>  }</pre><p>Use Long.hashCode(value), which performs an XOR shift: (int)(value ^ (value &gt;&gt;&gt; 32)).</p><pre>final class LongWrapper implements Comparable&lt;LongWrapper&gt; {<br>  private final long value;<br>  private final int length;<br>  ...<br>  @Override<br>  public int hashCode() {<br>    return Long.hashCode(value) * 31 + length;<br>  }</pre><h3>Hidden Varargs Allocations</h3><p>Java’s varargs including Objects.hash(Object... values) implicitly creates a new array object on every call.</p><pre>// Hidden array allocation on every call<br>@Override<br>public int hashCode() {<br>    return Objects.hash(value, length);  // Creates new Object[]{value, length}<br>}<br><br>// More efficient for hot paths<br>@Override<br>public int hashCode() {<br>    return value * 31 + length;  // No allocation<br>}</pre><h3>JMH and Memory Barriers</h3><p>Blackhole.consume() acts as a memory barrier. Calling it frequently inside tight loops can become a bottleneck; it is better to update a local variable and consume() it once at the end.</p><h3>Closing Thoughts</h3><p>Implementing numeric key support in Wormhole4j required carefully balancing abstraction with performance. While the techniques described here — flattening structures, avoiding allocations, breaking abstractions — are specific to computation-bound workloads, they demonstrate how understanding JVM behavior can unlock significant performance gains when they matter most.</p><p>For most applications, clean abstractions remain the right choice. But when you’re building a library that sits in the hot path of high-performance systems, knowing when and how to bend the rules becomes valuable.</p><p>Source code: <a href="https://github.com/komamitsu/wormhole4j">github.com/komamitsu/wormhole4j</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e4424aa9000" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Wormhole4j: A Fast Ordered Map for Java]]></title>
            <link>https://komamitsu.medium.com/wormhole4j-a-fast-ordered-map-for-java-724dc7f88e6e?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/724dc7f88e6e</guid>
            <category><![CDATA[data-structures]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[index]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[algorithms]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Tue, 26 Aug 2025 14:11:25 GMT</pubDate>
            <atom:updated>2025-08-26T14:11:25.388Z</atom:updated>
            <content:encoded><![CDATA[<p>When building fast in-memory data systems, the speed of data retrieval is critical. Hash tables provide O(1) time for point lookups, making them ideal for finding single items. However, they do not naturally support range queries — such as retrieving all keys between “apple” and “banana”. For that, ordered indexes like B+ trees or skip lists are commonly used, but they usually come with O(log N) lookup cost, which can become expensive at scale.</p><p>Tries offer an alternative where lookup time depends on key length (L) rather than the number of keys (N). While this can help with certain workloads, tries often consume more memory and may suffer from fragmented layouts, making them less efficient for real-world applications where keys can be moderately or significantly long.</p><h3>Wormhole: A Fast Ordered Index for In-memory Data Management</h3><p><strong>Wormhole</strong> was introduced in 2019 by Xingbo Wu, Fan Ni, and Song Jiang, in their research paper <a href="https://dl.acm.org/doi/10.1145/3302424.3303955">“Wormhole: A Fast Ordered Index for In-memory Data Management”</a>, to bridge the gap between hash tables and traditional ordered indexes. It provides lookup performance close to that of hash tables while still supporting ordered operations such as range and prefix scans.</p><p>Wormhole achieves lookups in O(log L), where L is the length of the key. For typical key sizes, this is close to constant time. This is a notable improvement over the O(log N) complexity of B+ trees or the O(L) complexity of traditional tries.</p><h3>How Wormhole Works Under the Hood</h3><p>Wormhole organizes its index into two main components: <strong>MetaTrieHT</strong> and <strong>LeafList</strong>.</p><p>The MetaTrieHT provides a highly optimized implementation of a logical trie, storing key prefixes to enable fast prefix-based lookups and reduce search complexity. This design minimizes unnecessary traversals while keeping memory use efficient.</p><p>Once the correct prefix or range starting point is found via MetaTrieHT, the LeafList (a doubly linked list of sorted leaf nodes) handles the actual keys and values. This structure enables efficient point lookups, range scans, and prefix scans while reducing cache misses and maintaining predictable performance.</p><h3>Introducing Wormhole4j: A Java Implementation</h3><p>Wormhole4j is a Java implementation of Wormhole designed for high-performance in-memory ordered indexes. It delivers very fast scan() performance for prefix and range scans, and fast point lookups via get(), along with efficient put() and delete() operations.</p><p>Currently, Wormhole4j supports only String keys and is not thread-safe. If you plan to use it in multi-threaded environments, you will need to implement external synchronization.</p><h3>Benchmark Overview</h3><p>In tests with 100,000 records and keys up to 128 characters, Wormhole4j has demonstrated significant performance improvements over standard java.util.TreeMap (Red-Black tree) and it.unimi.dsi.fastutil.objects.Object2ObjectAVLTreeMap (AVL tree), especially for scans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*CIOHB6EO2G2UP1zm0weMdw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*s61H8PmnWS0H3L6G.png" /></figure><h3>Future Directions</h3><p>Wormhole4j will evolve with features such as persistence for durability and thread-safe variants for concurrent workloads.</p><h3>Try Wormhole4j</h3><p>Add Wormhole4j to your project from Maven Central:</p><pre>implementation(&quot;org.komamitsu:wormhole4j:0.1.0&quot;)</pre><p>Source code and documentation are available on <a href="https://github.com/komamitsu/wormhole4j">GitHub</a>.<br>Feedback, issues, and contributions are welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=724dc7f88e6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside java.lang.String: Understanding and Optimizing Instantiation Performance]]></title>
            <link>https://komamitsu.medium.com/inside-java-lang-string-understanding-and-optimizing-instantiation-performance-15940a62105e?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/15940a62105e</guid>
            <category><![CDATA[benchmark]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Sun, 15 Dec 2024 02:50:58 GMT</pubDate>
            <atom:updated>2024-12-15T02:52:13.676Z</atom:updated>
            <content:encoded><![CDATA[<p>java.lang.String is probably one of the most used classes in Java. Naturally, it contains its string data internally.</p><p>Do you know how the data is actually stored in String, and what happens when instantiating a String from a byte array? In this post, we&#39;ll explore the internal structure of java.lang.String and discuss ways to improve instantiation performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*cFy6r0Y1zZzwDKFRRH7efw.png" /></figure><h3>Internal structure of java.lang.String in Java 8 or earlier</h3><p>In Java 8, <a href="https://github.com/openjdk/jdk8u/blob/4fa5109b8deee6539ba1765f27aa6cb641010221/jdk/src/share/classes/java/lang/String.java#L111-L114">java.lang.String</a> contains its string data as a 16-bit char array.</p><pre>public final class String<br>    implements java.io.Serializable, Comparable&lt;String&gt;, CharSequence {<br>    /** The value is used for character storage. */<br>    private final char value[];</pre><p>When instantiating a String from a byte array, StringCoding.decode() is <a href="https://github.com/openjdk/jdk8u/blob/4fa5109b8deee6539ba1765f27aa6cb641010221/jdk/src/share/classes/java/lang/String.java#L459-L464">called</a>.</p><pre>public String(byte bytes[], int offset, int length, Charset charset) {<br>    if (charset == null)<br>        throw new NullPointerException(&quot;charset&quot;);<br>    checkBounds(bytes, offset, length);<br>    this.value =  StringCoding.decode(charset, bytes, offset, length);<br>}</pre><p>In the case of US_ASCII, <a href="https://github.com/openjdk/jdk8u/blob/master/jdk/src/share/classes/sun/nio/cs/US_ASCII.java#L137-L148">sun.nio.cs.US_ASCII.Decoder.decode()</a> is finally called, which copies the bytes of the source byte array into a char array one by one.</p><pre>public int decode(byte[] src, int sp, int len, char[] dst) {<br>    int dp = 0;<br>    len = Math.min(len, dst.length);<br>    while (dp &lt; len) {<br>        byte b = src[sp++];<br>        if (b &gt;= 0)<br>            dst[dp++] = (char)b;<br>        else<br>            dst[dp++] = repl;<br>    }<br>    return dp;<br>}</pre><p>The newly created char array is used as the new String instance’s char array value.</p><p>As you notice, even if the source byte array contains only single byte characters, the byte-to-char copy iteration occurs.</p><h3>Internal structure of java.lang.String in Java 9 or later</h3><p>In Java 9 or later, <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/String.java#L125-L140">java.lang.String</a> contains its string data as a 8-bit byte array.</p><pre>public final class String<br>    implements java.io.Serializable, Comparable&lt;String&gt;, CharSequence {<br>    /**<br>     * The value is used for character storage.<br>     *<br>     * @implNote This field is trusted by the VM, and is a subject to<br>     * constant folding if String instance is constant. Overwriting this<br>     * field after construction will cause problems.<br>     *<br>     * Additionally, it is marked with {@link Stable} to trust the contents<br>     * of the array. No other facility in JDK provides this functionality (yet).<br>     * {@link Stable} is safe here, because value is never null.<br>     */<br>    @Stable<br>    private final byte[] value;</pre><p>When instantiating a String from a byte array, StringCoding.decode() is also <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/String.java#L502-L510">called</a>.</p><pre>public String(byte bytes[], int offset, int length, Charset charset) {<br>    if (charset == null)<br>        throw new NullPointerException(&quot;charset&quot;);<br>    checkBoundsOffCount(offset, length, bytes.length);<br>    StringCoding.Result ret =<br>        StringCoding.decode(charset, bytes, offset, length);<br>    this.value = ret.value;<br>    this.coder = ret.coder;<br>}</pre><p>In the case of US_ASCII, <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/StringCoding.java#L521-L534">StringCoding.decodeASCII()</a> is called, which copies the source byte array using Arrays.copyOfRange(), as both the source and destination are byte arrays. Arrays.copyOfRange() internally uses System.arrayCopy() that is a native method and significantly fast.</p><pre>private static Result decodeASCII(byte[] ba, int off, int len) {<br>    Result result = resultCached.get();<br>    if (COMPACT_STRINGS &amp;&amp; !hasNegatives(ba, off, len)) {<br>        return result.with(Arrays.copyOfRange(ba, off, off + len),<br>                           LATIN1);<br>    }<br>    byte[] dst = new byte[len&lt;&lt;1];<br>    int dp = 0;<br>    while (dp &lt; len) {<br>        int b = ba[off++];<br>        putChar(dst, dp++, (b &gt;= 0) ? (char)b : repl);<br>    }<br>    return result.with(dst, UTF16);<br>}</pre><p>You may notice COMPACT_STRINGS constant. This improvement introduced in Java 9 is called <a href="https://openjdk.org/jeps/254">Compact Strings</a>. The feature is enabled by default, but you can disable it if you want. See <a href="https://docs.oracle.com/en/java/javase/17/vm/java-hotspot-virtual-machine-performance-enhancements.html#GUID-D2E3DC58-D18B-4A6C-8167-4A1DFB4888E4">https://docs.oracle.com/en/java/javase/17/vm/java-hotspot-virtual-machine-performance-enhancements.html#GUID-D2E3DC58-D18B-4A6C-8167-4A1DFB4888E4</a> for detail.</p><h3>The performance of new String(byte[]) in Java 8, 11, 17 and 21</h3><p>I wrote this simple JMH benchmark code to evaluate the performance of new String(byte[]):</p><pre>@State(Scope.Benchmark)<br>@OutputTimeUnit(TimeUnit.MILLISECONDS)<br>@Fork(1)<br>@Measurement(time = 3, iterations = 4)<br>@Warmup(iterations = 2)<br>public class StringInstantiationBenchmark {<br>  private static final int STR_LEN = 512;<br>  private static final byte[] SINGLE_BYTE_STR_SOURCE_BYTES;<br>  private static final byte[] MULTI_BYTE_STR_SOURCE_BYTES;<br>  static {<br>    {<br>      StringBuilder sb = new StringBuilder();<br>      for (int i = 0; i &lt; STR_LEN; i++) {<br>        sb.append(&quot;x&quot;);<br>      }<br>      SINGLE_BYTE_STR_SOURCE_BYTES = sb.toString().getBytes(StandardCharsets.UTF_8);<br>    }<br>    {<br>      StringBuilder sb = new StringBuilder();<br>      for (int i = 0; i &lt; STR_LEN / 2; i++) {<br>        sb.append(&quot;あ&quot;);<br>      }<br>      MULTI_BYTE_STR_SOURCE_BYTES = sb.toString().getBytes(StandardCharsets.UTF_8);<br>    }<br>  }<br><br>  @Benchmark<br>  public void newStrFromSingleByteStrBytes() {<br>    new String(SINGLE_BYTE_STR_SOURCE_BYTES, StandardCharsets.UTF_8);<br>  }<br><br>  @Benchmark<br>  public void newStrFromMultiByteStrBytes() {<br>    new String(MULTI_BYTE_STR_SOURCE_BYTES, StandardCharsets.UTF_8);<br>  }<br>}</pre><p>The benchmark results are as follows:</p><ul><li>Java 8</li></ul><pre>Benchmark                        Mode  Cnt     Score     Error   Units<br>newStrFromMultiByteStrBytes     thrpt    4  1672.397 ±  11.338  ops/ms<br>newStrFromSingleByteStrBytes    thrpt    4  4789.745 ± 553.865  ops/ms</pre><ul><li>Java 11</li></ul><pre>Benchmark                        Mode  Cnt      Score      Error   Units<br>newStrFromMultiByteStrBytes     thrpt    4   1507.754 ±   17.931  ops/ms<br>newStrFromSingleByteStrBytes    thrpt    4  15117.040 ± 1240.981  ops/ms</pre><ul><li>Java 17</li></ul><pre>Benchmark                        Mode  Cnt      Score     Error   Units<br>newStrFromMultiByteStrBytes     thrpt    4   1529.215 ± 168.064  ops/ms<br>newStrFromSingleByteStrBytes    thrpt    4  17753.086 ± 251.676  ops/ms</pre><ul><li>Java 21</li></ul><pre>Benchmark                        Mode  Cnt      Score      Error   Units<br>newStrFromMultiByteStrBytes     thrpt    4   1543.525 ±   69.061  ops/ms<br>newStrFromSingleByteStrBytes    thrpt    4  17711.972 ± 1178.212  ops/ms</pre><p>The throughput of newStrFromSingleByteStrBytes() was drastically improved from Java 8 to Java 11. It&#39;s likely because of the change from the char array to the byte array in String class.</p><h3>Further performance improvement with zero copy</h3><p>Okay, <a href="https://openjdk.org/jeps/254">Compact Strings</a> is a great performance improvement. But there is no room to improve the performance of String instantiation from a byte array? String(byte bytes[], int offset, int length, Charset charset) in Java 9 or later copies the byte array. Even it uses System.copyArray() that is a native method and fast, it takes some time.</p><p>When I read the source code of <a href="https://github.com/apache/fury">Apache Fury</a> which is “a blazingly-fast multi-language serialization framework powered by JIT (just-in-time compilation) and zero-copy”, I found their <a href="https://github.com/apache/fury/blob/3865dcd0982c0ca9de04a8ba9635892b76288769/java/fury-core/src/main/java/org/apache/fury/serializer/StringSerializer.java">StringSerializer</a> achieves zero copy String instantiation. Let’s look into the implementation.</p><p>The usage of the StringSerializer is as follows:</p><pre>import org.apache.fury.serializer.StringSerializer;<br><br>...<br><br>    byte[] bytes = &quot;Hello&quot;.getBytes();<br>    String s = StringSerializer.newBytesStringZeroCopy(LATIN1, bytes);<br>    System.out.println(s);    // &gt;&gt;&gt; Hello</pre><p>What StringSerializer.newBytesStringZeroCopy() finally achieves is to call non-public String constructor <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/String.java#L3252-L3255">new String(byte[], byte coder)</a>, where the source byte array is directly set to String.value without copying bytes.</p><pre>   /*<br>    * Package private constructor which shares value array for speed.<br>    */<br>    String(byte[] value, byte coder) {<br>        this.value = value;<br>        this.coder = coder;<br>    }</pre><p>When <a href="https://github.com/apache/fury/blob/3865dcd0982c0ca9de04a8ba9635892b76288769/java/fury-core/src/main/java/org/apache/fury/serializer/StringSerializer.java#L633-L655">StringSerializer.newBytesStringZeroCopy()</a> is called, the method calls BYTES_STRING_ZERO_COPY_CTR BiFunction or LATIN_BYTES_STRING_ZERO_COPY_CTR Function.</p><pre>public static String newBytesStringZeroCopy(byte coder, byte[] data) {<br>  if (coder == LATIN1) {<br>    // 700% faster than unsafe put field in java11, only 10% slower than `new String(str)` for<br>    // string length 230.<br>    // 50% faster than unsafe put field in java11 for string length 10.<br>    if (LATIN_BYTES_STRING_ZERO_COPY_CTR != null) {<br>      return LATIN_BYTES_STRING_ZERO_COPY_CTR.apply(data);<br>    } else {<br>      // JDK17 removed newStringLatin1<br>      return BYTES_STRING_ZERO_COPY_CTR.apply(data, LATIN1_BOXED);<br>    }<br>  } else if (coder == UTF16) {<br>    // avoid byte box cost.<br>    return BYTES_STRING_ZERO_COPY_CTR.apply(data, UTF16_BOXED);<br>  } else {<br>    // 700% faster than unsafe put field in java11, only 10% slower than `new String(str)` for<br>    // string length 230.<br>    // 50% faster than unsafe put field in java11 for string length 10.<br>    // `invokeExact` must pass exact params with exact types:<br>    // `(Object) data, coder` will throw WrongMethodTypeException<br>    return BYTES_STRING_ZERO_COPY_CTR.apply(data, coder);<br>  }<br>}</pre><p>BYTES_STRING_ZERO_COPY_CTR is initialized to a BiFunction returned from <a href="https://github.com/apache/fury/blob/3865dcd0982c0ca9de04a8ba9635892b76288769/java/fury-core/src/main/java/org/apache/fury/serializer/StringSerializer.java#L681-L705">getBytesStringZeroCopyCtr()</a>:</p><pre>private static BiFunction&lt;byte[], Byte, String&gt; getBytesStringZeroCopyCtr() {<br>  if (!STRING_VALUE_FIELD_IS_BYTES) {<br>    return null;<br>  }<br>  MethodHandle handle = getJavaStringZeroCopyCtrHandle();<br>  if (handle == null) {<br>    return null;<br>  }<br>  // Faster than handle.invokeExact(data, byte)<br>  try {<br>    MethodType instantiatedMethodType =<br>        MethodType.methodType(handle.type().returnType(), new Class[] {byte[].class, Byte.class});<br>    CallSite callSite =<br>        LambdaMetafactory.metafactory(<br>            STRING_LOOK_UP,<br>            &quot;apply&quot;,<br>            MethodType.methodType(BiFunction.class),<br>            handle.type().generic(),<br>            handle,<br>            instantiatedMethodType);<br>    return (BiFunction) callSite.getTarget().invokeExact();<br>  } catch (Throwable e) {<br>    return null;<br>  }<br>}</pre><p>The method returns a BiFunction that receives byte[] value, byte coder as arguments. The function invokes a MethodHandle<br>for the String constructor new String(byte[] value, byte coder) via CallSite using LambdaMetafactory.metafactory(). It seems faster than directly calling MethodHandle.invokeExact(). I guess that&#39;s because of skipping bootstrap process by reusing the CallSite.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*knjOLWVBntbkbEC1.png" /></figure><p><a href="https://cr.openjdk.org/%7Entv/talks/eclipseSummit16/indyunderTheHood.pdf">https://cr.openjdk.org/~ntv/talks/eclipseSummit16/indyunderTheHood.pdf</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/0*au_0bk7SJhbyOJbw.png" /></figure><p><a href="https://cr.openjdk.org/%7Entv/talks/eclipseSummit16/indyunderTheHood.pdf">https://cr.openjdk.org/~ntv/talks/eclipseSummit16/indyunderTheHood.pdf</a></p><p>LATIN_BYTES_STRING_ZERO_COPY_CTR is initialized to a Function returned from <a href="https://github.com/apache/fury/blob/3865dcd0982c0ca9de04a8ba9635892b76288769/java/fury-core/src/main/java/org/apache/fury/serializer/StringSerializer.java#L707-L726">getLatinBytesStringZeroCopyCtr()</a>:</p><pre>private static Function&lt;byte[], String&gt; getLatinBytesStringZeroCopyCtr() {<br>  if (!STRING_VALUE_FIELD_IS_BYTES) {<br>    return null;<br>  }<br>  if (STRING_LOOK_UP == null) {<br>    return null;<br>  }<br>  try {<br>    Class&lt;?&gt; clazz = Class.forName(&quot;java.lang.StringCoding&quot;);<br>    MethodHandles.Lookup caller = STRING_LOOK_UP.in(clazz);<br>    // JDK17 removed this method.<br>    MethodHandle handle =<br>        caller.findStatic(<br>            clazz, &quot;newStringLatin1&quot;, MethodType.methodType(String.class, byte[].class));<br>    // Faster than handle.invokeExact(data, byte)<br>    return _JDKAccess.makeFunction(caller, handle, Function.class);<br>  } catch (Throwable e) {<br>    return null;<br>  }<br>}</pre><p>The method returns a Function that receives byte[] (coder isn&#39;t needed since it&#39;s only for LATIN1) as arguments like getBytesStringZeroCopyCtr(). This Function invokes a MethodHandle<br>for <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/StringCoding.java#L982-L986">StringCoding.newStringLatin1(byte[] src)</a> instead of the String constructor new String(byte[] value, byte coder). _JDKAccess.makeFunction() wraps the invocation of a MethodHandle with LambdaMetafactory.metafactory() as well as in getBytesStringZeroCopyCtr().</p><p>StringCoding.newStringLatin1() is removed at Java 17. So, BYTES_STRING_ZERO_COPY_CTR function is used in Java 17 or later, while LATIN_BYTES_STRING_ZERO_COPY_CTR function is used otherwise.</p><p>The points are:</p><ul><li>Call non-public <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/StringCoding.java#L982-L986">StringCoding.newStringLatin1()</a> or <a href="https://github.com/openjdk/jdk11u/blob/f12fc3feee66def311164b81d9d0170b550d22ad/src/java.base/share/classes/java/lang/String.java#L3249-L3255">new String(byte[] value, byte coder)</a> to avoid byte array copy</li><li>Minimize the cost of MethodHandle invocation via CallSite as much as possible.</li></ul><p>It’s time for the benchmark. I updated the JMH benchmark code as follows:</p><ul><li>build.gradle.kts</li></ul><pre>dependencies {<br>    implementation(&quot;org.apache.fury:fury-core:0.9.0&quot;)<br>    ...</pre><ul><li>org/komamitsu/stringinstantiationbench/StringInstantiationBenchmark.java</li></ul><pre>package org.komamitsu.stringinstantiationbench;<br><br>import org.apache.fury.serializer.StringSerializer;<br>import org.openjdk.jmh.annotations.*;<br>import java.nio.charset.StandardCharsets;<br>import java.util.concurrent.TimeUnit;<br>@State(Scope.Benchmark)<br>@OutputTimeUnit(TimeUnit.MILLISECONDS)<br>@Fork(1)<br>@Measurement(time = 3, iterations = 4)<br>@Warmup(iterations = 2)<br>public class StringInstantiationBenchmark {<br>  private static final int STR_LEN = 512;<br>  private static final byte[] SINGLE_BYTE_STR_SOURCE_BYTES;<br>  private static final byte[] MULTI_BYTE_STR_SOURCE_BYTES;<br>  static {<br>    {<br>      StringBuilder sb = new StringBuilder();<br>      for (int i = 0; i &lt; STR_LEN; i++) {<br>        sb.append(&quot;x&quot;);<br>      }<br>      SINGLE_BYTE_STR_SOURCE_BYTES = sb.toString().getBytes(StandardCharsets.UTF_8);<br>    }<br>    {<br>      StringBuilder sb = new StringBuilder();<br>      for (int i = 0; i &lt; STR_LEN / 2; i++) {<br>        sb.append(&quot;あ&quot;);<br>      }<br>      MULTI_BYTE_STR_SOURCE_BYTES = sb.toString().getBytes(StandardCharsets.UTF_8);<br>    }<br>  }<br><br>  @Benchmark<br>  public void newStrFromSingleByteStrBytes() {<br>    new String(SINGLE_BYTE_STR_SOURCE_BYTES, StandardCharsets.UTF_8);<br>  }<br><br>  @Benchmark<br>  public void newStrFromMultiByteStrBytes() {<br>    new String(MULTI_BYTE_STR_SOURCE_BYTES, StandardCharsets.UTF_8);<br>  }<br><br>  // Copied from org.apache.fury.serializer.StringSerializer.<br>  private static final byte LATIN1 = 0;<br>  private static final Byte LATIN1_BOXED = LATIN1;<br>  private static final byte UTF16 = 1;<br>  private static final Byte UTF16_BOXED = UTF16;<br>  private static final byte UTF8 = 2;<br><br>  @Benchmark<br>  public void newStrFromSingleByteStrBytesWithZeroCopy() {<br>    StringSerializer.newBytesStringZeroCopy(LATIN1, SINGLE_BYTE_STR_SOURCE_BYTES);<br>  }<br><br>  @Benchmark<br>  public void newStrFromMultiByteStrBytesWithZeroCopy() {<br>    StringSerializer.newBytesStringZeroCopy(UTF8, MULTI_BYTE_STR_SOURCE_BYTES);<br>  }<br>}</pre><p>And the result is as follows:</p><ul><li>Java 11</li></ul><pre>Benchmark                                  Mode  Cnt        Score      Error   Units<br>newStrFromMultiByteStrBytes               thrpt    4     1505.580 ±   13.191  ops/ms<br>newStrFromMultiByteStrBytesWithZeroCopy   thrpt    4  2284141.488 ± 5509.077  ops/ms<br>newStrFromSingleByteStrBytes              thrpt    4    15246.342 ±  258.381  ops/ms<br>newStrFromSingleByteStrBytesWithZeroCopy  thrpt    4  2281817.367 ± 8054.568  ops/ms</pre><ul><li>Java 17</li></ul><pre>Benchmark                                  Mode  Cnt        Score       Error   Units<br>newStrFromMultiByteStrBytes               thrpt    4     1545.503 ±    15.283  ops/ms<br>newStrFromMultiByteStrBytesWithZeroCopy   thrpt    4  2273566.173 ± 10212.794  ops/ms<br>newStrFromSingleByteStrBytes              thrpt    4    17598.209 ±   253.282  ops/ms<br>newStrFromSingleByteStrBytesWithZeroCopy  thrpt    4  2277213.103 ± 13380.823  ops/ms</pre><ul><li>Java 21</li></ul><pre>Benchmark                                  Mode  Cnt        Score        Error   Units<br>newStrFromMultiByteStrBytes               thrpt    4     1556.272 ±     16.482  ops/ms<br>newStrFromMultiByteStrBytesWithZeroCopy   thrpt    4  3698101.264 ± 429945.546  ops/ms<br>newStrFromSingleByteStrBytes              thrpt    4    17803.149 ±    204.987  ops/ms<br>newStrFromSingleByteStrBytesWithZeroCopy  thrpt    4  3817357.204 ±  89376.224  ops/ms</pre><p>The benchmark code failed with Java 8 due to NPE. Maybe I used the method in a wrong way.</p><p>The performance of StringSerializer.newBytesStringZeroCopy() was more than 100 times faster in Java 17 and more than 200 times faster in Java 21 than the normal new String(byte[] bytes, Charset charset). Maybe this is one on the secrets of why Fury is blazing-fast.</p><p>A possible concern of using such a zero-copy strategy and implementation is that the byte array passed to new String(byte[] value, byte coder) could be owned by multiple objects; the new String object and objects having reference to the byte array.</p><pre>  byte[] bytes = &quot;Hello&quot;.getBytes();<br>  String s = StringSerializer.newBytesStringZeroCopy(LATIN1, bytes);<br>  System.out.println(s);    // &gt;&gt;&gt; Hello<br>  bytes[4] = &#39;!&#39;;<br>  System.out.println(s);    // &gt;&gt;&gt; Hell!</pre><p>This mutability could cause an issue that a string content is unexpectedly changed.</p><h3>Conclusion</h3><ul><li>Use Java 9 or later as much as possible if you’re using Java 8, in terms of the performance of String instantiation.</li><li>There is a technique to instantiate a String from a byte array with zero copy. It’s blazing-fast.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15940a62105e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Let’s easily inject random delay to your Java application]]></title>
            <link>https://komamitsu.medium.com/lets-easily-inject-random-delay-to-your-java-application-9a31a6d53895?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a31a6d53895</guid>
            <category><![CDATA[chaos-testing]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[java-agent]]></category>
            <category><![CDATA[chaos-engineering]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Sat, 30 Mar 2024 13:23:53 GMT</pubDate>
            <atom:updated>2024-03-31T09:48:53.765Z</atom:updated>
            <content:encoded><![CDATA[<p>Finding race condition issues in tests can be challenging, as they occur only when an operation is unexpectedly executed between other operations. Such race condition issues could happen in production environment despite passing tests many times.</p><p>Consider a scenario where we’re implementing code to transfer money between accounts as follows.</p><pre>package org.komamitsu.database;<br><br>public class Account {<br>    public int balance;<br><br>    public Account(int balance) {<br>        this.balance = balance;<br>    }<br><br>    public int getBalance() {<br>        return balance;<br>    }<br><br>    public void setBalance(int balance) {<br>        this.balance = balance;<br>    }<br>}</pre><pre>package org.komamitsu.database;<br><br>public class Transfer {<br>    public void exec(Account from, Account to, int amount) {<br>        int balanceOfFrom = from.getBalance();<br>        from.setBalance(balanceOfFrom - amount);<br><br>        int balanceOfTo = to.getBalance();<br>        to.setBalance(balanceOfTo + amount);<br>    }<br>}</pre><pre>package org.komamitsu;<br><br>import org.komamitsu.database.Account;<br>import org.komamitsu.database.Transfer;<br><br>import java.util.ArrayList;<br>import java.util.List;<br>import java.util.concurrent.*;<br><br>public class RaceConditionable {<br>    void transfer() throws ExecutionException, InterruptedException {<br>        Account a = new Account(2000);<br>        Account b = new Account(2000);<br>        Transfer transfer = new Transfer();<br>        ExecutorService executorService = Executors.newFixedThreadPool(2);<br>        List&lt;Future&lt;?&gt;&gt; futures = new ArrayList&lt;&gt;(2);<br><br>        // 2 concurrent threads execute the transfer.<br>        futures.add(executorService.submit(() -&gt; transfer.exec(a, b, 100)));<br>        futures.add(executorService.submit(() -&gt; transfer.exec(a, b, 100)));<br>        executorService.shutdown();<br><br>        for (Future&lt;?&gt; future : futures) {<br>            future.get();<br>        }<br><br>        int balanceOfA = a.balance;<br>        int expectedBalanceOfA = 2000 - 2 * 100;<br>        if (balanceOfA != expectedBalanceOfA) {<br>            throw new RuntimeException(<br>                    String.format(&quot;Unexpected balance of `a`. Expected: %d, Actual: %d&quot;,<br>                            expectedBalanceOfA, balanceOfA));<br>        }<br><br>        int balanceOfB = b.balance;<br>        int expectedBalanceOfB = 2000 + 2 * 100;<br>        if (balanceOfB != expectedBalanceOfB) {<br>            throw new RuntimeException(<br>                    String.format(&quot;Unexpected balance of `b`. Expected: %d, Actual: %d&quot;,<br>                            expectedBalanceOfB, balanceOfB));<br>        }<br>    }<br><br>    public static void main(String[] args) throws ExecutionException, InterruptedException {<br>        RaceConditionable raceConditionable = new RaceConditionable();<br>        for (int i = 0; i &lt; 1000; i++) {<br>            raceConditionable.transfer();<br>        }<br>    }<br>}</pre><p>In this implementation, this application does the following operations:</p><ol><li>Instantiates two accounts, account <strong>a</strong> and <strong>b</strong>,<strong> </strong>each with an initial balance of 2000</li><li><strong>Concurrently</strong> transfers 100 from account <strong>a</strong> to account <strong>b </strong>with two threads</li><li>Verifies the balances of both accounts <strong>a</strong> and <strong>b</strong>. Account <strong>a</strong> should have 1800 and account <strong>b</strong> should have 2200</li><li>Repeats these steps 1000 times</li></ol><p>As you may notice, the implementation of <strong>Transfer</strong> class has a race condition issue (“lost update” anomaly). However, reproducing this issue in tests can be challenging due to the rapid execution of each line of code under light workload, minimizing the chances of thread interleaving. Even executing the above code multiple times may not reliably reproduce the race condition issue.</p><p>To address this, I developed a small Java agent library called Chaos Dukey (<a href="https://github.com/komamitsu/chaos-dukey">https://github.com/komamitsu/chaos-dukey</a>), designed to inject random delays into Java application. It internally uses <a href="https://github.com/raphw/byte-buddy">Byte Buddy</a>. Here’s how you can use it:</p><ol><li>Download the released JAR file (<strong>chaos-dukey-x.x.x-all.jar</strong>) from <a href="https://github.com/komamitsu/chaos-dukey/releases">https://github.com/komamitsu/chaos-dukey/releases</a></li><li>Pass <strong>-javaagent</strong> option pointing the downloaded JAR file with some parameters to inject delays into the target classes and methods</li><li>Execute the Java application with the <strong>javaagent</strong> option. You’ll observe random delays injected to your Java application.</li></ol><p>For example, if you execute the following code (class Foo and Bar are empty) without the Chaos Dukey Java agent library, you’ll observe no delays of course.</p><pre>    Foo foo = new Foo();<br>    for (int i = 0; i &lt; 4; i++) {<br>        long start = System.currentTimeMillis();<br>        int hashcode = foo.hashCode();<br>        System.out.printf(&quot;Foo: DURATION=%d, HASHCODE=%d\n&quot;, (System.currentTimeMillis() - start), hashcode);<br>    }<br>    <br>    Bar bar  = new Bar();<br>    for (int i = 0; i &lt; 4; i++) {<br>        long start = System.currentTimeMillis();<br>        int hashcode = bar.hashCode();<br>        System.out.printf(&quot;Bar: DURATION=%d, HASHCODE=%d\n&quot;, (System.currentTimeMillis() - start), hashcode);<br>    }</pre><pre>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Bar: DURATION=0, HASHCODE=120694604<br>Bar: DURATION=0, HASHCODE=120694604<br>Bar: DURATION=0, HASHCODE=120694604<br>Bar: DURATION=0, HASHCODE=120694604</pre><p>However, with passing <strong>-javaagent</strong> option with proper parameters like <strong>“-javaagent:/path/to/chaos-dukey-x.x.x-all.jar=typeNamePattern=^foo\..*\.Bar$,methodNamePattern=hashCode,waitMode=RANDOM,percentage=50,maxDelayMillis=1000”</strong>, random delays will be injected to the Java application.</p><pre>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Foo: DURATION=0, HASHCODE=1262822392<br>Bar: DURATION=148, HASHCODE=489349054<br>Bar: DURATION=729, HASHCODE=489349054<br>Bar: DURATION=0, HASHCODE=489349054<br>Bar: DURATION=689, HASHCODE=489349054</pre><p>Returning to the first example application, injecting random delays into <strong>Transfer.getBalance()</strong> and <strong>Transfer.setBalance()</strong> can help to reproduce the race conditoin issue. For instance, using the following option <strong>“-javaagent:/path/to/chaos-dukey-x.x.x-all.jar=typeNamePattern=^org\.komamitsu\.database\.Account$,methodNamePattern=.*Balance$,waitMode=RANDOM,percentage=10,maxDelayMillis=200”</strong>, the race condition issue would be easily reproduced.</p><pre>Exception in thread &quot;main&quot; java.lang.RuntimeException: Unexpected balance of `a`. Expected: 1800, Actual: 1900<br> at org.komamitsu.RaceConditionable.transfer(RaceConditionable.java:38)<br> at org.komamitsu.RaceConditionable.main(RaceConditionable.java:14)</pre><p>Although there are some other ways to do similar thing (e.g., write a Byteman script and execute a Java application with Byteman JAR file and the script). But I think <strong>chaos-dukey</strong> is simpler since it focuses on the target use case. I encourage you to try it out and share your feedback!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a31a6d53895" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Micro-benchmarks of some concurrency features in Java 8, 11, 17 and 21]]></title>
            <link>https://komamitsu.medium.com/micro-benchmarks-of-some-concurrency-features-in-java-8-11-17-and-21-9e4bec73bc49?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e4bec73bc49</guid>
            <category><![CDATA[benchmark]]></category>
            <category><![CDATA[concurrency-programming]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[microbenchmark]]></category>
            <category><![CDATA[concurrency-control]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Fri, 23 Feb 2024 12:20:47 GMT</pubDate>
            <atom:updated>2024-02-23T12:20:47.970Z</atom:updated>
            <content:encoded><![CDATA[<p>There are various concurrency features in Java. I sometimes observed a lock class had a good performance in a Java version while it had a poor performance in another Java version when I did micro-benchmarks. So, I wanted to know which concurrency control had a better performance in each Java version.</p><p>I created <a href="https://github.com/komamitsu/java-lock-bench">https://github.com/komamitsu/java-lock-bench</a> to satisfy my curiosity. It contains these 2 types of micro-benchmarks:</p><p><strong>Counter</strong></p><ul><li>100%: increment (read-modify-write) operation</li><li>I usually used <strong>java.util.concurrent.atomic.AtomicLong</strong> for this kind of operation. But it’s a really good choice?</li></ul><p><strong>Map</strong></p><ul><li>20%: Read-modify-write operation</li><li>80%: Read-only operation</li><li>I usually used <strong>java.util.concurrent.ConcurrentHashMap</strong> for this kind of operation. But it’s a really good choice?</li></ul><p>See <a href="https://github.com/komamitsu/java-lock-bench?tab=readme-ov-file#what-types-of-benchmarks">https://github.com/komamitsu/java-lock-bench?tab=readme-ov-file#what-types-of-benchmarks</a> for details</p><p>I executed those benchmarks on Linux instances of GitHub Actions using Java 8, 11, 17 and 21. The results are as follows:</p><p><strong>Counter</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EY22WnOSpwlzXhzG_kmWwQ.png" /></figure><ul><li><strong>java.util.concurrent.locks.StampedLock</strong> maintained strong performance across all the Java versions</li><li>synchronized keyword (intrinsic lock) generally exhibited a poor performance, except in Java 17</li></ul><p><strong>Map</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GZ9pnI4qisvysHtS0gBxdQ.png" /></figure><ul><li><strong>java.util.concurrent.locks.StampedLock</strong> maintained good performance across all the Java versions</li><li><strong>java.util.concurrent.locks.ReentrantReadWriteLock</strong> basically showed a poor performance in all the Java versions <strong>java.util.concurrent.locks.ReentrantLock</strong> was rather better than it</li><li><strong>java.util.concurrent.ConcurrentHashMap</strong> didn’t perform so well</li><li>The errors of the benchmarks for synchronized keyword (intrinsic lock) were basically large. But maybe the performance was good.</li></ul><p>This post doesn’t look into the reasons behind the performance differences. But I’ll investigate them when I have time later.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e4bec73bc49" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to reduce the state space when traversing a graph written in TLA+ using TLC]]></title>
            <link>https://komamitsu.medium.com/how-to-reduce-the-state-space-when-traversing-a-graph-written-in-tla-using-tlc-59a2819ac8bf?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/59a2819ac8bf</guid>
            <category><![CDATA[model-checking]]></category>
            <category><![CDATA[tla-plus]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Sun, 28 Jan 2024 06:38:46 GMT</pubDate>
            <atom:updated>2024-01-28T06:38:46.777Z</atom:updated>
            <content:encoded><![CDATA[<p>TLC is a very powerful tool to exhaustively check a specification written in TLA+. But, you may notice the state space to explore is easily extended and it takes a long time. Of course, the key is to reduce the number of state.</p><p>Probably you’re familiar with PlusCal, which is translated into TLA+ code and is also very useful when writing sequential imperative code, loop and so on. But using loop would increase the number of the states since it’s converted a goto jump to the label on the loop.</p><p>When I wrote a PlusCal code to traverse a graph structure represented as a record, the number of states was many. So, I tried to replace it with a recursion.</p><p>The original code using a loop is as follows:</p><pre>EXTENDS Integers, Sequences, TLC<br><br>(*--algorithm TestWithLoop<br><br>variables<br>  counter = 0,<br>  state = &quot;(init)&quot;<br><br>process Alpha = &quot;alpha&quot;<br>begin Start:<br>  print(&lt;&lt;counter, &quot;Alpha:start&quot;, &quot;state=&quot;, state&gt;&gt;);<br>  state := &quot;zero&quot;;<br>  counter := counter + 1;<br>  print(&lt;&lt;counter, &quot;Alpha:end&quot;, &quot;state=&quot;, state&gt;&gt;);<br>end process;<br><br>process Beta = &quot;beta&quot;<br>variables<br>  target = &quot;one&quot;,<br>  map = [one |-&gt; &quot;two&quot;, two |-&gt; &quot;three&quot;]<br>begin<br>  Loop: while target \in DOMAIN map do<br>    print(&lt;&lt;counter, &quot;Beta:start&quot;, &quot;state=&quot;, state&gt;&gt;);<br>    target := map[target];<br>    state := target;<br>    counter := counter + 1;<br>    print(&lt;&lt;counter, &quot;Beta:end&quot;, &quot;state=&quot;, state&gt;&gt;);<br>  end while;<br>end process;<br><br>end algorithm;*)</pre><p>There are 2 processes; Alpha and Beta. Alpha does nothing special and Beta traverses a graph map using while loop. Alpha should be invoked at an arbitrary timing.</p><p>The Loop label is converted into the following TLA+ code:</p><pre>Loop == /\ pc[&quot;beta&quot;] = &quot;Loop&quot;<br>        /\ IF target \in DOMAIN map<br>              THEN /\ PrintT((&lt;&lt;counter, &quot;Beta:start&quot;, &quot;state=&quot;, state&gt;&gt;))<br>                   /\ target&#39; = map[target]<br>                   /\ state&#39; = target&#39;<br>                   /\ counter&#39; = counter + 1<br>                   /\ PrintT((&lt;&lt;counter&#39;, &quot;Beta:end&quot;, &quot;state=&quot;, state&#39;&gt;&gt;))<br>                   /\ pc&#39; = [pc EXCEPT ![&quot;beta&quot;] = &quot;Loop&quot;]<br>              ELSE /\ pc&#39; = [pc EXCEPT ![&quot;beta&quot;] = &quot;Done&quot;]<br>                   /\ UNCHANGED &lt;&lt; counter, state, target &gt;&gt;<br>        /\ map&#39; = map</pre><p>The Loop operator updates the global variables pc and map in each evaluation.</p><p>The output of the TLC check is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vHbQSEwY0T8rzoNxNgA-Sw.png" /></figure><p>The numbers of the found states and the distinct are 19 and 11, respectively. The model check is executed in BFS, so the actual executions should be like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kudYozzeUMH4xVWAoZE0EA.png" /></figure><p>As you see, the execution timing of Alpha is arbitrary and the generated states are diverged.</p><p>To reduce the state space, the loop can be replaced with a recursion as follows (sorry, I didn’t put the debug print in Traverse operator as it was a bit tough):</p><pre>EXTENDS Integers, Sequences, TLC<br><br>Traverse(xs, target) == LET<br>        RECURSIVE Traverse_(_, _)<br>        Traverse_(xs_, target_) ==<br>            IF target_ \in DOMAIN xs_ THEN<br>                LET<br>                    v == xs_[target_]<br>                    new_keys == DOMAIN xs_ \ {target_}<br>                IN<br>                    Traverse_([ x \in new_keys |-&gt; xs_[x] ], v)<br>            ELSE<br>                target_<br>    IN Traverse_(xs, target)<br><br>(*--algorithm TestWithRecursion<br><br>variables<br>  counter = 0,<br>  state = &quot;(init)&quot;<br><br>process Alpha = &quot;alpha&quot;<br>begin Start:<br>  print(&lt;&lt;counter, &quot;Alpha:start&quot;, &quot;state=&quot;, state&gt;&gt;);<br>  state := &quot;zero&quot;;<br>  counter := counter + 1;<br>  print(&lt;&lt;counter, &quot;Alpha:end&quot;, &quot;state=&quot;, state&gt;&gt;);<br>end process;<br><br>process Beta = &quot;beta&quot;<br>variables<br>  map = [one |-&gt; &quot;two&quot;, two |-&gt; &quot;three&quot;]<br>begin Start:<br>  print(&lt;&lt;counter, &quot;Beta:start&quot;, &quot;state=&quot;, state&gt;&gt;);<br>  state := Traverse(map, &quot;one&quot;);<br>  counter := counter + 1;<br>  print(&lt;&lt;counter, &quot;Beta:end&quot;, &quot;state=&quot;, state&gt;&gt;);<br>end process;<br><br>end algorithm;*)</pre><p>The point is to use RECURSIVE directive. I used it inside the public operator to avoid an error “A VARIABLE declaration may not appear within a recursive definition section”. I’m not sure what was the actual cause of the error…</p><p>The output of the TLC check is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aP6gk_8D9Ap3l4wYxo4RbQ.png" /></figure><p>The numbers of the found states and the distinct are only 7 and 5 !!!!, respectively. As I wrote above, I didn’t put the debug print in Traverse operator and the number of outputs is very smaller than the original code using loop. But I guess the actual executions should be like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VE0j9JEB5E19no4nxGp0Lw.png" /></figure><p>With the recursive operator, the possible execution orders are only Alpha→Beta or Beta→Alpha resulting in the fewer states.</p><p>This optimization might be common, but I didn’t find that. So, I’ll record it here. I might be missing something and any feedback is welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59a2819ac8bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spring Data JDBC for ScalarDB]]></title>
            <link>https://medium.com/scalar-engineering-ja/spring-data-jdbc-for-scalardb-aab5404e227f?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/aab5404e227f</guid>
            <category><![CDATA[jdbc]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[transactions]]></category>
            <category><![CDATA[product]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Tue, 18 Jul 2023 05:50:06 GMT</pubDate>
            <atom:updated>2023-07-18T09:46:45.703Z</atom:updated>
            <content:encoded><![CDATA[<p>Spring Data JDBC for ScalarDBはScalarDB 3.8で最初にリリースされ、ScalarDB 3.9で幾つかの改善と新機能がリリースされました。本記事では、この機能が開発された背景や利用方法等について紹介します。</p><h3>Spring Data JDBC for ScalarDBが生まれた背景</h3><p><a href="https://medium.com/scalar-engineering-ja/scalar-db-universal-transaction-manager-f2eb06a6e566">ScalarDB</a>は、複数の様々なデータベースやマイクロサービスをまたいだ分散ACIDトランザクションを実現するための強力なツールです。しかし、直接ScalarDBを用いる場合、トランザクション用のAPI (例：<strong>rollback()</strong> や <strong>commit()</strong>) を「いつ・どのように」使うかを考慮しながら、多くのコードを書かなければいけないという問題がありました。この問題を解決すべく、より少ないコード量でより安全なScalarDBのアプリケーション開発を可能にするためSpring Data JDBC for ScalarDBの開発を始めました。</p><p>我々は「ScalarDBの大部分のユーザーはJavaや他のJVM言語を使って開発している」と想定しています。Spring Frameworkは、Java系言語による開発において最も人気のあるアプリケーション開発基盤の一つです。Spring Framework内で永続化層として利用できるSpring Data JDBC for ScalarDBを開発し、ScalarDBユーザーに使ってもらうことで、ScalarDBを用いたアプリケーション開発をより簡単に安全にできると考えています。</p><p>Springアプリケーション開発における永続化のインターフェースとしてはJDBC、Spring JDBC、Spring Data JDBC、Spring Data JPAといった選択肢があります。学習曲線や開発の効率化について考慮した上で、我々はSpring Data JDBCを選択しました。より強力で洗練されたSpring Data JPAも検討しましたが、機能の豊富さからくる複雑性や急勾配な学習曲線を懸念しました。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*MyXZ8JUKPbjHUrbP" /></figure><p><a href="https://hackmd.io/@ddubson/rkn-sR4wU">https://hackmd.io/@ddubson/rkn-sR4wU</a></p><h3>全体の構成</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1Lo0keqKberuhONx" /></figure><p>ScalarDBにおいては、データベースに直接アクセスするScalarDB Core (CRUD API) に加え、SQLを用いて開発者がScalarDB管理下のデータベースを操作できるScalarDB SQLも開発・提供されています。Spring Data JDBCもまたSQLを通してデータベースを操作する機能をもっており、SQLを経由することで、互いの連携がしやすそうです。しかし、Spring Data JDBCがデータベースに期待している文法や挙動と、実際にScalarDB SQLがどのように動作するか、の間には違いがありました。そのため、Spring Data JDBC for ScalarDB開発の主な目標の一つはこの相違を埋め、Spring Data JDBCとScalarDB SQLの機能を連携させることでした。</p><p>この連携により、開発者はSpring Data JDBC APIを呼び出し、ScalarDB層を通してRDBMSやNoSQLデータベースを操作できるようになります。</p><h3>Spring Data JDBC for ScalarDBの使い方</h3><p>Spring Data JDBC for ScalarDBの使用方法は、基本的に<a href="https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/">Spring Data JDBC</a>と同様の用法・文法となります。従って、Spring Data JDBCと同様の設定、使用方法でSpring Data JDBC for ScalarDBを扱うことができます。</p><h4>単一データベースの操作</h4><p>SQLを用いてScalarDBとその下層のデータベースに接続するには、まず <strong>resources/application.properties</strong> 内で<a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/configurations/">JDBC接続URL</a>を設定します。</p><pre>spring.datasource.driver-class-name=com.scalar.db.sql.jdbc.SqlJdbcDriver<br>spring.datasource.url=jdbc:scalardb:\<br>?scalar.db.sql.connection_mode=direct\<br>&amp;scalar.db.contact_points=jdbc:mysql://localhost:3306/my_app_ns\<br>&amp;scalar.db.username=root\<br>&amp;scalar.db.password=mysql\<br>&amp;scalar.db.storage=jdbc\<br>&amp;scalar.db.consensus_commit.isolation_level=SERIALIZABLE</pre><p>続いてJavaのソースコード内で、Spring Data JDBC for ScalarDBを有効にするため<strong>@EnableScalarDbRepositories</strong> アノテーションを設定します。これはSpring Data JDBCにおける <strong>@EnableJdbcRepositories</strong> と同等のアノテーションです。</p><pre>@SpringBootApplication<br>@EnableScalarDbRepositories<br>public class MyApplication {<br>    @Autowired private AccountRepository accountRepository;<br><br>    ...<br>}</pre><p>次は、エンティティのモデルやレポジトリークラスです。Spring Data JDBCの場合はレポジトリークラスの継承元として <strong>CrudRepository</strong> (または<strong>PagingAndSortingRepository</strong>) を用いますが、Spring Data JDBC for ScalarDBでは代わりに <strong>ScalarDbRepository</strong> を継承します。</p><pre>@Table<br>public class Account {<br>  @Id public int id;<br>  public int balance;<br><br>  public Account(int id, int balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}</pre><pre>public interface AccountRepository<br>    extends ScalarDbRepository&lt;Account, Integer&gt; {<br><br>  default void transfer(int srcAccountId, int dstAccountId, int amount) {<br>    Account src = findById(srcAccountId).get();<br>    Account dst = findById(dstAccountId).get();<br>    update(new Account(src.id, src.balance - amount));<br>    update(new Account(dst.id, dst.balance + amount));<br>  }<br>}</pre><p>これらの設定や実装により、ACIDトランザクション内で整合性を保ちながら、指定した金額を２つの口座間で移動することができます。</p><pre>// Transfer 5000 from account:42&#39;s balance to account:24&#39;s balance<br>accountRepository.transfer(42, 24, 5000);</pre><p>口座間の転送処理トランザクションは、処理の成功時に自動的にcommitされ、失敗時にはrollbackされます。</p><p>もし、上記と同様の処理をScalarDB APIを用いて書く場合、以下のように、やや冗長な実装になりそうです。</p><pre>DistributedTransaction tx = transactionManager.start();<br><br> try { <br>   Account src = tx.get(Get.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, 42)).build())<br>     .map(result -&gt;<br>       new Account(<br>         result.getInt(KEY_ID),<br>         result.getInt(KEY_BALANCE))).get();<br><br>   Account dst = tx.get(Get.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, 24)).build())<br>     .map(result -&gt;<br>       new Account(<br>         result.getInt(KEY_ID),<br>         result.getInt(KEY_BALANCE))).get();<br><br>   tx.put(Put.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, src.id))<br>       .intValue(KEY_BALANCE, src.balance - 5000).build());<br><br>   tx.put(Put.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, dst.id))<br>       .intValue(KEY_BALANCE, dst.balance + 5000).build());<br><br>   tx.commit();<br> } catch (Throwable e) { <br>   tx.abort();<br>   throw e;<br> }</pre><p>以上の例により、Spring Data JDBC for ScalarDBがどのようにSpringアプリケーションの開発を容易にするかの一部をお伝えできたかと思います。</p><h4>Multi-storage Transactionsを用いた複数データベースの操作</h4><p>複数データベースの操作をACIDトランザクション内で行いたい場合、<strong>resources/application.properties</strong> を適切に設定することにより、透過的にScalarDBの<a href="https://scalardb.scalar-labs.com/docs/3.9/multi-storage-transactions/">Multi-storage Transaction feature in ScalarDB</a>を利用することができます。</p><pre>spring.datasource.driver-class-name=com.scalar.db.sql.jdbc.SqlJdbcDriver<br>spring.datasource.url=jdbc:scalardb:\<br>?scalar.db.sql.connection_mode=direct\<br>&amp;scalar.db.storage=multi-storage\<br>&amp;scalar.db.multi_storage.storages=north,south\<br>&amp;scalar.db.multi_storage.namespace_mapping=north:north,south:south&amp;scalar.db.multi_storage.default_storage=south\<br>&amp;scalar.db.multi_storage.storages.north.storage=jdbc\<br>&amp;scalar.db.multi_storage.storages.north.contact_points=jdbc:mysql://localhost:3306/\<br>&amp;scalar.db.multi_storage.storages.north.username=${north_username}\<br>&amp;scalar.db.multi_storage.storages.north.password=${north_password}\<br>&amp;scalar.db.multi_storage.storages.south.storage=jdbc\<br>&amp;scalar.db.multi_storage.storages.south.contact_points=jdbc:postgresql://localhost:5432/\<br>&amp;scalar.db.multi_storage.storages.south.username=${south_username}\<br>&amp;scalar.db.multi_storage.storages.south.password=${south_password}\<br>&amp;scalar.db.consensus_commit.isolation_level=SERIALIZABLE\<br>&amp;scalar.db.sql.default_namespace_name=my_app</pre><p>この設定例では、２つのデータベース（MySQLとPostgreSQL）を扱うようになっています。これらのデータベースはそれぞれ論理的なnamespace <strong>north</strong> と <strong>south</strong> に対応しており、Spring Data JDBC for ScalarDBから両方のnamespace内のテーブルに１トランザクションで操作することができます。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*55-JCpEQYzo4hMbVpc2maQ.png" /></figure><p>上記設定後は、データベースのテーブルに対応するモデルやレポジトリークラスを実装することにより複数データベース内のテーブルにアクセスすることができます。</p><pre>// `schema` field is used to specify the underlying database<br>@Table(schema = &quot;north&quot;, value = &quot;account&quot;)<br>public class NorthAccount {<br>  @Id public final int id;<br>  public final Integer balance;<br>  public NorthAccount(int id, Integer balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}<br><br>@Table(schema = &quot;south&quot;, value = &quot;account&quot;)<br>public class SouthAccount {<br>  @Id public final int id;<br>  public final Integer balance;<br>  public SouthAccount(int id, Integer balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}</pre><pre>@Transactional<br>@Repository<br>public interface NorthAccountRepository<br>    extends ScalarDbRepository&lt;NorthAccount, Integer&gt; {<br><br> default void transferToSouthAccount(<br>      SouthAccountRepository southAccountRepository,<br>      int fromId, int toId, int value) {<br>    NorthAccount fromEntity =<br>        findById(fromId).orElseThrow(<br>            () -&gt; new AssertionError(&quot;Not found: &quot; + fromId));<br>    SouthAccount toEntity =<br>        southAccountRepository<br>            .findById(toId)<br>            .orElseThrow(() -&gt; new AssertionError(&quot;Not found: &quot; + toId));<br>    update(new NorthAccount(fromEntity.id, fromEntity.balance - value));<br>    southAccountRepository.update(<br>      new SouthAccount(toEntity.id, toEntity.balance + value));<br>  }<br>}<br><br>@Transactional<br>@Repository<br>public interface SouthAccountRepository<br>    extends ScalarDbRepository&lt;SouthAccount, Integer&gt; {}</pre><pre>  @Autowired private NorthAccountRepository northAccountRepository;<br>  @Autowired private SouthAccountRepository southAccountRepository;<br><br>  ...<br><br>  northAccountRepository.transferToSouthAccount(<br>        southAccountRepository, 42, 24, 5000);</pre><p>この他にも、​​動作可能なサンプルアプリケーションがありますので、興味のある方はこちらもご覧ください。</p><ul><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-sample">Sample application of Spring Data JDBC for ScalarDB</a></li><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-multi-storage-transaction-sample">Sample application of Spring Data JDBC for ScalarDB with Multi-storage Transactions</a></li><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-microservice-transaction-sample">Sample application of Spring Data JDBC for ScalarDB with Microservice Transactions</a></li></ul><p>各機能の詳細については<a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/spring-data-guide/">Guide of Spring Data JDBC for ScalarDB</a>をご覧ください。</p><h3>Spring Data JDBCからの拡張機能</h3><p>Spring Data JDBC for ScalarDBは、Spring Data JDBCから引き継いでいる機能に加え、新たにいくつかの機能を追加しています。</p><h4>CrudRepository#save(T t) 利用の抑止、および ScalarDbRepository への insert(T t)・update(T t) の追加</h4><p>Spring Data JDBCでは、レコードの追加と更新はともに <strong>CrudRepository#save(T t)</strong> によって処理されていました。このAPIは、引数で渡された当該モデルのインスタンスの <strong>@Id</strong> 変数がnullかどうかでレコード追加・更新のいずれを行うかを判断していますが、これは利用しているデータベースのautoincrement column機能に依存することになります (<a href="https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/#jdbc.entity-persistence.id-generation">ID Generation</a>)。具体的には引数で渡されたインスタンスの <strong>@Id</strong> 変数がnullの場合レコードの追加を、null以外の場合は当該レコードの更新を行います。しかし、ScalarDBではこのautoincrement column機能をサポートしておらず、<strong>CrudRepository#save(T t)</strong> が期待通りに動作しません。Spring Data JDBCでは幾つか<a href="https://spring.io/blog/2021/09/09/spring-data-jdbc-how-to-use-custom-id-generation">回避策</a>を用意しているものの、これらの回避策は直感的ではなく開発者に追加実装を強いるものになっています。</p><p>そこで我々はSpring Data JDBCの <strong>CrudRepository</strong> を継承し以下の拡張を取り入れた <strong>ScalarDbRepository</strong> を追加しました</p><ul><li>Spring Data JDBC for ScalarDBでは正常に機能しない <strong>ScalarDbRepository#save(T t)</strong> の呼び出しを、明示的に例外を発生させることにより抑止</li><li>新規API <strong>insert(T t)</strong> ・ <strong>update(T t)</strong> を追加。両APIとも、nonnullな <strong>@Id</strong> を持つモデルのインスタンスを引数として受け取る</li></ul><p><strong>ScalarDbRepository</strong> の利用により、Spring Data JDBC for ScalarDBでも直感的にScalarDBを操作することが可能になります。</p><h4>Two-phase Commit Transactions向けAPIの追加</h4><p>ScalarDBはmicroservice向けに、複数アプリケーションを跨いでトランザクションを実行する<a href="https://scalardb.scalar-labs.com/docs/3.9/two-phase-commit-transactions/">Two-phase Commit (2PC) Transactions</a>機能をサポートしています。Spring Data JDBC for ScalarDBからもその機能が使えるよう、以下の機能を追加しています。</p><p><a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/spring-data-guide/#primitive-2pc-apis">Primitive 2PC transaction API</a></p><ul><li>ScalarDBの2PC用APIが呼び出せるよう対応した原始的なAPI</li><li>2PCトランザクション処理を細かく柔軟に制御できる一方、ユーザーはどのAPIをいつ呼び出すべきか考慮する必要がある</li></ul><p><a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/spring-data-guide/#high-level-2pc-api">High-level 2PC transaction API</a></p><ul><li>一般的なユースケースを対象にした、使い勝手の良い抽象度の高いAPI。開発者はいつcommitやrollbackを実施すれば良いのか、意識する必要が無い</li><li>coordinator service開発用API<br>- <strong>executeTwoPcTransaction</strong></li><li>participant service開発用API<br>- <strong>joinTransactionOnParticipant</strong><br>- <strong>resumeTransactionOnParticipant</strong><br>- <strong>prepareTransactionOnParticipant</strong><br>- <strong>validateTransactionOnParticipant</strong><br>- <strong>commitTransactionOnParticipant</strong><br>- <strong>rollbackTransactionOnParticipant</strong></li></ul><p>Spring Data JDBCに組み込まれている標準transaction manager（<strong>JdbcTransactionManager</strong>）は、<strong>CrudRepository</strong> APIの呼び出し完了時に必ずトランザクションをcommitします。しかし、この挙動は、commitタイミングを細かく制御する必要があるScalarDBの2PCトランザクションとは上手く動作しません。そのため、我々は <strong>JdbcTransactionManager</strong> を拡張した独自transaction managerを用意し、トランザクションのcommitタイミングを内部的に細かく制御できるようにしました。</p><p>仮に、ScalarDBユーザーが以下のようなトランザクション処理を、commit・rollbackの漏れなく実装したい場合、</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HjeDhhki2UeRz2B6onWqmg.png" /></figure><p>Spring Data JDBC for ScalarDBのHigh-level 2PC APIを以下のように用いることで、開発者は複雑なトランザクション制御を意識せずにビジネスロジックの実装に集中できます。</p><p><strong>Account Service (coordinator)</strong></p><pre>@Transactional(transactionManager = &quot;scalarDbSuspendableTransactionManager&quot;)<br>@Repository<br>public interface AccountRepository<br>  extends ScalarDbTwoPcRepository&lt;Account, Integer&gt; {<br>  // findBydId(), insert(), update(), executeTwoPcTransaction()<br>  // and some more APIs are prepared automatically.<br>}</pre><pre>@Autowired private Account accountRepository;<br><br>...<br><br>accountRepository.executeTwoPcTransaction(<br>  // Business logic (CRUD operations) for local and<br>  // remote participants in execution phase.<br>  txId -&gt; {<br>    // [local] Read the account&#39;s balance<br>    Optional&lt;Account&gt; stored = accountRepository.findById(account.id);<br>    if (!stored.isPresent()) {<br>      // Cancel the transaction if the account doesn&#39;t exist.<br>      // No need to retry.<br>      throw new ScalarDbNonTransientException(<br>          &quot;The local state doesn&#39;t meet the condition.<br>                Aborting this transaction&quot;);<br>    }<br>    // [remote] Start a transaction with the transaction ID,<br>    //          read the item information and decrement the count<br>    Optional&lt;Integer&gt; price = stockService.purchaseItem(<br>                                             txId, account.id, itemName);<br>    if (price.isPresent()) {<br>      int currentBalance = stored.get().balance - price.get();<br>      if (currentBalance &lt; 0) {<br>        // Cancel the transaction<br>        // if the global state doesn&#39;t meet the condition.<br>        // No need to retry.<br>        throw new ScalarDbNonTransientException(<br>            &quot;The state of local and remote participants doesn&#39;t meet<br>                the condition. Aborting this transaction&quot;);<br>      }<br>      // [local] Decrease the account&#39;s balance for the item<br>      accountRepository.update(new Account2Pc(account.id, currentBalance));<br>      return currentBalance;<br>    }<br>    // Cancel the transaction<br>    // if the global state doesn&#39;t meet the condition.<br>    // No need to retry.<br>    throw new ScalarDbTransientException(<br>      &quot;The remote state doesn&#39;t meet the condition.<br>          Aborting this transaction&quot;);<br>  },<br><br>  // Remote operations for Prepare/Validate/Commit/Rollback.<br>  // These transactional operations will be automatically executed.<br>  Arrays.asList(<br>        RemotePrepareCommitPhaseOperations.createSerializable(<br>          stockService::callPrepareTransaction,<br>          stockService::callValidateTransaction,<br>          stockService::callCommitTransaction,<br>          stockService::callRollbackTransaction)<br>);</pre><p><strong>Stock Service (participant)</strong></p><pre>@Transactional(transactionManager = &quot;scalarDbSuspendableTransactionManager&quot;)<br>@Repository<br>public interface StockRepository<br>  extends ScalarDbTwoPcRepository&lt;Stock, String&gt; {<br>  // joinTransactionOnParticipant(), commitTransactionOnParticipant()<br>  // and some more APIs are prepared automatically.<br>}</pre><pre>@RestController<br>public class StockController {<br>  @Autowired private StockRepository stockRepository;<br><br>  @PostMapping(&quot;/purchaseItem&quot;)<br>  public Optional&lt;Integer&gt; purchaseItem(<br>               @RequestParam(&quot;transactionId&quot;) String transactionId,<br>               @RequestParam(&quot;accountId&quot;) int accountId,<br>               @RequestParam(&quot;itemName&quot;) String itemName) {<br>    // Join the global transaction and execute the CRUD operations in it<br>    return stockRepository.joinTransactionOnParticipant(txId, () -&gt; {<br>        Optional&lt;Item&gt; item = stockRepository.findById(itemName);<br><br>        ...<br><br>        return Optional.of(item.price);<br>    });<br>  }<br><br>  @PostMapping(&quot;/prepareTransaction&quot;)<br>  public void prepareTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.prepareTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/validateTransaction&quot;)<br>  public void validateTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.validateTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/commitTransaction&quot;)<br>  public void commitTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.commitTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/rollbackTransaction&quot;)<br>  public void rollbackTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.rollbackTransactionOnParticipant(txId);<br>  }<br>}</pre><h3>まとめ</h3><p>本記事では、Spring Data JDBC for ScalarDBを紹介しました。我々はこの機能によりScalarDBを用いたSpringアプリケーションの開発が簡単に安全になると信じています。現時点で、Spring Data JDBC本体およびScalarDBとの連携由来のいくつかの<a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/spring-data-guide/#limitations">制限事項</a>がありますが、将来的にはそれらの制限事項を解決し、ScalarDBユーザーがより簡単にアプリケーションを開発できるようにしていきたいです。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aab5404e227f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalar-engineering-ja/spring-data-jdbc-for-scalardb-aab5404e227f">Spring Data JDBC for ScalarDB</a> was originally published in <a href="https://medium.com/scalar-engineering-ja">Scalar Engineering (JA)</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Spring Data JDBC for ScalarDB]]></title>
            <link>https://medium.com/scalar-engineering/introducing-spring-data-jdbc-for-scalardb-a1d1532fc636?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1d1532fc636</guid>
            <category><![CDATA[transactions]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[jdbc]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Tue, 18 Jul 2023 05:49:43 GMT</pubDate>
            <atom:updated>2023-07-18T09:48:14.198Z</atom:updated>
            <content:encoded><![CDATA[<p>Spring Data JDBC for ScalarDB was first released in ScalarDB 3.8 and some improvements and new features were added in ScalarDB 3.9. This blog post introduces why we provide this feature and describes some details about usage.</p><h3>Why we provide Spring Data JDBC for ScalarDB</h3><p>ScalarDB is a very powerful tool to achieve <a href="https://medium.com/scalar-engineering/scalar-db-universal-transaction-manager-715379e07f34">database-agnostic and multidatabase/service ACID transactions</a>. However, directly using the ScalarDB API can be challenging because you need to write a significant amount of code and consider how and when to call the API (e.g., <strong>rollback()</strong> and <strong>commit()</strong>) for transactions. To address this issue, to help users develop ScalarDB applications with a lot less code and more safety, we started developing Spring Data JDBC for ScalarDB.</p><p>We assume most ScalarDB users develop their applications in Java or other JVM languages. The Spring Framework is one of the most popular application frameworks when developing in Java and other JVM languages. With Spring Data JDBC for ScalarDB, we are committed to helping you develop your applications more easily and securely.</p><p>Some module options to integrate with the persistent layer of the Spring Framework include JDBC, Spring JDBC, <a href="https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/">Spring Data JDBC</a>, and Spring Data JPA. After considering the learning curves and efficiencies, we chose to take advantage of Spring Data JDBC. We also considered Spring Data JPA, which has many features, but we noticed some complexities and a steep learning curve when researching the module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*IJnL6GPpCufI9nv2" /></figure><p><a href="https://hackmd.io/@ddubson/rkn-sR4wU">https://hackmd.io/@ddubson/rkn-sR4wU</a></p><h3>Overall architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s7s6GvNqoI1RiE05" /></figure><p>In addition to ScalarDB Core (CRUD API), which directly communicates with underlying databases, we also offer ScalarDB SQL, which has a SQL interface, so that you can communicate with the ScalarDB layer via SQL. Spring Data JDBC can also communicate with databases via SQL, so it may seem that the functionalities can relatively easily collaborate through SQL. But we noticed there are gaps between what Spring Data JDBC expects on databases and how ScalarDB SQL actually works. Because of this, one of the main challenges when implementing Spring Data JDBC for ScalarDB was to fill the gaps and integrate both solutions seamlessly.</p><p>With this integration, you can call the Spring Data JDBC API to manipulate an underlying RDBMS and/or a NoSQL database through ScalarDB.</p><h3>How to use Spring Data JDBC for ScalarDB</h3><p>Using Spring Data JDBC for ScalarDB basically follows the same semantics and syntax as Spring Data JDBC, as described in <a href="https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/">Spring Data JDBC — Reference Documentation</a>. Because of this, you can configure and use Spring Data JDBC for ScalarDB in a similar way to Spring Data JDBC.</p><h4>Manipulating a database</h4><p>To connect ScalarDB and its underlying databases via SQL, you must configure a <a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/configurations/">JDBC connection URL</a> in <strong>resources/application.properties</strong>.</p><pre>spring.datasource.driver-class-name=com.scalar.db.sql.jdbc.SqlJdbcDriver<br>spring.datasource.url=jdbc:scalardb:\<br>?scalar.db.sql.connection_mode=direct\<br>&amp;scalar.db.contact_points=jdbc:mysql://localhost:3306/my_app_ns\<br>&amp;scalar.db.username=root\<br>&amp;scalar.db.password=mysql\<br>&amp;scalar.db.storage=jdbc\<br>&amp;scalar.db.consensus_commit.isolation_level=SERIALIZABLE</pre><p>In the Java source code, the <strong>@EnableScalarDbRepositories</strong> annotation must be configured to enable Spring Data JDBC for ScalarDB instead of <strong>@EnableJdbcRepositories</strong> for Spring Data JDBC.</p><pre>@SpringBootApplication<br>@EnableScalarDbRepositories<br>public class MyApplication {<br>    @Autowired private AccountRepository accountRepository;<br><br>    ...<br>}</pre><p>Then, you can write entity model classes and repository classes. When using Spring Data JDBC, you would use <strong>CrudRepository</strong> (or <strong>PagingAndSortingRepository</strong>.) However, in Spring Data JDBC for ScalarDB, you would use <strong>ScalarDbRepository</strong> instead.</p><pre>@Table<br>public class Account {<br>  @Id public int id;<br>  public int balance;<br><br>  public Account(int id, int balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}</pre><pre>public interface AccountRepository<br>    extends ScalarDbRepository&lt;Account, Integer&gt; {<br><br>  default void transfer(int srcAccountId, int dstAccountId, int amount) {<br>    Account src = findById(srcAccountId).get();<br>    Account dst = findById(dstAccountId).get();<br>    update(new Account(src.id, src.balance - amount));<br>    update(new Account(dst.id, dst.balance + amount));<br>  }<br>}</pre><p>By following these configurations and this implementation, you can write code to transfer a specific amount between the balances of two accounts within a transaction, guaranteeing ACID compliance with strict serializability.</p><pre>// Transfer 5000 from account:42&#39;s balance to account:24&#39;s balance<br>accountRepository.transfer(42, 24, 5000);</pre><p>A transaction for the transfer operation will be automatically committed if the transaction finishes successfully or will be rolled back if the transaction fails.</p><p>If you implement the same operation as above by using the ScalarDB API in a straightforward way, the implementation would look like the following:</p><pre> DistributedTransaction tx = transactionManager.start();<br><br> try { <br>   Account src = tx.get(Get.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, 42)).build())<br>     .map(result -&gt;<br>       new Account(<br>         result.getInt(KEY_ID),<br>         result.getInt(KEY_BALANCE))).get();<br><br>   Account dst = tx.get(Get.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, 24)).build())<br>     .map(result -&gt;<br>       new Account(<br>         result.getInt(KEY_ID),<br>         result.getInt(KEY_BALANCE))).get();<br><br>   tx.put(Put.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, src.id))<br>       .intValue(KEY_BALANCE, src.balance - 5000).build());<br><br>   tx.put(Put.newBuilder()<br>       .namespace(NAMESPACE)<br>       .table(ACCOUNT_TABLE)<br>       .partitionKey(Key.ofInt(KEY_ID, dst.id))<br>       .intValue(KEY_BALANCE, dst.balance + 5000).build());<br><br>   tx.commit();<br> } catch (Throwable e) { <br>   tx.abort();<br>   throw e;<br> }</pre><p>We hope this example shows how Spring Data JDBC for ScalarDB simplifies development of Spring Framework–based applications that use ScalarDB.</p><h4>Manipulating databases by leveraging the Multi-storage Transactions feature</h4><p>If you want to manipulate multiple underlying databases within a transaction while guaranteeing ACID compliance with strict serializability, you can seamlessly leverage the <a href="https://scalardb.scalar-labs.com/docs/latest/multi-storage-transactions/">Multi-storage Transactions feature in ScalarDB</a> by configuring <strong>resources/application.properties</strong> as follows:</p><pre>spring.datasource.driver-class-name=com.scalar.db.sql.jdbc.SqlJdbcDriver<br>spring.datasource.url=jdbc:scalardb:\<br>?scalar.db.sql.connection_mode=direct\<br>&amp;scalar.db.storage=multi-storage\<br>&amp;scalar.db.multi_storage.storages=north,south\<br>&amp;scalar.db.multi_storage.namespace_mapping=north:north,south:south&amp;scalar.db.multi_storage.default_storage=south\<br>&amp;scalar.db.multi_storage.storages.north.storage=jdbc\<br>&amp;scalar.db.multi_storage.storages.north.contact_points=jdbc:mysql://localhost:3306/\<br>&amp;scalar.db.multi_storage.storages.north.username=${north_username}\<br>&amp;scalar.db.multi_storage.storages.north.password=${north_password}\<br>&amp;scalar.db.multi_storage.storages.south.storage=jdbc\<br>&amp;scalar.db.multi_storage.storages.south.contact_points=jdbc:postgresql://localhost:5432/\<br>&amp;scalar.db.multi_storage.storages.south.username=${south_username}\<br>&amp;scalar.db.multi_storage.storages.south.password=${south_password}\<br>&amp;scalar.db.consensus_commit.isolation_level=SERIALIZABLE\<br>&amp;scalar.db.sql.default_namespace_name=my_app</pre><p>This example configuration is designed to manage two underlying databases: MySQL and PostgreSQL. These databases correspond to the logical namespaces <strong>north</strong> and <strong>south</strong> respectively. Through Spring Data JDBC for ScalarDB, you can seamlessly access tables in the both namespaces, within a transaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*55-JCpEQYzo4hMbVpc2maQ.png" /></figure><p>Then, you can access tables over the multiple underlying databases by implementing model and repository classes.</p><pre>// `schema` field is used to specify the underlying database<br>@Table(schema = &quot;north&quot;, value = &quot;account&quot;)<br>public class NorthAccount {<br>  @Id public final int id;<br>  public final Integer balance;<br>  public NorthAccount(int id, Integer balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}<br><br>@Table(schema = &quot;south&quot;, value = &quot;account&quot;)<br>public class SouthAccount {<br>  @Id public final int id;<br>  public final Integer balance;<br>  public SouthAccount(int id, Integer balance) {<br>    this.id = id;<br>    this.balance = balance;<br>  }<br>}</pre><pre>@Transactional<br>@Repository<br>public interface NorthAccountRepository<br>    extends ScalarDbRepository&lt;NorthAccount, Integer&gt; {<br><br> default void transferToSouthAccount(<br>      SouthAccountRepository southAccountRepository,<br>      int fromId, int toId, int value) {<br>    NorthAccount fromEntity =<br>        findById(fromId).orElseThrow(<br>            () -&gt; new AssertionError(&quot;Not found: &quot; + fromId));<br>    SouthAccount toEntity =<br>        southAccountRepository<br>            .findById(toId)<br>            .orElseThrow(() -&gt; new AssertionError(&quot;Not found: &quot; + toId));<br>    update(new NorthAccount(fromEntity.id, fromEntity.balance - value));<br>    southAccountRepository.update(<br>      new SouthAccount(toEntity.id, toEntity.balance + value));<br>  }<br>}<br><br>@Transactional<br>@Repository<br>public interface SouthAccountRepository<br>    extends ScalarDbRepository&lt;SouthAccount, Integer&gt; {}</pre><pre>  @Autowired private NorthAccountRepository northAccountRepository;<br>  @Autowired private SouthAccountRepository southAccountRepository;<br><br>  ...<br><br>  northAccountRepository.transferToSouthAccount(<br>        southAccountRepository, 42, 24, 5000);</pre><p>We have a few sample applications that utilize Spring Data JDBC for ScalarDB, which you can execute. If you’re interested, try the following ones:</p><ul><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-sample">Sample application of Spring Data JDBC for ScalarDB</a></li><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-multi-storage-transaction-sample">Sample application of Spring Data JDBC for ScalarDB with Multi-storage Transactions</a></li><li><a href="https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-microservice-transaction-sample">Sample application of Spring Data JDBC for ScalarDB with Microservice Transactions</a></li></ul><p>For more details, see <a href="https://scalardb.scalar-labs.com/docs/latest/scalardb-sql/spring-data-guide/">Guide of Spring Data JDBC for ScalarDB</a>.</p><h3>Extensions from Spring Data JDBC</h3><p>Spring Data JDBC for ScalarDB adds some extensions to the original Spring Data JDBC module.</p><h4>Prohibiting CrudRepository#save(T t), and adding new insert(T t) and update(T t) methods in ScalarDbRepository</h4><p>In Spring Data JDBC, both insertion and update records are handled with <strong>CrudRepository#save(T t)</strong>, which decides which operation to do out of insertion and update based on whether the <strong>@Id</strong> field of a model instance received as an argument is null. <strong>CrudRepository#save(T t)</strong> inserts a new record if <strong>@Id</strong> field is null or updates an existing record otherwise. This behavior relies on the autoincrement column feature (<a href="https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/#jdbc.entity-persistence.id-generation">ID Generation</a>) in RDBMS, but ScalarDB doesn’t support this feature. There are <a href="https://spring.io/blog/2021/09/09/spring-data-jdbc-how-to-use-custom-id-generation">some workarounds</a> in Spring Data JDBC to address this issue, but those workarounds are not intuitive and require you to implement additional code.</p><p>We’ve introduced <strong>ScalarDbRepository</strong>, which inherits <strong>CrudRepository</strong> of Spring Data JDBC, and made the following customizations:</p><ul><li>Explicitly prohibit calling <strong>ScalarDbRepository#save(T t)</strong>, which may be confusing, by throwing an exception.</li><li>Add new <strong>insert(T t)</strong> and <strong>update(T t)</strong> methods to <strong>ScalarDbRepository</strong>. Both methods receive a model instance that has a non-null <strong>@Id</strong> field.</li></ul><p>By using <strong>ScalarDbRepository</strong>, you can intuitively add and update records even with ScalarDB.</p><h4>API for two-phase commit transactions</h4><p>ScalarDB supports <a href="https://scalardb.scalar-labs.com/docs/latest/two-phase-commit-transactions/">two-phase commit (2PC) transactions</a> to execute a transaction that spans multiple processes or applications (e.g., microservices). Spring Data JDBC for ScalarDB also supports this feature with the following extensions:</p><p><a href="https://scalardb.scalar-labs.com/docs/latest/scalardb-sql/spring-data-guide/#primitive-2pc-apis">Primitive 2PC transaction API</a></p><ul><li>Corresponds to the 2PC API (e.g., <strong>prepare()</strong>, <strong>commit()</strong> and <strong>rollback()</strong>) in ScalarDB.</li><li>Supports explicitly controlling 2PC transaction operations in flexible and fine-grained ways. On the other hand, you will need to consider which API to call in a proper order when using the API.</li></ul><p><a href="https://scalardb.scalar-labs.com/docs/latest/scalardb-sql/spring-data-guide/#high-level-2pc-api">High-level 2PC transaction API</a></p><ul><li>Provides a user-friendly API to cover common use cases. With this API, you don’t need to think about how and when to commit or rollback a transaction.</li><li>For coordinator service, the following API is provided<br>- <strong>executeTwoPcTransaction</strong></li><li>For participant services, the following APIs are provided<br>- <strong>joinTransactionOnParticipant</strong><br>- <strong>resumeTransactionOnParticipant</strong><br>- <strong>prepareTransactionOnParticipant</strong><br>- <strong>validateTransactionOnParticipant</strong><br>- <strong>commitTransactionOnParticipant</strong><br>- <strong>rollbackTransactionOnParticipant</strong></li></ul><p>The default transaction manager for Spring Data JDBC always commits the transaction when quitting a <strong>CrudRepository</strong> API call. But the behavior doesn’t work well with 2PC transactions in ScalarDB because the feature needs to control commit timing. Because of this, we’ve introduced a custom transaction manager that inherits <strong>JdbcTransactionManager</strong> and doesn’t automatically commit transactions.</p><p>Let’s say you want to implement the following transactional operations, while ensuring rollback in case of failure, over a set of microservices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HjeDhhki2UeRz2B6onWqmg.png" /></figure><p>With the high-level API of Spring Data JDBC for ScalarDB, you can focus on the business logic by hiding complicated transaction operations inside the API as follows:</p><p><strong>Account Service (coordinator)</strong></p><pre>@Transactional(transactionManager = &quot;scalarDbSuspendableTransactionManager&quot;)<br>@Repository<br>public interface AccountRepository<br>  extends ScalarDbTwoPcRepository&lt;Account, Integer&gt; {<br>  // findBydId(), insert(), update(), executeTwoPcTransaction()<br>  // and some more APIs are prepared automatically.<br>}</pre><pre>@Autowired private Account accountRepository;<br><br>...<br><br>accountRepository.executeTwoPcTransaction(<br>  // Business logic (CRUD operations) for local and<br>  // remote participants in execution phase.<br>  txId -&gt; {<br>    // [local] Read the account&#39;s balance<br>    Optional&lt;Account&gt; stored = accountRepository.findById(account.id);<br>    if (!stored.isPresent()) {<br>      // Cancel the transaction if the account doesn&#39;t exist.<br>      // No need to retry.<br>      throw new ScalarDbNonTransientException(<br>          &quot;The local state doesn&#39;t meet the condition.<br>                Aborting this transaction&quot;);<br>    }<br>    // [remote] Start a transaction with the transaction ID,<br>    //          read the item information and decrement the count<br>    Optional&lt;Integer&gt; price = stockService.purchaseItem(<br>                                             txId, account.id, itemName);<br>    if (price.isPresent()) {<br>      int currentBalance = stored.get().balance - price.get();<br>      if (currentBalance &lt; 0) {<br>        // Cancel the transaction<br>        // if the global state doesn&#39;t meet the condition.<br>        // No need to retry.<br>        throw new ScalarDbNonTransientException(<br>            &quot;The state of local and remote participants doesn&#39;t meet<br>                the condition. Aborting this transaction&quot;);<br>      }<br>      // [local] Decrease the account&#39;s balance for the item<br>      accountRepository.update(new Account2Pc(account.id, currentBalance));<br>      return currentBalance;<br>    }<br>    // Cancel the transaction<br>    // if the global state doesn&#39;t meet the condition.<br>    // No need to retry.<br>    throw new ScalarDbTransientException(<br>      &quot;The remote state doesn&#39;t meet the condition.<br>          Aborting this transaction&quot;);<br>  },<br><br>  // Remote operations for Prepare/Validate/Commit/Rollback.<br>  // These transactional operations will be automatically executed.<br>  Arrays.asList(<br>        RemotePrepareCommitPhaseOperations.createSerializable(<br>          stockService::callPrepareTransaction,<br>          stockService::callValidateTransaction,<br>          stockService::callCommitTransaction,<br>          stockService::callRollbackTransaction)<br>);</pre><p><strong>Stock Service (participant)</strong></p><pre>@Transactional(transactionManager = &quot;scalarDbSuspendableTransactionManager&quot;)<br>@Repository<br>public interface StockRepository<br>  extends ScalarDbTwoPcRepository&lt;Stock, String&gt; {<br>  // joinTransactionOnParticipant(), commitTransactionOnParticipant()<br>  // and some more APIs are prepared automatically.<br>}</pre><pre>@RestController<br>public class StockController {<br>  @Autowired private StockRepository stockRepository;<br><br>  @PostMapping(&quot;/purchaseItem&quot;)<br>  public Optional&lt;Integer&gt; purchaseItem(<br>               @RequestParam(&quot;transactionId&quot;) String transactionId,<br>               @RequestParam(&quot;accountId&quot;) int accountId,<br>               @RequestParam(&quot;itemName&quot;) String itemName) {<br>    // Join the global transaction and execute the CRUD operations in it<br>    return stockRepository.joinTransactionOnParticipant(txId, () -&gt; {<br>        Optional&lt;Item&gt; item = stockRepository.findById(itemName);<br><br>        ...<br><br>        return Optional.of(item.price);<br>    });<br>  }<br><br>  @PostMapping(&quot;/prepareTransaction&quot;)<br>  public void prepareTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.prepareTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/validateTransaction&quot;)<br>  public void validateTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.validateTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/commitTransaction&quot;)<br>  public void commitTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.commitTransactionOnParticipant(txId);<br>  }<br><br>  @PostMapping(&quot;/rollbackTransaction&quot;)<br>  public void rollbackTransaction(<br>                  @RequestParam(&quot;transactionId&quot;) String transactionId) {<br>    return stockRepository.rollbackTransactionOnParticipant(txId);<br>  }<br>}</pre><h3>Wrap-up</h3><p>In this blog post, we introduced Spring Data JDBC for ScalarDB. We believe this feature makes the development of Spring Framework–based applications that use ScalarDB easier and more secure. Although there are some <a href="https://scalardb.scalar-labs.com/docs/3.9/scalardb-sql/spring-data-guide/#limitations">limitations</a> that come from Spring Data JDBC itself and the integration of Spring Data JDBC and ScalarDB, our goal is to address those limitations in the future to help you simplify application development.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1d1532fc636" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalar-engineering/introducing-spring-data-jdbc-for-scalardb-a1d1532fc636">Introducing Spring Data JDBC for ScalarDB</a> was originally published in <a href="https://medium.com/scalar-engineering">Scalar Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Released Spring Data JDBC for SQLite]]></title>
            <link>https://komamitsu.medium.com/released-spring-data-jdbc-for-sqlite-a42cae0b1ecd?source=rss-880d4ea6ce25------2</link>
            <guid isPermaLink="false">https://medium.com/p/a42cae0b1ecd</guid>
            <category><![CDATA[spring-data]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[sqlite]]></category>
            <dc:creator><![CDATA[Mitsunori Komatsu]]></dc:creator>
            <pubDate>Sat, 21 Jan 2023 12:16:58 GMT</pubDate>
            <atom:updated>2023-01-21T12:16:58.486Z</atom:updated>
            <content:encoded><![CDATA[<p>Spring Data has released 2 types of component Spring Data JDBC and Spring Data JPA. I wanted to use some RDBMS on Spring application in a simple way for some reasons and chose Spring Data JDBC (Spring Data JPA seemed a bit non-straightforward around caching.) But I noticed Spring Data JDBC didn’t support SQLite. I found some workarounds on Stack Overflow and technical blog posts, but they’re a bit complicated and require some hassles.</p><p>I also noticed Spring Data JDBC depends on underlying database tables’ auto-generated ID columns. Without the feature <strong>CrudRepository#save(T)</strong> doesn’t work fine. This is really painful point for me as I want to use Spring Data JDBC for tables that don’t have auto-generated ID columns.</p><p>So, I developed <a href="https://github.com/komamitsu/spring-data-sqlite">https://github.com/komamitsu/spring-data-sqlite</a> that supports SQLite via Spring Data JDBC API and provides new <strong>insert(T)</strong> and <strong>update(T)</strong> to play Spring Data JDBC using database tables that don’t have auto-generated ID columns.</p><p>I hope those features will be developed by Spring team and merged into the official repository some day so that I can archive the GitHub repo.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a42cae0b1ecd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>