<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yorgos S. on Medium]]></title>
        <description><![CDATA[Stories by Yorgos S. on Medium]]></description>
        <link>https://medium.com/@smyrgeorge?source=rss-b36470932258------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hqn9ldRUZuqXWI5EYLd3mQ.jpeg</url>
            <title>Stories by Yorgos S. on Medium</title>
            <link>https://medium.com/@smyrgeorge?source=rss-b36470932258------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 00:06:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@smyrgeorge/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Sqlx4k — Interoperability between Kotlin and Rust (using FFI)]]></title>
            <link>https://medium.com/@smyrgeorge/sqlx4k-interoperability-between-kotlin-and-rust-using-ffi-aa9bc76d518b?source=rss-b36470932258------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa9bc76d518b</guid>
            <dc:creator><![CDATA[Yorgos S.]]></dc:creator>
            <pubDate>Thu, 01 Aug 2024 15:19:53 GMT</pubDate>
            <atom:updated>2024-08-01T20:16:05.604Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sqlx4k — Interoperability between Kotlin and Rust (using FFI)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/0*h1DMSG1TFSKLvX8i.png" /></figure><h3>Introduction</h3><p>Previously, in the <a href="https://smyrgeorge.github.io/posts/sqlx4k---introduction-to-the-kotlin-native-and-ffi-part-2/">Sqlx4k — Introduction to Kotlin Native and FFI (Part 2)</a>, the second part of this series, we attempted to provide an introduction to how C Interoperability functions in Kotlin, offering various examples from the <a href="https://github.com/smyrgeorge/sqlx4k">GitHub - smyrgeorge/sqlx4k</a> codebase.</p><p>Sqlx4k is a small, non-blocking PostgreSQL database driver written in Kotlin for the Native platform. It wraps the sqlx driver from the Rust ecosystem under the hood. The communication between the two languages is facilitated using FFI.</p><p>In this article, we are going to see in detail how it works and how we can benefit from the integration between the two.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/0*n7juxB5izD8qRtxM.png" /></figure><h3>Project Setup</h3><p>First of all, let’s take a look at the file tree:</p><ul><li>The Kotlin code is located under the src directory.</li><li>The Rust code is located in the rust_lib directory.</li></ul><pre>sqlx4k<br>├── build.gradle.kts<br>├── rust_lib<br>│   ├── Cargo.lock<br>│   ├── Cargo.toml<br>│   ├── build.rs<br>│   └── src<br>│       └── lib.rs<br>└── src<br>    ├── nativeInterop<br>    │   └── cinterop<br>    │       ├── aarch64-apple-darwin.def<br>    │       ├── aarch64-unknown-linux-gnu.def<br>    │       ├── x86_64-apple-darwin.def<br>    │       └── x86_64-unknown-linux-gnu.def<br>    └── nativeMain<br>        └── kotlin<br>            └── io<br>                └── github<br>                    └── smyrgeorge<br>                        └── sqlx4k<br>                            ├── Driver.kt<br>                            ├── Sqlx4k.kt<br>                            ├── Transaction.kt<br>                            └── impl<br>                                ├── Postgres.kt<br>                                └── extensions.kt</pre><p>For more details, you can always take a look at the project’s repository.</p><p>Now, let’s examine the Rust configuration. First, take a look at the Cargo.toml file:</p><pre># We going to show only the important parts here.<br># You can see the full file in the project&#39;s repoisory.<br>[lib]<br>crate-type = [&quot;staticlib&quot;]<br><br>[build-dependencies]<br># https://crates.io/crates/cbindgen<br>cbindgen = &quot;0.26.0&quot;</pre><p>So, we can make the following observations:</p><ul><li>crate-type must be staticlib. According to the documentation: *A static system library will be produced. This is different from other library outputs in that the compiler will never attempt to link to staticlib outputs. The purpose of this output type is to create a static library containing all of the local crate&#39;s code along with all upstream dependencies. This output type will create .a files on Linux, macOS and Windows (MinGW), and <em>.lib files on Windows (MSVC). This format is recommended for use in situations such as linking Rust code into an existing non-Rust application because it will not have dynamic dependencies on other Rust code.</em></li><li>We need the build-dependency cbindgen. *Creates C/C++11 headers for Rust libraries which expose a public C API.*</li></ul><h3>Kotlin Setup</h3><p>Now, let’s take a look at the build.gradle.kts file that configures the Kotlin part. We will focus on the most interesting sections.</p><pre>import org.gradle.nativeplatform.platform.internal.DefaultNativePlatform<br>import org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTarget<br>import java.lang.System.getenv<br><br>plugins {<br>    kotlin(&quot;multiplatform&quot;)<br>}<br><br>// Find the current OS and Architecture.<br>private val os = DefaultNativePlatform.getCurrentOperatingSystem()<br>private val arch = DefaultNativePlatform.getCurrentArchitecture()<br><br>// Appends `.exe` if the build is running in wingows.<br>private val exeExt: String<br>    get() = when {<br>        os.isWindows -&gt; &quot;.exe&quot;<br>        else -&gt; &quot;&quot;<br>    }<br><br>// Locate the cargo executable.<br>private val cargo: String<br>    get() = when {<br>        os.isWindows -&gt; getenv(&quot;USERPROFILE&quot;)<br>        else -&gt; getenv(&quot;HOME&quot;)<br>    }?.let(::File)<br>        ?.resolve(&quot;.cargo/bin/cargo$exeExt&quot;)<br>        ?.takeIf { it.exists() }<br>        ?.absolutePath<br>        ?: throw GradleException(<br>         &quot;Rust cargo binary is required to build project.&quot;)<br><br>// Build targets<br>val chosenTargets = (properties[&quot;targets&quot;] as? String)?.split(&quot;,&quot;)<br>    ?: listOf(&quot;macosArm64&quot;, &quot;macosX64&quot;, &quot;linuxArm64&quot;, &quot;linuxX64&quot;)<br><br>kotlin {<br>    fun KotlinNativeTarget.rust(target: String) {<br>        compilations.getByName(&quot;main&quot;).cinterops {<br>            create(&quot;librust_lib&quot;) {<br>             // Create a gradle task for each of one of the build targets.<br>                val cargo = tasks.create(&quot;cargo-$target&quot;) {<br>                    exec {<br>                        executable = cargo<br>                        args(<br>                            &quot;build&quot;,<br>                            &quot;--manifest-path&quot;,<br>                            projectDir<br>                                .resolve(&quot;rust_lib/Cargo.toml&quot;)<br>                                .absolutePath, // Path to the `Cargo.toml` file.<br>                            &quot;--package&quot;, &quot;rust_lib&quot;,<br>                            &quot;--lib&quot;,<br>                            &quot;--target=$target&quot;, // Define the Rust build target.<br>                            &quot;--release&quot;<br>                        )<br>                    }<br>                }<br><br>    // Build Rust code before call interop task.<br>                tasks.getByName(interopProcessingTaskName) {<br>                    dependsOn(cargo)<br>                }<br><br>    // Set the .def file for each platform.<br>    // The .def file describes what will be included into bindings.<br>                definitionFile.set(<br>                 projectDir<br>                 .resolve(&quot;src/nativeInterop/cinterop/$target.def&quot;)<br>             )<br>            }<br>        }<br>    }<br><br> // Setup the build targets.<br> // For each platform we need to define the rust target build.<br>    val availableTargets = mapOf(<br>        Pair(&quot;macosArm64&quot;) { macosArm64 { rust(&quot;aarch64-apple-darwin&quot;) } },<br>        Pair(&quot;macosX64&quot;) { macosX64 { rust(&quot;x86_64-apple-darwin&quot;) } },<br>        Pair(&quot;linuxArm64&quot;) { linuxArm64 { rust(&quot;aarch64-unknown-linux-gnu&quot;) } },<br>        Pair(&quot;linuxX64&quot;) { linuxX64 { rust(&quot;x86_64-unknown-linux-gnu&quot;) } },<br>    )<br><br> // Invoke build for each one of the selected targets.<br>    chosenTargets.forEach {<br>        println(&quot;Enabling target $it&quot;)<br>        availableTargets[it]?.invoke()<br>    }<br><br> // Other configs, non-related to this article (depedencies etc.).<br>    applyDefaultHierarchyTemplate()<br>    sourceSets {<br>        configureEach {<br>            languageSettings.progressiveMode = true<br>        }<br>        val nativeMain by getting {<br>            dependencies {<br>                // https://github.com/Kotlin/kotlinx.coroutines<br>                implementation(<br>                 &quot;org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1&quot;)<br>            }<br>        }<br>    }<br>}</pre><p>According to the Kotlin documentation, we need to create some .def files that define the bindings. In our case, for the aarch64-apple-darwin target, we have the following file:</p><pre>package = librust_lib<br>headers = rust_lib.h<br>compilerOpts = -I./rust_lib/target<br><br>staticLibraries = librust_lib.a<br>libraryPaths = ./rust_lib/target/aarch64-apple-darwin/release</pre><h3><strong>Build</strong></h3><p>If we build the project we should see something like</p><pre>&gt; Configure project :sqlx4k<br>Enabling target macosArm64<br> ...<br> ...<br>    Finished `release` profile [optimized] target(s) in 7.17s<br>Enabling target macosX64<br> ...<br> ...<br>    Finished `release` profile [optimized] target(s) in 7.14s<br>Enabling target linuxArm64<br> ...<br> ...<br>    Finished `release` profile [optimized] target(s) in 7.17s<br>Enabling target linuxX64<br> ...<br> ...<br>    Finished `release` profile [optimized] target(s) in 7.14s<br>...<br>BUILD SUCCESSFUL in 24s<br>35 actionable tasks: 3 executed, 32 up-to-date</pre><p>As we can see, gradle first triggers the build of the Rust code for each of the selected targets (in our case, macosArm64, macosX64, linuxArm64, linuxX64) and then builds the Kotlin code (located under the src folder).</p><h3>Generated Rust Files</h3><p>The Rust build generates a couple of files that are then used by the Kotlin compiler.</p><p>rust_lib/target/rust_lib.h: The generated (by cbindgen, that we saw earlier) C bindings (header files)</p><pre>// Only a small part of the original file is shown here.<br><br>#include &lt;stdarg.h&gt;<br>#include &lt;stdbool.h&gt;<br>#include &lt;stdint.h&gt;<br>#include &lt;stdlib.h&gt;<br><br>#define OK -1<br><br>#define ERROR_DATABASE 0<br><br>typedef struct Sqlx4kColumn {<br>  int ordinal;<br>  char *name;<br>  int kind;<br>  int size;<br>  void *value;<br>} Sqlx4kColumn;<br><br>void sqlx4k_query(const char *sql, void *callback, void (*fun)(struct Ptr, struct Sqlx4kResult*));<br><br>void sqlx4k_fetch_all(const char *sql,<br>                      void *callback,<br>                      void (*fun)(struct Ptr, struct Sqlx4kResult*));<br><br>void sqlx4k_free_result(struct Sqlx4kResult *ptr);</pre><p>rust_lib/target/aarch64-apple-darwin/release/librust_lib.a: The native binary that will be linked to our code. Of course, we should find the corresponding.a files for each platform for which we enabled the build.</p><h3>Let’s see a code example in action</h3><p>Assuming that you have the following Rust function:</p><pre>#[no_mangle]<br>pub extern &quot;C&quot; fn sqlx4k_pool_size() -&gt; c_int {<br>    unsafe { SQLX4K.get().unwrap() }.pool.size() as c_int<br>}</pre><p>Then, you can call the above Rust function from the Kotlin as follows:</p><pre>import librust_lib.sqlx4k_pool_size<br>fun poolSize(): Int = sqlx4k_pool_size()</pre><p>In this article we don’t going to explain how FFI works at the Rust part. You can easily review the available documentation. Also feel free to look at the references section below. You can find several other examples in the project’s codebase.</p><h3>Conclusion</h3><p>As we can see, it is actually very easy to “integrate” Kotlin with Rust.</p><p>At this moment, with the Kotlin ecosystem still in a very young stage, we can benefit by bringing well-tested libraries and functionality to the ecosystem.</p><p>In the next part of this series, we will see in detail how we can utilise the async features that both languages offer in order to create non-blocking IO code. Until then, you can review the previous articles here: <a href="https://smyrgeorge.github.io/">:: exploration and stuff ::</a> and also check out my GitHub repository: <a href="https://github.com/smyrgeorge">smyrgeorge (Yorgos S.) · GitHub</a>.</p><h3>References<a href="https://smyrgeorge.github.io/posts/sqlx4k---interoperability-between-kotlin-and-rust-using-ffi-part-1/#references">#</a></h3><p><em>Originally published at </em><a href="https://smyrgeorge.github.io/posts/sqlx4k---interoperability-between-kotlin-and-rust-using-ffi-part-1/"><em>https://smyrgeorge.github.io</em></a><em> on August 1, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa9bc76d518b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sqlx4k — Introduction to the Kotlin native and FFI (Part 2)]]></title>
            <link>https://medium.com/@smyrgeorge/sqlx4k-introduction-to-the-kotlin-native-and-ffi-part-2-0231160a7725?source=rss-b36470932258------2</link>
            <guid isPermaLink="false">https://medium.com/p/0231160a7725</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[ffi]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Yorgos S.]]></dc:creator>
            <pubDate>Tue, 16 Jul 2024 18:12:33 GMT</pubDate>
            <atom:updated>2024-07-17T07:09:42.665Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sqlx4k — Introduction to the Kotlin native and FFI (Part 2)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lxJO1bbZ06CNlS1Z.png" /></figure><h3>Introduction</h3><p>In the previous <a href="https://smyrgeorge.github.io/posts/sqlx4k---introduction-to-the-kotlin-native-and-ffi-part-1/">part</a> of this series, I attempted to explain the importance of the Kotlin FFI (Foreign Function Interface) compatibility layer. As I mentioned, I believe it is a very effective way (at least for now) to leverage other ecosystems (such as the Rust ecosystem) to bring functionality to the Kotlin/Native ecosystem, in which I see a great deal of potential.</p><p>Foreign Function Interface (FFI) is a programming feature that enables code written in one language to call and interact with code written in another language. It acts as a bridge, allowing developers to leverage the strengths of different programming languages within a single project. FFI is essential for integrating diverse software libraries and for accessing hardware or operating system features not directly available in the host language. Implementing FFI involves defining bindings that specify how functions from one language can be called from another, including the necessary data type conversions.</p><p>As of the time these lines are being written, Kotlin/Native Interop (FFI) is still considered experimental. However, based on the experience I have had so far, it has seemed to me to be really stable. Take a look at the official documentation here: <a href="https://kotlinlang.org/docs/native-c-interop.html">Interoperability with C | Kotlin Documentation</a></p><h3>Call a C function</h3><p>Let’s begin with a simple example. Assume we have a small C library:</p><pre>// This could be part of header file (e.g. lib.h)<br>// The actual implementation of the C functions is not shown here.<br><br>// Just a simple function that accepts a C string.<br>void hello_1(char *msg);<br>void hello_2(const char *msg);</pre><p>Here is the Kotlin part:</p><pre>// Import the C functions<br>import mylib.hello_1<br>import mylib.hello_2<br><br>fun hello() {<br>    hello_1(&quot;Hello from Kotlin&quot;.cstr)<br>    hello_2(&quot;Hello from Kotlin&quot;.cstr)<br>}</pre><h3>More examples</h3><p>Here again, we have the C code:</p><pre>int fun_that_returns_int(void);<br><br>// A LinkedList Node<br>typedef struct Node {<br>  char *a_string_value;<br>  int an_int_value;<br>  struct Node *next;<br>} Node;<br><br>struct Node *get_a_node();</pre><p>And the Kotlin:</p><pre>val r1: Int = fun_that_returns_int()<br><br>// Here we can access the properties of a struct.<br>val r2: CPointer&lt;Node&gt;? = get_a_node()<br>val node: Node? = r2?.pointed<br><br>if (node == null) {<br> println(&quot;null pointer returned&quot;)<br>}<br><br>node?.let {<br>    println(it.a_string_value?.toKString() ?: &quot;null value returned&quot;)<br>    println(it.an_int_value)<br>    <br>    // Same as above (handle the pointer).<br>    it.next?.let { it1 -&gt;<br>     // Handle the next node of the List.<br>    }<br>}</pre><h3>Pass a callback function to the C code</h3><p>The ability to “pass” functions to the C code is really helpful, especially if we want to create asynchronous flows. The basic idea is that we are going to call the foreign function from our code (in our case, the Kotlin code we will continue without blocking), and as soon as the result is ready, the foreign code will make the callback back to our code and provide the result data. In the case of sqlx4k, this was very important in order to create a non-blocking database driver.</p><p>The following examples are parts of sqlx4k, which wraps a database driver that is actually written in Rust. Although in this tutorial we are not going to see any Rust code, we will see a lot of C code. In reality, almost all the time we need to think about how the equivalent code would be in C. The reason is that almost all programming languages that provide FFI (Foreign Function Interface) offer bindings for C and C++.</p><p>Kotlin also provides interoperability with Swift and Objective-C (for more details, check the official documentation).</p><p>So, here is the C code.</p><pre>// Query result<br>typedef struct Sqlx4kResult {<br>  int error; // error code<br>  char *error_message;<br>  void *tx; // transaction pointer<br>  int size; // result set size<br>  struct Sqlx4kRow *rows; // pointer to rows<br>} Sqlx4kResult;<br><br>// Raw pointer wrapper<br>typedef struct Ptr {<br>  void *ptr;<br>} Ptr;<br><br>void sqlx4k_query(const char *sql,<br>      // Ignore for now.<br>      void *continuation,<br>      // Function pointer.<br>      void (*callback)(struct Ptr, struct Sqlx4kResult*));</pre><p>In the following example, we make a call to the C function sqlx4k_query and we pass the following arguments:</p><ul><li>sql: the query that we need to execute in the database</li><li>continuation: ignore for now (used for the async-io)</li><li>callback: the function that the C code will call as soon as the result is ready.</li></ul><p>For now, let’s try to ignore the continuation and the async-io part of the code, let&#39;s focus on the callback part.</p><pre>// It&#39;s just an example, we do not return the actual fetched data.<br>suspend fun query(sql: String) {<br>  suspendCoroutine { c: Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt; -&gt;<br>    // Create a [StableRef] of the [Continuation] instance.<br>    // StableRef&lt;Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt;&gt;<br>    val ref = StableRef.create(c)<br>    val continuation: CPointer&lt;out CPointed&gt; = ref.asCPointer()<br>    // Call the actual C function.<br>    sqlx4k_query(sql, continuation, callback)<br>  }<br>}<br><br>// [callback] is the Kotlin function that we need to pass to the C code.<br>val callback = staticCFunction&lt;APtr, ACPointer, Unit&gt; { c, r -&gt;<br>  val ref = c.useContents { ptr }!!<br>    .asStableRef&lt;Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt;&gt;()<br>   ref.get().resume(r)<br>   ref.dispose()<br>}<br><br>typealias APtr = CValue&lt;Ptr&gt;<br>typealias ACPointer = CPointer&lt;Sqlx4kResult&gt;?</pre><p>First of all, let’s take a look at the <a href="https://kotlinlang.org/docs/native-c-interop.html#callbacks">staticCFunction</a>. In the Kotlin code, it is defined (as part of the kotlinx.cinterop package) like:</p><pre>public external fun &lt;P1, P2, R&gt; staticCFunction(<br>  function: (P1, P2) -&gt; R<br>): CPointer&lt;CFunction&lt;(P1, P2) -&gt; R&gt;&gt;</pre><p>In our example, we use the <a href="https://kotlinlang.org/docs/native-c-interop.html#callbacks">staticCFunction</a> to declare a function that takes two arguments, of type APtr and CPointer, and returns Unit. The C code can call this function as if it was a pure C function.</p><h3>How to “not-block”</h3><p>In the previous example, we saw that we can pass a callback to the foreign code. In this section, we are going to explain how the “suspend” part works.</p><p>Kotlin offers the <a href="https://kotlinlang.org/api/core/kotlin-stdlib/kotlin.coroutines/suspend-coroutine.html">suspendCoroutine</a> function.</p><p><em>According to the documentation: Obtains the current continuation instance inside suspend functions and suspends the currently running coroutine. </em><em>Continuation: Interface representing a continuation after a suspension point that returns a value of type T.</em></p><p>In other words, it is a way to suspend the execution flow, and the <a href="https://smyrgeorge.github.io/posts/sqlx4k---introduction-to-the-kotlin-native-and-ffi-part-2/">Continuation</a> instance can provide us the way to continue the flow (in our case, after the data from the database are fetched).</p><p>So, as we saw:</p><pre>// The code will suspend here.<br>suspendCoroutine { c: Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt; -&gt;<br>  // Create a [StableRef] of the [Continuation] instance.<br>  // StableRef&lt;Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt;&gt;<br>  val ref = StableRef.create(c)<br>  val continuation: CPointer&lt;out CPointed&gt; = ref.asCPointer()<br>  // Call the actual C function.<br>  sqlx4k_query(sql, continuation, callback)<br>}</pre><p>We need to pass the <a href="https://kotlinlang.org/api/core/kotlin-stdlib/kotlin.coroutines/-continuation/">Continuation</a> to the foreign code and then the foreign code will provide back to us (as a C raw pointer).</p><p>The way that Kotlin provides in order to pass references from &quot;one world to another&quot; is with <a href="https://kotlinlang.org/api/core/kotlin-stdlib/kotlinx.cinterop/-stable-ref/">StableRef</a>:</p><pre>// Function pointer that will be passed to the foreign code.<br>val callback = staticCFunction&lt;APtr, ACPointer, Unit&gt; { c, r -&gt;<br>  // Cast raw pointer back to [StableRef]<br>  val ref = c.useContents { ptr }!!<br>    .asStableRef&lt;Continuation&lt;CPointer&lt;Sqlx4kResult&gt;?&gt;&gt;()<br>  // Call the resume, continue the suspended flow above.<br>  ref.get().resume(r)<br>  // Dispose (will free) the ref,<br> // thus clear the reference to the continuation<br>  ref.dispose()<br>}</pre><p>I believe the process is now a bit more understandable. We perform the reverse process:</p><ul><li>Access the returned struct from the foreign code using useContents.</li><li>Cast the raw pointer back to a StableRef.</li><li>Resume the suspended flow.</li><li>Dispose of the StableRef.</li></ul><h3>Memory management</h3><p>Although until now everything seems very straightforward, there are some caveats. At every moment of this process, we need to keep in mind the memory management and how each one of the “worlds” allocates and deallocates memory. This is very crucial in order to avoid memory leaks.</p><p>While I was developing the sqlx4k library, I created the following rule:</p><blockquote><strong><em>Each one of the “worlds” (language) needs to manage its own memory.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vExap7CNVWZzqjlF.png" /></figure><p>This simply means that each language should deallocate the memory it allocated. In other words, we need to provide some more functions that will do this job for us. For instance, in my case, I created the following function that frees the allocated memory:</p><pre>// Function that deallocates the memory for a Sqlx4kResult struct.<br>// Accepts a pointer to a struct [Sqlx4kResult].<br>void sqlx4k_free_result(struct Sqlx4kResult *ptr);</pre><p>In each language, this process could be a bit different. In my case, with Rust, I almost hacked the memory management by intentionally “leaking” memory and then calling the sqlx4k_free_result function from the Kotlin code in order to free the memory, thus avoiding a memory leak:</p><pre>fun &lt;T&gt; CPointer&lt;Sqlx4kResult&gt;?.use(f: (it: Sqlx4kResult) -&gt; T): T {<br>  return try {<br>    // &quot;Point&quot; to the pointer<br>    this?.pointed?.let { f(it) }<br>      ?: error(&quot;null pointer exception&quot;)<br>  } finally {<br>    // Ensure that allways we will free the memory.<br>    sqlx4k_free_result(this)<br>  }<br>}</pre><p>In the next part of this series, I will provide detailed examples on how I managed to do it (it wasn’t so difficult in the end).</p><h3>Conclusion</h3><p>Although we need to be a bit more careful, especially with memory management-even in the case of Kotlin and Rust, which are considered memory-safe languages-I believe the whole process is not that difficult. Moreover, if you manage to “wire” the two languages for a given example (like wrapping a database driver), then doing it for a different task will become much easier, since you will have gained the knowledge and “invented” the necessary mechanics (or tricks, if you prefer).</p><p>I think the benefits are numerous:</p><ul><li>Continue writing in Kotlin, as it is a modern and easy-to-learn language.</li><li>Benefit from applications with a small memory footprint.</li><li>Provide the Kotlin ecosystem with libraries that will help us develop more applications, especially on the backend.</li></ul><p>As always, take a look at the project’s repository on GitHub: <a href="https://github.com/smyrgeorge/sqlx4k">GitHub — smyrgeorge/sqlx4k: A small non-blocking database driver written in Kotlin for the Native platform.</a></p><p>All contributions are welcome. Thank you very much.</p><h3>References</h3><ul><li><a href="https://en.wikipedia.org/wiki/Foreign_function_interface">Foreign function interface — Wikipedia</a></li><li><a href="https://kotlinlang.org/docs/native-c-interop.html">Interoperability with C | Kotlin Documentation</a></li></ul><p><em>Originally published at </em><a href="https://smyrgeorge.github.io/posts/sqlx4k---introduction-to-the-kotlin-native-and-ffi-part-2/"><em>https://smyrgeorge.github.io</em></a><em> on July 16, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0231160a7725" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sqlx4k — Introduction to the Kotlin native and FFI (Part 1)]]></title>
            <link>https://medium.com/@smyrgeorge/sqlx4k-introduction-to-the-kotlin-native-and-ffi-part-1-2740c15392d5?source=rss-b36470932258------2</link>
            <guid isPermaLink="false">https://medium.com/p/2740c15392d5</guid>
            <dc:creator><![CDATA[Yorgos S.]]></dc:creator>
            <pubDate>Mon, 08 Jul 2024 11:27:03 GMT</pubDate>
            <atom:updated>2024-07-08T21:39:13.445Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sqlx4k — Introduction to the Kotlin native and FFI (Part 1)</h3><h3>Introduction</h3><p>Recently, I began experimenting with the <strong>Kotlin Native platform</strong>. I initiated a new repository and attempted to create a simple project utilizing the <strong>ktor</strong> libraries. The purpose of the project was to recreate a small service that integrates some basic libraries and compile it to a native target (macosArm64 in my case). The service aimed to offer support for:</p><ul><li>Dependency injection</li><li>HTTP server</li><li>Database access (PostgreSQL)</li><li>Additionally, RabbitMQ support (though it isn’t a priority for now)</li></ul><p>In this first article of the series, I aim to describe how I ended up writing “low-level” code (FFI between Kotlin and Rust) and also to highlight the importance of native compatibility in Kotlin Native. In the subsequent parts, I’ll attempt to provide more detailed examples of how I managed to implement a simple SQL driver for PostgreSQL, using the sqlx (Rust) library.</p><h3>Ktor</h3><p>I discovered that starting a new project with Ktor is very easy.</p><p>Ktor, developed by JetBrains, is a Kotlin-based framework optimized for asynchronous I/O operations, enabling efficient web and microservices development. Leveraging Kotlin’s coroutines, Ktor facilitates non-blocking I/O, which significantly enhances application scalability and performance. Its architecture supports seamless integration of async features, making it ideal for building responsive, high-performance applications with streamlined concurrency management.</p><p>I began with the <a href="https://start.ktor.io/settings">Ktor: Project Generator</a> and created a sample project.</p><p>Then, I only had to make a few changes to the <strong>build.gradle.kts</strong> file:</p><pre>plugins {<br>    kotlin(&quot;multiplatform&quot;) version &quot;2.0.0&quot;<br>}<br><br>repositories {<br>    mavenCentral()<br>}<br><br>group = &quot;io.github.smyrgeorge&quot;<br>version = &quot;0.1.0&quot;<br><br>val ktorVersion = &quot;3.0.0-beta-1&quot;<br>val koinVersion = &quot;3.5.6&quot;<br>kotlin {<br>    // Enable the according to your platform (you can enable them all):<br>    macosArm64 { binaries { executable() } }<br>    // linuxX64 { binaries { executable() } }<br>    // linuxX64 { binaries { executable() } }<br><br>    applyDefaultHierarchyTemplate()<br>    sourceSets {<br>        val nativeMain by getting {<br>            dependencies {<br>                // Ktor with CIO<br>                implementation(&quot;io.ktor:ktor-server-cio:$ktorVersion&quot;)<br>                // Koin (DI)<br>                implementation(&quot;io.insert-koin:koin-core:$koinVersion&quot;)<br>                // Database driver<br>                implementation(&quot;io.github.smyrgeorge:sqlx4k:0.4.0&quot;)<br>            }<br>        }<br>    }<br>}</pre><p>And then I changed the <strong>main</strong> function to be like the default example:</p><pre>fun main() {<br>    embeddedServer(CIO, port = 8080) {<br>        routing {<br>            get (&quot;/&quot;) {<br>                call.respondText(&quot;Hello, world!&quot;)<br>            }<br>        }<br>    }.start(wait = true)<br>}</pre><p>Then I build the project with:</p><pre>./gradlew build</pre><p>And finally, I started my sample service (ktor-native-sample):</p><pre>./build/bin/macosArm64/releaseExecutable/ktor-native-sample.kexe</pre><p>And I then I saw the following logs:</p><pre>[INFO] (io.ktor.server.Application): Application started in 0.0 seconds.<br>[INFO] (io.ktor.server.Application): Responding at http://0.0.0.0:8080</pre><p>As we can observe, the startup time was less than a second (a few milliseconds, I guess), which I find to be the interesting part. Additionally, I must mention that the memory usage was around <strong>~5MB</strong>.</p><h3>A little bit of the background.</h3><p>So, the reality is that over the last couple of years, I have developed numerous projects using the Spring Boot stack (using Kotlin instead of Java). I believe we’re all aware of the notably slow startup times associated with JVM-based applications, not to mention the memory consumption (approximately ~50MB at startup).</p><p>Naturally, those coming from other tech stacks (e.g., Rust, or something similar) might find this amusing. To a large extent, I almost agree with them. However, transitioning from one technology stack to another can sometimes be challenging.</p><p>For instance, I have been working for the same company for nearly 9 years now, and during this time, my team and I have implemented several projects and products using several technologies. In the back-end side, almost from the start, we chose the Spring Boot (Kotlin) stack. And I believe it has served us very well and also the code base has aged really well. Just to provide an example, we have a very large (and complex) insurance project, and we managed to migrate almost the entire project to Kotlin-async (coroutines) in just 5–6 months.</p><p>During the migration period, we had to modify the parts of the code that accessed the database, the queues, and, of course, the entry points (e.g., the HTTP layer). Apart from that, in almost the rest of the codebase (service layer), we only needed to add the <strong>suspend</strong> keyword to indicate that a function is asynchronous (similar to async in Rust or JavaScript). It was a relatively easy process, considering the volume of code we had written up to that point and the complexity of the business model. Therefore, I consider this a significant success story.</p><h3>Kotlin Native ecosystem</h3><p>So, I began to explore libraries that support multiplatform capabilities, specifically those that target to the native platform (e.g., to be compiled for linuxArm64). Very quickly, I realised that the ecosystem is still in a very early stage. Consequently, I contemplated developing some libraries myself. The most critical component for the type of applications my team has been building over the last few years is database access, specifically to PostgreSQL databases. However, as of now, there is no native driver available (or at least I couldn’t find one), prompting me to consider creating one. I soon realised that this task was not going to be easy. Additionally, during my research, I discovered some small projects that wrap the <strong>libpq</strong> (the official <strong>C</strong> database client) library using FFI.</p><p>I also forgot to mention that in several side projects over the past few years, I have used the Rust programming language. Thus, I thought that perhaps I could try wrapping one of the Rust database drivers. I opted for <strong>sqlx</strong> because I had used it in the past and I believe it offers decent performance.</p><h3>Sqlx4k</h3><p>So, I ended up creating <strong>sqlx4k</strong> (perhaps not the best name, but I wanted to convey that it’s a wrapper around the <strong>sqlx</strong> library). Sqlx4k is a minimal (and I believe it should remain minimal in the future) non-blocking database driver. Currently, it only supports PostgreSQL, but I plan to extend support to MySQL and eventually SQLite databases. By “minimal,” I mean it’s essentially a convenient wrapper around the Rust library, providing an idiomatic Kotlin API for database access.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/0*NSFMipYfSZDqlvF7.png" /></figure><p>As part of this article, I wanted to offer an introduction. In subsequent parts of this series, I will briefly describe how I managed to bridge the two different “worlds.”</p><p>If you found this introduction interesting and wish to experiment, you can add the dependency to your project and start exploring:</p><pre>implementation(&quot;io.github.smyrgeorge:sqlx4k:x.y.z&quot;)</pre><p>You can find the latest published version here: <a href="https://central.sonatype.com/artifact/io.github.smyrgeorge/sqlx4k">Maven Central: io.github.smyrgeorge:sqlx4k</a></p><p>Here is a small example:</p><pre>val pg = Postgres(<br>    host = &quot;localhost&quot;,<br>    port = 15432,<br>    username = &quot;postgres&quot;,<br>    password = &quot;postgres&quot;,<br>    database = &quot;test&quot;,<br>    maxConnections = 10 // set the max-pool-size here<br>)<br><br>// Named parameters:<br>pg.query(&quot;drop table if exists :table;&quot;, mapOf(&quot;table&quot; to &quot;sqlx4k&quot;)).getOrThrow()<br><br>pg.fetchAll(&quot;select * from :table;&quot;, mapOf(&quot;table&quot; to &quot;sqlx4k&quot;)) {<br>    val id: Sqlx4k.Row.Column = get(&quot;id&quot;)<br>    Test(id = id.value.toInt())<br>}<br><br>// Transactions:<br>val tx1: Transaction = pg.begin().getOrThrow()<br>tx1.query(&quot;delete from sqlx4k;&quot;).getOrThrow()<br>tx1.fetchAll(&quot;select * from sqlx4k;&quot;) {<br>    println(debug())<br>}<br>pg.fetchAll(&quot;select * from sqlx4k;&quot;) {<br>    println(debug())<br>}<br>tx1.commit().getOrThrow()</pre><p>Also, take a look at the project’s repository on GitHub: <a href="https://github.com/smyrgeorge/sqlx4k">GitHub — smyrgeorge/sqlx4k: A small non-blocking database driver written in Kotlin for the Native platform.</a></p><p>All contributions are welcome. Thank you for now.</p><h3>References</h3><ul><li><a href="https://kotlinlang.org/">Kotlin Programming Language</a></li><li><a href="https://ktor.io/">Ktor: Build Asynchronous Servers and Clients in Kotlin | Ktor Framework</a></li><li><a href="https://github.com/launchbadge/sqlx">GitHub — launchbadge/sqlx: 🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, and SQLite.</a></li></ul><p><em>Originally published at </em><a href="https://smyrgeorge.github.io/posts/sqlx4k---introduction-to-the-kotlin-native-and-ffi-part-1/"><em>https://smyrgeorge.github.io</em></a><em> on July 8, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2740c15392d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reverse proxy with Bearer Authorization (using Keycloak identity server)]]></title>
            <link>https://medium.com/@smyrgeorge/reverse-proxy-with-bearer-authorization-using-keycloak-identity-server-ed1ebd686696?source=rss-b36470932258------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed1ebd686696</guid>
            <category><![CDATA[authorization]]></category>
            <category><![CDATA[openresty]]></category>
            <category><![CDATA[keycloak]]></category>
            <category><![CDATA[reverse-proxy]]></category>
            <category><![CDATA[nginx]]></category>
            <dc:creator><![CDATA[Yorgos S.]]></dc:creator>
            <pubDate>Tue, 25 Aug 2020 08:46:53 GMT</pubDate>
            <atom:updated>2020-08-25T10:53:27.268Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LGj1xQaVCHR9B1Dy-9PeCA.png" /></figure><p><a href="https://github.com/smyrgeorge/openresty-keycloak-gateway"><strong>openresty-keycloak-gateway</strong></a> is a fully working example of a reverse proxy, that supports JWT authentication.</p><p>Technologies used:</p><ul><li>Keycloak, provides authentication, authorization, user management, etc</li><li>OpenResty (with lua-resty-openidc module), web platform (like nginx)</li></ul><p>Note that the reverse proxy needs to validate a JWT token in order to forward the request. In this case we need to provide an Authorization: Bearer bearer_token_here header.</p><p>Also in every request the gateway, replaces the Authorization with a X-Real-Name header to the request.</p><h4>Original request</h4><pre>GET /api/v1/test HTTP/1.0<br>Accept: */*<br>Accept-Encoding: gzip, deflate<br>Authorization: Bearer bearer_token_here<br>Accept-Language: en-us</pre><h4>Forwarded request</h4><pre>GET /api/v1/test HTTP/1.0<br>Host: localhost<br>X-Real-IP: 172.17.0.1<br>X-Forwarded-For: 172.17.0.1<br>X-Forwarded-Proto: http<br>Connection: close<br>Accept: */*<br>User-Agent: Rested/2009 CFNetwork/1128.0.1 Darwin/19.6.0 (x86_64)<br>Accept-Language: en-us<br>Accept-Encoding: gzip, deflate<br>X-Real-Name: ewoJImZpcnN0X25hbWUiOiAidXNlciIsCgkidXVpZCI6ICIwYmM5MTMyNS0xNjk3LTRiNmQtYjRiZi01MGYxZmNmZGMzZWIiLAoJInJvbGVzIjogWyJST0xFX1RFU1RfMSIsICJST0xFX1RFU1RfMiJdLAoJImxhc3RfbmFtZSI6ICJ1c2VyIiwKCSJ1c2VybmFtZSI6ICJ1c2VyIiwKCSJlbWFpbCI6ICJ1c2VyQHRlc3QudGVzdCIKfQ==</pre><h4>X-Real-Name header</h4><p>Is a base64 encoded json object, which contains user information.</p><pre>{<br> “first_name”: “user”,<br> “uuid”: “0bc91325–1697–4b6d-b4bf-50f1fcfdc3eb”,<br> “roles”: [“ROLE_TEST_1”, “ROLE_TEST_2”],<br> “last_name”: “user”,<br> “username”: “user”,<br> “email”: “user@test.test”<br>}</pre><h3>Prerequisites</h3><ul><li>A working Keycloak identity server. <a href="https://www.keycloak.org/docs/latest/getting_started/index.html">See here</a>.</li><li>A keycloak realm. <a href="https://www.keycloak.org/docs/latest/getting_started/index.html#creating-a-realm-and-a-user">See here</a></li><li>A keycloak client with Access type: public.</li><li>A simple http echo server (Optional)</li></ul><h3><strong>Docker</strong></h3><h4>Dockerfile</h4><p>See <a href="https://github.com/smyrgeorge/openresty-keycloak-gateway/blob/master/Dockerfile">Dockerfile</a></p><h4>docker build</h4><p>docker build -t authproxy .</p><h4>docker run</h4><p>docker run --name authproxy -d -it -p 8000:8000 -v $PWD/nginx.conf:/nginx.conf authproxy -c /nginx.conf</p><h3>nginx.conf</h3><p>See <a href="https://github.com/smyrgeorge/openresty-keycloak-gateway/blob/master/nginx.conf">nginx.conf</a></p><h4>keycloak</h4><ul><li>Change the client_id.</li><li>Change keycloak_base_url to locate your Keycloak server.</li></ul><h4>proxy_pass</h4><p>In the location section, proxy_pass is used to forward request to another service. In this case we forward the request to a simple echo http server.</p><h3>Test http server, for debug purposes</h3><p><em>Note: in each request http-echo-server adds 2 sec timeout.</em></p><p>npm i -g http-echo-server</p><h3>Links</h3><ul><li><a href="https://eclipsesource.com/blogs/2018/01/11/authenticating-reverse-proxy-with-keycloak/">Authenticating Reverse Proxy with KeyCloak - EclipseSource</a></li><li><a href="https://www.keycloak.org/docs/latest/getting_started/index.html">Getting Started Guide</a></li><li><a href="https://github.com/zmartzone/lua-resty-openidc">zmartzone/lua-resty-openidc</a></li><li><a href="https://openresty.org/en/">OpenResty® - Open source</a></li><li><a href="https://jwt.io/">JSON Web Tokens - jwt.io</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed1ebd686696" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>