<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shingo OKAWA on Medium]]></title>
        <description><![CDATA[Stories by Shingo OKAWA on Medium]]></description>
        <link>https://medium.com/@ognis1205?source=rss-97a02a587c73------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4cxR0J1JLMvx7I7F8mLSjA.jpeg</url>
            <title>Stories by Shingo OKAWA on Medium</title>
            <link>https://medium.com/@ognis1205?source=rss-97a02a587c73------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:11:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ognis1205/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Desugaring Rust async/.await #0]]></title>
            <link>https://medium.com/@ognis1205/desugaring-rust-async-await-0-6d0b2c4fe2b7?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d0b2c4fe2b7</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[asynchronous-programming]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Sat, 25 May 2024 17:06:35 GMT</pubDate>
            <atom:updated>2024-05-25T17:06:35.194Z</atom:updated>
            <content:encoded><![CDATA[<h4>Motivation</h4><p>As I progress with the work on several main projects, I’ve become interested in understanding how Rust’s <a href="https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html"><em>async/.await</em></a> syntax transforms into corresponding state machines.</p><p>Consequently, I’ve decided to start a <a href="https://github.com/ognis1205/desugaring-async-await-rs">project</a> to analyze the assembly code generated from <a href="https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html"><em>async/.await</em></a> syntax to explore how state machines are represented at the assembly level.</p><p>In starting this <a href="https://github.com/ognis1205/desugaring-async-await-rs">project</a>, I felt the need to deepen my understanding of the foundational <a href="https://doc.rust-lang.org/std/future/trait.Future.html"><em>Future</em></a> runtime underlying <a href="https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html"><em>async/.await</em></a> tools. Thus, I implemented my own <a href="https://doc.rust-lang.org/std/future/trait.Future.html"><em>Future</em></a> runtime based on <a href="https://en.wikipedia.org/wiki/Kqueue">BSD kqueue</a>, a network IO demultiplexer (event loop), specifically designed for single-threaded operation on MacOSX. Drawing inspiration from <a href="https://github.com/tokio-rs/tokio">tokio-rs/tokio</a> and <a href="https://github.com/tokio-rs/mio">tokio-rs/mio</a>, I named this crate <a href="https://github.com/ognis1205/desugaring-async-await-rs">little-tokio</a>.</p><p>In this blog series, the focus will be on <a href="https://github.com/ognis1205/desugaring-async-await-rs">little-tokio</a>, aiming ultimately to analyze the assembly code of state machines implemented with <a href="https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html"><em>async/.await</em></a> syntax.</p><h4>What You Can Expect to Learn</h4><p>Through this blog series, you can learn the following:</p><ul><li><a href="https://en.wikipedia.org/wiki/Multiplexing">IO mux/demux</a> (<a href="https://en.wikipedia.org/wiki/Reactor_pattern">Reactor pattern</a>)</li><li>Rust’s <a href="https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html"><em>async/.await</em></a> tools and related technologies including the <a href="https://doc.rust-lang.org/std/future/trait.Future.html"><em>Future</em></a> trait, <a href="https://doc.rust-lang.org/std/pin/struct.Pin.html"><em>Pin</em></a> trait, and <a href="https://doc.rust-lang.org/std/pin/"><em>Pin structural projection</em></a></li><li>Rust C/C++ <a href="https://en.wikipedia.org/wiki/Foreign_function_interface">FFI</a></li></ul><p>Let’s learn together about Rust’s async/.await and related technologies.</p><h4>Links</h4><ul><li><a href="https://github.com/ognis1205/desugaring-async-await-rs">Little Tokio</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d0b2c4fe2b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rust: Two Types of 'static]]></title>
            <link>https://medium.com/@ognis1205/rust-two-types-of-static-f244d3014ff3?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/f244d3014ff3</guid>
            <category><![CDATA[rustlang]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[rust-programming-language]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Fri, 26 Apr 2024 13:51:27 GMT</pubDate>
            <atom:updated>2024-04-26T13:51:27.749Z</atom:updated>
            <content:encoded><![CDATA[<p>For non-native Rust developers coming from another language (like myself!), Rust’s <em>lifetime</em> terminology can sometimes be challenging. I would say one of the most essential and/or basic (<em>lifetime</em>-related) keyword is &#39;static.</p><blockquote>&#39;static keyword? Piece of cake!</blockquote><p>Yes, certainly. Then, can you explain the difference between the following two examples of the usage of the &#39;static keyword?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b32bc930281c690c361b3155daef6dc7/href">https://medium.com/media/b32bc930281c690c361b3155daef6dc7/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2092581cd24bd5047aaea50bd719229/href">https://medium.com/media/d2092581cd24bd5047aaea50bd719229/href</a></iframe><p>Both are related but subtly different and this is a common source for confusion when learning Rust. The former is a<em> </em><strong><em>reference</em> <em>lifetime</em></strong>, whereas the latter is a <strong><em>trait bound</em></strong>.</p><blockquote>Hmm, the first one is simple and easy to explain but what does the second example actually mean?</blockquote><p>This blog was written for those kinds of people. If you can’t answer accurately, please give it a read.</p><h3>&#39;static as a reference lifetime</h3><p>This is fairly simple and easy to explain. As a <em>reference lifetime</em> &#39;static indicates that the data pointed to by the reference lives for the remaining <em>lifetime</em> of the running program. It can still be coerced to a shorter <em>lifetime</em>.</p><p>Examples of data with &#39;static <em>lifetime</em> are as follows:</p><ul><li>literals ( 1, &quot;abc” , … )</li><li>constants ( const X: i32 = 5, … )</li><li>global variables ( static X: i32 = 5 , … )</li></ul><h3>‘static as a trait bound</h3><p>This might be a bit more complicated than you expect. In <em>generics</em>, <em>trait bounds</em> are sometimes added, right? For example, T: Ord + Debug means, &quot;I don&#39;t care what type T is, but it must implement the Ord and Debug <em>traits</em>.&quot; Similarly, you can add <em>lifetime bounds</em> to your <em>traits</em> and specify their <em>lifetime</em> constraints.</p><p>First, I’ll mention the practically important points rather than strict details. <strong>If type </strong><strong>T has a </strong><strong>&#39;static <em>lifetime bound</em>, it generally implies that </strong><strong>T should not contain <em>references</em> (if </strong><strong>T is a <em>struct</em>, <em>enum</em>, <em>vector</em>, etc., its contents should also not be <em>references</em>). This is the most common usage.</strong></p><p>To be a bit more precise, if T: &#39;static, it means one of the following:</p><ul><li>T is a <em>scalar type</em> value. (e.g., T &lt;- 123)</li><li>T is a value of a <em>compound type</em> (such as <em>struct</em>, <em>enum</em>, <em>vector</em>, <em>array</em>, etc.) that has an accessible internal data without <em>references</em>. (e.g., T &lt;- struct S(u32), enum E { V1(u32), V2(i32)}, T &lt;- Vec&lt;u32&gt;)</li><li>T is a value of a <em>compound type</em> with an internal data that includes <em>references</em> with the &#39;static lifetime. (e.g., T &lt;- struct S(u32, &amp;&#39;static u32), T &lt;- Vec&lt;&amp;&#39;static str&gt;)</li><li>T is a <em>reference</em> with the &#39;static <em>lifetime</em> to any of the above values. (e.g., T &lt;- &amp;&#39;static 123, T &lt;- &amp;&#39;static S(u32))</li></ul><h3>Comprehension test</h3><p>Well, here is a comprehension test! Which will cause a compile error? Please select all options from (1) through (4):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/17a5e87109a53bd42ff76c8b963af614/href">https://medium.com/media/17a5e87109a53bd42ff76c8b963af614/href</a></iframe><p>🦀彡<br>ミ🦀<br>🦀彡<br>ミ🦀<br>🦀彡<br>ミ🦀<br>🦀彡<br>ミ🦀<br>🦀彡<br>ミ🦀</p><p><strong>The answer is only (4).</strong></p><p>You see that (1) and (2) can be accepted as T for generic. The former passes a <em>struct</em> containing only values, while the latter passes a <em>struct</em> containing <em>references</em> with the &#39;static <em>lifetime</em>.</p><p>It might be somewhat surprising that (3) does not result in a compilation error. When you don’t specify the <em>lifetime</em> of a <em>reference</em> passed as an argument during a function call, it gets allocated the maximum possible <em>lifetime</em>. In this case, since we’re specifying the <em>literal </em><em>&quot;abc”</em>, its value lives as long as the program runs. Therefore, it gets allocated the maximum &#39;static <em>lifetime</em>.</p><p>(4) is passing a <em>reference</em> to a value, x, which dies within the inner scope enclosed by {}. Since the <em>lifetime</em> of this <em>reference</em> is shorter than the &#39;static <em>lifetime</em>, it cannot be passed to generic, which requires the &#39;static <em>lifetime boundary</em>.</p><h3>Background of the Article</h3><p>As part of my review of my Rust knowledge, I stumbled upon the following thread on <a href="https://users.rust-lang.org/">users.rust-lang.org</a>:</p><p><a href="https://users.rust-lang.org/t/confusion-about-impl-lifetimes-in-return-values/64734">https://users.rust-lang.org/t/confusion-about-impl-lifetimes-in-return-values/64734</a></p><p>After reviewing my knowledge, I came up with the idea of writing a blog post about the ‘static keyword.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f244d3014ff3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenTableHub: Data Sharing Platform #1]]></title>
            <link>https://medium.com/@ognis1205/deltahub-data-sharing-platform-1-bdee60e74245?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdee60e74245</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[delta-lake]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[delta-sharing]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Wed, 29 Nov 2023 20:42:56 GMT</pubDate>
            <atom:updated>2023-12-31T04:03:58.043Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="DeltaHub Logo" src="https://cdn-images-1.medium.com/max/256/1*vZd_0TkvJg8D9qojEY32Nw.png" /></figure><p>I am currently working on a project named OpenTableHub for a data-sharing social networking service platform. In this series, I would like to share the design and implementation progress, challenges faced, and their solutions while implementing OpenTableHub with everyone. In this post, I would like to discuss the overview of the entire platform, its architecture, and future plans associated with it.</p><h4>1. Screenshot of the Mockup</h4><figure><img alt="Screenshot" src="https://cdn-images-1.medium.com/max/1024/1*wMbfOPXIyKISKtJjLr2u7w.gif" /></figure><h4>2. Overall Architecture</h4><p>In this section, after presenting the overall architecture, I would like to explain the workflow from the perspectives of both data providers and data recipients. Additionally, I aim to highlight the differences in implementation compared to the current <a href="https://github.com/delta-incubator/delta-sharing-rs">delta-sharing-rs</a>. Please refer to the <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md">official documentation</a> for detailed information on the Delta Sharing protocol.</p><figure><img alt="Architecture" src="https://cdn-images-1.medium.com/max/1024/1*7UFflrW3CW0dG04KuQCvfg.png" /></figure><p>The portion enclosed by the red rectangle on the left corresponds to the front end and BFF (Backend For Frontend) of the platform, implemented in the <a href="https://github.com/ognis1205/delta-hub-ts">delta-hub-ts</a> project using <a href="https://www.typescriptlang.org/">TypeScript</a>/<a href="https://nextjs.org/">Next.js</a>/<a href="https://chakra-ui.com/">Chakra UI</a>. On the right, the portion enclosed by the red rectangle corresponds to the backend of the project, i.e., <a href="https://github.com/delta-incubator/delta-sharing-rs">delta-sharing-rs</a> implemented in <a href="https://www.rust-lang.org/">Rust</a>/<a href="https://docs.rs/axum/latest/axum/">axum</a>/<a href="https://github.com/delta-io/delta-rs">delta-rs</a>.</p><h4>2.1. Provider Workflow</h4><p>The workflow of the provider is shown in the diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qc3bp2GG-F2fV9boiRooDg.png" /></figure><p><strong>Initiating Data Provision</strong></p><p><strong>(1) </strong>The data provider logs into the platform via <a href="https://oauth.net/2/">OAuth2.0</a> to initiate data provision. The data provider needs to prepare a dedicated bucket and assign the appropriate role to the cloud service account provided in advance by the <a href="https://github.com/ognis1205/delta-hub-ts">DeltaHub</a> platform (e.g., using permission delegation via AssumeRole for AWS). The details of the implementation are not yet determined, but currently, the plan is to add a simple object store crawler. This crawler will automatically detect all <a href="https://delta.io/">Delta tables</a> on the dedicated bucket and consider them as candidates for the catalog.</p><p><strong>Data Selection and Configuration</strong></p><p>In order to initiate data sharing, the data provider needs to create a data catalog for publication. The data catalog consists of multiple schemas logically grouping data tables. Tasks such as creating the data catalog, retrieving its metadata, and setting valid sharing periods are performed through the frontend API. <strong>(2)</strong> First, the frontend API is called, and <strong>(2&#39;)</strong> <a href="https://oauth.net/2/token-introspection/">OAuth 2.0 Token Introspection</a> is executed. If the token is valid, <strong>(3)</strong> it is reverse-proxied to the backend API, and <strong>(4)</strong> the <a href="https://www.databricks.com/dataaisummit/session/delta-kernel-simplifying-building-connectors-delta/">Delta Kernel API</a> is invoked by <a href="https://github.com/delta-io/delta-rs">delta-rs</a>. <strong>(5)(6)(7)</strong> After the corresponding instructions are executed, responses are returned.</p><h4>2.2. Recipient Workflow</h4><p>The workflow of the recipient is shown in the diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oCrZhOfSphJHMCjIerQfrg.png" /></figure><p><strong>Initiating Data Browsing</strong></p><p><strong>(1) </strong>The data recipient logs into the platform via <a href="https://oauth.net/2/">OAuth2.0</a> to initiate data browsing. To utilize the platform as a data recipient, all that is required is a social account.</p><p><strong>Catalog Selection and Request</strong></p><p>The data recipient <strong>(2)</strong> sends a request to the frontend and searches for the desired catalog from the list provided by the data provider. After finding the catalog they want to share, <strong>(2)</strong> they call the frontend API to <strong>(3)</strong> proxy the backend and <strong>(4)</strong> send a sharing request email to the data provider from the SMTP host. Upon obtaining permission from the data provider, <strong>(5)</strong> they acquire the <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#profile-file-format">Delta Sharing profile</a> via email.</p><p><strong>Share via Connectors</strong></p><p>In <a href="https://delta.io/">Delta Lake OSS</a>, numerous <a href="https://github.com/delta-io/delta-sharing">Delta Sharing connectors</a> are available, accessible not only from <a href="https://www.python.org/">Python</a> but also from <a href="https://www.microsoft.com/en-us/power-platform/products/power-bi">Power BI</a> or <a href="https://www.microsoft.com/en-us/microsoft-365/excel">Excel</a>. <strong>(6)</strong> The data recipient invokes the frontend API, proxying to the backend REST API implementing the <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md">Delta Sharing Protocol</a>. <strong>(7)</strong> Requests reverse-proxied to the backend API, similar to the data provider’s workflow, <strong>(8)</strong> are processed by <a href="https://github.com/delta-io/delta-rs">delta-rs</a>, which calls the <a href="https://www.databricks.com/dataaisummit/session/delta-kernel-simplifying-building-connectors-delta/">Delta Kernel API</a>, creates a presigned URL for the table on the corresponding cloud object storage, <strong>(9)(10) </strong>and returns it as a response. <strong>(11)(12)</strong> By requesting the obtained presigned URL, the data recipient can retrieve the shared data.</p><h4>3. Conclusion</h4><p>In this post, I have shared an overview of <a href="https://github.com/ognis1205/delta-hub-ts">DeltaHub</a>’s architecture and workflow. Even from this overview, it is evident that there are features that differ from or are not present in the existing <a href="https://github.com/delta-incubator/delta-sharing-rs">delta-sharing-rs</a>, such as authentication flows, search functionality, email features, and more. In the next post, I plan to share the record of adding and modifying these features on an issue-by-issue basis.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdee60e74245" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[delta-rs as a Python/Rust FFI example Part 2]]></title>
            <link>https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-2-f0cb386a04bd?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/f0cb386a04bd</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[delta-lake]]></category>
            <category><![CDATA[pyo3]]></category>
            <category><![CDATA[maturin]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Tue, 13 Jun 2023 17:55:57 GMT</pubDate>
            <atom:updated>2023-06-14T20:48:59.167Z</atom:updated>
            <content:encoded><![CDATA[<h4>0.Summary of the Previous Episode</h4><p>In the previous episode, I explained the concept of the <a href="https://en.wikipedia.org/wiki/Foreign_function_interface"><strong>FFI</strong></a>, which is an essential component for creating bindings to foreign languages. Let’s first review what was covered in the previous episode:</p><ul><li>A language-level interface is known as an <a href="https://en.wikipedia.org/wiki/API"><strong>API</strong></a>.</li><li>An <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>ABI</strong></a><strong> </strong>specifies the rules for interaction between different object files.</li><li>An <a href="https://en.wikipedia.org/wiki/Foreign_function_interface"><strong>FFI</strong></a><strong> </strong>is a mechanism that allows a program written in one programming language to call routines or utilize services written or compiled in another language.</li></ul><p>In this blog post, let’s delve into the <a href="https://www.rust-lang.org/">Rust</a> ecosystem and examine its characteristics. Additionally, we’ll explore how the <a href="https://github.com/PyO3/pyo3">PyO3</a> <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crate</a> offers a straightforward example of managing <a href="https://www.python.org/">Python</a>/<a href="https://www.rust-lang.org/">Rust</a> <a href="https://en.wikipedia.org/wiki/Foreign_function_interface">FFI</a> functionality. I will demonstrate how to examine the generated “glue code” produced by the <a href="https://github.com/PyO3/pyo3">PyO3</a> <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crate</a>.</p><h4>1.Rust Ecosystem</h4><p>If you are not familiar with <a href="https://www.rust-lang.org/">Rust</a> development, please install <a href="https://www.rust-lang.org/tools/install">rustup</a> first. The <a href="https://www.rust-lang.org/tools/install">rustup</a> is a toolchain installer that helps you install The <a href="https://www.rust-lang.org/">Rust</a> Programming Language from the official release channels. The following is a brief explanation of Rust development. You can skip this section if you are already familiar with it. If you are using <a href="https://en.wikipedia.org/wiki/Linux">Linux</a> or <a href="https://en.wikipedia.org/wiki/MacOS">macOS</a>, open a terminal and enter the following command:</p><pre>$ curl --proto &#39;=https&#39; --tlsv1.2 https://sh.rustup.rs -sSf | sh<br>$ rustc --version<br>rustc 1.67.0 (fc594f156 2023-01-24)</pre><h4>Cargo</h4><p><a href="https://doc.rust-lang.org/cargo/">Cargo</a> is <a href="https://www.rust-lang.org/">Rust</a>’s build system and package manager. Most Rustaceans use this tool to manage <a href="https://www.rust-lang.org/">Rust</a> projects. <a href="https://doc.rust-lang.org/cargo/">Cargo</a> handles tasks such as building code, downloading dependencies (libraries that code depends on, referred to as “dependencies”), and building those libraries. It takes care of many tasks related to managing <a href="https://www.rust-lang.org/">Rust</a> projects. If you are using the official <a href="https://www.rust-lang.org/tools/install">rustup</a> installer in the previous section, <a href="https://doc.rust-lang.org/cargo/">Cargo</a> will be automatically installed for you:</p><pre>$ cargo --version<br>cargo 1.67.0 (8ecd4f20a 2023-01-10)</pre><h4>cargo-expand</h4><p><a href="https://github.com/dtolnay/cargo-expand">cargo-expand</a> is a subcommand that allows you to view the results of <a href="https://rustc-dev-guide.rust-lang.org/macro-expansion.html">macro expansion</a> in <a href="https://www.rust-lang.org/">Rust</a> code, which will be used in this blog post to explore the generated “glue code” produced by the <a href="https://github.com/PyO3/pyo3">PyO3</a> <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crate</a>. To install <a href="https://github.com/dtolnay/cargo-expand">cargo-expand</a>, open a terminal and enter the following command:</p><pre>$ cargo install cargo-expand</pre><h4>2.Rust Compiler Train Schedule</h4><p><a href="https://www.rust-lang.org/">Rust</a> development operates on a train schedule. That is, all development is done on the master branch of the <a href="https://www.rust-lang.org/">Rust</a> repository. Releases follow a <a href="https://medium.com/tiket-com/release-train-in-software-development-6794fb1939f1">software release train model</a>, which has been used by <a href="https://en.wikipedia.org/wiki/Cisco_IOS">Cisco IOS</a> and other software projects. There are three release channels for Rust:</p><ul><li>Nightly</li><li>Beta</li><li>Stable</li></ul><p>To execute cargo-expand, you need the nightly compiler. If you haven&#39;t installed it yet, please use rustup to install the toolchain.</p><pre>$ rustup toolchain install nightly</pre><h4>3.Simple PyO3 Project</h4><p>Putting the long introduction aside, let’s create a simple <a href="https://github.com/PyO3/pyo3">PyO3</a> project to explore the “glue code.” To introduce <a href="https://github.com/PyO3/pyo3">PyO3</a>, it is convenient to use a command called <a href="https://github.com/PyO3/maturin">maturin</a> which provides functionality for building and publishing <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crates</a> with <a href="https://github.com/PyO3/pyo3">PyO3</a> and other <a href="https://en.wikipedia.org/wiki/Foreign_function_interface">FFI</a> bindings, as well as <a href="https://www.rust-lang.org/">Rust</a> binaries as <a href="https://www.python.org/">Python</a> packages. First, install <a href="https://github.com/PyO3/maturin">maturin</a> using pip:</p><pre>$ python -m pip install maturin</pre><p>When you execute maturin new, it generates a <a href="https://www.python.org/">Python</a> package. Let’s create a package called my-first-pyo3 as an example:</p><pre>$ maturin new my-first-pyo3</pre><p>After executing the command, you will be prompted with the following questions:</p><pre>Which kind of bindings to use? ›</pre><p>If you select pyo3 at this point, a project utilizing <a href="https://github.com/PyO3/pyo3">PyO3</a> will be created:</p><pre>✨ Done! New project created my-first-pyo3</pre><p>Once the command is complete, a directory named my-first-pyo3 will be created, and the following files will be placed within it:</p><pre>$ tree ./my-first-pyo3<br>./my-first-pyo3<br>├── Cargo.toml<br>├── pyproject.toml<br>└── src<br>    └── lib.rs<br><br>2 directories, 3 files</pre><p>Cargo.toml contains metadata and dependencies for the <a href="https://www.rust-lang.org/">Rust</a> project:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/20b54d707fac31b14e00d482b764d680/href">https://medium.com/media/20b54d707fac31b14e00d482b764d680/href</a></iframe><p>pyproject.toml is a <a href="https://toml.io/en/">TOML</a> file that specifies build configurations for the <a href="https://www.python.org/">Python</a> project, including the <a href="https://github.com/PyO3/pyo3">PyO3</a> <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crate</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/752fbbb92aa8111be5176bcd89217fde/href">https://medium.com/media/752fbbb92aa8111be5176bcd89217fde/href</a></iframe><p>src/lib.rs contains the implementation of the <a href="https://github.com/PyO3/pyo3">PyO3</a> based functionality. You can modify this file to define your desired behavior:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6530256db18fd7a53d49b6beb420be9/href">https://medium.com/media/f6530256db18fd7a53d49b6beb420be9/href</a></iframe><p>Let me provide some explanation regarding the code mentioned above. A package named my_first_pyo3 has been created, which offers a <a href="https://www.rust-lang.org/">Rust</a> function called sum_as_string for use in <a href="https://www.python.org/">Python</a>.</p><p>What I would like to draw your attention to is the [#pyfunction] and [#pymodule] declarations in lines <a href="https://gist.github.com/ognis1205/87da134b80cf1f9b64e2725c32c5f8af#file-lib-rs-L4">4</a> and <a href="https://gist.github.com/ognis1205/87da134b80cf1f9b64e2725c32c5f8af#file-lib-rs-L10">10</a>. These are features known as <a href="https://doc.rust-lang.org/reference/procedural-macros.html">procedural macros</a> in <a href="https://www.rust-lang.org/">Rust</a>. A macro is a mechanism that generates code from code. <a href="https://doc.rust-lang.org/reference/procedural-macros.html">Procedural macros</a> generate code through the following steps:</p><ol><li>The <a href="https://www.rust-lang.org/">Rust</a> source code is parsed into an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> using the <a href="https://crates.io/crates/syn">syn</a> <a href="https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html">crate</a>.</li><li>The relevant information is extracted from the <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> to assemble the desired code.</li></ol><p>The principle is quite simple. If you have experience creating <a href="https://babeljs.io/docs/plugins">Babel</a> plugins or have used the <a href="https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API">TypeScript Compiler API</a>, you can think of it as performing similar tasks.</p><p>Let’s try running sum_as_string as a test. You can accomplish this by executing maturin develop. To utilize maturin develop, it is necessary to have either <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a> or <a href="https://docs.conda.io/en/latest/">conda</a> installed. In this case, let’s proceed with <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a>. Let’s create a <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a> named venv:</p><pre>$ python -m venv venv<br>$ . venv/bin/activate</pre><p>After running maturin develop, a <a href="https://pypi.org/project/wheel/">wheel</a> will be created, and the my_first_pyo3 package will be installed in the venv environment:</p><pre>$ maturin develop<br>    Updating crates.io index<br>🔗 Found pyo3 bindings<br>🐍 Found CPython 3.8 at /Users/shin/.local/share/virtualenvs/pyo3-playground-AibpzOrO/bin/python<br>📡 Using build options features from pyproject.toml<br>💻 Using `MACOSX_DEPLOYMENT_TARGET=10.7` for x86_64-apple-darwin by default<br>   Compiling target-lexicon v0.12.7<br>   Compiling autocfg v1.1.0<br>   Compiling proc-macro2 v1.0.60<br>   Compiling once_cell v1.18.0<br>   Compiling libc v0.2.146<br>   Compiling unicode-ident v1.0.9<br>   Compiling quote v1.0.28<br>   Compiling syn v1.0.109<br>   Compiling pyo3-build-config v0.18.3<br>   Compiling lock_api v0.4.10<br>   Compiling parking_lot_core v0.9.8<br>   Compiling memoffset v0.8.0<br>   Compiling smallvec v1.10.0<br>   Compiling cfg-if v1.0.0<br>   Compiling scopeguard v1.1.0<br>   Compiling parking_lot v0.12.1<br>   Compiling indoc v1.0.9<br>   Compiling unindent v0.1.11<br>   Compiling pyo3-ffi v0.18.3<br>   Compiling pyo3 v0.18.3<br>   Compiling pyo3-macros-backend v0.18.3<br>   Compiling pyo3-macros v0.18.3<br>   Compiling my-first-pyo3 v0.1.0 (/Users/shin/Documents/github/pyo3-playground/my-first-pyo3)<br>    Finished dev [unoptimized + debuginfo] target(s) in 33.47s<br>📦 Built wheel for CPython 3.8 to /var/folders/90/kskh3wbj04d1cp3_crkb9cgr0000gn/T/.tmp3y52td/my_first_pyo3-0.1.0-cp38-cp38-macosx_10_7_x86_64.whl<br>🛠 Installed my-first-pyo3-0.1.0</pre><p>If you try executing sum_as_string, you will obtain the following output:</p><pre>&gt;&gt;&gt; import my_first_pyo3<br>&gt;&gt;&gt; my_first_pyo3.sum_as_string(100, 1)<br>&#39;101&#39;</pre><h4>4.What does PyO3 really do?</h4><p><a href="https://github.com/PyO3/pyo3">PyO3</a> generates glue code for exposing to <a href="https://www.python.org/">Python</a> when using macros like [#pyfunction] or [#pymodule]. You can view the generated code by using cargo-expand. Let’s give it a try and see it in action:</p><pre>$ cargo expand &gt; expand.rs<br>   Compiling pyo3-build-config v0.18.3<br>    Checking libc v0.2.146<br>    Checking smallvec v1.10.0<br>    Checking cfg-if v1.0.0<br>    Checking scopeguard v1.1.0<br>    Checking lock_api v0.4.10<br>    Checking memoffset v0.8.0<br>    Checking unindent v0.1.11<br>    Checking parking_lot_core v0.9.8<br>    Checking parking_lot v0.12.1<br>   Compiling pyo3-ffi v0.18.3<br>   Compiling pyo3 v0.18.3<br>    Checking my-first-pyo3 v0.1.0 (/Users/shin/Documents/github/pyo3-playground/my-first-pyo3)<br>    Finished dev [unoptimized + debuginfo] target(s) in 10.66s</pre><p>And the resulting expanded code will appear as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/261b2b07cd98af59684a791d22cf7821/href">https://medium.com/media/261b2b07cd98af59684a791d22cf7821/href</a></iframe><p>I understand what you feel right now. The generated code can be quite difficult to read with all the jargon. I’ve been through that as well. Let’s try to decipher the code a bit here. By doing so, we can get some hints about what <a href="https://github.com/PyO3/pyo3">PyO3</a> is doing.</p><p>First, let’s focus on line <a href="https://gist.github.com/ognis1205/b3939398213bad2c46ae733f59c786ae#file-expand-rs-L12">12</a>. This is where the definition of the function we want to provide to Python appears. In other words, the <a href="https://doc.rust-lang.org/reference/procedural-macros.html">procedural macros</a> has embedded code fragments here.</p><p>Next, let’s pay attention to the const _: = {...} statements on lines <a href="https://gist.github.com/ognis1205/b3939398213bad2c46ae733f59c786ae#file-expand-rs-L16">16</a> and <a href="https://gist.github.com/ognis1205/b3939398213bad2c46ae733f59c786ae#file-expand-rs-L104">104</a>. This is a commonly used technique where the const declaration ensures that the <a href="https://en.wikipedia.org/wiki/Value_(computer_science)">r-values</a> is evaluated at compile time. As you can see from the following {...} block, <a href="https://github.com/PyO3/pyo3">PyO3</a> wraps the sum_as_string function with a <a href="https://github.com/PyO3/pyo3">PyO3</a> wrapper object to convert it into a <a href="https://docs.python.org/3/c-api/index.html">CPython (PyPy) ABI</a> compatible binary at compile time.</p><h4>3.Conclusion</h4><p>Thank you for reading this far. I hope you found this (possibly long and tedious) article that explores the <a href="https://github.com/PyO3/pyo3">PyO3</a> “glue code” informative and enjoyable. In my next blog post, I will explain the workflow of building with <a href="https://github.com/PyO3/maturin">maturin</a>, and finally, I would like to explain using the example repository of an actual project how <a href="https://github.com/PyO3/pyo3">PyO3</a>/<a href="https://github.com/PyO3/maturin">maturin</a> is utilized in <a href="https://github.com/delta-io/delta-rs">delta-rs</a>, which is probably of most interest to you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f0cb386a04bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[delta-rs as a Python/Rust FFI example Part 1]]></title>
            <link>https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-1-b11f53b40f9c?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/b11f53b40f9c</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[delta-lake]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Thu, 01 Jun 2023 09:31:18 GMT</pubDate>
            <atom:updated>2023-06-13T17:56:28.821Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>0.delta-rs</strong></h3><p>As stated by the official <a href="https://github.com/delta-io/delta-rs">repository</a>, the <a href="https://github.com/delta-io/delta-rs">delta-rs</a> is a library providing low level access to <a href="https://delta.io/">Delta</a> tables in <a href="https://www.rust-lang.org/">Rust</a>, which can be used with data processing frameworks like <a href="https://github.com/apache/arrow-datafusion">datafusion</a>, <a href="https://github.com/apache/arrow-datafusion/tree/master/ballista">ballista</a>, <a href="https://github.com/pola-rs/polars">polars</a>, <a href="https://github.com/rajasekarv/vega">vega</a>, etc. In the documentation, the following statement intrigues me a lot:</p><blockquote>It also provides bindings to other higher level language Python.</blockquote><p>What does it really mean? And how is it achieved?</p><h3><strong>1.API and ABI</strong></h3><p>Before delving into the <a href="https://en.wikipedia.org/wiki/Foreign_function_interface"><strong>Foreign Function Interface (FFI)</strong></a>, which is an essential component of creating bindings for foreign languages, let’s first examine linking using a simple example in the C language.</p><p>Suppose you are writing the following code. The a_function is defined in a_function.c, and you are trying to call it from the main function residing in main.c.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76134bec4292b545e7f4efb9f354683b/href">https://medium.com/media/76134bec4292b545e7f4efb9f354683b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/75dc808f1dc6bae5631093c1c9e06d7b/href">https://medium.com/media/75dc808f1dc6bae5631093c1c9e06d7b/href</a></iframe><p>In the C language, you can link these two separate files by using a build system or a compilation command. Here’s an example of how you can compile and link the files main.c and a_function.c using the <a href="https://gcc.gnu.org/">GCC</a> compiler:</p><pre>$ gcc -c a_function.c<br>$ gcc -c main.c<br>$ gcc main.o a_function.o<br>$ ./a.out<br>This is a_function!</pre><p>By linking the files, the system starts the execution from the main function. The program flow then transfers to the a_function, which calls printf. After executing the necessary operations in a_function, the control returns to the main function again.</p><p>In the C language, the declaration void a_function(); in the main.c file serves as a forward declaration or function prototype. It informs the compiler about the existence of a function named a_function that takes no arguments and returns void. This declaration allows the compiler to correctly handle the function call to a_function() within the main function. This language-level interface like this is called <a href="https://en.wikipedia.org/wiki/API"><strong>Application Programming Interface (API)</strong></a>.</p><p>At this level, regardless of whether C language is compiled using a <a href="https://www.linux.org/">Linux</a> or <a href="https://www.microsoft.com/en-us/windows">Windows</a> processing system, the <a href="https://en.wikipedia.org/wiki/API"><strong>API</strong></a> has portability. However, it becomes a concept within the C language, so it cannot be called from other languages.</p><p>On the other hand, in <a href="https://www.linux.org/">Linux</a>/<a href="https://gcc.gnu.org/">GCC</a>, this declaration will result in the symbol a_function being included in the object file main.o as an undefined symbol. It is during the linking phase that the method of calling a_function is determined. In the case of a shared library, the dynamic linker <a href="https://linux.die.net/man/8/ld-linux.so">ld-linux.so</a> is responsible for locating it at runtime.</p><p>This is where the <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>Application Binary Interface (ABI)</strong></a> comes into play. It defines how to access the a_function function from a_function.o within the machine code (e.g., <a href="https://en.wikipedia.org/wiki/X86-64">x86_64</a> or <a href="https://en.wikipedia.org/wiki/AArch64">aarch64</a>) present in main.o. The <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>ABI</strong></a> specifies the rules for interacting with functions across different object files, ensuring proper communication and interoperability.</p><h3><strong>2.Foreign Function Interface</strong></h3><p>Summarizing the above, the diagram below illustrates the relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-sp-r_3DAQFwD-7fZWvzsg.png" /></figure><p>A <a href="https://en.wikipedia.org/wiki/Foreign_function_interface"><strong>Foreign Function Interface (FFI)</strong></a><strong> </strong>is a mechanism by which a program written in one programming language can call routines or make use of services written or compiled in another one. An FFI is often used in contexts where calls are made into <a href="https://en.wikipedia.org/wiki/Dynamic-link_library">binary dynamic-link library</a>.</p><p>The noteworthy characteristic of an <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>ABI</strong></a> is its machine-specific but language-agnostic nature. In theory, once source code is compiled into a binary, it can be called by any programming language through the operating system’s <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>ABI</strong></a>. This means that as long as the <a href="https://en.wikipedia.org/wiki/Application_binary_interface"><strong>ABI</strong></a><strong> </strong>is adhered to, different programming languages can interact with the compiled binary without being restricted to the language in which the code was originally written. Hence, the statement in the <a href="https://github.com/delta-io/delta-rs">delta-rs</a> project:</p><blockquote>It also provides bindings to other higher level language Python.</blockquote><p>implies that the crate incorporates Python bindings using the Python <a href="https://en.wikipedia.org/wiki/Foreign_function_interface"><strong>Foreign Function Interface (FFI)</strong></a> provided by the third-party library <a href="https://github.com/PyO3/pyo3">PyO3</a>.</p><h3>3.Conclusion</h3><p>Thank you for reading this far. I hope you enjoyed this short article that explains the fundamental concepts behind the <a href="https://github.com/delta-io/delta-rs">delta-rs</a> Python bindings feature. In my <a href="https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-2-f0cb386a04bd">next blog post</a>, I will share some <a href="https://www.rust-lang.org/">Rust</a> code snippets from my playground repository, specifically focusing on illustrating the structure of the “glue code” generated by <a href="https://github.com/PyO3/pyo3">PyO3</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b11f53b40f9c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to deploy self-hosted Delta Sharing server on AWS]]></title>
            <link>https://medium.com/@ognis1205/how-to-deploy-self-hosted-delta-sharing-server-on-aws-7aa2a7f6f46?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/7aa2a7f6f46</guid>
            <category><![CDATA[rustlang]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[delta-lake]]></category>
            <category><![CDATA[delta-sharing]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Wed, 17 May 2023 00:13:33 GMT</pubDate>
            <atom:updated>2023-05-17T00:13:33.014Z</atom:updated>
            <content:encoded><![CDATA[<p>I have been asked by several people, via LinkedIn message, how to deploy <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing</a> on cloud computing platforms such as AWS, GCP, and Azure. Therefore, I am providing a step-by-step guide on how to deploy your <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing</a> server with HTTPS enabled in an AWS environment.</p><h4><strong>System requirements and the target architecture</strong></h4><p>The system requirements for <a href="https://delta.io/sharing/">Delta Sharing</a> are minimal, and a cluster is not required for its deployment. The <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing</a> server acts as a control plane and is responsible for <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#concepts">provider</a> and <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#concepts">recipient</a> authorization, as well as managing <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#concepts">share</a> information.</p><p>One of the key features of <a href="https://delta.io/sharing/">Delta Sharing</a> is secure and efficient data sharing, and therefore, the <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md">protocol</a> assumes that the server is deployed on HTTPS. As I mentioned earlier, the <a href="https://delta.io/sharing/">Delta Sharing</a> server itself is just a control plane. It does not require any infrastructure other than a database server for managing authorization and <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#concepts">share</a> information.</p><p>Hence, the requirements and the target architecture is as follows:</p><ol><li>An EC2 instance with HTTPS traffic enabled</li><li>An RDS instance for authorization and <a href="https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#concepts">share</a> management</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WrRnTwT9PIDuSLipA1KuIw.png" /></figure><h4>Terraform files for provisioning</h4><p>Don’t worry, I’ve already prepared <a href="https://www.terraform.io/">Terraform</a> files for you! All you need to do is clone the code from <a href="https://github.com/kotosiro/sharing/tree/main/examples/aws">here</a> and follow the instructions below.</p><h4>Acquiring your domain</h4><p>Before proceeding with the instructions, you will need to acquire your own domain name to enable HTTPS. We purchased one from <a href="https://aws.amazon.com/route53/">Amazon Route 53</a>, but you can also find another (free sub)domain registry, such as <a href="https://netfy.domains/">Netfy</a>.</p><h4>Provisioning AWS infrastructure</h4><p>After cloning the code from <a href="https://github.com/kotosiro/sharing/tree/main/examples/aws">the provided link</a>, you can provision the AWS infrastructure by running the following commands in the corresponding directory:</p><pre> $ terraform init<br> $ terraform apply</pre><p>You need to configure at least the following Terraform variables in terraform.tfvars:</p><pre>name                = &quot;kotosiro&quot;<br>region              = &quot;your-aws-region&quot;<br>availability_zone_1 = &quot;your-aws-availability-zone-a&quot;<br>availability_zone_2 = &quot;your-aws-availability-zone-b&quot;<br>ssh_cidr_blocks     = [&quot;0.0.0.0/0&quot;]<br>domain              = &quot;your.domain.org&quot;</pre><p>Please note that ssh_cidr_blocks must be configured to only accept ssh connections from proper IP addresses for the EC2 instances. To configure other variables, such as the Postgres admin password, please refer to the corresponding <a href="https://github.com/kotosiro/sharing/tree/main/examples/aws/module">Terraform modules</a> and set the preferred values as you wish.</p><h4>Deploying AWS credentials to EC2 instances</h4><p>This is not a great way to handle AWS credentials. However, to deploy your AWS credentials to the EC2 servers, you can follow these steps for each EC2 instance:</p><pre> $ chmod 400 creds/key.id_rsa<br> $ ssh -i creds/key.id_rsa ec2-user@YOUR_EC2_IP_ADDRESS<br> [ec2-user@ip-xxx-xxx-xxx-xxx ~]$ mkdir .aws<br> [ec2-user@ip-xxx-xxx-xxx-xxx ~]$ logout<br> $ scp -i creds/key.id_rsa path/to/aws/credentials ec2-user@YOUR_EC2_IP_ADDRESS:~/.aws/credentials</pre><h4>Setting Up Kotosiro Sharing server</h4><p>To configure the <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing </a>server, follow these steps for each EC2 instance:</p><pre>$ ssh -i creds/key.id_rsa ec2-user@YOUR_EC2_IP_ADDRESS<br> [ec2-user@ip-xxx-xxx-xxx-xxx ~]$ git clone https://github.com/kotosiro/docker-compose-files.git<br> [ec2-user@ip-xxx-xxx-xxx-xxx ~]$ cd docker-compose-files/sharing<br> [ec2-user@ip-xxx-xxx-xxx-xxx sharing]$ vi docker-compose.yaml</pre><p>The docker-compose.yaml file should look like this:</p><pre>version: &quot;3.9&quot;<br>services:<br>  app:<br>    image: kotosiro/sharing:latest<br>    restart: always<br>    environment:<br>      - AWS_SHARED_CREDENTIALS_FILE=/.aws/credentials<br>      - AWS_PROFILE=YOUR_AWS_PROFILE_NAME<br>      - AWS_REGION=YOUR_AWS_REGION<br>      - GOOGLE_APPLICATION_CREDENTIALS=/.gcp/service-account-file.json<br>      - KOTOSIRO_DB_URL=postgres://postgres:secretpassword@YOUR_AWS_RDS_ENDPOINT:5432/sharing<br>      - KOTOSIRO_SERVER_ADDR=https://your.domain.org<br>      - KOTOSIRO_SERVER_BIND=0.0.0.0:80<br>      - KOTOSIRO_ADMIN_NAME=kotosiro<br>      - KOTOSIRO_ADMIN_EMAIL=kotosiro@email.com<br>      - KOTOSIRO_ADMIN_PASSWORD=password<br>      - KOTOSIRO_ADMIN_NAMESPACE=kotosiro<br>      - KOTOSIRO_ADMIN_TTL=28800<br>      - KOTOSIRO_SIGNED_URL_TTL=28800<br>      - KOTOSIRO_JWT_SECRET=secret<br>      - KOTOSIRO_USE_JSON_LOG=true<br>      - KOTOSIRO_LOG_FILTER=warn,kotosiro_sharing=info<br>    ports:<br>      - 80:80<br>    volumes:<br>      - ~/.aws/credentials:/.aws/credentials<br>      - ~/.gcp/service-account-file.json:/.gcp/service-account-file.json<br>    command:<br>      server</pre><p>After finishing the configuration of your docker-compose.yaml file, run the following command to start the <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing</a> daemon:</p><pre>[ec2-user@ip-xxx-xxx-xxx-xxx sharing]$ docker-compose up -d</pre><p>I hope you have enjoyed this short journey and have seen how it is easy to deploy your own <a href="https://delta.io/sharing/">Delta Sharing</a> server. The official open protocol specification is available <a href="https://github.com/delta-io/delta-sharing">here</a>. I also welcome contributions to my <a href="https://github.com/kotosiro/sharing">Kotosiro Sharing</a> project. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7aa2a7f6f46" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My first post.]]></title>
            <link>https://medium.com/@ognis1205/my-first-post-bc23bd2682af?source=rss-97a02a587c73------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc23bd2682af</guid>
            <category><![CDATA[test]]></category>
            <dc:creator><![CDATA[Shingo OKAWA]]></dc:creator>
            <pubDate>Sun, 14 Aug 2022 21:08:17 GMT</pubDate>
            <atom:updated>2022-08-14T21:08:17.204Z</atom:updated>
            <content:encoded><![CDATA[<p>Test.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc23bd2682af" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>