<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by koray sariteke on Medium]]></title>
        <description><![CDATA[Stories by koray sariteke on Medium]]></description>
        <link>https://medium.com/@ksaritek?source=rss-e5e1af2488fb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*0Z_1E4rBd-h5AdY1qzXeFQ.jpeg</url>
            <title>Stories by koray sariteke on Medium</title>
            <link>https://medium.com/@ksaritek?source=rss-e5e1af2488fb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:31:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ksaritek/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Database from Scratch in Rust]]></title>
            <link>https://medium.com/@ksaritek/building-a-database-from-scratch-in-rust-04bd742368f0?source=rss-e5e1af2488fb------2</link>
            <guid isPermaLink="false">https://medium.com/p/04bd742368f0</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[koray sariteke]]></dc:creator>
            <pubDate>Sat, 31 Jan 2026 23:42:57 GMT</pubDate>
            <atom:updated>2026-01-31T23:42:57.078Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>This project follows Build Your Own Database from Scratch by James Smith, implemented in Rust. Not a wrapper around SQLite. Not an ORM. A database with B+Trees, memory-mapped files, transactions, and a SQL parser.</blockquote><p>This post covers what I learned and the key details.</p><p><a href="https://github.com/ksaritek/byodb">https://github.com/ksaritek/byodb</a></p><p><strong>## Why Build a Database?</strong></p><p>Databases are black boxes until you build one. I wanted to understand:</p><p>-How data is stored on disk<br>- How B+Trees work in practice<br>- How SQL gets executed<br>- How transactions maintain consistency</p><p><strong>## Architecture Overview</strong></p><pre>SQL Query<br>    |<br>    v<br>[Lexer] -&gt; [Parser] -&gt; [Executor]<br>                            |<br>                            v<br>                       [Table Layer]<br>                            |<br>                            v<br>                       [KV Store]<br>                            |<br>                            v<br>                       [B+Tree]<br>                            |<br>                            v<br>                       [Mmap File]</pre><p><strong>## The B+Tree</strong></p><p>The core data structure. Each node is a fixed-size page (4KB) containing.</p><p>- Header: node type, key count<br>- Pointers: child page numbers (internal nodes) or values (leaf nodes)<br>- Keys: sorted for binary search</p><p>Key insight: The tree uses copy-on-write. Modifications create new nodes instead of mutating existing ones. This simplifies crash recovery.</p><p><strong>## Memory-Mapped I/O</strong></p><p>Instead of read/write syscalls, the database maps the file directly into memory.</p><p>The OS handles paging data in and out. Writes go to the page cache and get flushed to disk asynchronously (or on explicit sync).</p><p><strong>## Free List</strong></p><p>Deleted pages need to be reused. The free list is a linked list of available page numbers, stored in the pages themselves.</p><p>When allocating, pop from the free list. When deallocating, push to it.</p><p><strong>## Transactions</strong></p><p>ACID transactions use a simple approach:</p><p>1. Buffer all writes in memory<br>2. On commit: write all pages, update master page, sync<br>3. On rollback: discard buffer</p><p>The master page contains the root pointer and is written last. If crash occurs before master page write, the old tree is still valid.</p><p><strong>## SQL Parser</strong></p><p>A recursive descent parser. Tokenize first, then parse.<br>Supports SELECT, INSERT, UPDATE, DELETE with WHERE clauses.</p><p><strong>## Query Execution</strong></p><p>The executor walks the AST and calls into the table layer.</p><ul><li>No query optimizer.</li><li>Full table scans.</li></ul><p><strong>## What’s Missing</strong></p><p>This is not production-ready:</p><p>- No WAL (write-ahead logging)<br>- No MVCC (multi-version concurrency control)<br>- No query optimizer<br>- No network protocol<br>- Limited SQL syntax</p><p>The goal was understanding, not building PostgreSQL :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=04bd742368f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lighting Up Rust on Bare Metal]]></title>
            <link>https://medium.com/@ksaritek/lighting-up-rust-on-bare-metal-ea992748f84b?source=rss-e5e1af2488fb------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea992748f84b</guid>
            <dc:creator><![CDATA[koray sariteke]]></dc:creator>
            <pubDate>Wed, 26 Nov 2025 12:27:05 GMT</pubDate>
            <atom:updated>2025-11-26T12:27:05.405Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Booting xv6 in Rust sounds exotic until you step through the pieces that actually make it work. The heart of this project is a clean-room `no_std` kernel that boots directly in QEMU using only rust-lld and a sprinkle of inline assembly (no GNU RISC-V toolchain, no C shim). In this opening post, we wire up the foundation: a linker script, an `_start` trampoline, and a Rust entry point that makes the first privileged decisions.</blockquote><p><strong>Why Rust for xv6?</strong></p><p>xv6 is a teaching operating system designed to illustrate the core concepts of OS design. The original is written in C, which gives you direct control but also plenty of footguns. Rust brings memory safety, expressive types, and modern tooling to this space without sacrificing the low-level control you need when talking to hardware.</p><p>More importantly, Rust’s `no_std` ecosystem is mature enough that we can build a complete kernel without pulling in libc or other runtime dependencies. Every instruction that runs is code we wrote or explicitly called. That’s the kind of clarity that makes learning OS internals actually enjoyable.</p><p><strong>The Toolchain Story</strong></p><p>The first win comes from Rust’s built-in RISC-V support. By targeting `riscv64gc-unknown-none-elf`, we get a freestanding binary without pulling in `newlib` or external GCC bits. The entire workflow lives inside `cargo build — release`. No cross-compiler setup, no PATH juggling, just Rust.</p><p>The linker script defines our boot layout. QEMU’s `-kernel` flag expects code at `0x80000000`, so we anchor everything there:</p><pre>OUTPUT_ARCH( &quot;riscv&quot; )<br>ENTRY( _start )<br><br>SECTIONS<br>{<br>  . = 0x80000000;<br><br>  .text : {<br>    *(.text.entry)<br>    *(.text .text.*)<br>    ...<br>  }<br>  <br>  .bss : {<br>    ...<br>    *(.bss.stack0)<br>  }<br><br>  PROVIDE(STACK0 = .);<br>}</pre><p>This gives us a predictable memory map: code at the entry point, read-only data, writable data, uninitialized BSS, and finally a stack arena that grows downward. The `STACK0` symbol marks where our per-hart stacks begin.</p><p><strong>The Assembly Trampoline</strong></p><p>When QEMU starts, it jumps to `0x80000000` in Machine mode. We need to set up a stack before Rust can do anything useful. That’s where our `_start` trampoline comes in:</p><pre>global_asm!(<br>    r#&quot;<br>    .section .text.entry<br>    .global _start<br>    _start:<br>        # Get hartid<br>        csrr a1, mhartid<br>        # Calculate stack offset: (hartid + 1) * 4096<br>        addi a1, a1, 1<br>        slli a0, a1, 12  # Multiply by 4096 (2^12)<br>        # Load STACK0 address<br>        la sp, STACK0<br>        # Add offset to base address<br>        add sp, sp, a0<br>        # jump to start() in start.rs<br>        call start<br>    spin:<br>        j spin<br>    &quot;#<br>);</pre><p>Each hardware thread (hart) gets its own 4KB stack. Hart 0 uses the first 4KB, hart 1 uses the next, and so on. This prevents stack collisions when multiple harts boot simultaneously. Once the stack pointer is set, we call into Rust’s `start()` function.</p><p><strong>Waking Up in Rust</strong></p><p>`start()` is where the kernel truly wakes up. This function runs in Machine mode and prepares the transition to Supervisor mode — the privilege level where most of the kernel will live. Rust functions manipulate Control Status Registers (CSRs) through tiny inline helpers:</p><pre>#[unsafe(no_mangle)]<br>pub extern &quot;C&quot; fn start() -&gt; ! {<br>    let mut x = read_mstatus();<br>    x &amp;= !MSTATUS_MPP_MASK;<br>    x |= MSTATUS_MPP_S;<br>    write_mstatus(x);<br><br>    write_mepc(main as u64);<br>    write_satp(0);<br>    write_medeleg(0xffff);<br>    write_mideleg(0xffff);<br>    write_sie(read_sie() | SIE_SEIE | SIE_STIE);<br>    <br>    write_pmpaddr0(0x3fffffffffffff);<br>    write_pmpcfg0(0xf);<br>    <br>    timerinit();<br>    <br>    let id = read_mhartid();<br>    write_tp(id);<br><br>    unsafe {<br>        core::arch::asm!(&quot;mret&quot;);<br>    }<br>    unreachable!()<br>}</pre><p>Let’s break down what’s happening here. We clear the Machine Previous Privilege (MPP) bits and set them to Supervisor mode. We seed `mepc` with the address of `main()`, which is where execution will resume after the privilege drop. We zero `satp` (no virtual memory yet), delegate exceptions and interrupts to Supervisor mode, enable Supervisor interrupts, and configure Physical Memory Protection (PMP) to allow access to all memory.</p><p>The timer initialization arms the hardware timer, and we stash the hart ID in the thread pointer register. Then comes the magic: `mret` drops us from Machine mode into Supervisor mode, and execution resumes at `main()`.</p><p><strong>The First Kernel Loop</strong></p><p>Right now, `main()` is about as exciting as it gets:</p><pre>#[unsafe(no_mangle)]<br>pub extern &quot;C&quot; fn main() -&gt; ! {<br>    loop {<br>        unsafe {<br>            core::arch::asm!(&quot;wfi&quot;);<br>        }<br>    }<br>}</pre><p>The `wfi` (Wait For Interrupt) instruction puts the CPU into a low-power state until an interrupt arrives. Our timer is armed, so interrupts will fire, but we don’t have a trap handler yet. That’s fine — the kernel boots, reaches Supervisor mode, and sits there waiting. It’s alive.</p><p><strong>What You Get</strong></p><p>By the end of this post, the kernel compiles and boots. You can run:</p><pre>make kernel<br>qemu-system-riscv64 -machine virt -kernel target/riscv64gc-unknown-none-elf/release/rxv6-riscv -nographic</pre><p>Watch the machine reach Supervisor mode, and know that every instruction up to that point is owned by your Rust code. No C runtime, no libc, no hidden initialization — just the kernel doing exactly what you told it to do.</p><p>This is the launchpad for everything else: UART output, trap handlers, allocators, schedulers, and eventually user programs. Next time, we’ll teach the kernel to breathe by handling interrupts and exposing its first observable behavior.</p><p><strong>Code:</strong> <a href="https://github.com/ksaritek/rxv6-riscv">https://github.com/ksaritek/rxv6-riscv</a><br><strong>Reference:</strong> <a href="https://github.com/mit-pdos/xv6-riscv">https://github.com/mit-pdos/xv6-riscv</a><br><strong>RISC-V Book:</strong> <a href="https://github.com/riscv/riscv-isa-manual/releases">https://github.com/riscv/riscv-isa-manual/releases</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea992748f84b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Local RAG with Rust and MCP: Private Document Search for Claude Desktop]]></title>
            <link>https://medium.com/@ksaritek/local-rag-with-rust-and-mcp-private-document-search-for-claude-desktop-6fccb37c024e?source=rss-e5e1af2488fb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6fccb37c024e</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[rust-sdk]]></category>
            <dc:creator><![CDATA[koray sariteke]]></dc:creator>
            <pubDate>Wed, 11 Jun 2025 00:25:10 GMT</pubDate>
            <atom:updated>2025-06-11T00:36:08.285Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Building a local document search system that integrates with Claude Desktop using the Model Context Protocol</blockquote><p>The Model Context Protocol (MCP) enables AI assistants like Claude to interact with external tools and data sources. Instead of being limited to training data, MCP allows Claude to call functions, access local files, and integrate with applications in real-time.</p><p>The <a href="https://github.com/ksaritek/rust-local-rag">rust-local-rag</a> project demonstrates this capability by providing semantic search across PDF documents directly within Claude Desktop conversations—all while keeping your data completely local.</p><h3>What is MCP?</h3><p>MCP is a standardized protocol that connects AI assistants to external tools. It works through a simple stdin/stdout interface, allowing Claude Desktop to:</p><ul><li>Call external functions during conversations</li><li>Access real-time data from local applications</li><li>Maintain context across tool interactions</li><li>Work with private data without sending it to external services</li></ul><h3>The Architecture</h3><p>The rust-local-rag system creates an MCP server that exposes document search capabilities to Claude:</p><pre>┌─────────────────┐ MCP Protocol ┌──────────────────┐<br>│                 │ (stdin/stdout) │                  │<br>│ Claude Desktop  │ ◄─────────────► │ Rust RAG        │<br>│                 │                │ MCP Server      │<br>└─────────────────┘                └──────────────────┘<br>                                           │<br>                                           ▼<br>                                  ┌──────────────────┐<br>                                  │ • PDF Parser     │<br>                                  │ • Ollama         │<br>                                  │ • Vector Store   │<br>                                  └──────────────────┘</pre><h3>MCP Integration with rmcp</h3><p>Using the rmcp crate (Rust&#39;s official MCP SDK), the application exposes three tools to Claude:</p><pre>#[tool(description = &quot;Search through uploaded documents using semantic similarity&quot;)]<br>async fn search_documents(&amp;self, query: String, top_k: Option&lt;usize&gt;) -&gt; Result&lt;CallToolResult, McpError&gt;<br><br>#[tool(description = &quot;List all uploaded documents&quot;)]<br>async fn list_documents(&amp;self) -&gt; Result&lt;CallToolResult, McpError&gt;<br><br>#[tool(description = &quot;Get RAG system statistics&quot;)]<br>async fn get_stats(&amp;self) -&gt; Result&lt;CallToolResult, McpError&gt;</pre><p>When you ask Claude something like “Search my documents for information about machine learning,” Claude automatically invokes the search_documents tool, processes the results, and provides relevant insights from your document collection.</p><h3>Ollama’s Role: Local Embeddings</h3><p>The system uses Ollama to generate embeddings locally. Here’s how it works:</p><ol><li><strong>Document Processing</strong>: PDFs are parsed and split into chunks</li><li><strong>Embedding Generation</strong>: Each chunk is sent to Ollama (using models like nomic-embed-text)</li><li><strong>Vector Storage</strong>: Embeddings are stored in a local vector database</li><li><strong>Semantic Search</strong>: User queries are embedded and matched against stored vectors</li><li><strong>Results</strong>: Relevant document chunks are returned with similarity scores</li></ol><p>This approach ensures that:</p><ul><li>No document content is sent to external APIs</li><li>Embeddings are generated using high-quality local models</li><li>Search quality rivals cloud-based solutions</li><li>Everything runs offline after initial setup</li></ul><h3>MCP vs Traditional Approaches</h3><blockquote>Integration <br>— MCP: Native Claude Desktop support<br> — HTTP: Requires custom client</blockquote><blockquote>Security <br>— MCP: Process isolation, no network <br>— HTTP: Network exposure required</blockquote><blockquote>Performance <br>— MCP: Direct stdin/stdout IPC <br>— HTTP: Network overhead</blockquote><blockquote>User Experience<br> — MCP: Seamless tool integration <br>— HTTP: Manual API management</blockquote><h3>The Document Search Flow</h3><ol><li><strong>User Query</strong>: “Find information about neural networks in my research papers”</li><li><strong>MCP Tool Call</strong>: Claude automatically calls search_documents(&quot;neural networks&quot;, 5)</li><li><strong>Embedding</strong>: Query is embedded using Ollama</li><li><strong>Vector Search</strong>: System finds most similar document chunks</li><li><strong>Results</strong>: Relevant passages are returned to Claude</li><li><strong>Response</strong>: Claude synthesizes findings into a natural answer</li></ol><h3>Privacy and Performance Benefits</h3><p><strong>Local Processing</strong>: All operations happen on your machine. Documents, embeddings, and search queries never leave your system.</p><p><strong>Rust Performance</strong>: The Rust implementation provides:</p><ul><li>Memory safety without garbage collection overhead</li><li>Async/await for non-blocking operations</li><li>Fast vector operations and search</li></ul><p><strong>Ollama Integration</strong>: Local embedding models provide:</p><ul><li>High-quality semantic understanding</li><li>No dependency on external services</li><li>Consistent performance regardless of internet connectivity</li></ul><h3>Why This Matters</h3><p>This implementation shows how MCP can bridge AI assistants with specialized local tools. Rather than building monolithic systems, developers can create focused MCP servers that excel at specific tasks while integrating seamlessly with Claude Desktop.</p><p>The combination of MCP’s standardized protocol, Rust’s performance, and Ollama’s local AI capabilities creates a powerful foundation for private, efficient document intelligence.</p><h3>Looking Forward</h3><p>MCP enables a new ecosystem of AI-integrated tools. Similar patterns can be applied to database queries, file system operations, API integrations, and other specialized tasks — all while maintaining the natural conversational interface that makes AI assistants useful.</p><p>The rust-local-rag project demonstrates that sophisticated AI capabilities don&#39;t require cloud dependencies or privacy compromises. Local processing, standardized protocols, and thoughtful architecture can deliver powerful results.</p><p><em>Explore the </em><a href="https://github.com/ksaritek/rust-local-rag"><em>rust-local-rag repository</em></a><em> to see MCP integration patterns and local RAG implementation details.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6fccb37c024e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ownership, Borrowing, and Shared Data in Rust: Beyond Concurrency]]></title>
            <link>https://medium.com/@ksaritek/ownership-borrowing-and-shared-data-in-rust-beyond-concurrency-302c9d2557ef?source=rss-e5e1af2488fb------2</link>
            <guid isPermaLink="false">https://medium.com/p/302c9d2557ef</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[data-race]]></category>
            <category><![CDATA[ownership]]></category>
            <category><![CDATA[rustlang]]></category>
            <category><![CDATA[borrow]]></category>
            <dc:creator><![CDATA[koray sariteke]]></dc:creator>
            <pubDate>Sun, 25 May 2025 20:07:46 GMT</pubDate>
            <atom:updated>2025-05-25T20:07:46.227Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Rust’s approach to memory safety and data access is built on two fundamental pillars: <strong>ownership</strong> and <strong>borrowing</strong>. These concepts are at the heart of all Rust code. However, their real power becomes evident when you need to share data safely between parts of your code, especially across threads.</blockquote><h3>Ownership</h3><p><strong>Ownership</strong> is about who is responsible for a value — who can move it, drop it, or share it. In concurrent programming, Rust gives you several tools to manage ownership safely:</p><p><strong>Transferring ownership to threads (move)</strong></p><p>When spawning a thread, you can transfer ownership of data to that thread using the move keyword. Once ownership moves, the original owner cannot use the data anymore. This prevents multiple threads from unexpectedly accessing and mutating the same value.</p><p><strong>Static values (static X: …)</strong></p><p>A static item is “owned” by the entire program and lives for its whole duration. Any thread can access it, but its value must be a compile-time constant and cannot be dropped or changed at runtime.</p><p><strong>Leaking with Box::leak</strong></p><p>This technique gives up ownership of heap-allocated memory, turning it into a reference that lives as long as the program (a ‘static reference). This can be handy for sharing data across threads without worrying about lifetimes — but leaks memory forever, so use sparingly.</p><p><strong>Reference Counting (Rc, Arc)</strong></p><p>For cases where you need multiple owners:</p><ul><li>Rc&lt;T&gt; (Reference Counted): For single-threaded shared ownership.</li><li>Arc&lt;T&gt; (Atomically Reference Counted): For thread-safe shared ownership.</li></ul><p>These smart pointers keep track of how many owners exist. The value is cleaned up only when the last owner is dropped. Arc uses atomic operations to make it safe across threads.</p><p><strong>Thread safety traits (Send and Sync)</strong></p><p>These auto traits tell the compiler which types are safe to move or share between threads:</p><ul><li><strong>Send</strong>: Types that can be safely transferred to another thread.</li><li><strong>Sync</strong>: Types whose references can be safely shared between threads.</li></ul><p>If a type isn’t Send or Sync, you cannot move or share it across threads, which prevents undefined behavior.</p><blockquote>Thread safety traits (Send and Sync) aren’t strictly “ownership” traits, but they control whether a type’s ownership or references can safely cross thread boundaries. That’s why they’re essential for safe data sharing and are usually discussed alongside Rust’s system.</blockquote><h3>Borrowing</h3><p><strong>Borrowing</strong> is about using data temporarily without taking ownership — ensuring only the right kind of access happens at the right time.</p><p><strong>Borrowing rules (&amp;T, &amp;mut T)</strong></p><p>Rust enforces that you can have any number of immutable references (&amp;T), or one mutable reference (&amp;mut T) — but never both at once. This prevents data races at compile time.</p><p><strong>Prevent data races</strong></p><p>The borrow checker ensures you never have unsynchronized reads and writes to the same data, which are a common source of bugs in other languages.</p><p><strong>Determine when you can safely read or mutate data</strong></p><p>The borrow checker enforces access rules at compile time, so you always know when your data is safe to read or write.</p><p><strong>Lifetime of references</strong></p><p>Borrowed references cannot outlive the data they point to. Rust’s lifetime system guarantees that no reference ever becomes dangling or invalid, making memory safety airtight.</p><p><strong>Interior mutability types</strong></p><p>Sometimes you need to mutate data even when you only have a shared reference. Rust provides special types to allow this, with safety guarantees:</p><ul><li><strong>Single-threaded:<br>- Cell&lt;T&gt;:</strong> Allows replacing or copying values; only for types that implement Copy.<br><strong>- RefCell&lt;T&gt;:</strong> Allows dynamic borrowing checks at runtime, panicking if rules are broken.</li><li><strong>Multi-threaded:<br>- Mutex&lt;T&gt;:</strong> Ensures exclusive access (only one thread at a time can access data).<br><strong>- RwLock&lt;T&gt;:</strong> Allows many readers or one writer at a time.</li></ul><blockquote><strong>Under the Hood: UnsafeCell</strong></blockquote><blockquote>All interior mutability types in Rust — such as Cell, RefCell, Mutex, and RwLock — are ultimately built on top of a low-level primitive called UnsafeCell.<br>UnsafeCell is the only legal way in Rust to obtain mutable access to data through a shared reference (&amp;T). The standard library types wrap UnsafeCell and add additional safety guarantees, either at compile time (like Cell), at runtime (like RefCell), or by using synchronization primitives (like Mutex and RwLock).<br>Most Rustaceans will never need to use UnsafeCell directly, but it’s useful to know it’s the foundation that makes safe interior mutability possible.</blockquote><h3>Blending Ownership and Borrowing</h3><p>Some types and patterns in Rust combine both ownership and borrowing, offering flexibility for complex concurrency scenarios:</p><p><strong>Mutexes and RwLocks</strong></p><p>You <strong>own</strong> the lock (e.g., an Arc&lt;Mutex&lt;T&gt;&gt;), but you only <strong>borrow</strong> the data it protects when you acquire the lock. This ensures that only one thread can access or mutate the data at a time (mutex), or that multiple threads can read (RwLock).</p><p><strong>Condition variables (Condvar)</strong></p><p>Used alongside a mutex to allow threads to wait for certain conditions. While you hold the lock, you borrow the data; when waiting, the lock is temporarily released and reacquired when notified. This pattern is crucial for efficient communication and coordination between threads.</p><h3>Summary</h3><p><strong>Ownership</strong> controls who is responsible for data and ensures only one owner at a time (unless using smart pointers like Arc/Rc).</p><p><strong>Borrowing</strong> allows multiple safe accesses, with clear rules about who can read or write and when.</p><p>Some concurrency primitives blend both concepts to provide safe, flexible multi-threaded programming.</p><p>By organizing Rust’s concurrency tools in this way, you can more easily design programs that are safe, performant, and free from classic bugs like data races and memory leaks.</p><h3><strong>Further Reading &amp; References:</strong></h3><ul><li><a href="https://doc.rust-lang.org/book/">The Rust Programming Language (Book)</a></li><li><a href="https://doc.rust-lang.org/std/">Rust Standard Library Documentation</a></li><li><a href="https://marabos.nl/atomics/">Rust Atomics and Locks</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=302c9d2557ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Lightweight Coroutines in Rust: Introducing rust-fibers]]></title>
            <link>https://medium.com/@ksaritek/building-lightweight-coroutines-in-rust-introducing-rust-fibers-53b91625a9de?source=rss-e5e1af2488fb------2</link>
            <guid isPermaLink="false">https://medium.com/p/53b91625a9de</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[coroutine]]></category>
            <category><![CDATA[system-programming]]></category>
            <dc:creator><![CDATA[koray sariteke]]></dc:creator>
            <pubDate>Sun, 18 May 2025 23:43:01 GMT</pubDate>
            <atom:updated>2025-05-18T23:43:01.323Z</atom:updated>
            <cc:license>https://creativecommons.org/publicdomain/mark/1.0/</cc:license>
            <content:encoded><![CDATA[<blockquote>How do you build Go-like goroutines in Rust? What’s the difference between preemptive and cooperative multitasking, or stackful and stackless coroutines? Let’s dive into the world of fibers with rust-fibers.</blockquote><p>Rust is famous for its safety, performance, and fearless concurrency. But what if you want to experiment with lightweight, stackful coroutines — something like Go’s goroutines, but running on a single thread with cooperative multitasking? That’s exactly what I set out to explore with rust-fibers: a playground for building and understanding fibers in Rust.</p><p>In this post, I’ll walk you through the motivation, the architecture, and the code behind rust-fibers, and explain some key concepts in concurrency and coroutine design.</p><h3>What Are Fibers?</h3><p>Fibers are lightweight threads of execution that use cooperative multitasking. Unlike OS threads, which are scheduled preemptively by the operating system, fibers:</p><ul><li>Run on a single thread</li><li>Manually yield control to other fibers</li><li>Have their own stack</li><li>Can be suspended and resumed at will</li></ul><p>This model is powerful for certain classes of problems — especially when you want to avoid the overhead of OS threads, or need fine-grained control over scheduling.</p><h3>Preemptive vs. Cooperative Multitasking</h3><p>Before we go further, let’s clarify two important multitasking models:</p><ul><li>Preemptive multitasking is what most operating systems use. The OS scheduler can interrupt (preempt) a running thread at almost any time, saving its state and switching to another thread. This ensures fairness, but introduces complexity: you have to worry about race conditions, locks, and other synchronization issues.</li><li>Cooperative multitasking relies on each task (or fiber) to voluntarily yield control back to the scheduler. The scheduler never interrupts a running fiber; instead, the fiber must call a function like yield_now() to give up the CPU. This model is simpler in some ways — since context switches only happen at well-defined points, you can avoid many concurrency bugs. However, if a fiber forgets to yield, it can block the entire system.</li></ul><h3>Stackful vs. Stackless Coroutines</h3><p>Coroutines come in two main flavors: stackful and stackless.</p><ul><li>Stackful coroutines (like the fibers in rust-fibers) have their own stack. This means they can be suspended and resumed from deep within nested function calls, making them very flexible. You can yield from anywhere in the call stack.</li><li>Stackless coroutines (like Rust’s async/await or Python’s generators) do not have their own stack. They can only be suspended at specific points (e.g., at an await or yield statement). This makes them lighter weight, but less flexible — you can’t yield from arbitrary places in the call stack.</li></ul><h3>Why Build Fibers in Rust?</h3><p>Rust’s type system and ownership model make it a great language for systems programming, but also a challenging one for implementing low-level concurrency primitives. By building a fiber system from scratch, I wanted to:</p><ul><li>Learn more about stack management and context switching</li><li>Explore how Rust’s safety guarantees interact with unsafe code</li><li>Provide a simple, educational playground for others interested in coroutines</li></ul><h3>Architecture Overview</h3><p>The core of rust-fibers is a single-threaded scheduler that manages a set of fibers, each with its own stack. Here’s a high-level view:</p><pre>┌───────────────────┐<br>│     Scheduler     │◄───┐<br>├───────────────────┤    │<br>│ - Ready Queue     │    │<br>│ - Fibers List     │    │<br>└────────┬──────────┘    │<br>         │               │<br>         │ schedules     │ yields<br>         ▼               │<br>┌───────────────────┐    │<br>│       Fiber       │    │<br>├───────────────────┤    │<br>│ - Stack           │────┘<br>│ - Callback        │<br>│ - State           │<br>└───────────────────┘</pre><p>Each fiber is a struct containing:</p><ul><li>A unique ID</li><li>Its own 2MB stack (allocated manually)</li><li>A callback (the function to run)</li><li>State (Ready, Running, or Completed)</li></ul><p>The scheduler keeps a queue of ready fibers and runs them one by one. When a fiber calls yield_now(), it voluntarily gives up control, allowing the scheduler to run another fiber.</p><h3>How Stack Management Works in rust-fibers</h3><p>One of the defining features of stackful coroutines (like fibers) is that each coroutine has its own stack. In rust-fibers, this is implemented manually, giving you a hands-on look at how low-level stack management works in Rust.</p><p>Let’s walk through the relevant code:</p><h3>1. Defining the Stack Size</h3><p>At the top of the file, we define the size of each fiber’s stack. Here, each fiber gets a 2MB stack:</p><pre>const STACK_SIZE: usize = 1024 * 1024 * 2;</pre><p>This constant ensures that every fiber has enough space for deep call stacks, but you could adjust it depending on your needs.</p><h3>2. The Stack Struct</h3><p>The Stack struct is responsible for managing the memory for each fiber’s stack:</p><pre>struct Stack {<br>    ptr: NonNull&lt;u8&gt;,<br>    size: usize,<br>}</pre><ul><li>ptr is a non-null pointer to the start of the allocated memory.</li><li>size is the size of the stack (in bytes).</li></ul><h3>3. Allocating and Deallocating the Stack</h3><p>When a new stack is created, we allocate a block of memory using Rust’s low-level allocation APIs:</p><pre>impl Stack {<br>    fn new(size: usize) -&gt; Self {<br>        let layout = Layout::from_size_align(size, 16)<br>            .expect(&quot;Failed to create stack layout&quot;);<br>        <br>        let ptr = unsafe {<br>            NonNull::new(alloc(layout)).expect(&quot;Failed to allocate stack&quot;)<br>        };<br>        <br>        Stack { ptr, size }<br>    }<br>    <br>    // Get the top of the stack (highest address)<br>    fn top(&amp;self) -&gt; *mut u8 {<br>        unsafe { self.ptr.as_ptr().add(self.size) }<br>    }<br>}</pre><p>When the stack is no longer needed, it’s deallocated:</p><pre>impl Drop for Stack {<br>    fn drop(&amp;mut self) {<br>        unsafe {<br>            let layout = Layout::from_size_align(self.size, 16)<br>                .expect(&quot;Failed to create stack layout for deallocation&quot;);<br>            dealloc(self.ptr.as_ptr(), layout);<br>        }<br>    }<br>}</pre><p>This ensures that each fiber’s stack memory is properly cleaned up, preventing memory leaks.</p><h3>4. Each Fiber Gets Its Own Stack</h3><p>When you spawn a new fiber, it gets its own stack:</p><pre>impl Fiber {<br>    fn new&lt;F&gt;(id: usize, f: F) -&gt; Self <br>    where <br>        F: FnMut() -&gt; () + &#39;static <br>    {<br>        Fiber {<br>            id,<br>            stack: Stack::new(STACK_SIZE),<br>            state: State::Ready,<br>            callback: Callback::new(f),<br>        }<br>    }<br>}</pre><p>This design means that each fiber can be suspended and resumed from anywhere in its call stack — a key advantage of stackful coroutines.</p><h3>Why Manual Stack Management?</h3><p>By managing the stack manually, rust-fibers gives you a taste of what’s happening under the hood in coroutine implementations. This approach is more flexible than stackless coroutines (which can only yield at specific points), but it also requires careful memory management and a deeper understanding of how stacks work.</p><h3>How to Use rust-fibers</h3><p>Here’s a quick example of how you can use rust-fibers:</p><pre>fn main() {<br>    let mut scheduler = Scheduler::new();<br><br>    scheduler.spawn(|| {<br>        println!(&quot;Fiber 1 running&quot;);<br>        println!(&quot;Fiber 1 yielding&quot;);<br>        yield_now();<br>        println!(&quot;Fiber 1 resumed after yielding&quot;);<br>        for i in 0..3 {<br>            println!(&quot;Fiber 1: counting {}&quot;, i);<br>        }<br>        println!(&quot;Fiber 1 complete&quot;);<br>    });<br><br>    scheduler.spawn(|| {<br>        println!(&quot;Fiber 2 running&quot;);<br>        for i in 0..3 {<br>            println!(&quot;Fiber 2: counting {}&quot;, i);<br>            if i == 1 {<br>                println!(&quot;Fiber 2 yielding&quot;);<br>                yield_now();<br>                println!(&quot;Fiber 2 resumed after yielding&quot;);<br>            }<br>        }<br>        println!(&quot;Fiber 2 complete&quot;);<br>    });<br><br>    run_scheduler(scheduler);<br>}</pre><p>When you run this, you’ll see the fibers taking turns, yielding and resuming as expected.</p><h3>Limitations and Future Directions</h3><p><strong>This is a playground project, so there are some important limitations:</strong></p><ul><li>Single-Threaded: All fibers run on a single thread — no multicore support (yet).</li><li>Fixed Stack Size: Each fiber gets a fixed 2MB stack, which may be overkill for simple tasks.</li><li>Manual Yielding: Fibers must explicitly yield; there’s no automatic yielding during blocking operations.</li><li>No Built-in Communication: Unlike Go’s channels, there’s no built-in way for fibers to communicate.</li></ul><p><strong>Some ideas for future improvements:</strong></p><ul><li>Add channels for fiber communication</li><li>Implement automatic yielding during blocking operations</li><li>Support dynamic stack sizes</li><li>Explore work-stealing scheduling</li><li>Multi-threaded support</li></ul><h3>Conclusion</h3><p>rust-fibers is a fun, educational project for anyone interested in coroutines, stack management, or Rust’s low-level capabilities. If you’re curious about how cooperative multitasking works under the hood, or want to experiment with your own fiber-based scheduler, check out the GitHub repo and try it out!</p><p><em>Have thoughts or suggestions? Feel free to open an issue or PR on </em><a href="https://github.com/ksaritek/rust-fibers"><em>GitHub</em></a><em>!</em></p><p>References:</p><ul><li><a href="https://en.wikipedia.org/wiki/Coroutine#Stackful_and_stackless_coroutines">Wikipedia: Coroutine — Stackful and Stackless</a></li><li><a href="https://github.com/ksaritek/rust-fibers">rust-fibers on GitHub</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53b91625a9de" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>