<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aarambh Dev Hub on Medium]]></title>
        <description><![CDATA[Stories by Aarambh Dev Hub on Medium]]></description>
        <link>https://medium.com/@aarambhdevhub?source=rss-0811e9432b8e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6ru5B2BnsgAsJeHSb3x6Xg.png</url>
            <title>Stories by Aarambh Dev Hub on Medium</title>
            <link>https://medium.com/@aarambhdevhub?source=rss-0811e9432b8e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 03:05:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aarambhdevhub/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Stop Watching Tutorials: How Building Git, Redis and Docker From Scratch Made Me a Better…]]></title>
            <link>https://aarambhdevhub.medium.com/stop-watching-tutorials-how-building-git-redis-and-docker-from-scratch-made-me-a-better-11d902dfbf49?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/11d902dfbf49</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[learn-to-code]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 02:44:14 GMT</pubDate>
            <atom:updated>2026-10-25T02:44:14.615Z</atom:updated>
            <content:encoded><![CDATA[<h3>Stop Watching Tutorials: How Building Git, Redis and Docker From Scratch Made Me a Better Programmer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GHUvOjlxLr-FwyV5hmjAjg.jpeg" /></figure><p>There’s a moment every self-taught developer knows.</p><p>You finish a tutorial. You feel great. You close the tab, open a blank file, and try to build something on your own. Nothing comes. The cursor just blinks at you.</p><p>I lived in that loop for a long time. Video after video, course after course, and I could follow along with anything. But the second the instructor stopped talking, I was stuck. That’s tutorial hell, and it’s a quiet kind of frustrating because it feels like you’re making progress.</p><p>What finally broke the loop for me wasn’t another course. It was building real software, the kind of software I use every day but never understood, with nobody typing the answer for me.</p><p>That’s what CodeCrafters is. Let me explain what it is, how it works, what I’ve learned, and who it’s actually for, including who should skip it.</p><h3>What is CodeCrafters? (The short answer)</h3><p><strong>CodeCrafters is a platform where you learn programming by rebuilding real developer tools from scratch.</strong> Instead of watching someone build a to-do app, you build your own version of Git, Redis, Docker, SQLite, a shell, an HTTP server, Kafka or a BitTorrent client, in the programming language of your choice.</p><p>The format is called “Build your own X.” The platform was founded in 2022 by Sarup Banskota and Paul Kuruvilla, and it’s backed by Y Combinator.</p><p>The key difference from a normal course: <strong>there’s no video lecture walking you through every line.</strong> You get a project broken into stages, a clear description of what each stage needs, and automated tests that tell you whether your code works.</p><h3>Why tutorials stop working (and what replaces them)</h3><p>Tutorials are great for one thing: getting a first look at a topic. But they have a hidden flaw. The instructor already made all the hard decisions. They chose the structure, hit the bugs off-camera, and edited out the dead ends.</p><p>You only practice typing. You don’t practice thinking.</p><p>Real learning happens in the messy middle, when your code fails, the error message makes no sense, and you have to read documentation and a protocol spec to figure out what’s going on. That’s exactly the part tutorials remove, and it’s exactly the part that makes you a developer.</p><p>Building something with a clear goal but no hand-holding puts you right in that messy middle, in a safe way, because the project is split into small stages so it never feels impossible.</p><h3>How CodeCrafters actually works</h3><p>Here’s the flow, step by step:</p><ol><li><strong>Pick a challenge.</strong> For example, build your own Redis or build your own Git.</li><li><strong>Pick your language.</strong> Rust, Go, Python, JavaScript, Java, C++ and many more are supported, over twenty at the time of writing.</li><li><strong>Clone the starter repo</strong> to your own machine. You work in your own editor and terminal, not inside a browser box.</li><li><strong>Read the stage instructions.</strong> Each stage asks for one small, specific piece of behavior.</li><li><strong>Write the code, then push it.</strong> Pushing to the repo runs automated tests against your solution and gives you feedback right away.</li><li><strong>Move to the next stage.</strong> Stages get harder, and by the end you’ve built a working system.</li></ol><p>After you finish a stage, you can also look at how other people solved the same problem. I’ve found that’s one of the best parts. You solve it your own way first, then compare, and suddenly you notice patterns and language idioms you’d never have found alone.</p><h3>What building these projects teaches you</h3><p>I believed in this way of learning long before I found CodeCrafters. Almost everything useful I know in Rust came from building, not from watching. I built a language model from scratch in Rust, with no Python and no PyTorch. I built an animation library and a 3D scene library. Every one of those projects got me stuck somewhere, and every time the way out was the same: read the docs, break things, fix them, repeat.</p><p>That’s exactly the habit CodeCrafters trains, just with more structure. The project is already split into stages, and automated tests tell you whether you got each one right, so you spend your energy on the thinking, not on guessing what to build next.</p><p>Take a challenge like building your own Redis. The first stages feel easy: accept a connection, reply to a ping, handle a couple of simple commands. Then a stage asks for something you’ve never done, like serving many clients at once or reading a protocol byte by byte, and your tests turn red. There’s no video to rescue you. You read the spec, add some print statements, make a guess and try again. When the test finally goes green, you haven’t just finished a stage. You understand that piece of the system in a way you never would from watching someone else do it.</p><p>Beyond the specific story, here’s what building systems like this teaches that tutorials usually don’t:</p><ul><li><strong>How the tools you use every day really work.</strong> After building a version of Git, commands like commit and branch stop feeling like magic. You know there are objects and hashes underneath.</li><li><strong>How to read specs and documentation.</strong> Real protocols and file formats come with dense documents. Learning to read them is a career skill.</li><li><strong>How to debug without a safety net.</strong> When a test fails, you can’t ask the instructor. You learn to isolate problems and form hypotheses.</li><li><strong>How to structure a bigger codebase.</strong> A tutorial app is a few hundred lines. These projects grow, and you feel the pain of bad structure and learn to fix it.</li><li><strong>How to think about concurrency, parsing and networking.</strong> These topics show up in backend and systems jobs constantly, and most courses skip them.</li></ul><p>And there’s a bonus: you end up with real projects on your GitHub that show employers you can build something hard, not just follow instructions.</p><h3>CodeCrafters vs tutorials vs LeetCode</h3><p>They solve different problems, so it’s not really a fight:</p><ul><li><strong>Tutorials</strong> are best for a first introduction to a new topic.</li><li><strong>LeetCode-style sites</strong> are best for interview-style algorithm practice.</li><li><strong>CodeCrafters</strong> is best for learning how real software is built, and for building the skill of finishing a large project.</li></ul><p>If your goal is to become a better engineer, not just pass a screening round, project-based practice is the piece most people are missing.</p><h3>The honest downsides</h3><p>I wouldn’t trust a review with no downsides, so here are mine:</p><ul><li><strong>It’s a paid platform.</strong> There’s a way to try it before paying, but full access is a subscription. Check their pricing page for the current price and what’s free.</li><li><strong>It’s not for absolute beginners.</strong> You should be comfortable with a language’s basics, the terminal and Git. CodeCrafters itself says people early in their careers on tight budgets may want to focus on fundamentals first, and I agree.</li><li><strong>You will get stuck.</strong> That’s the point, but it can be frustrating if you expect step-by-step answers.</li><li><strong>The challenges take time.</strong> A serious challenge can take days or weeks of evenings, not an hour.</li></ul><h3>Who should try it</h3><p>CodeCrafters is a great fit if you:</p><ul><li>Know the basics of a language and want to go deeper</li><li>Feel stuck in tutorial hell</li><li>Want backend or systems programming skills</li><li>Are learning Rust, Go or another language and want real projects to practice in</li><li>Want portfolio projects that stand out</li></ul><p>It’s probably not the right time if you’re still learning what a variable or a loop is.</p><h3>How to get started</h3><p>If you want to try it, here’s the simplest path:</p><ol><li>Create an account using my link: <a href="https://app.codecrafters.io/join?via=aarambh-darshan">https://app.codecrafters.io/join?via=aarambh-darshan</a></li><li>Pick one challenge that sounds interesting, not the hardest one.</li><li>Choose a language you already know a little.</li><li>Aim for one stage a day. Small and steady beats a big weekend burst.</li><li>After each stage, compare your code with other solutions and learn from the differences.</li></ol><h3>Your first 30 days: a simple plan</h3><p>The hardest part isn’t the code, it’s staying consistent. Here’s a plan that keeps the pressure low.</p><ol><li><strong>Week 1: pick one small challenge and one language.</strong> Choose a language you already know a little, so you’re learning how the system works, not fighting syntax.</li><li><strong>Week 2: aim for consistency, not speed.</strong> One stage a day, or every other day, is plenty. When you’re stuck, step away for an hour before you look at other solutions.</li><li><strong>Week 3: read before you write.</strong> Read each stage’s instructions twice and the related docs once before you type anything. This is where most of the learning happens.</li><li><strong>Week 4: review and share.</strong> Compare your code with other solutions, clean yours up, and push it to GitHub with a short README that explains what you built and what you learned.</li></ol><p>After 30 days you’ll have a finished project, a daily habit and something real to show. That beats a folder of half-watched courses.</p><h3>Final thoughts</h3><p>The biggest shift for me was simple: I stopped asking “which course should I watch next?” and started asking “what should I build next?”</p><p>If you’re tired of tutorials and want to see what you can really do, try one challenge. Pick something small, get stuck, figure it out, and watch what happens to your confidence.</p><p>If you want to give it a go, here’s my link again: <a href="https://app.codecrafters.io/join?via=aarambh-darshan">Start your first CodeCrafters challenge</a></p><p>And if you build something with it, I’d love to hear about it in the comments.</p><p><em>Thanks for reading, and happy coding!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11d902dfbf49" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Architected a FLUX Alternative in Pure Rust — No Python, No PyTorch.]]></title>
            <link>https://aarambhdevhub.medium.com/i-architected-a-flux-alternative-in-pure-rust-no-python-no-pytorch-3a571cadfa0d?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a571cadfa0d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Wed, 16 Sep 2026 02:35:21 GMT</pubDate>
            <atom:updated>2026-09-16T02:37:02.089Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Architected a FLUX Alternative in Pure Rust — No Python, No PyTorch. And I’m Handing the Blueprint to the World.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4e7-g8R9LutmnnhisgWt2g.jpeg" /></figure><p>There is a particular feeling that comes when a system you have been designing for months finally crystallizes into something coherent. Not working code — architecture. The moment when every design decision connects logically to every other design decision and the whole thing has a kind of internal gravity.</p><p>I felt that feeling with <a href="https://github.com/AarambhDevHub/aarambh-studio">aarambh-studio</a> — the complete Rust LLM built from scratch. I felt it again with <a href="https://github.com/AarambhDevHub/aarambh-voice-studio">aarambh-voice-studio</a> — the AI audio studio with zero Python dependencies. And I felt it a third time a few weeks ago, completing the architectural design for what I believe is the most technically ambitious open-source blueprint I have ever published.</p><p>aarambh-vision-studio.</p><p>A complete image generation, editing, and understanding studio designed entirely in pure Rust. Twenty-five modular crates. A Multimodal Diffusion Transformer backbone with Rectified Flow Matching. A continuous latent VAE. Spatial conditioning equivalent to ControlNet. Zero-shot reference image prompting equivalent to IP-Adapter. Reinforcement learning alignment with GRPO and DPO. Online self-learning with catastrophic forgetting prevention.</p><p>I am not actively building it anymore. The entire architecture, starter code, and 27-phase roadmap are published as an open blueprint for the community.</p><p>Here is the complete architectural story — and why I think giving this away is the most valuable thing I can do with it.</p><h3>The State of Open-Source Image Generation in 2026</h3><p>Image generation has a Python problem that’s even worse than the text LLM ecosystem.</p><p>The standard setup for running any diffusion model — FLUX, Stable Diffusion 3, SDXL — requires PyTorch, torchvision, the diffusers library, transformers, accelerate, xformers, and another dozen supporting packages. A clean installation consumes ten to fifteen gigabytes of disk space. Getting those libraries to agree on CUDA driver versions, compiler versions, and flash attention kernels is the kind of exercise that wastes entire engineering afternoons.</p><p>But the Python dependency mess is just the surface problem. The deeper issue is architectural incoherence.</p><p>Most Python diffusion codebases were not designed — they evolved. Features were bolted onto features. ControlNet was added as a separate model with a different loading mechanism. IP-Adapter was integrated through adapter injection. Inpainting was handled by a different checkpoint entirely. The result is a codebase where changing one component frequently breaks three others in ways that only surface at runtime.</p><p>Deployment makes it worse. Shipping a Python diffusion service to any target other than a developer GPU workstation means containerizing an entire Python environment, gigabytes of model weights, and a chain of CUDA-specific compiled extensions. Cold start times measured in tens of seconds. Memory footprints that make edge deployment impractical.</p><p>By building aarambh-vision-studio in pure Rust on top of Candle, we eliminate the interpreter entirely. Memory allocation is deterministic and governed by Rust’s ownership model — when an intermediate latent tensor goes out of scope, its memory is freed immediately. The entire studio compiles to a single native binary that boots in milliseconds and communicates with GPU hardware through native CUDA or Metal bindings. And — crucially — the architecture was designed as a system, not assembled from parts.</p><h3>Why MMDiT and Not a U-Net</h3><p>Before describing the architecture, it helps to understand the architectural choice that drives every subsequent decision: why we are building a Multimodal Diffusion Transformer and not the convolutional U-Net that powered Stable Diffusion 1.5 and 2.1.</p><p>U-Nets were remarkable engineering achievements. The downsampling-bottleneck-upsampling structure with skip connections made image generation practical at a time when transformers were computationally out of reach for dense image inputs. Text conditioning injected through cross-attention layers gave them meaningful prompt following.</p><p>But U-Nets have a hard architectural ceiling. Convolutions have fixed, localized receptive fields. They are excellent at local texture — the grain of fabric, the pattern of brickwork — but they struggle fundamentally with global spatial reasoning. This is why SDXL models consistently struggled with prompts like “a red ball to the left of a blue cube” or correct hand anatomy across large compositional scenes.</p><p>The breakthrough that FLUX, Stable Diffusion 3, and Sana demonstrated: if you represent an image as a sequence of patch tokens — exactly like representing a sentence as word tokens — you can apply full bidirectional transformer attention across the entire visual field simultaneously. No local receptive field limitation. Full global spatial reasoning from the first attention layer.</p><p>This architecture is called a Multimodal Diffusion Transformer, or MMDiT. Image patches and text tokens are processed in the same attention operation — not sequentially, not through separate networks that exchange information through adapters, but jointly, in a shared representation space where visual features and linguistic semantics reason about each other at every layer.</p><p>aarambh-vision-studio is built entirely on this foundation.</p><h3>The Mathematics of Rectified Flow Matching</h3><p>The second fundamental architectural decision is the training objective. Traditional diffusion models — DDPM and its variants — add Gaussian noise to images according to a complex curved noise schedule over hundreds of discrete timesteps and train a network to denoise. At inference time, you reverse this process by solving a stochastic differential equation backward through time, requiring fifty to one hundred sampling steps to avoid trajectory collapse.</p><p>aarambh-vision-studio uses <strong>Rectified Flow Matching</strong> instead.</p><p>The core mathematical insight is elegant: instead of learning to denoise along a curved stochastic path, train the network to predict a <strong>straight-line constant velocity vector</strong> from pure noise to clean image.</p><p>At any timestep t ∈ [0, 1], the intermediate noisy state is simply:</p><pre>x_t = (1 - t) · x_0 + t · x_1</pre><p>where x_0 is Gaussian noise and x_1 is the clean target image latent. The velocity vector the network must predict is:</p><pre>v = x_1 - x_0</pre><p>That velocity is <strong>constant</strong> — it does not change with t. The network is trained with a simple MSE loss:</p><pre>L_flow = E[ ‖ v_θ(x_t, t, text) - (x_1 - x_0) ‖² ]</pre><p>Because the trajectory is a straight line, a simple Euler ODE integrator can take large confident strides across the timeline:</p><pre>x_{t + Δt} = x_t + v_θ(x_t, t, c) · Δt</pre><p>This directly slashes sampling steps from fifty-plus to twenty steps for full quality, and down to four or two steps after distillation — without the quality degradation that affects previous fast-sampling methods. Faster generation, simpler code, dramatically lower latency on commodity hardware.</p><h3>Layer One: The Continuous Latent VAE</h3><p>Running the MMDiT directly on raw image pixels is computationally impossible. A 512×512 image contains 786,432 individual color values. Cut that into 2×2 pixel patches and you have nearly 200,000 tokens — transformer attention would run out of memory instantly.</p><p>The solution is a Variational Autoencoder (aarambh-vision-tokenizer) that compresses high-resolution images into a compact continuous latent space before the transformer ever sees them.</p><p>Our VAE uses 8× spatial downsampling. A 512×512 image becomes a 64×64×16 latent grid — 65,536 values. That is a 12× reduction in sequence length, turning a computationally impossible sequence into a manageable transformer input.</p><p>The VAE training has one of the most important de-risking insights in the entire project: we do not train the full VAE with adversarial and KL divergence losses from day one. Instead:</p><p><strong>Stage 0A:</strong> Train reconstruction-only with L₁ pixel loss and LPIPS perceptual loss:</p><pre>L_0A = ‖x - x̂‖₁ + λ_lpips · L_LPIPS(x, x̂)</pre><p>This guarantees the encoder and decoder master sharp spatial reconstruction and clean edge boundaries before we introduce any additional pressure.</p><p><strong>Stage 0B:</strong> Only after reconstruction passes a strict PSNR threshold do we advance to Stage 0B — adding the PatchGAN discriminator and the KL divergence penalty:</p><pre>L_KL = (1/2) · Σ(μᵢ² + σᵢ² - log σᵢ² - 1)</pre><p>The KL penalty ensures the latent space remains smooth and Gaussian-shaped, so that noise sampled at inference time maps coherently to meaningful image content.</p><p>Once trained, the VAE is frozen permanently and shared across the entire workspace. It is the immutable ground truth that every downstream component depends on.</p><h3>Layer Two: Reusing aarambh-studio as the Text Encoder</h3><p>Here is a cross-project architectural decision that saved months of work: instead of training a separate text encoder from scratch — which would require another multi-gigabyte pretraining run — we load the decoder-only transformer checkpoint from aarambh-studio directly.</p><p>The insight: a language model trained to predict the next token in a sequence has already learned extraordinarily rich contextual semantic representations of language in its intermediate hidden layers. By creating a thin wrapper in aarambh-vision-textencoder, we load the aarambh-studio weights, pass the prompt through the model, and instead of reading the output vocabulary logits, we extract the hidden activation vectors from an intermediate transformer layer.</p><p>This gives us rich, world-grounded, multilingual semantic embeddings for any text prompt — without requiring a separate 10GB external text encoder dependency. It is pure cross-project reuse through SafeTensors weight loading, enabled by the consistent architecture choices across the aarambh ecosystem.</p><h3>Layer Three: The MMDiT Backbone</h3><p>The MMDiT backbone (aarambh-vision-nn and aarambh-vision-model) is the central engineering challenge of the entire project. It consists of two types of transformer blocks:</p><p><strong>Dual-Stream Blocks (early layers):</strong> Image patch tokens and text tokens are processed through separate QKV projections and separate MLP layers. They interact during cross-attention, but maintain distinct representations — this allows each modality to develop modality-specific feature hierarchies before being merged.</p><p><strong>Single-Stream Blocks (deep layers):</strong> Image tokens and text tokens are concatenated into a unified sequence. They now attend to each other without any distinction — a text token and an image patch token are peers in the same attention operation. This is where the deep semantic fusion happens.</p><p>Every block uses <strong>Adaptive Layer Normalization with Zero Initialization (adaLN-Zero):</strong></p><pre>x̂ = α · Block((1 + γ) · LayerNorm(x) + β) + x</pre><p>A conditioning MLP projects the timestep embedding and the pooled text embedding into scale (γ), shift (β), and residual gate (α) vectors. The gate α is initialized to exactly zero — meaning at the start of training, every block is an identity function. This allows training 24-layer transformers on consumer hardware without gradient explosion.</p><h3>Aspect Ratio Bucketing and 2D-RoPE</h3><p>Standard image generation models operate at fixed square resolutions, distorting landscape photographs into cropped squares and portrait shots into letterboxed rectangles. aarambh-vision-studio solves this with native multi-resolution aspect ratio bucketing paired with 2D Rotary Position Embeddings.</p><p>Bucketing groups training images by aspect ratio — 1:1, 16:9, 9:16, 4:3, 3:4, 21:9 — and trains the model on each batch at the native aspect ratio of its bucket. No cropping. No distortion.</p><p><strong>2D-RoPE</strong> makes this work for attention. Standard 1D RoPE encodes sequential position for text. For images on a 2D spatial grid, we split each attention head’s embedding dimension in half: the first half encodes the horizontal x-coordinate, the second half encodes the vertical y-coordinate.</p><pre>R_{x, y} = | R_Θ(x)    0     |<br>            |    0   R_Θ(y)  |</pre><p>The critical property: the dot product between any two rotated token pairs depends only on their relative spatial offset (Δx, Δy), not their absolute positions. Translation invariance is mathematically guaranteed — two features have the same relative attention weight wherever they appear on the canvas.</p><h3>Production Control: ControlNet and IP-Adapter in Rust</h3><p>Two production capabilities that separate aarambh-vision-studio from a pure text-to-image system:</p><p><strong>Structural Spatial Conditioning (</strong><strong>aarambh-vision-structure):</strong> Users can provide Canny edge maps, monocular depth estimates, or OpenPose skeleton keypoints as spatial conditioning inputs. These pass through a lightweight convolutional feature extractor with zero-initialized first-layer weights — ensuring the conditioning path begins as an identity function, just like adaLN-Zero — and inject spatial features into the MMDiT stream. This is the architectural equivalent of ControlNet, but native to the Rust workspace from the ground up.</p><p><strong>Reference-Image Prompting (</strong><strong>aarambh-vision-refprompt):</strong> Users provide a reference photograph alongside their text prompt. The understanding encoder extracts visual feature tokens representing the subject&#39;s identity, style, and aesthetic. These feed into a <strong>decoupled cross-attention</strong> operation:</p><pre>Z = Softmax(Q K_text^T / √d) V_text + λ · Softmax(Q K_ref^T / √d) V_ref</pre><p>The text cross-attention and reference cross-attention run independently, with a scalar λ controlling reference influence versus prompt adherence. At λ=0 the reference is ignored entirely. At λ=1 it exerts full visual influence. Precise, independent control without entanglement — this is the architectural equivalent of IP-Adapter, native to Rust.</p><h3>Inpainting, GRPO Alignment, and Self-Learning</h3><p>The workspace’s remaining four production systems:</p><p><strong>Inpainting (</strong><strong>aarambh-vision-edit):</strong> The model receives a 33-channel input tensor — 16 channels of noisy latent + 1 channel binary mask + 16 channels of VAE-encoded unmasked context. The expanded input channels let the transformer observe precise pixel boundaries between masked and unmasked regions, guaranteeing coherent lighting, perspective, and focal depth across edit boundaries.</p><p><strong>GRPO Alignment (</strong><strong>aarambh-vision-align):</strong> Per prompt, generate G candidate images under different noise seeds. Score each with CLIP alignment score, aesthetic predictor, and edge consistency. Compute relative advantages within the group and optimize with a clipped surrogate objective. No separate value/critic network — saving roughly 12GB VRAM — making RL alignment viable on Kaggle free tiers.</p><p><strong>Online Self-Learning (</strong><strong>aarambh-vision-selflearn):</strong> Given three to four reference images of a new visual subject, the system extracts their embeddings into an associative memory bank and computes a compact LoRA adapter delta. Adapter updates are gradient-orthogonalized against prior knowledge:</p><pre>g_ortho = g - Σₖ (⟨g, uₖ⟩ / ‖uₖ‖²) · uₖ</pre><p>Any update in Null(W_base) has exactly zero dot product with existing model knowledge. Catastrophic forgetting is eliminated mathematically.</p><p><strong>Provenance Watermarking (</strong><strong>aarambh-vision-safety):</strong> Every image passes through a spread-spectrum DCT watermark before export:</p><pre>C&#39;_{u,v} = C_{u,v} + α · Wᵢ · |C_{u,v}|    for (u, v) ∈ Ω_mid-freq</pre><p>Mid-frequency DCT coefficients carry enough perceptual energy to survive JPEG/WebP recompression while remaining completely imperceptible to human eyes. Every image is cryptographically traceable.</p><h3>The Four Model Scales: From Core i3 to Kaggle GPU</h3><p>Four model scales designed for commodity hardware progression:</p><pre>| Scale | Parameters | Resolution | Target Hardware |<br>|---|---|---|---|<br>| Tiny | ~40M | 64×64 | Intel Core i3 CPU |<br>| Small | ~120M | 128×128 | Kaggle T4 GPU (free) |<br>| Medium | ~350M | 256×256 | Kaggle P100 GPU (free) |<br>| Large | ~900M | 512×512 | Multi-GPU or fine-tuning from Medium |</pre><p>The Tiny scale is specifically designed to verify the full training pipeline — codec forward pass, transformer forward pass, loss computation, gradient update — on an everyday laptop in minutes. This is the Phase Zero milestone command that tells you the foundation is solid before you invest weeks in larger training runs.</p><h3>The 25-Crate Workspace</h3><pre>aarambh-vision-core       → types, config, error handling, SafeTensors I/O<br>aarambh-vision-tokenizer  → continuous latent VAE (the image codec)<br>aarambh-vision-textprep   → BPE tokenizer, multilingual text handling<br>aarambh-vision-data       → dataset loading, aspect-ratio bucketing<br>aarambh-vision-understand → CLIP-style vision-language understanding encoder<br>aarambh-vision-nn         → MMDiT blocks, adaLN-Zero, 2D-RoPE<br>aarambh-vision-kernel     → fused CUDA kernels for patchify/unpatchify<br>aarambh-vision-textencoder→ aarambh-studio LLM checkpoint wrapper<br>aarambh-vision-model      → full MMDiT model assembly<br>aarambh-vision-weights    → weight loading, conversion, checkpointing<br>aarambh-vision-train      → training loop, rectified flow matching loss<br>aarambh-vision-quant      → GGUF/INT8 quantization<br>aarambh-vision-finetune   → LoRA, DreamBooth-style adapter fine-tuning<br>aarambh-vision-align      → GRPO + DPO alignment<br>aarambh-vision-selflearn  → associative memory, gradient orthogonalization<br>aarambh-vision-edit       → inpainting, instruction-based editing<br>aarambh-vision-structure  → ControlNet-equivalent spatial conditioning<br>aarambh-vision-refprompt  → IP-Adapter-equivalent reference image prompting<br>aarambh-vision-upscale    → learned super-resolution upscaling<br>aarambh-vision-safety     → content filtering + DCT provenance watermarking<br>aarambh-vision-eval       → FID, CLIP score, LPIPS, aesthetic predictor<br>aarambh-vision-control    → runtime configuration management<br>aarambh-vision-inference  → streaming generation, Euler/DDIM/DPM++ samplers<br>aarambh-vision-serve      → Axum REST API, WebSocket streaming endpoint<br>aarambh-vision-studio     → CLI binary entry point</pre><h3>The 27-Phase Roadmap</h3><p>The full roadmap covers twenty-seven phases, each with:</p><ul><li>Exact engineering goals expressed in file-level task checklists</li><li>Automated unit and integration tests that verify correctness</li><li>A single milestone command that tells you whether the phase is complete</li><li>Hardware-specific guidance (which phases run on i3, which need Kaggle GPU)</li></ul><p>Phases 0–5 establish the pure infrastructure layer: workspace setup, VAE, text preparation, data loading, NN primitives, and model assembly. This gives you a verifiable, runnable MMDiT on your laptop before any training happens.</p><p>Phases 6–12 build the training, distillation, and production control pipeline: text-to-image training, few-step distillation, quantization, spatial conditioning, and reference prompting.</p><p>Phases 13–27 build the advanced production features: instruction editing, RL alignment, self-learning, safety watermarking, evaluation, inference optimization, and REST API serving.</p><h3>Why I’m Giving It Away</h3><p>The same reason I gave with aarambh-studio and aarambh-voice-studio.</p><p>I have spent months designing these systems properly — not rushing to a demo, but working through every architectural decision, every failure mode, every subsystem interaction. The 2D-RoPE coordinate decomposition. The staged VAE de-risking. The decoupled cross-attention for reference prompting. The GRPO advantage normalization. The gradient orthogonalization for self-learning.</p><p>Every piece was designed to be correct before being designed to be fast.</p><p>But I have another system that needs my attention. And the Rust AI community — which has grown remarkably over the past year, largely because of projects like this one — is better positioned than I am to bring this blueprint to life. They have the time. They have the motivation. And now they have a complete, production-grade architectural foundation that took months to design properly.</p><p>The hardest part of any ambitious open-source project is the architecture. Knowing which design decisions to make, which failure modes to anticipate, which mathematical approaches are proven versus experimental. This repository removes that uncertainty completely.</p><p>Fork it. Pick up Phase Zero. Run the milestone command. Tell me what you build.</p><h3>Resources</h3><ul><li><strong>GitHub:</strong> <a href="https://github.com/AarambhDevHub/aarambh-vision-studio">github.com/AarambhDevHub/aarambh-vision-studio</a></li><li><strong>YouTube Walkthrough:</strong> <a href="https://youtu.be/MONQdH5Bn_M">I Architected a FLUX Alternative in Pure Rust — And Why I’m Giving It Away</a></li><li><strong>Discord Community:</strong> <a href="https://discord.gg/HDth6PfCnp">discord.gg/HDth6PfCnp</a></li><li><strong>LinkedIn:</strong> <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">Darshan Vichhi — Rust Developer</a></li><li><strong>aarambh-studio (LLM):</strong> <a href="https://github.com/AarambhDevHub/aarambh-studio">github.com/AarambhDevHub/aarambh-studio</a></li><li><strong>aarambh-voice-studio:</strong> <a href="https://github.com/AarambhDevHub/aarambh-voice-studio">github.com/AarambhDevHub/aarambh-voice-studio</a></li></ul><p><em>Aarambh Dev Hub builds real AI systems from first principles in pure Rust. Language. Sound. Vision. The complete multimodal trinity — zero Python, zero PyTorch, zero compromises.</em></p><p><em>If this post helped you understand how modern diffusion models actually work beneath the API surface, the clap button is appreciated. And if you are forking this — I genuinely want to know which phase you are starting with.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a571cadfa0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Fully Designed an AI Voice Studio in Rust — No Python, No PyTorch. Why Won’t I Build It?]]></title>
            <link>https://aarambhdevhub.medium.com/i-fully-designed-an-ai-voice-studio-in-rust-no-python-no-pytorch-why-wont-i-build-it-8f702b727809?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f702b727809</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Tue, 15 Sep 2026 02:18:21 GMT</pubDate>
            <atom:updated>2026-09-15T02:18:21.031Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Aw1gRLSyyX8xEOoqhIZfWQ.jpeg" /></figure><p>Six months ago I sat down and made myself a promise. I was going to build a complete Large Language Model in pure Rust — not a wrapper, not a tutorial toy, not an API client. A real pretraining pipeline, attention kernels, GRPO alignment, Flash Attention — the whole thing. No Python. No PyTorch. No excuses.</p><p>That project became <a href="https://github.com/AarambhDevHub/aarambh-studio">aarambh-studio</a>, and it changed how I think about systems engineering.</p><p>When it reached its planned end-state at version four — fifty-five phases, four major versions, production-grade training on free Kaggle GPUs — the obvious question appeared. Language models generate tokens. But the world doesn’t only communicate through text. The world communicates through <strong>sound</strong>.</p><p>That question became aarambh-voice-studio.</p><p>And now I’ve designed it completely. Twenty-four modular Rust crates. A twenty-eight phase build roadmap. Full starter code. Three generative engines capable of synthesizing speech, cloning voices in three seconds, generating instrumental music, and composing full songs with verse-chorus-bridge structure.</p><p>I am not actively building it anymore. I am giving the entire blueprint away.</p><p>Here is exactly why — and exactly how it works.</p><h3>The Problem With Audio AI in Python (That Nobody Talks About Honestly)</h3><p>Search GitHub for “text to speech open source” right now and you’ll find the same pattern repeating itself endlessly. A requirements.txt with seventy-five dependencies. Three gigabytes of PyTorch wheels. CUDA driver conflicts. Unmaintained C extensions from 2017. Six gigabytes of RAM consumed before a single millisecond of audio is generated.</p><p>In the text LLM world, engineers have tolerated this because model size dwarfs runtime overhead. But audio is categorically different.</p><p><strong>Audio is real-time.</strong> A fifty millisecond garbage collector pause in Python doesn’t cause a missed word in a chat response — it causes an audible click or pop in a voice stream that the listener immediately notices and judges. Latency and determinism in audio aren’t nice-to-haves. They are the product.</p><p>Worse, the entire Python audio AI ecosystem is fragmented. Bark for speech. MusicGen for music. EnCodec for compression. None of them share a common tensor representation. None share a config format. If you want to take an instrumental track from MusicGen, mix it with a singing voice from a separate model, and master it into a stereo FLAC — you are gluing shell scripts together and bouncing temporary files on disk.</p><p>aarambh-voice-studio was designed to reject this entire paradigm from the first line of Cargo.toml.</p><h3>Why Rust, and Why Candle</h3><p>By building on top of <a href="https://github.com/huggingface/candle">Candle</a> — Hugging Face’s pure-Rust minimalist tensor framework — the entire studio has zero Python runtime dependency. Memory allocation is deterministic and bound by Rust’s ownership model. When an audio buffer goes out of scope, the memory is reclaimed instantly, without a GC pause.</p><p>The entire system compiles to a single native binary. No interpreter startup. No dependency hell. On edge devices, serverless containers, or embedded systems — just ship the binary.</p><p>Most importantly: voice, music, singing, and mixing are not separate tools bolted together. They are three engines built on top of a <strong>single, unified transformer core</strong> and a <strong>single shared neural audio codec</strong>. A voice token, an instrumental token, and a pitch tracking token all live in the same mathematical latent space, governed by the same Rust type system. That unification is what makes a true studio — not a collection of scripts — possible.</p><h3>The Foundation: Why 44,100 Samples Per Second Breaks Transformers</h3><p>To understand the architecture, you first need to understand why audio is fundamentally harder than text for a neural network to model.</p><p>A text LLM like aarambh-studio tokenizes an English sentence into twenty or thirty token IDs using Byte Pair Encoding. A model generating five tokens per second is producing readable text at conversational speed.</p><p>Now consider audio. CD-quality sound is sampled at 44,100Hz. One second of audio is 44,100 individual floating-point amplitude values. A ten-second clip is nearly half a million data points. Feed those raw samples into an autoregressive transformer and the sequence length explodes — and since attention scales quadratically with sequence length, even two seconds of audio would bring a high-end GPU to a crawl.</p><p>Early systems like WaveNet tried generating one sample at a time at 16kHz. Real-time inference was nearly impossible. Spectrogram-based approaches like HiFi-GAN were faster but discarded phase information, producing metallic artifacts and unnatural robotic timbres.</p><p>The modern breakthrough: <strong>Neural Audio Codecs</strong>.</p><p>Instead of generating raw waveforms, a codec encoder-decoder network compresses continuous audio into discrete acoustic tokens — and reconstructs perfectly on the other end. Think of it as the audio equivalent of a tokenizer, except the “tokenizer” is a deep neural network trained with perceptual losses to preserve every nuance of human timbre and instrumental character.</p><h3>The 12.5Hz Codec: The Most Critical Crate in the Entire Workspace</h3><p>Most neural audio codecs — EnCodec, DAC, SoundStream — operate between 50Hz and 75Hz. That means 50 to 75 acoustic tokens per second. For a ten-second audio clip, that is 500 to 750 tokens. Already stressful for a transformer backbone.</p><p>The architecture decision that separates aarambh-voice-studio from everything else: we compress to <strong>12.5Hz</strong>.</p><p>That is twelve and a half tokens per second. A ten-second clip becomes 125 tokens — small enough for a standard transformer running on an Intel Core i3 laptop to process in real time, without a GPU.</p><p>This is not magic. It requires a carefully designed codec architecture:</p><ul><li>A <strong>Residual Vector Quantizer (RVQ)</strong> operating across multiple codebooks at progressively finer acoustic resolutions</li><li>A <strong>transformer bottleneck layer</strong> between the convolutional encoder and the RVQ, which compresses rich acoustic features into a more compact semantic representation</li><li><strong>Semantic distillation</strong> from self-supervised speech representations (trained on Whisper-style contrastive signals), which teaches the codec to preserve linguistic and phonemic information rather than just raw waveform shape at extremely low frame rates</li></ul><p>If this codec fails, everything downstream fails. That is why aarambh-vision-codec is Phase Zero of the roadmap, and it has the most extensive test suite of any crate in the workspace.</p><h3>The Three-Engine Architecture</h3><p>Once the codec foundation is solid, the rest of the studio is built on top of a <strong>shared transformer backbone</strong> (aarambh-vision-nn) that is directly adapted from the same decoder-only architecture used in aarambh-studio. Here is how the three engines are layered:</p><h3>Engine One: The Voice Engine</h3><p>The voice engine (aarambh-voice-model) handles three related tasks unified by the same codec token vocabulary:</p><p><strong>Text-to-Speech (TTS):</strong> The phoneme preparation crate (aarambh-voice-textprep) converts raw text through a multilingual Grapheme-to-Phoneme conversion pipeline, producing an international phoneme alphabet representation. This allows accurate pronunciation across English, Hindi, and code-switched multilingual speech without retraining. The transformer generates acoustic tokens that the codec decoder converts to waveforms.</p><p><strong>Zero-Shot Voice Cloning:</strong> Given as little as three seconds of a speaker’s voice, the system extracts a speaker embedding from the codec’s latent space and conditions the transformer through cross-attention. The model then generates new speech in that voice without any fine-tuning or per-speaker training. The key insight: because speaker identity is captured in the codec’s compressed latent space, you only need a few hundred acoustic tokens — not thousands of raw waveform samples — to characterize a speaker completely.</p><p><strong>Continuous Emotional Control:</strong> Rather than discrete emotion labels (happy, sad, angry), the voice engine uses an eight-dimensional continuous emotional control space derived from the valence-arousal emotional circumplex model. The conditioning MLP projects this eight-dimensional vector into cross-attention keys and values, giving the model precise, smooth, expressive control over prosody, pitch range, and speaking pace — the difference between a system that can simulate warmth versus one that actually sounds human.</p><h3>Engine Two: The Music Engine</h3><p>The music engine introduces a critical architectural insight that most music AI projects skip entirely: <strong>you cannot generate good music without first understanding music.</strong></p><p>Before the music transformer generates a single token, a dedicated music understanding module (aarambh-voice-understand) runs on the input prompt to classify:</p><ul><li><strong>Genre:</strong> Electronic, orchestral, jazz, folk, ambient</li><li><strong>Tempo:</strong> Beats-per-minute range</li><li><strong>Key and Mode:</strong> Major, minor, Dorian, Lydian</li><li><strong>Mood:</strong> Energetic, melancholic, tense, peaceful</li><li><strong>Instrumentation Density:</strong> Solo, chamber, ensemble, full production</li></ul><p>These classifiers are trained first — in Phase Ten of the roadmap — using auto-labeled datasets. They then serve as conditioners for the music generation transformer, which produces acoustic tokens corresponding to instrumental sounds. The separation of understanding from generation is what prevents the common failure mode where music AI systems produce technically playable but emotionally incoherent output.</p><h3>Engine Three: The Singing Engine</h3><p>The singing engine is the most complex subsystem in the entire workspace, for one reason: singing requires the system to simultaneously control phonemes (what words to say), pitch (what melody note to hit), duration (how long to hold each syllable), and breath support timing.</p><p>The architecture layers:</p><ol><li>A <strong>melody pitch tracker</strong> that converts a MIDI sequence or fundamental frequency contour into frame-level pitch conditioning vectors</li><li>A <strong>lyrics-to-duration aligner</strong> that stretches phoneme sequences to match the musical rhythm using attention-based forced alignment</li><li>The shared transformer backbone generating codec tokens conditioned on both pitch and phoneme streams simultaneously</li><li>An optional <strong>diffusion refinement head</strong> — a four-step Rectified Flow Matching pass that post-processes the raw codec tokens to add breath sounds, vibrato, and natural vocal articulation that autoregressive sampling alone rarely captures</li></ol><h3>The Song Composer: Structure Before Sound</h3><p>One of the most neglected problems in AI music generation is structure. When you or I listen to a song we love, we don’t just hear a continuous stream of pleasant sounds. We hear a narrative arc — an intro that builds tension, a verse that tells a story, a chorus that releases emotion, a bridge that surprises.</p><p>Most open-source music AI systems generate audio as one flat continuous sequence. The result sounds like someone improvising endlessly without ever arriving at a point.</p><p>The aarambh-voice-studio Song Composer (aarambh-voice-compose) separates structure planning from audio generation:</p><p><strong>Phase 1 — Structure Planning:</strong> Given a prompt like “a melancholic indie folk song about missing home,” the structure planner predicts the full song layout — intro (8 bars), verse 1 (16 bars), chorus (8 bars), verse 2 (16 bars), chorus (8 bars), bridge (8 bars), outro (4 bars). It also predicts per-section instrumentation density and mood targets.</p><p><strong>Phase 2 — Stem Generation:</strong> Each structural section is generated independently by the music engine and singing engine, conditioned on its section-specific mood and instrumentation plan.</p><p><strong>Phase 3 — Multi-Track Mixing:</strong> The individual audio stems are mixed using a learned gain and EQ prediction network that balances vocal presence against instrumental density, applies dynamic range compression appropriate to the genre, and produces a final stereo master at the correct loudness target.</p><p>This architecture is why the Song Composer is one of the most exciting and underexplored research directions in the entire roadmap. If you want to build one part of this studio and publish a paper, the Structure Planner is likely where the highest-impact contribution lives.</p><h3>Online Self-Learning: New Voices in Seconds, Zero Forgetting</h3><p>One of the most practically valuable features of aarambh-voice-studio is its online self-learning subsystem (aarambh-voice-selflearn), which borrows architectural principles from our separate Manas continual learning project.</p><p>The problem it solves: a user records three seconds of a new speaker’s voice and wants the system to remember that voice permanently — not just for the current session, but the next time they open the application, even after the model has been updated with new data in the meantime.</p><p>Naive fine-tuning would cause catastrophic forgetting — learning the new voice by overwriting old knowledge.</p><p>The aarambh approach:</p><ol><li>The new speaker recording is passed through the codec encoder, producing a small sequence of high-level speaker embeddings</li><li>These embeddings are stored in an <strong>indexed associative memory bank</strong> alongside a compact LoRA adapter delta computed from the few-shot examples</li><li>At inference time, the memory bank performs nearest-neighbor lookup to retrieve the relevant speaker adapter, which is dynamically applied on top of the frozen base model weights</li><li>New adapter updates are <strong>gradient-orthogonalized</strong> against the subspace spanned by existing stored adapters, ensuring that learning a new voice does not degrade any previously stored voice</li></ol><p>The result: unlimited voice registration over time, with no catastrophic forgetting and no need to ever retrain the base model.</p><h3>Alignment: GRPO and DPO Without a Human Labeler in Sight</h3><p>Getting audio models to produce genuinely good output — not just technically valid sequences — requires alignment beyond supervised training. aarambh-voice-studio’s alignment crate (aarambh-voice-align) implements both Group Relative Policy Optimization (GRPO) and Direct Preference Optimization (DPO), adapted for discrete acoustic token sequences.</p><p>For GRPO: given a prompt, the model generates a group of G candidate audio outputs with different noise seeds. An automatic reward model scores each candidate on:</p><ul><li><strong>Speaker similarity:</strong> cosine distance between generated and reference speaker embeddings</li><li><strong>Intelligibility:</strong> phoneme error rate as measured by a lightweight CTC speech recognizer</li><li><strong>Naturalness:</strong> a learned MOS (Mean Opinion Score) predictor trained on human preference labels</li><li><strong>Prosody adherence:</strong> pitch contour correlation with an emotional target</li></ul><p>The relative advantage A_i = (R_i — mean(R)) / std(R) is computed within the group, and the policy is updated via a clipped surrogate objective — the same mathematical machinery from language model RLHF, but operating over acoustic token policies rather than text token distributions.</p><p>No human annotators needed. No reward hacking risks from a single omnipotent reward model. Clean, mathematically principled improvement.</p><h3>Ethical Engineering: Safety Baked Into the Type System</h3><p>The safety architecture of aarambh-voice-studio (aarambh-voice-safety) treats consent and provenance not as policy decisions implemented by runtime checks, but as Rust type-system constraints enforced at compile time.</p><p>Every synthesized voice output must carry a <strong>ConsentToken</strong> in its type signature — a cryptographically signed structure that attests the speaker whose voice was used in cloning provided explicit informed consent for this use case. Code that attempts to synthesize audio without a valid ConsentToken literally cannot compile. There is no runtime bypass. The type checker enforces it.</p><p>Additionally, every audio file produced by the studio has a <strong>psychoacoustic watermark</strong> imperceptibly embedded in the frequency domain. The watermark encodes a SHA-256 hash of the generation session metadata (prompt, speaker identity, timestamp, model version) in frequency bands that survive typical lossy recompression to MP3 or OGG, but remain completely inaudible to human listeners.</p><p>This is not ethical theater. These are engineering decisions that make abuse materially harder at the systems level.</p><h3>The 24-Crate Architecture at a Glance</h3><p>The full workspace spans twenty-four library crates plus one CLI binary:</p><pre>| Layer | Crate | Responsibility |<br>|---|---|---|<br>| L0 | `aarambh-voice-core` | Types, config, errors, SafeTensors I/O |<br>| L0 | `aarambh-voice-kernel` | Low-level DSP primitives, resampling |<br>| L1 | `aarambh-voice-codec` | Neural audio codec — RVQ encoder/decoder |<br>| L1 | `aarambh-voice-textprep` | G2P, phoneme sequences, multilingual |<br>| L2 | `aarambh-voice-data` | Dataset loading, augmentation, bucketing |<br>| L2 | `aarambh-voice-understand` | Music understanding classifiers |<br>| L2 | `aarambh-voice-nn` | Transformer primitives, RoPE, SwiGLU |<br>| L3 | `aarambh-voice-model` | Three-engine model definition |<br>| L3 | `aarambh-voice-weights` | Weight loading, conversion, checkpointing |<br>| L4 | `aarambh-voice-train` | Training loop, loss functions |<br>| L4 | `aarambh-voice-quant` | GGUF/GPTQ quantization |<br>| L5 | `aarambh-voice-finetune` | LoRA, adapter, voice personalization |<br>| L5 | `aarambh-voice-align` | GRPO + DPO acoustic alignment |<br>| L5 | `aarambh-voice-selflearn` | Associative memory, gradient orthogonalization |<br>| L6 | `aarambh-voice-dsp` | Mastering chain: EQ, compression, LUFS |<br>| L6 | `aarambh-voice-compose` | Song Structure Planner + stem mixing |<br>| L7 | `aarambh-voice-safety` | Consent tokens + psychoacoustic watermarking |<br>| L7 | `aarambh-voice-eval` | MOS, PER, UTMOS, speaker similarity |<br>| L8 | `aarambh-voice-inference` | Streaming generation, speculative decoding |<br>| L8 | `aarambh-voice-serve` | Axum REST API, WebSocket streaming |<br>| L9 | `aarambh-voice-studio` | CLI binary — the entry point |</pre><p>Four model scales designed for commodity hardware:</p><ul><li><strong>Tiny (~10M params):</strong> Verifies on CPU, Intel Core i3, no GPU needed</li><li><strong>Small (~55M params):</strong> Trains on free Kaggle T4 GPU tier in days</li><li><strong>Medium (~170M params):</strong> Full production quality on Kaggle P100</li><li><strong>Large (~450M params):</strong> High-fidelity synthesis for studio mastering</li></ul><h3>The 28-Phase Roadmap: How to Actually Build This</h3><p>The repository includes a detailed twenty-eight phase roadmap, with each phase defining exact engineering goals, file-by-file task checklists, automated tests you can run to verify milestone completion, and a single command milestone test that proves the phase is done.</p><p>The first seven phases build the Mini 7-Crate TTS Variant — a stripped-down build path targeting a working zero-shot voice cloning engine in one to two weeks. This is designed for engineers who want to contribute meaningfully without committing to the full twenty-four crate vision immediately.</p><p>Phases Eight through Twenty-Eight complete the full studio: music understanding, music generation, singing synthesis, the song composer, GRPO alignment, online self-learning, and production-grade inference serving.</p><p>Every phase is verifiable. You run the milestone command, and the system tells you whether it passed.</p><h3>Why I’m Giving It Away</h3><p>The honest answer is the same one I gave at the end of aarambh-studio: I think the most valuable thing I can do with these architectural designs is make them completely free and available to people who have the time, skills, and motivation to actually build them out.</p><p>I spent months designing this system properly — not rushing to a demo, but thinking through every architectural decision, every failure mode, every interaction between subsystems. The codec design. The emotional control space. The song structure planner. The safety type system. Every piece of this was designed to be correct before it was designed to be fast.</p><p>But designing a system and building a system are two different projects. And I have another system calling for my attention.</p><p>The open-source community — particularly the growing ecosystem of Rust engineers who are serious about systems programming for AI — is better positioned than I am to take these blueprints and realize them. That is not a resignation. That is a deliberate decision about where leverage lives.</p><p>The repository is ready. The roadmap is complete. The starter code is in place.</p><p>If you want to build an AI audio studio in pure Rust from first principles — one that actually understands music, actually remembers voices, and actually treats safety as a systems concern rather than an afterthought — this is your starting point.</p><h3>Resources</h3><ul><li><strong>GitHub:</strong> <a href="https://github.com/AarambhDevHub/aarambh-voice-studio">github.com/AarambhDevHub/aarambh-voice-studio</a></li><li><strong>YouTube Walkthrough:</strong> <a href="https://youtu.be/l0bYU1G0twI">I Fully Designed an AI Voice Studio in Rust — No Python, No PyTorch</a></li><li><strong>Discord Community:</strong> <a href="https://discord.gg/HDth6PfCnp">discord.gg/HDth6PfCnp</a></li><li><strong>LinkedIn:</strong> <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">Darshan Vichhi — Rust Developer</a></li></ul><p><em>Aarambh Dev Hub engineers real AI systems from first principles in pure Rust. No Python. No PyTorch. No compromises.</em></p><p><em>If this post was useful, clapping helps other engineers find it. And if you are planning to fork and build — I genuinely want to hear which part you are starting with.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f702b727809" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Gave My From-Scratch Rust LLM Ears, a Memory, and a Council of Agents — Then I Called It Finished…]]></title>
            <link>https://aarambhdevhub.medium.com/i-gave-my-from-scratch-rust-llm-ears-a-memory-and-a-council-of-agents-then-i-called-it-finished-53f374e0fdb9?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/53f374e0fdb9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sat, 22 Aug 2026 02:48:34 GMT</pubDate>
            <atom:updated>2026-08-22T02:48:34.398Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Gave My From-Scratch Rust LLM Ears, a Memory, and a Council of Agents — Then I Called It Finished (aarambh-studio v4.0.0, Still Zero Python)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*43ijsTjOUFUgsW7UGHodTw.png" /></figure><p>At the end of my v3.0.0 post, I wrote a short “what comes next” list: crates.io publishing “for real this time,” sparse/grouped MoE dispatch, multi-node training, fuller OpenAI API parity, and audio — the one modality nobody had touched yet.</p><p>Four of those five shipped in this release.</p><p>The fifth one I reversed on purpose. And that reversal is a bigger part of this story than any single feature.</p><p><strong>aarambh-studio v4.0.0</strong> is out — fifteen phases, eighteen pull requests, one architecture rebrand, and a decision I didn’t see coming when I wrote that list two releases ago: this is the last planned version. Not paused. Not “on hold while I figure out v5.” Finished, on purpose, with the reasoning written down.</p><p>Still Rust. Still Candle as the only tensor backend. Still not one line of Python anywhere in the dependency graph.</p><h3>Quick Summary</h3><ul><li><strong>aarambh-studio v4.0.0</strong> (formerly aarambh-ai) is the fourth and <strong>final planned</strong> release of a complete LLM pipeline built from scratch in Rust — pretraining through serving, zero Python.</li><li>It finishes the attention family with <strong>Multi-Head Latent Attention</strong>, adds <strong>native audio understanding</strong>, <strong>sparse MoE dispatch on CUDA</strong>, <strong>multi-node training</strong>, <strong>test-time compute scaling</strong>, <strong>RLAIF</strong>, <strong>sandboxed multi-agent tool execution</strong>, <strong>from-scratch RAG</strong>, <strong>model merging</strong>, a <strong>public inference server</strong>, <strong>red-team evaluation</strong>, and <strong>auto-generated model cards</strong>.</li><li>The workspace grew to <strong>21 crates</strong>, up from 19 — two genuinely new, two extended, zero renamed, zero removed.</li><li>The project’s crates.io policy got <strong>reversed</strong>, not extended: source-only, forever, confirmed as final.</li><li>No pretrained weights, checkpoints, adapters, or GGUF files ship with any version, including this one. You train and align your own.</li></ul><h3>What Is aarambh-studio, in One Paragraph?</h3><p><strong>aarambh-studio</strong> — Sanskrit <em>aarambh</em>, “beginning” — is a decoder-only transformer LLM application built as a Rust workspace on top of the <a href="https://github.com/huggingface/candle">Candle</a> tensor backend. It used to be called aarambh-ai. v1.0.0 was the text-only foundation: pretraining, SFT, LoRA/QLoRA, GRPO alignment, INT4/GGUF quantization, Flash Attention v2 CUDA kernels, a thinking engine, safety guardrails, and CPU-capable self-learning. v2.0.0 made it multimodal and self-serving — vision, MoE, multi-GPU training, DPO, speculative decoding, grammar-constrained tool calling, its own OpenAI-compatible server. v3.0.0 rebuilt attention itself — Gated DeltaNet, DeepSeek Sparse Attention, fine-grained MoE, native video and document understanding, long-horizon tool chains.</p><p><strong>v4.0.0 finishes the arc.</strong> The rename happened in this release, in the very first pull request — because by this point the project had stopped being a research repo you’d cargo add pieces of and become an actual, self-contained application you run.</p><h3>The Reversal Nobody Asked For</h3><p>Here’s the part of this release I think is more interesting than any single phase.</p><p>Back in the v3.0.0 post, I listed “crates.io publishing, for real this time” as a known boundary and a v4 candidate. I meant it. It felt like the obvious next milestone — a Rust project that never publishes to crates.io looks unfinished by Rust community standards.</p><p>Then I actually sat down to plan v4 and asked the honest question: publish <em>what</em>? aarambh-studio isn’t a library someone imports into their own Cargo.toml. It’s an application — a CLI binary with 20 internal crates behind it, none of which are designed to be consumed independently. Publishing them to crates.io wouldn’t make the project more usable. It would just be checking a box that doesn’t apply to what this actually is.</p><p>So v4.0.0 doesn’t publish to crates.io. It <strong>confirms, permanently, that it never will.</strong> All 21 workspace packages ship publish = false. Every prior version&#39;s release policy pointed toward this eventually changing. v4.0.0 is the version that says: no, this was the answer the whole time, we just hadn&#39;t stated it as finally as we could.</p><p>I’m including this near the top because it’s the same discipline that runs through the rest of the release — measure, don’t assume, and be willing to say a plan you announced publicly was wrong.</p><h3>Finishing the Attention Family: Multi-Head Latent Attention</h3><p>v3.0.0 replaced most of the model’s attention layers with Gated DeltaNet (constant-size recurrent state) and kept a minority on DeepSeek Sparse Attention. Phase 41 adds a third kind: <strong>Multi-Head Latent Attention (MLA)</strong>, the mechanism DeepSeek-V2 popularized for cutting inference-time KV cache cost.</p><p>The idea, at a conceptual level: instead of caching full-size key and value projections per head, down-project them into a much smaller shared latent vector before the cache ever stores anything, then reconstruct what attention needs from that latent representation at read time. The cache holds the compressed latent, not the full K/V tensors — so the memory a long conversation costs shrinks without touching what the model can attend to.</p><p>The detail that mattered most to me practically, because it’s the same principle that made Gated DeltaNet adoptable in v3: <strong>MLA is retrofittable onto existing checkpoints via continued pretraining.</strong> You don’t throw away a v1, v2, or v3 training run to adopt it. With MLA in place, the v4.0 attention stack is a complete, four-way family — full scaled dot-product, Gated DeltaNet, DSA, and MLA — selectable per layer through TOML, with dense fallback retained for correctness-first configurations. A new eval --kv-cache-report flag quantifies the saving per layer, so &quot;materially cuts KV memory&quot; is a number you can print, not a claim you take on faith.</p><h3>Teaching It to Listen: Native Audio</h3><p>Vision landed in v2. Video and documents landed in v3. Phase 42 closes the modality gap that was left: <strong>audio.</strong></p><p>A frozen audio spectrogram transformer paired with a trainable projector turns raw sound into the same kind of token-space embedding vision already produces — the fusion mechanism is deliberately identical to how images and video frames get spliced into the sequence, so the decoder doesn’t need to know or care that some of its context came from a waveform instead of a picture.</p><p>The honesty boundary I want to be upfront about: decoding is pure-Rust WAV PCM only. No FFmpeg binding, no MP3, no FLAC, no Ogg. That’s not an oversight — it’s the same “no shelling out to a system codec, no hidden C dependency” rule that governed video decoding in v3, applied consistently to a new format. A new reserved &lt;audio&gt; token marks the boundary in the stream, and audio DoRA/QDoRA tuning shares the exact same VLM training code path every other modality already uses. One training path, four modalities now instead of three.</p><h3>Making MoE Actually Sparse — Where It Counts</h3><p>v2 and v3 both shipped Mixture of Experts with <strong>dense masked dispatch</strong>: every expert computes on every token, gets masked, then weighted by the router. Correct, simple, and deliberately not the efficient version — I called that out explicitly in both prior posts as a known, documented tradeoff.</p><p>Phase 43 resolves it, on CUDA: real sparse dispatch, where only the tokens actually routed to a given expert touch that expert’s weights. The CPU path keeps the original dense masked-matmul implementation, because sparse dispatch’s real advantage only shows up with hardware-level gather/scatter, and correctness-first CPU portability still matters. The two paths are proven equivalent with a per-layer dense-vs-sparse numerical test — the same “measure the claim, don’t just ship the idea” instinct that’s run through every version so far.</p><h3>Test-Time Compute, RLAIF, and a Second Set of Eyes on Alignment</h3><p>Phase 45 adds a family of inference-time techniques that trade extra compute for better answers without touching model weights: best-of-N sampling, self-consistency (majority vote across N generations), verifier-based selection, and a heuristic process-reward scorer with a trait left open for a future trained head. It’s text-only in this release, and I’m calling that a heuristic on purpose rather than dressing it up as a trained reward model it isn’t.</p><p>Phase 46 adds <strong>RLAIF</strong> as a third alignment signal alongside GRPO and DPO: a judge model scores preference pairs, and those scores feed the existing DPO pipeline rather than requiring a new training loop. It’s deliberately kept separate from the self-learning loop — same framing v4’s model-merging feature uses: offline, measured, never assumed to help until the eval harness says so.</p><h3>Agents That Can’t Escalate Their Own Permissions</h3><p>This is the phase I think has the most interesting engineering story in the whole release.</p><p>Phase 47 adds sandboxed tool execution — opt-in via agent --execute-tools, restricted to a closed-world set of operator-authorized named capabilities like read_file_in_workdir. Not arbitrary code. Not shell access. A fixed menu of things the model is allowed to do, decided by whoever&#39;s running it.</p><p>Phase 48 builds on that with <strong>multi-agent orchestration</strong>: one orchestrating process can delegate independent sub-tasks to multiple parallel sandboxed sub-chains, then merge their results back through the existing tool-result ingestion path, applied recursively. The moment you allow that kind of delegation, you’ve created a way for permissions to quietly expand — a sub-agent doing something its orchestrator was never authorized to do in the first place.</p><p>The fix is a single call: AuthorizationScope::intersect. A sub-agent&#39;s scope is only ever a subset of its parent&#39;s. It can never grow through delegation, only shrink. Alongside that, there are two more hard, operator-set bounds — max sub-agent count and a max total execution-time budget — and sub-chains run sequentially by default. True parallelism would need a Send + Sync ChainDecoder, which is out of scope for this release; running sequentially and saying so honestly beat quietly cutting a corner on thread safety.</p><h3>RAG That Doesn’t Need a Trained Model to Start Working</h3><p>Phase 49 adds a new crate, aarambh-studio-retrieve — chunking, embedding heads, a navigable small-world graph ANN index, and prompt augmentation before generation, with no external vector database dependency for the core index.</p><p>The detail I like most: the default tested embedder is <strong>weight-free</strong> — a hashing-based embedding rather than a trained one. That means the entire retrieval pipeline runs end-to-end without requiring a trained embedding checkpoint first. You can stand up RAG on day one, before you’ve trained anything, and swap in a learned embedder later once you have one worth training.</p><h3>Merging Models Without a New Crate</h3><p>Phase 50 adds a full model-merging toolkit — linear averaging (Model Soups), SLERP, task-vector arithmetic, TIES-Merging, and DARE — with hard shape and schema validation before any write, CPU-first f32 math, and quality judged by the eval harness, never assumed.</p><p>The part worth calling out structurally: this didn’t need a new crate. It’s a new merge.rs module inside the existing aarambh-studio-weights crate, which already owned SafeTensors and GGUF handling. Same instinct that kept vision from touching the transformer core in v2, and kept forgetting diagnostics from needing a new dependency in v3 — new capability, existing layer, zero disruption to anything above or below it.</p><h3>A Server You Can Actually Put Behind Other People</h3><p>Phase 51 extends the OpenAI-compatible server from v2 with opt-in multi-tenant API-key auth, per-key RPM/TPM rate limiting, per-tenant in-flight request isolation, and prompt-prefix KV caching with longest-prefix reuse under an LRU eviction policy and a configurable memory ceiling. The loopback-only, unauthenticated single-user mode from v2 stays the default — none of this is forced on, and none of it turns the project into a hosted product. No billing. No auto-scaling. Still self-hosted, still your hardware.</p><h3>A Reserved Token, and the Discipline Behind It</h3><p>Phase 52 gives the model a first-class system role marker. Sounds trivial until you remember the project’s own rule, in force since v2: <strong>token IDs are append-only, never reassigned.</strong> ID 7 has been IMAGE since v2.0.0 and stays IMAGE forever — so the new SYSTEM marker had to take the next free ID, 17, paired with a chat_template_version tag on tokenizer config and checkpoint metadata, and a fail-loud startup check if a checkpoint and its tokenizer ever disagree about which version they&#39;re speaking.</p><p>I shipped the first version of that marker, then had to go back and rename the literal string itself — it originally didn’t read as an actual chat-template token — before it landed as &lt;|system|&gt;. One-line fix. Took more careful thought than the multi-agent orchestrator did, which tells you something about where the real difficulty in a project like this tends to hide.</p><h3>Checking Your Own Work Before You Call It Safe</h3><p>Phase 53 runs a single, systematic, end-to-end red-team pass against the complete v4.0 attack surface — the safety layer and system-turn precedence, the closed-world tool-execution boundary, the orchestrator’s hard bounds, and the public server’s auth/rate-limit/tenant-isolation surface. It’s a 24-case corpus, hand-authored or sourced from free public material only, with a labeled expected outcome per case. A failing case gets surfaced plainly in the report — never silently dropped.</p><p>Phase 54 then makes that report load-bearing: model cards are <strong>assembled</strong>, not hand-written, from an eval-harness scorecard, the red-team report, and static metadata — so a card can’t silently drift out of sync with the numbers behind it. Generation <strong>fails loudly if no red-team report is present, or if the report isn’t clean.</strong> A checkpoint cannot get a model card without a clean red-team pass first. That’s a real gate, not a formality.</p><h3>Closing the Loop: Phase 55</h3><p>The last phase is the release itself. Every one of the 21 workspace packages gets bumped to 4.0.0 — the version jumps from 4.0.0-alpha.14, Cargo.lock regenerates with 21 matching member entries, and every release command runs --locked. The release audit script gets extended to gate ROADMAP_V4.md for unchecked tasks and assert that all four v4-touched crates — aarambh-studio-audio, aarambh-studio-retrieve, aarambh-studio-agent, aarambh-studio-serve — are actually present in the workspace, so a v4 surface can&#39;t quietly get dropped before release. Documentation across ARCHITECTURE_V4.md, ROADMAP_V4.md, SELF_LEARNING_V4.md, README.md, and CHANGELOG.md gets finalized with the complete v1 → v4 arc.</p><p>RELEASE.md itself got a small, honest correction here too — it was still pointing at a stale phase40_release_audit.sh script name from an earlier draft and got fixed to the actual scripts/phase28_release_audit.sh. Small detail. The kind that&#39;s easy to let slide in a &quot;final&quot; release and exactly the kind I didn&#39;t want sliding in this one.</p><h3>The Hard Parts Nobody Warns You About in v4</h3><p><strong>Permissions have to shrink, never grow, across delegation.</strong> The AuthorizationScope::intersect design for multi-agent orchestration sounds obvious once it&#39;s named. Getting the boundary condition right — a sub-agent&#39;s scope as strictly a subset, never a sibling or superset, of its parent&#39;s — is the difference between a sandboxing feature and a privilege-escalation bug wearing a sandboxing feature&#39;s clothes.</p><p><strong>A red-team gate only works if it can actually block something.</strong> It would have been easy to make model-card generation a soft warning when the red-team report is missing or dirty. Making it fail loudly, with no override, is what turns “we ran a safety pass” into “a checkpoint literally cannot claim to be documented as safe without one.”</p><p><strong>Weight-free doesn’t mean assumption-free.</strong> Choosing a hashing-based default embedder for RAG so the pipeline works without a trained checkpoint was the easy part. Making sure that decision was explicit — documented as the <em>default</em>, not silently treated as the only option — mattered more, because “weight-free” quietly becomes “permanently untrained” if nobody writes down that it’s meant to be swappable.</p><p><strong>Reversing a public commitment is harder than making one.</strong> Writing “crates.io publishing, for real this time” in the v3 post was easy. Going back and explaining, in the v4 release notes, exactly why that was the wrong plan — rather than quietly dropping it and hoping nobody noticed — took more words than most of the actual features did.</p><h3>How to Run It</h3><pre># Clone and build<br>git clone https://github.com/AarambhDevHub/aarambh-studio.git<br>cd aarambh-studio<br>git checkout v4.0.0<br>cargo build --release -p aarambh-studio --locked<br><br># Verify the version<br>target/release/aarambh-studio --version<br># aarambh-studio 4.0.0<br># Run the red-team evaluation pass<br>target/release/aarambh-studio eval --redteam --help<br># Generate a model card (requires a clean red-team report first)<br>target/release/aarambh-studio eval --generate-model-card --help<br># Merge two compatible checkpoints<br>target/release/aarambh-studio merge --help<br># Start the multi-tenant-capable inference server<br>target/release/aarambh-studio serve --help</pre><p>Same distribution policy as v1 through v3, now stated as final rather than provisional: all 21 workspace crates stay publish = false, GitHub&#39;s automatic source archive is the only release asset, and no pretrained checkpoint, adapter, tokenizer, optimizer state, or GGUF file ships with the release. You train, align, quantize, and serve your own.</p><h3>What v4.0.0 Deliberately Does Not Do</h3><ul><li>MoE sparse dispatch is CUDA-only; CPU keeps the correctness-first dense masked path.</li><li>Multi-node training is data-parallel only — not model- or pipeline-parallel — with a single-retry fault policy.</li><li>Test-time compute scaling is text-only, and its process-reward scorer is a documented heuristic, not a trained model.</li><li>Tool execution is strictly closed-world, named-capability only — never arbitrary code or shell access.</li><li>Multi-agent sub-chains run sequentially by default; true parallel execution is explicitly out of scope for this release.</li><li>RAG’s core index is the from-scratch pure-Rust graph ANN — no external vector database is permitted for it.</li><li>Video stays visual-only H.264 MP4; audio stays WAV PCM only; documents stay pixel-based with no OCR or table parser.</li><li>The public server makes self-hosted multi-tenancy <em>possible</em> — it is not a hosted product. No billing, no auto-scaling.</li><li>crates.io publishing is not deferred to a future version. It’s not happening, ever, by design.</li></ul><h3>Why v4.0.0 Is Also the Last One</h3><p>Twenty-one crates now, up from nineteen in v3. Two genuinely new (aarambh-studio-audio, aarambh-studio-retrieve), two extended in place (aarambh-studio-agent, aarambh-studio-serve), zero renamed, zero removed. Model merging — a feature big enough that I&#39;d have expected it to need its own crate — landed as one new module inside a crate that already existed. That&#39;s the same payoff strict, compiler-enforced crate layering has produced every release so far: growth in one direction doesn&#39;t quietly break another.</p><p>But the crate count isn’t really the point of this section. Here’s the actual one.</p><p>When v1.0.0 shipped, the open question was whether you could build a working LLM pipeline in Rust at all, without Python anywhere in the stack. v2 asked whether that codebase could grow in multiple directions without the architecture collapsing. v3 asked a harder version of the same question by rebuilding the part everything else depends on. v4 asks the question none of the earlier releases needed to: <strong>when do you stop?</strong></p><p>The honest answer, written into ROADMAP_V4.md itself, is that this roadmap doesn&#39;t carry forward the &quot;out of scope, next version&quot; section the way v2&#39;s and v3&#39;s roadmaps both did. There&#39;s no seeded v5 candidate list at the bottom of this one. That&#39;s not an oversight — it&#39;s the point. The core engineering question the project set out to answer — can a complete LLM pipeline, including alignment, multimodality, efficient serving, and safe tool use, be built entirely from scratch in Rust, without a single line of Python — has been answered, demonstrated, and documented, phase by phase, across fifty-five of them.</p><p>Not every project needs an indefinitely-growing roadmap. Some are supposed to reach a finish line and actually stop there.</p><h3>Final Thought</h3><p>The garbled 50-step output from the very first post — “Secondprophebreathety,seeask,argweremanlyremories” — is still the honest starting point of this whole project, four releases later. What’s different now isn’t that the discipline changed. It’s the same one: understand exactly why something breaks, build the smallest thing that fixes that specific failure, measure whether it actually helped before calling it done. That discipline built the tokenizer in v1, the vision path in v2, Gated DeltaNet in v3, and the authorization-scope boundary in v4’s multi-agent orchestrator.</p><p>The difference this time is that the same discipline is also what told me to stop. A project that only knows how to keep growing isn’t more honest than one that knows how to finish. If anything, knowing when you’re done is the harder discipline to practice.</p><p>Four versions, fifty-five phases, and a garbled first sentence later — I’d take that trade again.</p><h3>Frequently Asked Questions</h3><p><strong>What is aarambh-studio?</strong> aarambh-studio (formerly aarambh-ai) is a decoder-only large language model application built entirely from scratch in Rust, using the Candle tensor backend. It covers pretraining, fine-tuning, alignment, quantization, inference, agents, retrieval, and serving in one codebase, with no Python or PyTorch dependency anywhere.</p><p><strong>What’s new in aarambh-studio v4.0.0?</strong> v4.0.0 adds Multi-Head Latent Attention as a third attention kind, native audio understanding, sparse/grouped MoE dispatch on CUDA, multi-node distributed training, test-time compute scaling, RLAIF alignment, sandboxed multi-agent tool execution, from-scratch retrieval-augmented generation, a model-merging toolkit, a public multi-tenant inference server, a first-class system role with chat-template versioning, red-team adversarial evaluation, and auto-generated model cards.</p><p><strong>Is aarambh-studio v4.0.0 really the final version?</strong> Yes. It’s declared the final planned release. No v5 roadmap exists as of this release, and ROADMAP_V4.md deliberately does not carry forward the &quot;next version&quot; scoping section that the v2 and v3 roadmaps both included.</p><p><strong>Does aarambh-studio v4.0.0 ship pretrained model weights?</strong> No. It’s a source-only release, same as every prior version. All 21 workspace crates remain publish = false, and GitHub&#39;s automatic source archive is the only release asset — no checkpoints, adapters, tokenizers, optimizer state, or GGUF files are included.</p><p><strong>Will aarambh-studio ever be published to crates.io?</strong> No. v4.0.0 confirms this permanently. aarambh-studio is an application, not a library, and that policy is now final rather than provisional.</p><p><strong>Can aarambh-studio run without a GPU?</strong> Yes, at the Tiny scale. The pipeline trains and runs inference on CPU-only hardware, with CUDA support — including sparse MoE dispatch and multi-node training — available for larger scales.</p><p><strong>Where can I find the aarambh-studio source code?</strong> The full source, release notes, and documentation are on GitHub at github.com/AarambhDevHub/aarambh-studio.</p><h3>Links</h3><p><strong>GitHub — aarambh-studio v4.0.0 source and release notes:</strong> <a href="https://github.com/AarambhDevHub/aarambh-studio/releases/tag/v4.0.0">https://github.com/AarambhDevHub/aarambh-studio/releases/tag/v4.0.0</a></p><p><strong>LinkedIn — systems programming, Rust, and ML:</strong> <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">https://www.linkedin.com/in/darshan-vichhi-rust-developer/</a></p><p><strong>YouTube — Aarambh Dev Hub (Rust tutorials and deep-dives):</strong> <a href="https://www.youtube.com/@AarambhDevHub">https://www.youtube.com/@AarambhDevHub</a></p><p><em>If this was useful, a star on the GitHub repo genuinely helps other developers find it. Thanks for reading and happy coding!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53f374e0fdb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is Topcoat Any Good? An Honest First Look at Tokio’s New Full-Stack Rust Framework]]></title>
            <link>https://aarambhdevhub.medium.com/is-topcoat-any-good-an-honest-first-look-at-tokios-new-full-stack-rust-framework-2e847f84a38a?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e847f84a38a</guid>
            <category><![CDATA[tokio]]></category>
            <category><![CDATA[full-stack]]></category>
            <category><![CDATA[top-coat]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[axum]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Thu, 13 Aug 2026 03:23:41 GMT</pubDate>
            <atom:updated>2026-08-13T03:23:41.768Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yMDNzKkRDqGeT6oYcXe9lQ.png" /></figure><p>The team behind Tokio — the async runtime running underneath most of the Rust web ecosystem, Axum included — just shipped something new. Not a runtime feature. Not another crate for the ecosystem to build on top of. A full-stack web framework, built by the Tokio team itself, called <strong>Topcoat</strong>. Its own tagline calls it “the full full-stack framework for Rust.”</p><p>I’m not affiliated with this project, and I haven’t shipped anything real with it yet. So this isn’t a “why I chose Topcoat” post. This is an honest first look: what it actually does, how its routing and reactivity work under the hood, and where the real skepticism around it is coming from — because a review that only lists upsides isn’t a review, it’s a press release.</p><h3>What problem is Topcoat actually trying to solve?</h3><p>To understand why Topcoat exists, you have to understand the trade-off it’s reacting to.</p><p>Frameworks like Leptos and Dioxus let you write highly interactive Rust web apps by compiling your code to WebAssembly and running it in the browser. That’s genuinely powerful — but it comes with real cost: a separate compile target, bundle size to manage, and data that has to be serialized across the client-server boundary.</p><p>Topcoat’s bet is that a lot of applications don’t actually need that level of interactivity. So instead of compiling to WebAssembly, it renders everything on the server. Components can be asynchronous and query a database directly, which removes the need for a separate API layer entirely. No REST endpoints just to move data from your database to your frontend — because there’s no separate frontend making that request in the first place.</p><h3>Feature one: reactivity without shipping WebAssembly</h3><p>The part of Topcoat that’s actually novel is how it handles interactivity without WebAssembly at all.</p><p>You write ordinary, type-checked Rust expressions for things like button clicks and state updates. Topcoat evaluates that same expression on the server for the very first render, and separately cross-compiles it to JavaScript so it re-runs instantly in the browser afterward. Roughly, it looks like this:</p><pre>view! {<br>    signal open = false;<br>&lt;button @click=$(|_| open.set(!open.get()))&gt;&quot;What does Topcoat do?&quot;&lt;/button&gt;<br>    &lt;p :hidden=$(!open.get())&gt;&quot;Renders on the server, reacts in the browser.&quot;&lt;/p&gt;<br>}</pre><p>That $(...) block is the interesting part. It&#39;s real, type-checked Rust — not a string of JavaScript hiding in your codebase — and Topcoat runs it in two places: once on the server to produce the first HTML the browser ever sees, and again (translated to JavaScript) so the click handler works instantly client-side afterward. No WebAssembly bundle, no separate client build step, no serialization boilerplate between two different codebases, because it&#39;s genuinely one codebase.</p><p>For updates that do need the server — search results changing as someone types, say — Topcoat has a concept called a <strong>shard</strong>. You mark a component as a shard, and whenever its inputs change, Topcoat re-renders just that piece on the server and swaps the new HTML into the page:</p><pre>#[shard]<br>async fn search_results(cx: &amp;Cx, query: String) -&gt; Result {<br>    view! {<br>        &lt;ul&gt;<br>            for item in search_products(cx, &amp;query).await? {<br>                &lt;li&gt;(item.name)&lt;/li&gt;<br>            }<br>        &lt;/ul&gt;<br>    }<br>}</pre><p>It’s a deliberate middle ground: instant client-side interactions where you don’t need the server, and clean server round-trips where you do — without you having to hand-wire either path yourself.</p><h3>Feature two: module-based routing, explained properly</h3><p>This is the part I actually want to slow down on, because it’s one of the more interesting design decisions in the whole framework.</p><p>In a lot of Rust web frameworks, you build your route table by hand — registering every path in one central place, mapping each URL string to the function that handles it. Topcoat offers something different: it can optionally look at how your Rust files are organized on disk and infer the entire route tree directly from that structure, no separate build step required.</p><p>Here’s roughly what that looks like:</p><pre>src/<br>├── app.rs              -&gt; /            (root layout)<br>└── app/<br>    ├── about.rs         -&gt; /about<br>    ├── _marketing.rs                   (layout, no URL segment)<br>    ├── _marketing/<br>    │   └── pricing.rs   -&gt; /pricing<br>    ├── posts.rs         -&gt; /posts<br>    ├── posts/<br>    │   └── id.rs        -&gt; /posts/{post_id}<br>    └── api/<br>        └── health.rs    -&gt; GET /api/health</pre><p>A few things worth unpacking there:</p><ul><li>The <strong>root file</strong> in your source folder defines both the home page and the outer HTML layout wrapping every other page.</li><li>A plain file automatically becomes a route named after itself — about.rs becomes /about, with zero manual registration.</li><li>Files whose names start with an <strong>underscore</strong> act purely as shared layouts — they wrap the routes nested inside their matching folder without adding their own segment to the URL. A pricing page nested under a marketing layout still lives at /pricing, not /marketing/pricing, because the underscore-prefixed folder is invisible to the URL itself.</li><li>Nesting folders nests URL segments the same way nesting files does, and a file named after a parameter (like id) becomes a dynamic route segment — so a post page under posts/ becomes /posts/{post_id} automatically.</li><li>The same system covers <strong>API routes</strong>, not just pages — a health-check file inside an api folder becomes a real GET /api/health endpoint, sitting in the same file tree as your pages instead of a separate routing system bolted on beside it.</li></ul><p>If you’ve used a framework like Next.js with its file-based app router, the shape of this will feel familiar — Topcoat is bringing that pattern into Rust, backed by the compiler instead of a bundler.</p><h3>Feature three: everything else is batteries-included</h3><p>Beyond reactivity and routing, Topcoat leans hard into the batteries-included philosophy:</p><ul><li>Its <strong>templating macro</strong> stays close to real HTML while letting you use ordinary Rust control flow — loops, conditionals — directly inside your markup, instead of learning a separate templating language.</li><li>A <strong>component library</strong> modeled on the shadcn approach ships with the framework. Instead of pulling in a black-box dependency, components get copied directly into your own project, so you fully own and can edit the code.</li><li>An <strong>asset system</strong> scans your compiled binary for referenced files and serves them with content-hashed URLs for aggressive caching, with ready-made integrations for web fonts and icon sets.</li><li>A <strong>mail macro</strong> lets you declare and send email through real SMTP, a file, or purely in-memory — genuinely convenient for testing without standing up a mail server.</li><li>Topcoat doesn’t insist you only use its own reactivity system — it ships integrations for a few well-known partial-page-update tools from outside the Rust world too: htmx, Alpine AJAX, and Datastar, for teams who’d rather bring existing patterns over than relearn everything.</li><li>Alongside shards, there’s a separate concept called <strong>procedures</strong> — plain async functions that live on the server but can be called directly from your client-side code, better suited to things like form submissions than a full page re-render.</li></ul><p>One small design detail says a lot about the framework’s philosophy: Topcoat steers you toward modeling things like authentication checks as plain functions you call explicitly, rather than a stack of framework middleware running invisibly before your code. It’s a minor choice, but it’s consistent with how transparent the rest of the framework tries to be.</p><h3>The honest take</h3><p>Here’s where I want to be direct instead of just excited.</p><p>Topcoat is MIT licensed, sitting at a healthy early number of GitHub stars and commits for a project this young, and it’s genuinely backed by people who work on Tokio itself. That’s a real signal — this isn’t a solo side project.</p><p>But the roadmap is long, and it’s worth actually reading instead of skimming past. There’s already real session and cookie infrastructure — signed and encrypted cookies, bring-your-own-storage sessions with login/logout handling, even sliding expiration and token rotation — but a complete, ready-to-use <strong>authentication system</strong> built on top of those primitives isn’t there yet. Same story for background jobs, static export, pre-rendering, streaming server rendering, client-side navigation with prefetching, image optimization, markdown support, localization, sitemaps, and automatic OpenAPI endpoints. The README itself says to expect breaking changes.</p><p>And the skepticism floating around Topcoat isn’t just about missing features. Some of the most pointed community discussion isn’t about whether the reactivity model is clever — most people seem to agree it is — it’s about <strong>scope</strong>: whether bundling something like an ORM into the framework’s direction is the right call, and a real worry that a project like this could gain outsized adoption purely on the strength of the Tokio name, ahead of technical merit. That’s a fair thing to sit with before betting a real project on it.</p><h3>How it actually compares</h3><p>Worth being precise here, because comparisons like this get thrown around loosely.</p><p><strong>Topcoat isn’t competing with Axum.</strong> It’s built on top of Axum, so if you already like Axum’s foundations, you’re not giving them up.</p><p>The real comparison is with <strong>Leptos and Dioxus</strong>, and it’s not that one is simply better — it’s a different bet. Full client-side interactivity through WebAssembly, versus a simpler, fully server-rendered model that trades some interactivity ceiling for a dramatically simpler build and deployment story. If your app genuinely needs rich, WebAssembly-level client interactivity, Topcoat isn’t trying to be that. If most of what you’re building is fairly standard server-rendered pages with pockets of interactivity, this is exactly the gap Topcoat is aiming at — and until now, that gap didn’t have a strong Rust-native answer.</p><h3>The real numbers</h3><p>A few concrete facts instead of vibes:</p><ul><li>Topcoat comes directly from the <strong>tokio-rs</strong> GitHub organization — the same team behind Tokio itself.</li><li><strong>MIT licensed</strong>, matching Tokio’s own license.</li><li>Launched only a few weeks before this piece was written, and already past <strong>1,000 GitHub stars</strong> with dozens of forks and active daily commits.</li><li>The community is already building with it in public — tutorials, first-impression videos, and interviews with people behind the project have all shown up online within the first few weeks.</li></ul><p>For a brand-new, explicitly early-stage framework, that’s a genuinely fast start — even with a long roadmap still ahead of it.</p><h3>FAQ</h3><p><strong>What is Topcoat?</strong> A batteries-included, full-stack Rust web framework built by the Tokio team, built on top of Axum. It renders everything server-side and adds client reactivity without WebAssembly.</p><p><strong>Is Topcoat a replacement for Axum?</strong> No — Topcoat is built on top of Axum, not a competitor to it.</p><p><strong>How does Topcoat’s routing work?</strong> It infers your entire route tree from your file structure instead of a central route table. Files become pages, underscore-prefixed files become layouts, nested folders nest URL segments, and the same system covers API routes too.</p><p><strong>Is Topcoat better than Leptos or Dioxus?</strong> Not strictly better — it’s a different trade-off. Leptos and Dioxus compile to WebAssembly for full client interactivity; Topcoat stays fully server-rendered with lighter reactivity and no WASM bundle.</p><p><strong>Is Topcoat production-ready?</strong> Not yet. It’s MIT licensed and actively developed, but early-stage — a full authentication system, background jobs, and streaming server rendering are all still on the roadmap, and the README explicitly warns of breaking changes.</p><h3>Final thoughts</h3><p>Topcoat is a genuinely interesting bet from the Tokio team on what full-stack Rust could look like without WebAssembly in the loop — a reactivity model, a module-based routing system, and a batteries-included toolkit that all fit together with real technical ideas behind them, backed by people who know this ecosystem deeply.</p><p>It’s also, right now, an early-stage project with a long roadmap still ahead of it and real open questions about scope and adoption. I’d adopt it today for the direction it’s heading, not for what it can fully do right now — and I’d say the same thing to anyone asking whether to bet a real project on it this week.</p><p>I made a full video breakdown of Topcoat, with more detail on the reactivity model and routing than fits cleanly in a written post — link below, along with the project itself and where to find more of my work.</p><p><strong>Watch the full video breakdown:</strong> <a href="https://youtu.be/i9fIPSaoKHw">Topcoat: Tokio’s New Full-Stack Rust Framework — Honest First Look</a> <strong>Topcoat on GitHub:</strong> <a href="https://github.com/tokio-rs/topcoat">https://github.com/tokio-rs/topcoat</a> <strong>My work:</strong> <a href="https://github.com/AarambhDevHub/aarambh-studio">https://github.com/AarambhDevHub/aarambh-studio</a> <strong>Join the community:</strong> <a href="https://discord.gg/HDth6PfCnp">Discord</a> · <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">LinkedIn</a></p><p><em>If this was useful, consider supporting the channel through </em><a href="https://github.com/sponsors/aarambh-darshan"><em>GitHub Sponsors</em></a><em> or </em><a href="https://buymeacoffee.com/aarambhdevhub"><em>Buy Me a Coffee</em></a><em> — it directly funds more deep-dives like this one.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e847f84a38a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Rebuilt My From-Scratch Rust LLM’s Attention Mechanism So It Never Runs Out of Memory — Then…]]></title>
            <link>https://aarambhdevhub.medium.com/i-rebuilt-my-from-scratch-rust-llms-attention-mechanism-so-it-never-runs-out-of-memory-then-90895b18b6e8?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/90895b18b6e8</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sat, 25 Jul 2026 10:39:04 GMT</pubDate>
            <atom:updated>2026-07-25T10:39:04.524Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Rebuilt My From-Scratch Rust LLM’s Attention Mechanism So It Never Runs Out of Memory — Then Taught It to Watch Video and Read PDFs (aarambh-ai v3.0.0, Still Zero Python)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9z7C0QToMgxAt4tu19IFeg.png" /></figure><p>Two weeks after v2.0.0 shipped, I posted a progress update titled “Phase 32 Complete — Multi-Token Prediction” and said, plainly, that it wasn’t a full v3 release yet. Just a milestone.</p><p>Twelve phases, thirteen pull requests, and one architecture rewrite later, it is.</p><p><strong>aarambh-ai v3.0.0</strong> is out, and it answers a question I didn’t fully know how to answer when I started: what happens to a from-scratch attention mechanism once you actually try to push context length, reasoning depth, and modality count all at the same time? The honest answer turned out to be — it breaks, unless you change what “attention” means in your architecture. So that’s what this release does.</p><p>Still Rust. Still Candle as the only tensor backend. Still not one line of Python anywhere in the dependency graph.</p><h3>What Is aarambh-ai, in One Paragraph?</h3><p><strong>aarambh-ai</strong> (Sanskrit for <em>beginning</em>) is a decoder-only transformer LLM application built as a Rust workspace. v1.0.0 was the text-only foundation — pretraining, SFT, LoRA/QLoRA, GRPO alignment, INT4/GGUF quantization, Flash Attention v2 CUDA kernels, a thinking engine, and CPU-capable self-learning. v2.0.0 made it multimodal and self-serving — vision, Mixture of Experts, multi-GPU training, DPO, speculative decoding, grammar-constrained tool calling, and its own OpenAI-compatible HTTP server. <strong>v3.0.0 rebuilds the part of the model that everything else sits on top of: attention itself.</strong></p><h3>The Problem v3 Actually Set Out to Solve</h3><p>Every feature I added in v2 quietly made the same assumption worse. Longer context, vision tokens, tool-call transcripts — they all grow the sequence length, and standard multi-head attention’s KV cache grows linearly right along with it. Push the context far enough and you stop being memory-bound by your model size and start being memory-bound by how long the conversation has been running.</p><p>You can patch around that for a while. At some point you have to change the recurrence itself.</p><p>v3 does that, and then uses the headroom it buys to go after four other problems at once: a model that gets a denser training signal per step, a model that can watch video and read documents without a second architecture, a model that can plan and execute multi-step tool chains instead of single calls, and — critically — a model with the instrumentation to tell you what it’s quietly forgetting while it learns all of the above.</p><h3>Gated DeltaNet: Linear Attention With a Constant-Size Memory</h3><p>Phase 29 replaces most of the model’s attention layers with <strong>Gated DeltaNet</strong>, a hybrid linear-attention mechanism that carries a fixed-size recurrent state between tokens instead of an ever-growing cache of past keys and values.</p><p>The core recurrence is a gated delta rule:</p><pre>S_t = α_t ⊙ S_{t−1} + β_t · (v_t − S_{t−1} k_t) kᵀ_t<br>o_t = S_t q_t</pre><p>S_t is a [d_k × d_v] state matrix — and no matter how long the sequence gets, that matrix never grows. α_t is a learned decay gate deciding how much of the old state to keep, β_t is a learned write-strength gate deciding how hard to overwrite it with new information, and q, k, v pass through a causal depthwise convolution before ever entering the recurrence, giving the model a small amount of local context awareness for free.</p><p>The part that mattered most for me practically: Gated DeltaNet isn’t a “start over” architecture. It’s <strong>retrofitted onto existing pretrained checkpoints via continued pretraining</strong> — you don’t throw away v1/v2 training runs to adopt it.</p><h3>DeepSeek Sparse Attention: Keeping Full Attention Honest</h3><p>Linear attention alone tends to lose some of the precise, long-range retrieval that dense attention is good at. So v3 doesn’t go all-in on Gated DeltaNet — it keeps a minority of layers on full attention, and makes <em>those</em> layers sparse instead of dense, using an approach modeled on <strong>DeepSeek Sparse Attention (DSA)</strong>.</p><pre>for each query position i:<br>    block_scores = lightning_indexer(q_i, all_key_blocks)   // cheap, low-rank scoring pass<br>    top_k_blocks = select_top_k(block_scores, k)<br>if seq_len &lt;= dense_fallback_threshold:<br>        attend_dense(q_i, all_keys)          // not worth the overhead on short sequences<br>    else:<br>        attend(q_i, top_k_blocks)            // full attention, restricted to selected blocks</pre><p>A lightweight “lightning indexer” scores key blocks cheaply before the expensive matmul ever runs, then the real attention computation only touches the top-K blocks it selected. On short sequences the indexing overhead isn’t worth it, so DSA falls back to plain dense attention automatically — no config flag required.</p><p>Put the two together and the default v3.0 attention stack is: mostly Gated DeltaNet, a minority of DSA-sparse layers, and <strong>zero fully-dense attention layers</strong> left by default. Dense attention is still available as a config fallback, but it’s no longer the default anywhere in the stack.</p><h3>Fine-Grained Mixture of Experts, With a Shared Expert That Never Sleeps</h3><p>v2’s MoE used a handful of large experts. Phase 31 rebuilds it as <strong>fine-grained MoE</strong>: many small experts instead of a few large ones, plus a small number of <strong>shared experts every token passes through regardless of what the router decides.</strong></p><pre>output = Σ shared_experts(token)<br>       + Σ_{i in top_k(routed_experts)} gate_i · expert_i(token)</pre><p>The shared experts absorb the general-purpose knowledge every token needs, freeing the routed experts to specialize harder — and because the shared experts always fire, they’re excluded from the load-balancing auxiliary loss that penalizes uneven routing; penalizing something that’s supposed to always activate would just be fighting the design.</p><p>One detail I was glad I planned for: the new, finer-grained router doesn’t start from nothing. It <strong>warm-starts from v2’s already-trained coarse router’s weights</strong> wherever the dimensions allow, instead of re-learning routing behavior from a random init.</p><h3>Multi-Token Prediction: A Denser Signal, and a Free Draft Model</h3><p>Phase 32 — the one I wrote a standalone progress post about — adds lightweight auxiliary heads that predict more than one future token per position during training:</p><pre>main_head    → predicts t+1<br>mtp_head_1   → predicts t+2<br>loss = ce(main_head) + λ · ce(mtp_head_1)</pre><p>The obvious win is a denser training signal per forward pass — the model isn’t just learning “what comes next,” it’s learning a short window of where the sequence is heading. The less obvious win is what this does for inference: those same MTP heads <strong>double as a free draft source for speculative decoding</strong>. v2’s speculative decoding needed a separate small Tiny checkpoint to draft tokens for the Large model to verify. v3 doesn’t need a second checkpoint at all when MTP is enabled — the model drafts its own continuation.</p><h3>On-Policy Distillation: Teaching the Student From Its Own Mistakes</h3><p>Phase 33 adds a new crate, aarambh-ai-distill, built around a specific complaint about offline distillation: training a student on a fixed, pre-generated dataset from a teacher model doesn&#39;t match what the student actually does at inference time, because the student&#39;s own generations drift from that dataset the moment training starts.</p><p><strong>On-policy distillation</strong> closes that gap by having the student generate its own rollouts, live, and scoring <em>those</em>:</p><pre>rollout = student.generate(prompt)                 // on-policy — student&#39;s current weights<br>score   = teacher.score(rollout)                    // KL-style or reward-style, behind one shared trait<br>loss    = kl(student_logprobs, teacher_logprobs)     // or: -reward * logprob(rollout)</pre><p>Both a KL-style scoring path and a reward-style scoring path sit behind one shared trait, and the whole thing blends cleanly with the MTP auxiliary loss when both are enabled at once.</p><h3>Native Quantization-Aware Training: Making Training and Export Agree</h3><p>v1 and v2 quantized <em>after</em> training — GPTQ on a finished checkpoint. Phase 34 moves quantization <em>into</em> the training loop itself with device-native fake-quantization for INT4/INT8, using a straight-through estimator so gradients still flow:</p><pre>fake_quantize(w) = round_ste(clamp(w / scale, qmin, qmax)) * scale</pre><p>QAT wraps attention, FFN/expert, MoE-router, DeltaNet, and DSA-indexer projections — embeddings, norms, convolutions, and scalars stay full precision, because those are cheap to keep exact and expensive to get wrong. The genuinely hard part is what I’ll get to in the next section, but the short version: the fake-quantization boundaries used during training have to line up <strong>exactly</strong> with the real GGUF Q4_K_M export format, or you’ve trained robustness against a rounding error that doesn’t exist in the file you actually ship.</p><h3>Teaching the Model to Watch Video and Read Documents</h3><p>Phases 35 and 36 extend v2’s vision path — a frozen CLIP-style encoder plus a trainable projector — to two new modalities, without building a second architecture for either.</p><p><strong>Video</strong> decodes native H.264 MP4 through bundled OpenH264 — pure Rust, no FFmpeg binding, no shelling out to a system codec. Frames get sampled deterministically, either uniformly or scene-aware, and each sampled frame gets a learned or sinusoidal <em>temporal</em> offset on top of its normal patch position — so the model has a notion of “this happened at second four,” not just “this patch sits here in the grid.” Frame embeddings reuse v2’s exact frozen ViT path, one frame at a time.</p><p><strong>Documents</strong> rasterize PDF pages through Hayro, a pure-Rust PDF renderer, then project each page with layout-aware 2D positional information per patch. Multi-page documents get &lt;page_sep&gt; tokens between pages so the model knows where one page ends and the next begins.</p><p>Both modalities share the same DoRA training path v2 already had for images — one training code path, three data types now instead of one. New reserved tokens (&lt;video&gt;, &lt;video_end&gt;, &lt;frame_sep&gt;, &lt;document&gt;, &lt;document_end&gt;, &lt;page_sep&gt;) mark the boundaries in the token stream.</p><h3>Long-Horizon Tool-Use Chains</h3><p>v2’s tool calling was single-call, emit-only: the model produces one structurally valid JSON call and stops. Phase 37 adds a new aarambh-ai-agent crate for <strong>multi-step chains</strong> — the model calls a tool, the result gets fed back into context as the next turn&#39;s input, and it can call another tool, and another, up to an explicit step budget.</p><pre>ToolResult::Text(String)<br>ToolResult::Image(ImageData)<br>ToolResult::Video(VideoData)<br>ToolResult::Document(DocumentData)</pre><p>That typed result wrapper is what makes it work across the new modalities — a tool can hand back “here’s a screenshot,” and the next step in the chain reasons over it natively, using the exact same vision path from Phases 35–36, instead of the chain being restricted to text-only tool outputs.</p><p>Step budgets run 1 to 64, there’s an eviction/summarization policy for when a long chain starts pressuring the context window, and multi-step SFT training uses loss-masking across the full multi-turn transcript rather than just the final response. One boundary stayed exactly where it was in v2: aarambh-ai proposes and structures tool calls — it does not execute them. That responsibility stays with the caller.</p><h3>Forgetting Diagnostics: Measuring What Training Quietly Costs You</h3><p>This is the phase I think matters most long-term, and it’s the one tied to my other project, <strong>Manas</strong>, a self-growing neural network with its own associative-memory anti-forgetting system.</p><p>Phase 38 adds persistent, per-capability forgetting curves — math, code, reasoning, factual recall, vision, video, document understanding, and tool-use — measured across training checkpoints as the model keeps learning new things.</p><pre>CapabilityProbe reuses the eval harness&#39;s existing task subsets as probes.<br>Per probe: accuracy over time + MoE routing signature over time.<br>Export: shared JSONL format, directly importable by Manas.</pre><p>CapabilityProbe doesn&#39;t invent new benchmark data — it reuses the same task subsets the eval harness from v2 already had. Alongside accuracy, it captures each probe example&#39;s <strong>MoE routing signature</strong>, so you can distinguish &quot;the model got worse at math&quot; from &quot;the model&#39;s routing quietly shifted away from the experts that used to handle math,&quot; which are very different failure modes with very different fixes. The whole thing exports to a shared JSONL format that Manas can import directly — a data contract between the two projects, not a runtime or filesystem dependency.</p><h3>Max Thinking Mode: A Fifth Reasoning Budget</h3><p>Phase 39 is the smallest phase in scope and, honestly, my favorite for how little new code it needed. It adds a fifth thinking-budget tier — ThinkingMode::Max, 16,384 tokens — continuing the same roughly 4x progression the earlier modes already followed: None → Low (256) → Medium (1,024) → High (4,096) → Max (16,384).</p><p><strong>Zero structural changes to </strong><strong>ThinkingController.</strong> Same forced-token mechanism, same budget tracking, same collapse-on-force-close behavior every earlier mode already had — Max just runs it with a bigger number. Per-mode sampling defaults kick in only when the caller doesn&#39;t specify their own (Max defaults to temperature 0.85, top_p 0.97), the none|low|medium|high|max vocabulary is centralized and shared by the CLI, the serving API, GRPO, and distillation, and there&#39;s now a dedicated hard-problems eval task specifically for comparing High-vs-Max accuracy — because a bigger thinking budget only earns its place in the release if the eval harness says it actually helps on the problems that need it.</p><h3>The Hard Parts Nobody Warns You About in v3</h3><p><strong>QAT and the GGUF exporter have to agree bit-for-bit.</strong> Fake-quantization during training simulates INT4/INT8 rounding, but if the simulated block boundaries don’t land on the exact same blocks the real GGUF Q4_K_M exporter produces afterward, you spend hours training the model to be robust to a rounding error that isn’t in the file you actually ship. Getting ExportAligned mode to hit the same Q4_K_M blocks and global Q8 absmax as the real quantizer took longer than the DeltaNet layer did.</p><p><strong>DSA’s dense fallback threshold isn’t a nice-to-have.</strong> Block-sparse indexing has real overhead, and on short sequences that overhead costs more than it saves — worse, on very short sequences there simply aren’t enough blocks for top-K selection to mean anything. The automatic fallback to dense attention below a sequence-length threshold isn’t an optimization; without it, short-context quality regresses.</p><p><strong>MTP and distillation losses will fight each other if you don’t scale them carefully.</strong> Blend the multi-token-prediction auxiliary loss with the on-policy distillation loss at the wrong relative weight and one term dominates — either the model stops using its future-token heads meaningfully, or it stops listening to the teacher. This needed explicit loss-scale tuning, not just “add both terms and hope.”</p><p><strong>A frozen per-frame ViT has no idea what “frame 4 of 12” means.</strong> Reusing v2’s frozen image encoder for video was the fast path, but the encoder itself processes each frame independently — it has zero notion of temporal order. All of that has to be injected afterward, through the learned/sinusoidal temporal offsets. Skip that step and a ten-second clip looks to the model like a shuffled bag of unrelated photos.</p><p><strong>Forgetting diagnostics needed a stable export schema before Manas could depend on it.</strong> The JSONL format Phase 38 exports is a data contract, not a shared library — deliberately versioned so that adding a new eval task to aarambh-ai’s harness doesn’t silently break Manas’s anti-forgetting tracker on the other side of the import.</p><h3>How to Run It</h3><pre># Clone and build<br>git clone https://github.com/AarambhDevHub/aarambh-ai.git<br>cd aarambh-ai<br>git checkout v3.0.0<br>cargo build --release -p aarambh-ai --locked<br><br># Verify the version<br>target/release/aarambh-ai --version<br># aarambh-ai 3.0.0<br><br># Train with quantization-aware training (enable qat in config TOML)<br>target/release/aarambh-ai train \<br>  --config configs/tiny_v3.toml<br><br># Run inference over a video file, with Max thinking mode<br>target/release/aarambh-ai infer \<br>  --model checkpoints/vlm_v3/best.safetensors \<br>  --video clip.mp4 \<br>  --prompt &quot;What happens in this clip?&quot; \<br>  --thinking max<br><br># Run inference over a PDF document<br>target/release/aarambh-ai infer \<br>  --model checkpoints/vlm_v3/best.safetensors \<br>  --document report.pdf \<br>  --prompt &quot;Summarize page 3&quot; \<br>  --thinking high<br><br># Run a long-horizon tool-use agent chain<br>target/release/aarambh-ai agent \<br>  --model checkpoints/tiny_v3/best.safetensors \<br>  --tools tools.json \<br>  --max-steps 16 \<br>  --prompt &quot;Find the latest release notes and summarize what changed&quot;<br><br># Check per-capability forgetting curves after a training run<br>target/release/aarambh-ai eval \<br>  --model checkpoints/tiny_v3/best.safetensors \<br>  --forgetting-jsonl forgetting_report.jsonl \<br>  --forgetting-manifest configs/forgetting_probes.toml</pre><p>Same distribution policy as v1 and v2: all 19 workspace crates stay publish = false, GitHub&#39;s automatic source archive is the only release asset, and no pretrained checkpoint, adapter, tokenizer, optimizer state, or GGUF file ships with the release. You train, quantize, and serve your own.</p><h3>What v3.0.0 Deliberately Does Not Do</h3><ul><li>No pretrained model quality claim is made anywhere, because no weights are distributed.</li><li>crates.io publishing is still deferred — all 19 crates remain publish = false; that milestone is pushed to v4.0.0.</li><li>Multi-GPU support is still single-node NCCL only. Multi-node training remains out of scope.</li><li>MoE still uses correctness-first dense masked dispatch, not sparse/grouped dispatch — same tradeoff v2 made, still unresolved by design.</li><li>The inference server is still local and self-hosted. It doesn’t claim parity with every OpenAI endpoint or parameter.</li><li>Audio is not supported. Video understanding is visual frames only — no audio track processing.</li></ul><h3>What This Proves About Building ML Infrastructure in Rust</h3><p>v2 proved the codebase could grow in multiple directions without the crate layering collapsing. v3 is a harder test of the same claim, because this release doesn’t just add a new layer on top — it replaces something structural underneath everything else.</p><p>Nineteen crates now, up from sixteen in v2, including dedicated new crates for on-policy distillation (aarambh-ai-distill) and long-horizon tool orchestration (aarambh-ai-agent). And despite rebuilding the attention mechanism itself: existing v1/v2 dense model configs stay valid whenever the new v3-only fields (attention_schedule, dsa_config, mtp, qat) are simply absent. Existing tokenizer IDs and checkpoint paths still work. Existing LoRA/DoRA adapter merges and SafeTensors/GGUF inference paths are all still available, unchanged.</p><p>That’s not an accident. It’s what strict, compiler-enforced crate layering buys you even when the change touches the deepest layer in the stack — new capability doesn’t have to mean broken compatibility.</p><h3>What Comes Next</h3><p>The known boundaries above are the honest v4 candidate list: crates.io publishing for real this time, sparse/grouped MoE dispatch instead of dense masked dispatch, multi-node training, fuller OpenAI API parity, and — the modality nobody’s touched yet — audio.</p><p>Same rule as always: prove it, document it, measure it with the eval harness, before it ships.</p><h3>Final Thought</h3><p>The garbled 50-step output from the very first post is still the honest starting point of this whole project, three releases later. What changed with v3 isn’t that I got braver about touching the deepest part of the architecture — it’s that the same discipline that worked on RoPE scaling and grammar-constrained JSON in v2 works just as well on gated linear attention and quantization-aware training in v3: understand exactly why the old approach breaks, build the smallest thing that fixes that specific failure, and don’t call it done until the eval harness says it actually helped.</p><p>Twelve phases in, replacing the thing everything else depends on, I’d take that trade again.</p><h3>Frequently Asked Questions</h3><p><strong>What is aarambh-ai?</strong> aarambh-ai is a decoder-only large language model application built entirely from scratch in Rust, using the Candle tensor backend. It covers pretraining, fine-tuning, alignment, quantization, inference, and serving in one codebase, with no Python or PyTorch dependency anywhere.</p><p><strong>What’s new in aarambh-ai v3.0.0?</strong> v3.0.0 replaces most of the model’s attention layers with Gated DeltaNet (constant-size recurrent state) and DeepSeek Sparse Attention, adds fine-grained Mixture of Experts with shared experts, multi-token prediction, on-policy distillation, native quantization-aware training, native video and document understanding, long-horizon tool-use chains, per-capability forgetting diagnostics, and a fifth “Max” thinking mode with a 16,384-token budget.</p><p><strong>Does aarambh-ai v3.0.0 ship pretrained model weights?</strong> No. It’s a source-only release. All 19 workspace crates remain publish = false, and GitHub&#39;s automatic source archive is the only release asset — no checkpoints, adapters, tokenizers, or GGUF files are included.</p><p><strong>Can aarambh-ai run without a GPU?</strong> Yes, at the Tiny scale. The pipeline is designed to train and run inference on CPU-only hardware, with GPU support (CUDA, multi-GPU via NCCL) available for larger scales through Kaggle-style free sessions.</p><p><strong>Where can I find the aarambh-ai source code?</strong> The full source, release notes, and documentation are on GitHub at github.com/AarambhDevHub/aarambh-ai.</p><h3>Links</h3><p><strong>GitHub — aarambh-ai v3.0.0 source and release notes:</strong> <a href="https://github.com/AarambhDevHub/aarambh-ai/releases/tag/v3.0.0">https://github.com/AarambhDevHub/aarambh-ai/releases/tag/v3.0.0</a></p><p><strong>LinkedIn — systems programming, Rust, and ML:</strong> <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">https://www.linkedin.com/in/darshan-vichhi-rust-developer/</a></p><p><strong>YouTube — Aarambh Dev Hub (Rust tutorials and deep-dives):</strong> <a href="https://www.youtube.com/@AarambhDevHub">https://www.youtube.com/@AarambhDevHub</a></p><p><em>If this was useful, a star on the GitHub repo genuinely helps other developers find it. Thanks for reading and happy coding!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90895b18b6e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[16 Nobel Laureates Just Warned Us About AI Jobs. Here’s What the Evidence Actually Says]]></title>
            <link>https://aarambhdevhub.medium.com/16-nobel-laureates-just-warned-us-about-ai-jobs-heres-what-the-evidence-actually-says-c9640b07d491?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9640b07d491</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 02:19:09 GMT</pubDate>
            <atom:updated>2026-07-23T02:27:14.776Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cIwjNLtvQaXWVW9lOq1EZg.png" /></figure><p>On July 13, 2026, a short statement appeared with an unusually heavy list of names behind it.</p><p>Sixteen Nobel laureates. Leading economists. AI researchers. Technology executives. People connected to Stanford, MIT, Harvard, OpenAI, Anthropic, Google and other major institutions.</p><p>The title was not subtle:</p><blockquote><strong><em>We Must Act Now.</em></strong></blockquote><p>The statement warned that artificial intelligence may become radically more powerful over the next decade. It said the resulting economic transformation could be larger than the Industrial Revolution, while unfolding over a much shorter period. It also raised the possibility of large-scale job displacement alongside major gains in productivity and living standards.</p><p>That is the kind of language that immediately produces two reactions.</p><p>One group hears it and thinks: <em>AI is about to destroy the job market.</em></p><p>The other group hears it and thinks: <em>Another dramatic open letter with no evidence.</em></p><p>Both reactions miss the most important part of the story.</p><p>The statement itself is not a research paper. It contains no dataset, no equations, no forecast model and no estimate of how many jobs will disappear.</p><p>But the warning did not appear in a vacuum.</p><p>There is real evidence that AI is increasing productivity. There are also early signs that young workers are finding it harder to enter some AI-exposed careers. At the same time, there is no convincing evidence yet of economy-wide mass unemployment caused by AI.</p><p>The truth is not as dramatic as “AI will take every job.”</p><p>It may be more subtle — and more difficult to solve.</p><h3>The quick answer</h3><p><strong>What is “We Must Act Now”?</strong></p><p>It is a short statement organized through the Stanford Digital Economy Lab calling on economists, policymakers and technology leaders to prepare for the possibility that advanced AI could transform the economy unusually quickly.</p><p><strong>Does it prove AI will cause mass unemployment?</strong></p><p>No. The statement is an agenda-setting warning, not a research result.</p><p><strong>What does current evidence show?</strong></p><p>AI is already improving productivity in some workplaces. Employment and hiring pressure may be emerging among young workers entering highly exposed occupations. However, current data does not show broad AI-driven unemployment across the economy.</p><p><strong>What is the biggest near-term risk for developers?</strong></p><p>Not necessarily that software engineering disappears. The more immediate risk is that companies hire fewer beginners because AI can perform many of the routine tasks through which junior developers traditionally learn.</p><h3>What the Stanford-linked statement actually says</h3><p>The full title is <strong>“We Must Act Now: A Statement on AI’s Transformation of the Economy.”</strong></p><p>It was organized by Erik Brynjolfsson, Ajay Agrawal, Anton Korinek and Tom Cunningham. The launch included more than 200 economists, AI researchers and technology leaders, including 16 Nobel laureates.</p><p>The three-point statement says:</p><ol><li>AI may become radically more powerful over the next ten years.</li><li>This could produce an economic transformation larger than the Industrial Revolution, creating both large-scale job displacement and major gains in living standards.</li><li>Economists, policymakers and technology leaders should build the incentives, guardrails and institutions needed to make AI complement humans and benefit society.</li></ol><p>The wording matters.</p><p>It says <strong>may</strong> and <strong>could</strong>.</p><p>It does not claim certainty. It does not say all jobs will disappear. It does not propose a specific AI law, automation tax, universal basic income program or pause on model development.</p><p>That broad wording helped people with very different political and economic views sign the same document.</p><p>Daron Acemoglu, Joseph Stiglitz, Ben Bernanke, Paul Krugman, Simon Johnson, Yoshua Bengio, Eric Schmidt, Reid Hoffman, Jeff Dean, Yann LeCun and many others could agree that the issue deserves preparation — even if they strongly disagree about the most likely outcome or the correct policy response.</p><p>That makes the statement influential.</p><p>It also makes it vague.</p><p>“Act now” can mean better labor-market measurement. It can mean worker retraining. It can mean model regulation, public investment, stronger competition policy, wage insurance or new social-safety systems.</p><p>The letter creates urgency without resolving the hard choices.</p><h3>This is not a research paper</h3><p>The authority of the signatories should make us pay attention.</p><p>It should not make us stop asking questions.</p><p>The statement includes no methodology that another economist can reproduce. It does not assign a probability to the Industrial Revolution comparison. It does not define what “larger” means.</p><p>Larger in gross domestic product?</p><p>Larger in productivity growth?</p><p>Larger in job displacement?</p><p>Larger in inequality?</p><p>Larger in its effect on daily life?</p><p>The Industrial Revolution was not one clean event. It unfolded through factories, steam power, railways, electrification, urbanization and mass production over decades.</p><p>For AI to produce something larger in ten years, several uncertain things would all need to happen.</p><p>Models would need to keep improving. They would need to become reliable on long, ambiguous, real-world tasks. Companies would need to integrate them into existing organizations. Security, privacy, legal responsibility and data-access problems would need to be solved. Firms would need to redesign workflows. Workers and customers would need to accept those changes.</p><p>A benchmark improvement does not automatically become an economic revolution.</p><p>A successful agent demo does not automatically become a reliable business process.</p><p>And automating one task does not automatically eliminate the entire job.</p><p>So why take the statement seriously at all?</p><p>Because some of the evidence underneath the warning is already visible.</p><h3>AI is producing real productivity gains</h3><p>One of the strongest workplace studies followed 5,179 customer-support agents after a generative AI assistant was introduced.</p><p>The result was not small.</p><p>Workers using the assistant resolved nearly 14 percent more issues per hour on average. The lowest-skilled and least-experienced workers improved by roughly 35 percent, while the strongest and most experienced workers saw little benefit.</p><p>That finding can support two completely different stories.</p><p>The optimistic story is that AI spreads expert knowledge.</p><p>A beginner can access better responses, better phrasing and better problem-solving patterns. The tool helps less-experienced workers become useful faster. It can reduce the gap between beginners and top performers.</p><p>The worrying story is that companies may need fewer beginners.</p><p>If one new employee becomes productive much faster, a team may not hire as many people. A technology can help individual workers and reduce total hiring at the same time.</p><p>This is the tension at the center of the AI jobs debate.</p><p>Productivity and displacement are not opposites.</p><p>They can happen together.</p><h3>The strongest warning signal is appearing among young workers</h3><p>The most concerning labor-market evidence does not show millions of people suddenly losing jobs.</p><p>It shows something quieter.</p><p>A Stanford Digital Economy Lab working paper examined high-frequency payroll data from a major US payroll provider. The researchers looked at workers aged 22 to 25 in occupations with high exposure to generative AI.</p><p>They found a <strong>16 percent relative decline in employment</strong> for early-career workers in the most exposed occupations.</p><p>More experienced workers in the same occupations remained stable or continued growing.</p><p>The researchers also reported that the decline was concentrated in occupations where AI is more likely to automate tasks rather than augment human work.</p><p>This does not mean 16 percent of all young workers lost their jobs.</p><p>It is a relative change within a specific comparison of highly exposed occupations.</p><p>It also does not prove that AI caused every part of the decline. Interest rates, post-pandemic hiring corrections, outsourcing, slower technology investment and company over-hiring can all affect entry-level employment.</p><p>But the age pattern is difficult to dismiss.</p><p>If the entire occupation were collapsing for unrelated reasons, experienced workers might be expected to decline in the same way.</p><p>Instead, the weakness appears most clearly at the entrance.</p><p>Anthropic’s 2026 labor-market analysis found a similar signal.</p><p>Its researchers did not find a statistically significant rise in unemployment for highly exposed occupations overall. But when they examined workers aged 22 to 25 entering those occupations, they found a roughly <strong>14 percent decline in the job-finding rate</strong> compared with 2022.</p><p>Anthropic described the result as tentative and barely statistically significant. The report also listed alternative explanations: young people may be taking different jobs, staying in education or being measured imperfectly in survey data.</p><p>That caution is important.</p><p>But two different analyses are pointing toward the same possible problem:</p><blockquote><em>The first visible impact of AI may not be mass layoffs. It may be fewer first jobs.</em></blockquote><h3>The missing first rung of the career ladder</h3><p>Entry-level work is often treated as cheap, replaceable labor.</p><p>That is a mistake.</p><p>Entry-level work is how people become experienced.</p><p>A junior developer writes tests, fixes small bugs, updates documentation, handles simple features, reads production code and learns what happens when a system fails.</p><p>A new analyst prepares reports, checks data, summarizes documents and learns how experienced colleagues make decisions.</p><p>A young support agent handles common questions before moving toward more difficult cases.</p><p>Many of these tasks are exactly where generative AI is improving fastest.</p><p>From a company’s perspective, replacing routine work with AI can look efficient. One senior engineer using powerful tools may produce work that previously required several people.</p><p>But this creates a long-term question:</p><p><strong>Where does the next senior engineer come from?</strong></p><p>You cannot remove the learning stage and still expect experienced workers to appear five years later.</p><p>This is why the AI labor debate cannot be reduced to a simple question like “Will the profession survive?”</p><p>A profession can survive while the path into it becomes much narrower.</p><p>Software engineering may continue to employ millions of people. But if companies reduce junior hiring, require entry-level candidates to perform at a mid-level standard and use AI to absorb routine work, an entire generation may struggle to enter.</p><p>The job remains.</p><p>The ladder loses its first rung.</p><h3>Evidence against immediate panic</h3><p>The alarming findings are real, but they are not the whole picture.</p><p>A major study in Denmark connected workplace AI-adoption surveys with administrative labor records. Workers and employers were using AI chatbots. Tasks were changing. People reported productivity benefits.</p><p>Yet after two years, the researchers found no large measurable effect on earnings or recorded working hours. Their estimates were precise enough to rule out average effects larger than roughly two percent.</p><p>That is strong evidence against the claim that broad labor-market collapse has already begun.</p><p>Anthropic’s analysis reached a similar overall conclusion: the unemployment rate for the most exposed occupations had not risen significantly compared with less-exposed work.</p><p>The International Labour Organization estimated that about one in four jobs worldwide has some exposure to generative AI. But the ILO’s conclusion was not that one in four jobs will disappear.</p><p>It said <strong>transformation is more likely than complete replacement</strong> for most exposed occupations.</p><p>That distinction is critical.</p><p>Exposure means AI can affect some tasks inside a job.</p><p>It does not mean the whole occupation can be removed.</p><p>A lawyer does more than summarize documents.</p><p>A doctor does more than classify symptoms.</p><p>A software engineer does more than generate functions.</p><p>Jobs also involve responsibility, negotiation, domain knowledge, trust, communication, physical action and decisions made under uncertainty.</p><p>AI can automate a meaningful share of work without eliminating the profession.</p><h3>Exposure, assistance, automation and replacement are different things</h3><p>The public conversation often mixes four separate ideas:</p><ul><li>A task is exposed to AI.</li><li>A worker is assisted by AI.</li><li>A task is automated by AI.</li><li>An entire job is eliminated.</li></ul><p>These are not the same.</p><p>Imagine an AI system that generates the first draft of a legal contract.</p><p>That exposes one task.</p><p>A lawyer may use the draft and finish the work faster. That is assistance.</p><p>A company may automate the first draft completely. That is task automation.</p><p>But the lawyer may still negotiate terms, advise the client, evaluate risk, appear in court and accept professional responsibility.</p><p>The job changes without disappearing.</p><p>Software development works the same way.</p><p>An AI agent can implement a feature, generate tests, explain an unfamiliar module and propose a refactor.</p><p>Production engineering still requires someone to decide what should be built, understand the business constraint, notice when the specification is wrong, evaluate security and performance, debug failures across multiple services and accept responsibility when the generated code causes damage.</p><p>However, we should not use this as a comfortable excuse.</p><p>People often say:</p><blockquote><em>“AI will not replace developers. Developers using AI will replace developers who do not.”</em></blockquote><p>That sentence hides the economic question.</p><p>A company may decide that three developers using AI can perform work previously done by five.</p><p>The remaining developers are augmented.</p><p>The two missing positions are still displaced.</p><p>Both things can be true.</p><h3>What does it mean for AI to “complement humans”?</h3><p>The most important phrase in the Stanford-linked statement is not “job displacement.”</p><p>It is <strong>complements humans</strong>.</p><p>Complementary technology increases the value of human work.</p><p>A coding assistant that helps an engineer test more carefully can be complementary.</p><p>A medical system that gives a doctor a second opinion can be complementary.</p><p>A teaching tool that helps create personalized exercises can be complementary.</p><p>A support assistant that helps a new employee answer difficult questions can be complementary.</p><p>Substitutive technology is designed mainly to remove the person from the workflow.</p><p>The same model can be used in either direction.</p><p>A coding agent can help a junior developer learn faster — or become the reason that the company does not hire the junior developer.</p><p>A support model can make workers more capable — or replace the first layer of the support team.</p><p>The final outcome is not determined only by model capability.</p><p>It is determined by business incentives, organizational design, labor institutions and policy.</p><p>Reducing head count creates an immediate, measurable cost saving.</p><p>Building systems that make workers more capable may require training, workflow redesign and patience.</p><p>If companies optimize only for short-term labor reduction, the broader economy can end up with fewer career pathways and a greater concentration of income.</p><p>That is the economic concern behind the letter.</p><h3>The strongest criticism of “act now”</h3><p>There is a serious argument on the other side.</p><p>Governments are not automatically good at steering new technology.</p><p>The statement does not define which policies should be created. Acting before understanding the problem can produce rules that protect the largest companies rather than workers.</p><p>A frontier AI company can afford legal departments, compliance teams, security audits and licensing processes.</p><p>A small startup, independent developer, university lab or open-source project may not.</p><p>Regulation presented as public protection can become a barrier to competition.</p><p>That is why “act now” should not mean “write the largest possible AI law immediately.”</p><p>The most responsible early actions are less dramatic:</p><ul><li>Measure entry-level hiring, wages, hours and occupational movement.</li><li>Track how firms are actually using AI rather than relying on anecdotes.</li><li>Support independent research.</li><li>Protect competition and open technical ecosystems.</li><li>Build apprenticeships and faster learning pathways.</li><li>Help workers transition when roles change.</li><li>Test policies before applying them at national scale.</li><li>Make obligations proportional to the capability and deployment risk of the system.</li></ul><p>Preparation does not require panic.</p><h3>What this means for software developers</h3><p>The letter does not mean you should stop learning software engineering.</p><p>It means that being only a code producer is becoming a weaker career position.</p><p>Routine implementation is becoming cheaper.</p><p>Boilerplate is becoming cheaper.</p><p>Basic test generation is becoming cheaper.</p><p>Simple documentation is becoming cheaper.</p><p>The first draft of many technical tasks is becoming cheaper.</p><p>If your entire value is converting a perfectly written ticket into predictable code, pressure will increase.</p><p>The defensible work is moving upward and downward at the same time.</p><p><strong>Upward</strong> means architecture, product understanding, system design, trade-offs, communication and responsibility.</p><p><strong>Downward</strong> means understanding what actually happens at runtime: memory, concurrency, networking, databases, operating systems, compilers, inference, security, observability and failure recovery.</p><p>AI performs best when the task is clearly specified and easily verified.</p><p>It is weaker when the specification is incomplete, the system is unfamiliar and failure has real consequences.</p><p>This is where systems engineering becomes valuable.</p><p>Learning Rust can help — but Rust alone is not a career shield.</p><p>A collection of syntax tutorials is not a moat. Another small command-line project is not enough.</p><p>The value comes from using the language to solve a real problem:</p><ul><li>a fast inference server,</li><li>a secure agent sandbox,</li><li>a memory-efficient model runtime,</li><li>a reliable data pipeline,</li><li>an evaluation system,</li><li>a distributed service with measurable latency,</li><li>or a production tool with tests, benchmarks and documented trade-offs.</li></ul><p>The language matters because of what you can build and prove with it.</p><p>For junior developers, the correct response is not to compete with AI on typing speed.</p><p>Use AI to move through routine work faster.</p><p>But do not outsource your understanding.</p><p>Read the generated code. Break it. Test it. Measure it. Explain why it works. Learn to identify when the model is confidently wrong.</p><p>The developer who only accepts generated code becomes easier to replace.</p><p>The developer who can define constraints, verify output, debug failure and own production outcomes becomes more valuable.</p><h3>Open source may become part of the new career ladder</h3><p>If companies hire fewer beginners, traditional credentials will become less convincing on their own.</p><p>Employers will need better evidence that someone can work on real systems.</p><p>A credible open-source repository can show more than a résumé bullet.</p><p>It can show architecture decisions, benchmarks, issues, pull requests, test discipline, documentation, security thinking and long-term maintenance.</p><p>But this only works when the project solves a real problem.</p><p>Building something impressive-looking only to collect stars is not enough.</p><p>The strongest portfolio projects are useful, measurable and honest about their limitations.</p><p>In a labor market shaped by AI, proof of judgment may matter more than proof that you can produce a large amount of code.</p><h3>My verdict</h3><p>“We Must Act Now” is directionally credible but analytically incomplete.</p><p>It is correct that AI capability and adoption are moving faster than many economic institutions.</p><p>It is correct that entry-level workers may face the earliest pressure.</p><p>It is correct that productivity gains and job displacement can happen together.</p><p>And it is correct that waiting for perfect certainty could leave governments reacting after the damage begins.</p><p>But the statement does not prove that AI will create a transformation larger than the Industrial Revolution.</p><p>It does not prove that mass unemployment is close.</p><p>It does not define the policies it wants.</p><p>And it does not solve the conflict between innovation, worker protection, open research and competition.</p><p>The statement should be treated as an early-warning document.</p><p>Not as a forecast.</p><p>Not as a research result.</p><p>And not as permission for fear-based headlines.</p><p>The most honest conclusion is this:</p><p>AI is already changing tasks.</p><p>It is already increasing productivity in some workplaces.</p><p>There are early warning signs around young workers and hiring.</p><p>But the economy has not yet experienced broad AI-driven unemployment.</p><p>We should prepare seriously.</p><p>We should not pretend we know exactly what happens next.</p><h3>What happens next may look surprisingly ordinary</h3><p>The transition may not arrive as one dramatic day when millions of jobs disappear.</p><p>It may look quieter.</p><p>A company does not replace an employee who leaves.</p><p>A team hires three graduates instead of five.</p><p>A senior developer supervises several agents.</p><p>A support department handles more customers with fewer workers.</p><p>A junior position requires skills that used to be expected after three years.</p><p>Each individual decision looks small.</p><p>Across thousands of companies, the labor market changes.</p><p>At the same time, new work will appear: AI evaluation, model security, agent infrastructure, data governance, workflow design, reliability engineering and specialized domain systems.</p><p>The difficult question is whether those opportunities arrive quickly enough, in the right places and for the same people who lose access to older roles.</p><p>History says technology can create more prosperity.</p><p>History also says the transition can be painful and unequal.</p><p>The people who do best will not necessarily be those who use the largest number of AI tools.</p><p>They will be the people who combine AI leverage with deep ability.</p><p>They will know how to define the task, design the system, verify the result, measure performance, protect users and take responsibility when the model is wrong.</p><p>AI makes implementation faster.</p><p>It does not make consequences disappear.</p><h3>Frequently asked questions</h3><h3>Did Stanford publish a research paper saying AI will take jobs?</h3><p>No. “We Must Act Now” is a short statement organized through the Stanford Digital Economy Lab. It calls for preparation but does not provide a dataset or forecast model.</p><h3>Did 16 Nobel laureates sign it?</h3><p>Stanford’s July 13, 2026 announcement said the statement launched with 16 Nobel laureates among more than 200 economists, researchers and technology leaders.</p><h3>Is AI already causing mass unemployment?</h3><p>Current evidence does not show economy-wide mass unemployment caused by AI. Some studies do show weaker employment or hiring among young workers entering highly exposed occupations.</p><h3>Will AI replace software developers?</h3><p>AI is likely to automate and accelerate many software tasks. That does not mean all software-engineering jobs disappear. Roles may shift toward architecture, verification, security, performance, deployment and ownership of production outcomes.</p><h3>What should junior developers learn?</h3><p>Strong fundamentals, debugging, testing, system design, databases, networking, security, performance and the ability to verify AI-generated work. Build credible projects that solve real problems.</p><h3>Is learning Rust enough to protect a developer career?</h3><p>No. Rust can support valuable systems and AI-infrastructure work, but the language itself is not a guarantee. The value comes from using it to build reliable, measurable systems.</p><h3>Watch the full video</h3><p>I also published a full video breakdown with the research, visual explanations and a developer-focused conclusion:</p><p><strong>YouTube:</strong> <a href="https://youtu.be/dmRarwTJVsw">https://youtu.be/dmRarwTJVsw</a></p><h3>Primary sources</h3><ul><li><a href="https://www.wemustactnow.ai/">We Must Act Now — official statement and signatories</a></li><li><a href="https://digitaleconomy.stanford.edu/">Stanford Digital Economy Lab — announcement and AI economic research</a></li><li><a href="https://digitaleconomy.stanford.edu/publication/canaries-in-the-coal-mine-six-facts-about-the-recent-employment-effects-of-artificial-intelligence/">Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence</a></li><li><a href="https://www.anthropic.com/research/labor-market-impacts">Anthropic — Labor Market Impacts of AI: A New Measure and Early Evidence</a></li><li><a href="https://www.nber.org/papers/w31161">NBER — Generative AI at Work</a></li><li><a href="https://www.nber.org/papers/w33777">NBER — Large Language Models, Small Labor Market Effects</a></li><li><a href="https://www.nber.org/papers/w32487">NBER — The Simple Macroeconomics of AI</a></li><li><a href="https://www.ilo.org/publications/generative-ai-and-jobs-refined-global-index-occupational-exposure">International Labour Organization — Generative AI and Jobs</a></li></ul><p><em>I write about Rust, AI systems, open-source engineering and the future of software development through Aarambh Dev Hub.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9640b07d491" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Wrote Down Everything I Learned About LLMs — Then Open-Sourced It]]></title>
            <link>https://aarambhdevhub.medium.com/i-wrote-down-everything-i-learned-about-llms-then-open-sourced-it-86efde0580a8?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/86efde0580a8</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sat, 18 Jul 2026 02:53:12 GMT</pubDate>
            <atom:updated>2026-07-18T02:53:52.213Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sm33hY2DBnWCgY_taK7pGQ.png" /></figure><p>For a long time, I could use large language models without feeling that I truly understood them.</p><p>I knew how to call an API. I could read model documentation, experiment with prompts, and use existing libraries. But many important questions were still sitting underneath the surface:</p><ul><li>How does raw text become something a neural network can process?</li><li>What do token embeddings actually contain?</li><li>Why do Transformers use queries, keys, and values?</li><li>How does causal attention stop a model from seeing future tokens?</li><li>What exactly changes during backpropagation?</li><li>Why is training different from inference?</li><li>What is stored inside a KV cache?</li><li>How do quantization, LoRA, DPO, GRPO, RAG, and tool use fit into the larger system?</li><li>How do you prepare a real dataset without accidentally creating duplicates, leakage, or licensing problems?</li></ul><p>I did not want to memorize disconnected definitions.</p><p>I wanted to see the complete path — from text entering a system to a model producing the next token — and understand what each component was doing along the way.</p><p>So while learning and building my own AI systems, I started writing Markdown notes.</p><p>At first, those files were only for me. I wrote down formulas in simpler language, traced tensor shapes, created diagrams, added small code examples, and documented the mistakes that confused me.</p><p>The notes kept growing.</p><p>Eventually, they became a public GitHub repository:</p><p><strong>LLM from Scratch — From Fundamentals to AI Systems</strong></p><p>Repository: <a href="https://github.com/AarambhDevHub/llm-from-scratch">https://github.com/AarambhDevHub/llm-from-scratch</a></p><p>This is not something I am presenting as a production-ready LLM framework.</p><p>It is the organized record of what I learned while trying to understand large language models from first principles, shared so other developers can learn from the same material.</p><h3>Why I Needed to Build This for Myself</h3><p>There is no shortage of information about large language models.</p><p>The problem is that the information is scattered.</p><p>One article explains tokenization. Another explains self-attention. A research paper introduces an optimization but assumes you already understand the architecture. A framework tutorial shows ten lines of code that work, but hides almost everything happening underneath.</p><p>Each resource may be useful on its own, but connecting them into one mental model is difficult.</p><p>I repeatedly found myself asking questions such as:</p><blockquote><em>Where does this component sit in the full pipeline?</em></blockquote><blockquote><em>What shape does this tensor have before and after the operation?</em></blockquote><blockquote><em>Is this example performing real training, or only simulating the idea?</em></blockquote><blockquote><em>Is this technique used during pretraining, fine-tuning, or inference?</em></blockquote><blockquote><em>Is this an established method, a simplified teaching example, or an experimental proposal?</em></blockquote><p>Those questions changed how I wrote the notes.</p><p>Instead of collecting definitions, I started organizing the material around the lifecycle of a language model:</p><pre>raw data<br>   ↓<br>cleaning and tokenization<br>   ↓<br>embeddings<br>   ↓<br>Transformer blocks<br>   ↓<br>loss and backpropagation<br>   ↓<br>trained parameters<br>   ↓<br>inference and sampling<br>   ↓<br>serving, retrieval, tools, and applications</pre><p>Once I could see that path clearly, the advanced topics became easier to place.</p><p>Quantization belongs mainly to efficient inference and deployment.</p><p>LoRA is a parameter-efficient fine-tuning method.</p><p>DPO and GRPO belong to post-training and alignment.</p><p>RAG adds external retrieval around the model.</p><p>Agents add a control loop, tools, state, and execution around model generation.</p><p>Multimodal systems connect text models with representations from images, audio, video, or documents.</p><p>The goal of the repository is to make those relationships visible.</p><h3>What Does “LLM from Scratch” Mean Here?</h3><p>“From scratch” can mean several different things, so I want to be precise.</p><p>In this repository, it means learning the major components from first principles and building a small decoder-only Transformer that performs real training.</p><p>The runnable implementation includes:</p><ul><li>A deterministic character tokenizer</li><li>Token and positional embeddings</li><li>Causal multi-head self-attention</li><li>Transformer blocks</li><li>Feed-forward networks</li><li>Layer normalization and residual connections</li><li>Cross-entropy loss</li><li>A real backward pass</li><li>AdamW parameter updates</li><li>Training and validation loss</li><li>Checkpoint saving and loading</li><li>Autoregressive text generation</li></ul><p>It does <strong>not</strong> mean that the repository provides a production-scale replacement for systems such as PyTorch, CUDA, distributed training frameworks, inference servers, or frontier language models.</p><p>The advanced chapters explain how those systems work, often with simplified examples, pseudocode, diagrams, and references. The small mini-transformer is the cohesive runnable implementation.</p><p>That distinction matters to me.</p><p>I would rather describe a small implementation honestly than call a collection of demonstrations a complete production LLM.</p><h3>The Learning Path I Built</h3><p>The repository currently contains 37 structured chapters, along with two practical dataset guides, a runnable mini-transformer, unit tests, references, and validation tooling.</p><p>The material moves from basic concepts to full AI-system design.</p><h3>1. Foundations: AI, learning, mathematics, and neural networks</h3><p>The first section answers the questions that are often skipped too quickly:</p><ul><li>What is artificial intelligence?</li><li>What is the difference between AI, machine learning, and deep learning?</li><li>What does it mean for a computer to “learn”?</li><li>What are parameters, features, labels, training, and inference?</li><li>Which parts of linear algebra, calculus, and probability are actually needed?</li><li>How does a perceptron become a neural network?</li><li>What happens during a forward pass?</li></ul><p>I wanted this section to be readable even for developers who have not formally studied machine learning.</p><p>The goal is not to turn every reader into a mathematician. The goal is to make the later equations understandable instead of mysterious.</p><h3>2. Text, tokenization, embeddings, and attention</h3><p>A language model cannot consume a sentence directly.</p><p>Text must first become tokens, and tokens must become vectors.</p><p>The repository walks through:</p><ul><li>Byte Pair Encoding</li><li>WordPiece</li><li>SentencePiece</li><li>Vocabulary construction</li><li>Token IDs</li><li>Embedding tables</li><li>Positional information</li><li>Queries, keys, and values</li><li>Scaled dot-product attention</li><li>Multi-head attention</li><li>Causal masking</li></ul><p>The attention chapter was especially important for me because attention is often explained with a single formula:</p><pre>Attention(Q, K, V) = softmax(QKᵀ / √dₖ)V</pre><p>The formula is compact, but it hides many questions.</p><p>Where do Q, K, and V come from?</p><p>What are their shapes?</p><p>Why do we transpose the keys?</p><p>What does each value in the score matrix represent?</p><p>Why do we divide by the square root of the head dimension?</p><p>Where is the causal mask applied?</p><p>What comes out of the attention layer?</p><p>The notes trace those steps with concrete dimensions and diagrams instead of stopping at the equation.</p><h3>3. The complete Transformer block</h3><p>After tokenization, embeddings, and attention are clear, the repository combines them into a decoder-only Transformer.</p><p>This section covers:</p><ul><li>Pre-normalization</li><li>Residual connections</li><li>Multi-head self-attention</li><li>Feed-forward networks</li><li>GELU and SwiGLU</li><li>Layer normalization and RMSNorm</li><li>Output logits</li><li>Weight tying</li><li>Mixture-of-Experts routing</li></ul><p>The main idea is to show that a Transformer is not one magical operation.</p><p>It is a repeated composition of understandable operations.</p><p>Each block transforms the current representation of every token while preserving useful information through residual paths.</p><h3>4. Training: loss, gradients, optimizers, and data pipelines</h3><p>A model architecture alone does not learn.</p><p>Training requires data, an objective, gradients, and an optimizer.</p><p>The training chapters explain:</p><ul><li>Next-token prediction</li><li>Cross-entropy loss</li><li>Forward and backward passes</li><li>Gradient accumulation</li><li>Gradient clipping</li><li>SGD</li><li>Momentum</li><li>Adam</li><li>AdamW</li><li>Learning-rate schedules</li><li>Batching and sequence construction</li><li>Data loading and preprocessing</li><li>Distributed-training concepts</li></ul><p>The runnable mini-transformer performs genuine backpropagation and optimizer updates. The tests verify that the parameters change after training.</p><p>That may sound like a small detail, but it was important to me because many “from scratch” examples calculate a loss and then print simulated metrics without updating the model.</p><p>I wanted at least one path in the repository where the complete training loop is real and inspectable.</p><h3>Dataset Engineering Deserves More Attention</h3><p>One of the biggest lessons I learned is that model code is only one part of the system.</p><p>A weak or undocumented dataset can invalidate everything built on top of it.</p><p>That is why I added two dedicated practical guides.</p><h3>Using external datasets from Hugging Face and other sources</h3><p>The first guide explains how to work with existing data from:</p><ul><li>Hugging Face Datasets</li><li>Kaggle</li><li>Direct HTTPS-hosted files</li><li>GitHub release assets</li><li>Common Crawl</li><li>Local CSV, JSONL, text, and Parquet files</li></ul><p>But loading a dataset is not just one function call.</p><p>Before using external data, developers need to inspect:</p><ul><li>The license</li><li>The source and provenance</li><li>Dataset configurations and splits</li><li>Column schemas</li><li>Revision pinning</li><li>Access restrictions</li><li>Personally identifiable information</li><li>Duplicates and contamination</li><li>Streaming behavior</li><li>Storage and checksum requirements</li></ul><p>The guide also shows how to normalize different sources into a consistent internal schema.</p><h3>Building your own dataset from scratch</h3><p>The second guide covers the full process of creating a dataset from authorized raw material:</p><pre>source definition<br>   ↓<br>collection<br>   ↓<br>immutable raw storage<br>   ↓<br>normalization<br>   ↓<br>cleaning<br>   ↓<br>deduplication<br>   ↓<br>quality checks<br>   ↓<br>leakage-safe splits<br>   ↓<br>JSONL export<br>   ↓<br>manifest and checksums</pre><p>It includes a standard-library Python builder that:</p><ul><li>Reads local text and Markdown files</li><li>Normalizes content</li><li>Rejects empty or invalid records</li><li>Removes exact duplicates</li><li>Produces deterministic train, validation, and test splits</li><li>Writes JSONL files</li><li>Generates SHA-256 checksums</li><li>Creates a manifest with dataset statistics</li></ul><p>I included this because “create a dataset” is often reduced to scraping text and saving it into one file.</p><p>That is not enough for reproducible machine learning.</p><p>A useful dataset needs provenance, structure, quality checks, versioning, and a clear explanation of what it contains.</p><h3>Inference Is a Different Engineering Problem</h3><p>Training and inference use the same model parameters, but they create very different systems problems.</p><p>During training, the model processes many tokens in parallel and stores intermediate activations for backpropagation.</p><p>During autoregressive inference, the model repeatedly generates one new token at a time.</p><p>That introduces topics such as:</p><ul><li>Prefill and decode phases</li><li>KV caching</li><li>Memory bandwidth</li><li>Continuous batching</li><li>Temperature</li><li>Top-k sampling</li><li>Top-p sampling</li><li>Repetition behavior</li><li>Speculative decoding</li><li>Quantization</li><li>Serving latency and throughput</li></ul><p>Understanding inference helped me see why a model with fewer parameters is not automatically faster in every situation, and why memory movement can matter as much as arithmetic.</p><p>It also made techniques such as grouped-query attention easier to understand. Sharing key/value heads is not only an architectural choice; it can reduce KV-cache size and memory bandwidth during decoding.</p><h3>Beyond Pretraining: Fine-Tuning, Alignment, and Distillation</h3><p>Modern language models are rarely shipped immediately after pretraining.</p><p>They usually go through additional stages that change how they respond, follow instructions, or specialize in a domain.</p><p>The repository includes explanations of:</p><ul><li>Supervised fine-tuning</li><li>LoRA</li><li>QLoRA</li><li>DoRA</li><li>Preference data</li><li>Reward models</li><li>Direct Preference Optimization</li><li>Group Relative Policy Optimization</li><li>Reinforcement learning from human or AI feedback</li><li>Offline and on-policy distillation</li></ul><p>These topics are easy to overstate.</p><p>A few equations and a toy reward function do not reproduce a complete alignment pipeline.</p><p>For that reason, the chapters label whether the code is a runnable educational implementation, a simplified illustration, a systems simulation, or an experimental proposal.</p><p>I added those labels after realizing that readers need to know not only <em>what</em> a code block shows, but also <em>what it does not prove</em>.</p><h3>RAG, Agents, Safety, and Production Systems</h3><p>A useful AI application is usually more than a model.</p><p>The surrounding system may include:</p><ul><li>Retrieval</li><li>Embedding models</li><li>Vector indexes</li><li>Tool definitions</li><li>Sandboxed execution</li><li>Conversation state</li><li>Authentication</li><li>Rate limits</li><li>Caching</li><li>Observability</li><li>Safety checks</li><li>Prompt-injection defenses</li><li>Multi-tenant serving</li></ul><p>The repository covers Retrieval-Augmented Generation, tool use, agent loops, serving, safety, and model merging because these components often determine whether an AI feature is useful in practice.</p><p>A language model generates tokens.</p><p>An AI system decides what context the model receives, which tools it can call, what actions are permitted, how outputs are checked, and how failures are handled.</p><p>That distinction is important for anyone moving from model experiments to real applications.</p><h3>Multimodal Systems and Continual Learning</h3><p>The later chapters explore how language models connect with:</p><ul><li>Images</li><li>Audio</li><li>Video</li><li>Documents</li><li>Frozen encoders</li><li>Projection layers</li><li>Temporal representations</li><li>Layout-aware processing</li></ul><p>They also discuss continual-learning problems such as:</p><ul><li>Catastrophic forgetting</li><li>Replay buffers</li><li>Online updates</li><li>Self-critique</li><li>Memory retention</li><li>Evaluation before and after adaptation</li></ul><p>These chapters are broader and more experimental than the core mini-transformer.</p><p>They are included to show how the same fundamental ideas extend into larger systems, not to claim that the repository contains production implementations of every method.</p><h3>Why Every Diagram Has a Written Explanation</h3><p>Technical diagrams often look useful while quietly assuming that the reader already understands them.</p><p>An arrow may represent a tensor flow, a residual connection, a training dependency, a control path, or a conceptual relationship.</p><p>Those are not the same thing.</p><p>For every major diagram, I added a written reading guide that explains:</p><ul><li>Where to start</li><li>What moves from one block to another</li><li>What each arrow represents</li><li>Which details have been simplified</li><li>What the diagram should not be used to conclude</li></ul><p>This may be one of the less visible parts of the repository, but I think it makes the material much more useful.</p><p>A diagram should reduce confusion, not decorate the page.</p><h3>The Runnable Mini-Transformer</h3><p>The repository includes a small decoder-only Transformer implemented in PyTorch.</p><p>You can clone the repository and run it on a normal CPU:</p><pre>git clone https://github.com/AarambhDevHub/llm-from-scratch.git<br>cd llm-from-scratch<br><br>python3 -m venv .venv<br>source .venv/bin/activate<br>python -m pip install --upgrade pip<br>make install<br>make check<br>make demo</pre><p>make check validates the Markdown and runs the unit tests.</p><p>make demo trains a deliberately small model and generates text.</p><p>The model is intentionally tiny because the goal is not to produce a useful chatbot.</p><p>The goal is to make the full path visible:</p><pre>text<br>  → token IDs<br>  → embeddings<br>  → causal Transformer<br>  → logits<br>  → cross-entropy loss<br>  → gradients<br>  → AdamW update<br>  → checkpoint<br>  → generated tokens</pre><p>Once that path works at a small scale, it becomes easier to understand what larger frameworks and distributed systems are doing.</p><h3>Who Is This Repository For?</h3><p>I wrote the material for several kinds of readers.</p><h3>Developers new to machine learning</h3><p>You can begin with the first chapter and move in order. The early sections explain the required mathematics and neural-network concepts without assuming formal ML experience.</p><h3>Machine-learning engineers who use frameworks</h3><p>You may already know how to train or fine-tune models, but want to understand the internals hidden behind framework APIs. Starting from tokenization, attention, or the Transformer block may be more useful.</p><h3>Systems programmers</h3><p>Developers working in Rust, C++, CUDA, inference, compilers, distributed systems, or performance engineering can use the later chapters to connect model architecture with memory, kernels, serving, and deployment.</p><h3>Developers building AI applications</h3><p>The RAG, dataset, safety, serving, and agent sections help connect language models to complete application architecture.</p><h3>Researchers and technical writers</h3><p>The repository can serve as a map across related topics, with implementation labels and primary references that make it easier to identify where deeper study is needed.</p><h3>What I Learned From Writing It</h3><p>Writing these notes changed how I learn technical subjects.</p><p>Reading creates familiarity.</p><p>Writing exposes gaps.</p><p>The moment I tried to explain attention step by step, I noticed which tensor transitions I could not justify.</p><p>When I tried to draw a training pipeline, I saw where data preprocessing ended and optimization began.</p><p>When I tried to write a DPO example, I had to distinguish the policy model from the reference policy.</p><p>When I tried to describe inference performance, I had to separate attention intermediates from the persistent KV cache.</p><p>When I tried to explain dataset creation, I realized that cleaning, provenance, and leakage prevention deserved as much attention as model code.</p><p>The repository is better because I made mistakes while writing it and then corrected them.</p><p>I expect more corrections will be needed as the field changes and as experienced readers review the material.</p><p>That is one reason I published it openly.</p><p>Technical knowledge improves when people can question it.</p><h3>Frequently Asked Questions</h3><h3>What is a large language model?</h3><p>A large language model is a neural network trained to predict and generate sequences of tokens. Modern LLMs usually use Transformer architectures and learn statistical patterns from large text or multimodal datasets.</p><h3>Does this repository build an LLM from scratch?</h3><p>It builds a small decoder-only Transformer with real tokenization, causal attention, loss calculation, backpropagation, optimizer updates, checkpointing, and text generation. The broader production topics are explained through educational code, diagrams, simulations, and research references.</p><h3>Do I need a GPU?</h3><p>No. The default mini-transformer demonstration is intentionally small and can run on a CPU. A GPU becomes useful for larger models, longer sequences, bigger datasets, and faster experimentation.</p><h3>Is this written in Rust?</h3><p>The learning repository primarily uses Python, NumPy, and PyTorch so the examples are accessible to more developers. Some systems discussions include C++ or kernel sketches. The broader systems thinking is connected to my Rust-based Aarambh AI work.</p><h3>Can I use a Hugging Face dataset?</h3><p>Yes. The external-dataset guide explains how to inspect licenses and schemas, pin dataset revisions, stream large datasets, handle gated access, and normalize records before training.</p><h3>Can I create my own LLM dataset?</h3><p>Yes. The from-scratch dataset guide covers collection, provenance, cleaning, exact deduplication, deterministic splits, JSONL export, checksums, manifests, privacy, and licensing considerations.</p><h3>Does it include RAG and AI agents?</h3><p>Yes, as systems-level learning material. The repository explains retrieval, vector search, tool calling, execution loops, safety boundaries, and multi-agent concepts. These chapters are not presented as a production agent platform.</p><h3>Is it a production-ready framework?</h3><p>No. It is a learning and reference repository with one runnable mini-transformer. Production systems require much more work in distributed training, data governance, performance, serving, observability, evaluation, and security.</p><h3>Where should a beginner start?</h3><p>Start with Chapter 1 and follow the learning path in order. If you already understand basic machine learning, you can begin with tokenization, embeddings, attention, or the Transformer architecture.</p><h3>Why I Am Sharing It</h3><p>Open-source work does not have to begin with a new algorithm.</p><p>Sometimes it begins with making difficult knowledge easier to navigate.</p><p>I am sharing these Markdown files because they are the notes I wanted while learning how language models fit together.</p><p>They are not final.</p><p>They are not proof that I have solved every topic inside them.</p><p>They are a structured, inspectable, and continuously improving record of the concepts I studied, the implementations I tested, and the questions I am still exploring.</p><p>My hope is that another developer can open the repository, choose a topic, follow the explanation, inspect the code, and leave with a clearer understanding than they had before.</p><p>And if someone finds a technical error, an unclear diagram, or a better primary reference, I want them to point it out.</p><p>That is how the repository should improve.</p><h3>Links</h3><p>GitHub — LLM from Scratch notes, diagrams, dataset guides, and runnable mini-transformer: <a href="https://github.com/AarambhDevHub/llm-from-scratch">https://github.com/AarambhDevHub/llm-from-scratch</a></p><p>Aarambh AI — Rust LLM source code, docs, and systems research: <a href="https://github.com/AarambhDevHub/aarambh-ai">https://github.com/AarambhDevHub/aarambh-ai</a></p><p>LinkedIn — systems programming, Rust, and machine learning: <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">https://www.linkedin.com/in/darshan-vichhi-rust-developer/</a></p><p>YouTube — Aarambh Dev Hub, with Rust tutorials and technical deep-dives: <a href="https://www.youtube.com/@AarambhDevHub">https://www.youtube.com/@AarambhDevHub</a></p><p>If this was useful, a star on the <strong>LLM from Scratch</strong> GitHub repository means more than you might think for an open-source project. It helps other developers find it.</p><p>Thanks for reading, and happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86efde0580a8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Gave My From-Scratch Rust LLM Eyes, Tools, and a Second GPU — Still Zero Lines of Python…]]></title>
            <link>https://aarambhdevhub.medium.com/i-gave-my-from-scratch-rust-llm-eyes-tools-and-a-second-gpu-still-zero-lines-of-python-653a43d83949?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/653a43d83949</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sun, 12 Jul 2026 09:35:16 GMT</pubDate>
            <atom:updated>2026-07-12T09:35:16.546Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Gave My From-Scratch Rust LLM Eyes, Tools, and a Second GPU — Still Zero Lines of Python (aarambh-ai v2.0.0)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iTKfqsocmHBtxE7ZoSn4Wg.png" /></figure><p>Two weeks ago I published the story of building a complete LLM pipeline from scratch in Rust — pretraining, GRPO, Flash Attention v2, self-learning — with not a single line of Python anywhere in the stack. The most common question in the comments wasn’t “does it work?” It was “okay, but is it just a toy, or can this actually grow?”</p><p><strong>aarambh-ai v2.0.0</strong> is my answer. Thirteen pull requests, thirteen phases, and one production release later, the same architecture that once struggled to spell “Shakespeare” now sees images, calls functions with structurally-valid JSON, trains Mixture-of-Experts layers, splits work across two GPUs, and serves its own OpenAI-compatible HTTP API. Same 8GB i3 laptop for the parts that fit. Same free Kaggle sessions for the parts that don’t. Same rule: still zero Python, still zero pretrained weights shipped.</p><p>This post is the deep-dive on what changed, why each phase was sequenced the way it was, and the specific engineering problems that made each feature hard.</p><h3>What Is aarambh-ai, in One Paragraph?</h3><p><strong>aarambh-ai</strong> (Sanskrit for <em>beginning</em>) is a decoder-only transformer LLM application built as a Rust workspace on top of the <a href="https://github.com/huggingface/candle">Candle</a> tensor backend — no PyTorch, no ONNX Runtime, no Python FFI anywhere in the dependency graph. v1.0.0 covered the full text-only vertical: pretraining, SFT, LoRA/QLoRA, GRPO alignment, INT4/GGUF quantization, Flash Attention v2 CUDA kernels, a thinking engine, safety guardrails, and CPU-capable self-learning. <strong>v2.0.0 answers the natural next question: does this scale past “one modality, one GPU, one training recipe”?</strong></p><p>Short answer: yes. Here’s how.</p><h3>The Three Directions v2.0 Grew In</h3><p>v2.0 isn’t one feature — it’s thirteen phases (16 through 28) organized around three axes:</p><ol><li><strong>Deeper on text</strong> — longer context via RoPE scaling, a real evaluation harness instead of perplexity-only, a stronger fine-tuning method (DoRA), a second alignment path (DPO) alongside GRPO, and faster inference (speculative decoding, grammar-constrained tool calling).</li><li><strong>Wider — a new modality</strong> — a frozen vision encoder plus a trainable projector gives the model the ability to see and reason about images, trained entirely on free public datasets.</li><li><strong>Bigger training infrastructure</strong> — Mixture of Experts and multi-GPU data parallelism, both deliberately framed as systems-engineering exercises whose payoff is <em>measured</em>, not assumed.</li></ol><p>That third point matters more than it sounds. Let me explain why.</p><h3>The Discipline That Shaped Everything: Measure, Don’t Assume</h3><p>v1.0.0 judged model quality with one number: perplexity on a holdout set. That’s a blunt signal — it tells you the model is getting better at predicting the next token, but not whether it can actually answer a multiple-choice question, solve a word problem, or write code that runs.</p><p>Phase 17 fixed that with aarambh-ai-eval — a proper evaluation harness with four free, public benchmark subsets sitting alongside PPL:</p><pre>| Task | What it measures | Scoring |<br>|---|---|---|<br>| MMLU-lite | General knowledge, multiple choice | Next-token logprob comparison over A/B/C/D |<br>| HellaSwag | Commonsense sentence completion | Logprob ranking over candidate completions |<br>| GSM8K-subset | Grade-school math word problems | Exact-match, reusing the existing `MathVerifier` |<br>| HumanEval-lite | Python code generation | pass@1 via sandboxed execution |</pre><p>Every task implements one shared EvalTask trait, so new tasks — like Phase 20&#39;s vqa_subset for visual question answering — plug into the same harness runner without touching it:</p><pre>trait EvalTask {<br>    fn name(&amp;self) -&gt; &amp;&#39;static str;<br>    fn run(&amp;self, context: &amp;EvalContext, config: &amp;EvalConfig) -&gt; TaskScore;<br>}</pre><p>Here’s why this comes before Mixture of Experts and multi-GPU in the roadmap, not after: <strong>every phase from 16 onward that changes model behavior has to report a before/after scorecard.</strong> aarambh-ai eval --compare before.json after.json is how Phase 22 (MoE) and Phase 24 (DPO) are judged — not by whether the feature <em>sounds</em> like it should help, but by whether the numbers actually move. At Tiny/Small scale, that&#39;s an honest question, and the harness is what makes it answerable instead of assumed.</p><h3>Longer Context Without Retraining From Zero</h3><p>v1’s RoPE computed fixed rotation frequencies once at max_seq_len. It works well inside the trained context window and degrades sharply past it — the model has literally never seen rotation angles for positions beyond what it trained on.</p><p>Phase 16 adds two RoPE scaling methods:</p><p><strong>YaRN</strong> interpolates between the original and a scaled frequency, using a ramp function that leaves high-frequency (short-range) dimensions mostly untouched while stretching low-frequency (long-range) dimensions more aggressively:</p><pre>For each RoPE dimension pair i:<br>  if i is high frequency (&lt; beta_fast):  keep original θᵢ<br>  if i is low frequency  (&gt; beta_slow):  use θᵢ / scale_factor<br>  in between:                            ramp linearly between the two</pre><p><strong>NTK-aware</strong> scaling is the simpler one-line alternative — just rescale rope_theta itself:</p><pre>θ_new = rope_theta × factor^(d / (d − 2))</pre><p>NTK-aware is a config change you can try in five minutes. YaRN needs a short continued-pretraining pass on long documents but generalizes better. Both are implemented behind RopeScalingConfig.method, and method: None reproduces v1.0.0&#39;s RoPE output <em>exactly</em> — enforced by a dedicated regression test, because &quot;we added a feature and silently changed old behavior&quot; is exactly the kind of bug that hides for months.</p><p>The practical upshot: context grows through a progressive 4K → 8K → 16K schedule during continued pretraining, KV caches are now preallocated at max_seq_len instead of growing by concatenation on every token, and the model never materializes a full [max_seq_len × max_seq_len] causal mask anymore — attention dispatch just passes causal = true straight to the kernel.</p><h3>DoRA: A Stronger LoRA, Not a Replacement</h3><p>v1’s LoRA adapts a frozen weight W₀ with a low-rank update: W = W₀ + BA. The problem is that this update changes a weight vector&#39;s <em>magnitude</em> and <em>direction</em> together, coupled through the same low-rank matrices.</p><p>DoRA (Weight-Decomposed Low-Rank Adaptation) decomposes the weight first:</p><pre>W₀ = m · (V / ||V||_c)     where m = magnitude, V/||V||_c = unit direction<br><br>DoRA update:<br>  direction&#39; = (V + BA) / ||V + BA||_c    ← low-rank update, but only to direction<br>  m&#39;         = trainable magnitude vector  ← full-rank, trained independently<br>W&#39;         = m&#39; · direction&#39;</pre><p>Magnitude and direction can now move independently instead of being locked together. Published results show this consistently beats plain LoRA at the same rank, for the cost of one extra magnitude vector per adapted layer — a negligible memory addition on top of LoRA’s footprint. finetune dora and finetune qdora sit right alongside finetune sft, finetune lora, and finetune qlora as genuine alternatives, not replacements. The eval harness decides which one wins for a given run — that&#39;s not a rhetorical flourish, it&#39;s Phase 18&#39;s actual acceptance criterion.</p><h3>Teaching a Text-Only Model to See</h3><p>This is the phase I was most nervous about, because “add vision to an LLM” is where a lot of from-scratch projects either quietly reach for PyTorch bindings or quietly give up.</p><p>The constraint I set for myself: <strong>no </strong><strong>tch-rs, no ONNX Runtime, no Python FFI, anywhere.</strong> So Phase 19 follows the LLaVA recipe instead of trying to train a vision encoder from scratch — which would need dataset scale and compute far beyond what free Kaggle sessions offer:</p><pre>Image (H×W×3)<br>   │<br>   ▼<br>Patchify + linear patch embed (frozen) → N patches × vit_d_model<br>   │<br>   ▼<br>ViT transformer blocks (frozen, pretrained CLIP-B/32 weights, ~86M params)<br>   │<br>   ▼<br>Projector MLP (trainable): vit_d_model → hidden → llm_d_model<br>   │<br>   ▼<br>N &quot;image tokens&quot; in llm_d_model space<br>   │<br>   ▼<br>Spliced into the text sequence at the &lt;image&gt; special token position<br>   │<br>   ▼<br>...the rest of aarambh-ai&#39;s decoder, completely unmodified from v1...</pre><p>A frozen, pretrained CLIP-B/32-scale encoder turns an image into a grid of patch embeddings. A small trainable projector MLP maps those into the language model’s d_model space, so each image patch looks — to the rest of the decoder — like an ordinary token embedding. The CLIP weights load as SafeTensors directly through candle-core, the exact same loading path aarambh-ai-weights already used for the language model in v1. The only new dependency is the image crate, used purely for local decode/resize/normalize. No network calls, no external service, no policy violated.</p><p>I chose <strong>prefix fusion</strong> (image tokens spliced directly into the input sequence) over a Flamingo-style cross-attention mechanism. It’s the deliberately simpler option: zero changes to TransformerBlock, GroupedQueryAttention, or the KV cache — as far as the decoder is concerned, image tokens are just tokens. The trade-off is that images now consume context budget alongside text, which is part of why long context (Phase 16) is sequenced <em>before</em> vision — more room for image and text to coexist.</p><p>Training happens in two stages, mirroring LLaVA’s own recipe: first the projector trains alone with everything else frozen (Phase 19, captioning only), then the projector trains alongside a DoRA-adapted LLM for open-ended visual question answering (Phase 20). Using DoRA rather than full fine-tuning here isn’t a stylistic choice — full fine-tuning of even the Small scale on image-text pairs would blow straight through free Kaggle quota.</p><p>The thinking engine needed zero changes for any of this. Because fusion happens before the decoder even sees the sequence, a &lt;think&gt; block generated after image tokens behaves identically to one generated after text-only context. The ThinkingController doesn&#39;t know or care that some of the preceding context came from a picture.</p><h3>Mixture of Experts: An Honest Framing</h3><p>Here’s where I made a deliberate choice to be more careful with claims than most MoE writeups are. MoE’s whole value proposition is routing to <em>specialized</em> experts across a <em>large</em> total parameter count while keeping <em>active</em> compute per token low. At Tiny/Small scale (25M–117M dense-equivalent), there isn’t a lot of room for experts to meaningfully specialize.</p><p>So Phase 22 is documented, explicitly, as a systems-engineering exercise first and a quality improvement second — with the eval harness deciding which framing turns out to be true for a given run, rather than me asserting it in the release notes.</p><pre>MoeConfig { num_experts, top_k, expert_ffn_dim, aux_loss_weight, every_n_layers }<br><br>Router: linear gate, d_model → num_experts logits per token<br>top_k_gating(logits) → indices + softmax weights, over the top_k experts only<br>MoeFfn: num_experts independent SwiGLU FFNs, each token dispatched to its<br>        top_k experts and weighted-summed<br>Load-balancing auxiliary loss: penalizes uneven expert utilization,<br>        added to the main cross-entropy loss</pre><p>MoeFfn is a drop-in replacement for the dense SwiGluFfn at the block level — same input/output shape, selected every N layers via moe: Option&lt;MoeConfig&gt; on the model config. moe: None reproduces v1&#39;s dense FFN exactly. v2.0 ships a dense masked-matmul dispatch — every expert computes on every token, then gets masked and weighted by the router — as the first, simplest-<em>correct</em> implementation. Sparse/grouped dispatch, where only the selected experts actually compute, is called out explicitly as a future optimization, not a Phase 22 requirement. Correctness first, efficiency second.</p><h3>Two GPUs, One Model: Multi-GPU Data Parallelism</h3><p>Kaggle occasionally grants 2×T4 sessions instead of the usual single GPU, and I didn’t want that extra hardware sitting idle. Phase 23 adds single-node NCCL data-parallel training: deterministic sharded data loaders so each rank sees a disjoint slice of the batch, gradient averaging across ranks, rank-zero logging and checkpointing so you don’t get duplicate log spam or checkpoint races, and a clean single-GPU fallback when the second card isn’t available. Multi-node training is explicitly out of scope — this is single-node, multi-GPU only.</p><h3>DPO: A Second Alignment Path</h3><p>GRPO (from v1) needs a verifier — something that can check whether a completion is objectively correct, like exact-match for math or compile-and-run for code. That’s great when correctness is checkable. It’s useless for “which of these two chat responses is better,” where “better” is a judgment call, not a fact.</p><p>Phase 24 adds Direct Preference Optimization for exactly that gap:</p><pre>dpo_loss = -log σ(β × [(log π(chosen|prompt) − log π_ref(chosen|prompt))<br>                      − (log π(rejected|prompt) − log π_ref(rejected|prompt))])</pre><p>π is the model being trained, π_ref is a frozen reference checkpoint. Both responses tokenize against one shared chat-template prefix, then chosen and rejected rows concatenate into a single dynamically-padded batch — one policy forward pass scores the complete preference batch instead of two separate ones. The pairwise objective uses a numerically stable two-class log_softmax rather than evaluating log(sigmoid(x)) directly, which is a classic source of silent NaN in naive DPO implementations.</p><p>finetune dpo uses DoRA; finetune qdpo uses a quantized QDoRA base. Going forward, GRPO stays the preferred path for math/code/format tasks where correctness is checkable, and DPO becomes the preferred path for open-ended chat quality. They&#39;re complementary, not competing — both remain available.</p><h3>Speculative Decoding: Two Model Sizes You Already Have</h3><p>This phase needed zero new models — it reuses the Tiny (25M) and Large (1.3B) scales that already existed in v1’s config matrix:</p><pre>Draft model (Tiny):  proposes K tokens ahead, cheaply<br>Target model (Large): verifies all K tokens in a single forward pass</pre><p>For greedy decoding, accept the longest prefix where draft tokens match the target’s argmax. For sampled decoding, a proposal x is accepted with probability min(1, p(x)/q(x)); on the first rejection, resample from the normalized residual max(0, p − q); if every proposal in a block is accepted, sample one bonus token straight from the target. This modified rejection-sampling scheme is what keeps the <em>final output distribution</em> identical to running the target model alone — speculative decoding is a speed trick, not an approximation.</p><p>Draft and target need to share a tokenizer exactly — Phase 25 checks the complete token-to-id mapping and ordered BPE merges rather than just trusting that vocabulary sizes match, which is a much easier bug to ship than it sounds. Validated two ways: wall-clock speedup (target ≥1.8× tokens/second) and, more importantly, a statistical test that the output distribution is genuinely unchanged from target-only decoding.</p><h3>Tool Calling That Can’t Emit Broken JSON</h3><p>SFT alone gets a model to <em>usually</em> emit valid JSON for tool calls. “Usually valid” breaks every downstream parser eventually. Phase 26 adds a grammar layer that constrains sampling itself — a JsonSchemaGrammar compiles a JSON Schema into a per-step valid-next-token mask, applied during the existing sampling step:</p><pre>Decode loop:<br>  ...normal token-by-token generation...<br>  on entering a tool_call span:<br>    sampler restricted to grammar&#39;s currently-valid next tokens<br>  on exiting the span:<br>    sampler reverts to normal Top-K/Top-P/Temperature</pre><p>The model becomes <em>structurally incapable</em> of emitting invalid JSON inside a tool-call span, rather than merely trained to usually avoid it. ToolCallController wraps the existing ThinkingController rather than replacing it — the model can think first, bounded by its existing budget, then either answer directly or emit a grammar-constrained call. Masking happens before temperature/top-k/top-p, and when speculative decoding is active, the same grammar state applies to both draft and target distributions, cloned for proposals and committed only for accepted tokens — which is what keeps Phase 25&#39;s exact rejection-sampling guarantee intact even with grammar constraints layered on top.</p><p>The grammar covers the practical function-schema subset: nested objects and arrays, required/optional properties, primitives, enums, nullable types, length and count bounds. Schemas using $ref, composition, regex, or recursion fail at compile time with a clear schema path, rather than silently producing wrong output at inference time.</p><h3>Serving Itself: An OpenAI-Compatible Local Server</h3><p>Phase 27 wraps everything above in aarambh-ai-serve, built on Axum 0.8.9, exposing an OpenAI-compatible HTTP/SSE API over your own locally-loaded checkpoints:</p><pre>POST /v1/chat/completions<br>POST /v1/completions<br>GET  /v1/models<br>GET  /healthz<br>GET  /readyz<br>GET  /metrics</pre><p>This is explicitly local and self-hosted — same “source, not artifacts” policy as the rest of v2.0. /v1/models only ever lists checkpoints already present on the machine it&#39;s running on; nothing gets published. Rather than static batching (wait for a fixed batch, then run one forward pass), the server uses continuous batching: a dedicated inference worker admits new requests at every decode iteration, and each request&#39;s KV cache, sampler, thinking controller, tool grammar, and stop matcher live in an isolated GenerationSession. QKV projections, output projections, norms, and FFN/MoE work batch across active rows; ragged attention stays cache-isolated per row. Prompt prefill is chunked to cap individual forward-pass size, so one huge prompt can&#39;t stall every other in-flight request.</p><p>Streaming safety scans rolling cross-token windows before releasing text, redacts PII inline, and can terminate toxic output mid-stream with a content_filter finish reason — all without breaking tool-call JSON atomicity, which is buffered and released as a whole only after validation.</p><h3>The Hard Parts Nobody Warns You About in v2</h3><p><strong>DoRA’s magnitude vector is per output row, not per weight.</strong> Candle stores linear weights as [out_dim, in_dim]. Getting the column-wise vs. row-wise magnitude/norm computation backwards silently produces a DoRA layer that trains but never actually decouples magnitude from direction — the exact bug DoRA exists to prevent.</p><p><strong>GPTQ-style precision problems reappear in DPO.</strong> Evaluating log(sigmoid(x)) directly underflows for large negative x — exactly the regime a well-trained DPO policy pushes toward. The fix is the same numerically-stable two-class log-softmax formulation you&#39;d use anywhere else logits get pushed to extremes.</p><p><strong>Speculative decoding’s KV cache truncation is easy to get subtly wrong.</strong> On rejection, both the draft and target caches need truncating to the accepted prefix — but the target cache also needs to keep the replacement or bonus token <em>pending</em> and prepend it to the next verification block. Get this wrong and you either duplicate a token or silently drop one, and it only shows up as a statistical distribution shift, not a crash.</p><p><strong>Grammar masking order matters.</strong> Masking has to happen <em>before</em> temperature/top-k/top-p, not after — apply it after and you can end up renormalizing a probability distribution over tokens that were never valid in the first place, which produces JSON that’s usually fine and occasionally, maddeningly, not.</p><p><strong>Vision self-learning needed explicit hardware gating.</strong> A frozen ViT forward pass adds real CPU cost per turn that the i3 self-learning loop from v1 was never budgeted for. Rather than let it silently degrade the CPU path, vision-mode self-learning is gated to CUDA-only and fails clearly on i3 instead of quietly getting slower every turn.</p><h3>How to Run It</h3><pre># Clone and build<br>git clone https://github.com/AarambhDevHub/aarambh-ai<br>cd aarambh-ai<br>git checkout v2.0.0<br>cargo build --release -p aarambh-ai --locked<br><br># Verify the version<br>target/release/aarambh-ai --version<br># aarambh-ai 2.0.0<br># Evaluate a checkpoint against the new harness<br>target/release/aarambh-ai eval \<br>  --config configs/tiny_shakespeare.toml \<br>  --model checkpoints/tiny_shakespeare/best.safetensors \<br>  --tasks ppl,mmlu_lite,hellaswag,gsm8k_subset \<br>  --output eval_before.json<br># Fine-tune with DoRA instead of LoRA<br>target/release/aarambh-ai finetune dora \<br>  --config configs/tiny_shakespeare.toml \<br>  --model checkpoints/tiny_shakespeare/best.safetensors \<br>  --data data/sft_examples.jsonl<br># Run inference with an image<br>target/release/aarambh-ai infer \<br>  --model checkpoints/vlm_small/best.safetensors \<br>  --image photo.jpg \<br>  --prompt &quot;What is happening in this image?&quot; \<br>  --thinking medium<br># Start the local OpenAI-compatible server<br>target/release/aarambh-ai serve \<br>  --config configs/tiny_shakespeare.toml \<br>  --model checkpoints/tiny_shakespeare/step_000050/model.safetensors \<br>  --tokenizer checkpoints/tiny_shakespeare/tokenizer.json \<br>  --host 127.0.0.1 \<br>  --port 8080</pre><p>The default loopback bind needs no authentication; anything non-loopback requires a bearer token. Same distribution policy as v1: source only, publish = false on every crate, no checkpoint, adapter, tokenizer, or GGUF file attached to the release. You train, fine-tune, quantize, and serve your own checkpoints.</p><h3>What v2.0.0 Deliberately Does Not Do</h3><p>Being upfront about scope is part of what makes the eval-harness discipline mean anything:</p><ul><li>No pretrained checkpoint, adapter, or GGUF file is released — same as v1.</li><li>Multi-node distributed training is out of scope; multi-GPU is single-node only.</li><li>The inference server is local-only — there’s no public/hosted deployment story.</li><li>Vision covers images only, via a frozen ViT. No video, no audio — yet.</li><li>MoE ships dense masked-matmul dispatch, not sparse/grouped dispatch. Simpler and correct first; faster is a documented follow-up.</li><li>Tool calls are single-call and emit-only. Aarambh does not execute commands, hit URLs, or call APIs — it produces a validated, structured request and stops there.</li></ul><p>These are the natural v3 candidates, and several of them are already scoped in the v3 roadmap I’m drafting now.</p><h3>What This Proves About Building ML Infrastructure in Rust</h3><p>When v1.0.0 shipped, the open question was “can you even build a working LLM pipeline in Rust without Python?” v2.0.0 answers a harder question: <strong>can that codebase grow in multiple directions at once without the layering collapsing under its own weight?</strong></p><p>Sixteen library crates now, up from thirteen in v1 — three new ones (aarambh-ai-eval, aarambh-ai-vision, aarambh-ai-serve) added, zero renamed, zero removed. The dependency-layered architecture from v1 is the reason that was possible: vision slots in as a new Layer 3 crate that produces token embeddings the decoder already knows how to consume. The eval harness slots in as a new Layer 5 crate that reuses verifiers the finetune crate already had. The inference server slots in as a new Layer 6 crate that wraps generation logic the inference crate already exposed. Nothing about adding a new modality or a new serving surface required touching the transformer core.</p><p>That’s the actual payoff of Rust’s ownership model and strict crate layering here — not just “no garbage collector pauses,” but a codebase where growth in one direction doesn’t quietly break another.</p><h3>What Comes Next</h3><p>v3 is already fully planned — ten features, phases 29 through 40, with ROADMAP_V3.md, ARCHITECTURE_V3.md, and SELF_LEARNING_V3.md written before a line of implementation. On the list: Gated DeltaNet hybrid attention, DeepSeek Sparse Attention, fine-grained MoE with a shared expert, on-policy distillation, native quantization-aware training, multi-token prediction, native video and document understanding, long-horizon tool-use chains, and forgetting diagnostics tied back to my other project, Manas.</p><p>The same rule holds: prove it, document it, measure it with the eval harness — before it ships.</p><h3>Final Thought</h3><p>The garbled 50-step output from the v1 post is still the honest starting point of this whole project. What’s changed since then isn’t that the model got smarter by magic — it’s that every phase, from RoPE scaling to grammar-constrained JSON, got built the same way: understand the failure mode first, write the code that fixes exactly that failure mode, then measure whether it actually helped before calling it done.</p><p>That’s not a faster way to ship features. It’s a slower, more honest one. Thirteen phases in, I’d take that trade again.</p><h3>Links</h3><p><strong>GitHub — aarambh-ai source code, docs, and the v2.0.0 release:</strong> <a href="https://github.com/AarambhDevHub/aarambh-ai/releases/tag/v2.0.0">https://github.com/AarambhDevHub/aarambh-ai/releases/tag/v2.0.0</a></p><p><strong>LinkedIn — systems programming, Rust, and ML:</strong> <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">https://www.linkedin.com/in/darshan-vichhi-rust-developer/</a></p><p><strong>YouTube — Aarambh Dev Hub (Rust tutorials and deep-dives):</strong> <a href="https://www.youtube.com/@AarambhDevHub">https://www.youtube.com/@AarambhDevHub</a></p><p><em>If this was useful, a star on the GitHub repo genuinely helps other developers find it. Thanks for reading and happy coding!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=653a43d83949" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Deleted My Entire AI Project After One Comment — Then Rebuilt It From Scratch in Rust]]></title>
            <link>https://aarambhdevhub.medium.com/i-deleted-my-entire-ai-project-after-one-comment-then-rebuilt-it-from-scratch-in-rust-9df3d132fe80?source=rss-0811e9432b8e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9df3d132fe80</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Aarambh Dev Hub]]></dc:creator>
            <pubDate>Sat, 04 Jul 2026 11:40:13 GMT</pubDate>
            <atom:updated>2026-07-04T11:40:13.808Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IALrqKXMcVmVscBuXAcEuA.png" /></figure><p>Three weeks ago I shipped Manas v1.0.0.</p><p>It was a local-first AI memory system written in Rust. You could teach it facts from text files, ask it questions, and it would answer from a local source index. No cloud. No API key. No external LLM. I was proud of it.</p><p>Then someone left a comment on LinkedIn that I couldn’t shake.</p><blockquote><em>“The hard part of a memory system is not storing facts — it is reconciling them over time. When a new fact contradicts an old one, you want provenance and an append-only history, not a silent overwrite. Memory you can trace beats memory that is merely fast.”</em></blockquote><blockquote><em>— </em>A comment on the Manas v1.0.0 LinkedIn post</blockquote><p>That comment made me go back and actually test v1 properly for the first time.</p><p>I taught it a fact. Then I deleted the text sidecar file it used to answer questions. Then I asked again.</p><pre>Not enough local memory to answer this yet.</pre><p>Then it searched DuckDuckGo.</p><p>For something I had literally just taught it.</p><p>The neural network in v1 — all the backprop, all the neurons, all the importance scoring, all the protection levels — played <strong>zero role</strong> in answering questions. It was keyword search over a text file the whole time. The neural network was decoration.</p><p>So I deleted the entire codebase and started over.</p><h3>The Real Question</h3><p>Before writing a single line of v2, I forced myself to answer one question clearly:</p><blockquote><em>Can a neural network learn a fact, store it directly in its own weights, and retrieve it from those weights alone — with no text file, no vector database, no cloud API, no GPU — running on a normal laptop CPU?</em></blockquote><p>This is called <strong>continual learning</strong> or <strong>lifelong learning</strong> in academic research. It is one of the hardest unsolved problems in AI. The big labs haven’t solved it cleanly either — they just brute-force around it by retraining from scratch on massive datasets.</p><p>I wanted to build something small and honest that actually does it.</p><h3>What Was Wrong With v1 — In Detail</h3><p>To understand v2, you need to understand exactly how v1 failed.</p><p>When you ran manas teach &quot;A cat is a small animal&quot; in v1, two things happened:</p><ol><li>The text was saved to brain.manas.sources — a plain text file</li><li>The neural network was trained on next-token prediction — learning that “cat” is often followed by “is”, “is” by “a”, and so on</li></ol><p>When you ran manas ask &quot;What is a cat?&quot;, v1 did this:</p><pre>search brain.manas.sources by token overlap with &quot;cat&quot;<br>→ find the chunk with the most matching tokens<br>→ return that chunk verbatim</pre><p>The neural network was never consulted. It was an expensive decoration.</p><p>The proof:</p><pre># teach a fact<br>./manas teach &quot;A cat is a small domesticated animal with fur and whiskers.&quot;<br>./manas ask &quot;What is a cat?&quot;<br># → correct answer ✅<br><br># delete the text sidecar<br>rm brain.manas.sources brain.manas.sourceindex<br># ask again - neural weights only<br>./manas ask &quot;What is a cat?&quot;<br># → &quot;Not enough local memory to answer this yet.&quot; <br># → falls back to DuckDuckGo web search</pre><p>The network knew nothing. The file was everything.</p><h3>Starting From Scratch — The Architecture Decision</h3><p>When I decided to rebuild, I had one foundational decision to make:</p><p><strong>Build from scratch in Rust, or use an existing ML framework like Candle or HuggingFace?</strong></p><p>The answer was clear once I understood what Manas actually needed:</p><ul><li>A network that grows new neurons when it cannot represent something</li><li>Per-neuron protection levels that are enforced at the gradient level</li><li>Content-addressed memory retrieval — neurons that fire based on input similarity</li><li>A freshness system that tracks knowledge staleness at the neuron level</li><li>A custom binary format that stores everything in one file</li></ul><p><strong>None of this exists in any framework.</strong></p><p>Candle is excellent for transformer models. HuggingFace is excellent for working with pretrained weights. But neither gives you a “grow a new neuron when loss stays high” API. Neither has per-neuron protection built into apply_gradients(). Neither can store a self-growing brain in a custom binary format with CRC32 integrity.</p><p>These things had to be built from scratch because they are the invention. The matrix multiplication underneath them is maybe 200 lines of Rust. That is the easy part.</p><h3>The Core Problem: Catastrophic Forgetting</h3><p>Before building anything, I needed to prove the engine could work at all.</p><p>The #1 enemy of continual learning is <strong>catastrophic forgetting</strong>. When a standard neural network learns something new (fact B), it overwrites the weights that stored fact A. This happens because standard backpropagation distributes error signals across all weights equally — it has no concept of “this weight is important, protect it.”</p><p>I built a standalone proof in experiment.rs — a single file, no crates, no CLI, just the math:</p><pre>// teach 5 anchor facts<br>for (input, target) in &amp;anchors {<br>    for _ in 0..300 {<br>        network.backprop(&amp;encode(input), &amp;encode(target), lr);<br>    }<br>}<br><br><br>// measure similarity BEFORE new learning<br>let before = cosine(network.forward(&amp;encode(&quot;cat&quot;)), encode(&quot;small animal&quot;));<br>println!(&quot;cat before: {:.4}&quot;, before);  // 0.8432<br>// teach 50 completely different facts<br>for _ in 0..200 {<br>    for (input, target) in &amp;noise_facts {<br>        network.backprop(&amp;encode(input), &amp;encode(target), lr);<br>    }<br>}<br>// measure similarity AFTER new learning<br>let after = cosine(network.forward(&amp;encode(&quot;cat&quot;)), encode(&quot;small animal&quot;));<br>println!(&quot;cat after:  {:.4}&quot;, after);  // 0.1204 - catastrophic forgetting</pre><p>That 0.12 was the problem clearly visible. The network learned 50 new facts and completely forgot the first one.</p><h3>The Fix: Keyed Neurons + Gradient Orthogonalization</h3><p>The solution I arrived at combines two ideas:</p><p><strong>1. Keyed neurons</strong> — instead of standard tanh(w·x + b) activation, a keyed neuron computes:</p><pre>activation = exp(-KEY_SHARPNESS × ||normalized_input - key||²)</pre><p>This is a Gaussian RBF. The neuron fires strongly only when the input closely matches its stored key. It is content-addressed memory — like a biological neuron that responds to a specific stimulus pattern.</p><p><strong>2. Gradient orthogonalization</strong> — after anchoring key neurons to specific facts, the open neurons have the anchor directions projected out of their weight gradients. This means new learning literally cannot interfere with anchor directions — the gradient space for new learning is orthogonal to the anchor space.</p><p>The result in the proof:</p><pre>cat   before: 0.8432<br>cat   after:  0.8291  ← survived 50 new facts<br>forgetting:   0.0141  ← delta under 0.15 ✅<br><br>amazon after: 0.8847  ← new fact also learned well ✅</pre><p>Old knowledge preserved. New knowledge learned. Both strong simultaneously.</p><h3>The Architecture — 8 Crates</h3><p>The full v2 system is built across 8 Rust crates, each with a single responsibility:</p><h3>manas-core</h3><p>The neural network runtime. Neurons, layers, the network itself. Growth logic. The apply_gradients() function where protection is enforced structurally:</p><pre>pub fn apply_gradients(&amp;mut self, gradients: &amp;[(u64, NeuronGradients)], lr: f32) {<br>    for (neuron_id, grad) in gradients {<br>        let neuron = self.find_neuron_mut(*neuron_id);<br>        match neuron.protection_level {<br>            ProtectionLevel::Frozen  =&gt; continue,  // zero update, always<br>            ProtectionLevel::Guarded =&gt; apply_clamped(neuron, grad, lr),<br>            ProtectionLevel::Open    =&gt; apply_full(neuron, grad, lr),<br>        }<br>    }<br>}</pre><p>Frozen neurons never update. This is not a suggestion. It is structural — enforced in the one place where weights change. Nothing above this layer can bypass it.</p><p><strong>Dependencies:</strong> rand only.</p><h3>manas-store</h3><p>Custom binary persistence. The entire brain — weights, vocab, neuron metadata, protection levels, freshness categories, source tracking — lives in one .manas file.</p><pre>[MAGIC: MANS] [VERSION: 3] [HEADER]<br>[VOCAB SECTION] [LAYER + NEURON SECTION]<br>[CRC32 CHECKSUM]</pre><p>No sidecars. No .sources. No .sourceindex. The checksum catches corruption on load. The format is append-friendly — adding a new neuron takes microseconds, not a full rewrite.</p><p><strong>Dependencies:</strong> none (pure std).</p><h3>manas-learn</h3><p>The learning engine. This is the most complex crate and where most of the interesting decisions live.</p><p><strong>Tokenizer:</strong> Character prefix n-grams with max length 4. “cat” becomes [&quot;c&quot;, &quot;ca&quot;, &quot;cat&quot;, &quot;#cat&quot;]. &quot;cats&quot; becomes [&quot;c&quot;, &quot;ca&quot;, &quot;cat&quot;, &quot;cats&quot;, &quot;#cats&quot;]. They share 3 tokens — the model learns they are structurally related. Works for any language including Unicode.</p><p><strong>Embedder:</strong> Rotary-style positional embeddings — rotating the embedding vector using sin/cos per dimension pair. “cat eats dog” ≠ “dog eats cat”. Order matters.</p><p><strong>Trainer:</strong> Two learning paths depending on network state:</p><ul><li><strong>Gradient descent path</strong> — standard backprop with growth. Tries gradient updates first, grows a new neuron if loss stays above threshold.</li><li><strong>Bound memory path</strong> — keyed neurons with analytical output fitting using ridge regression. Structurally prevents forgetting.</li></ul><p><strong>Decoder:</strong> Two-path decoding. Primary path uses a packed answer codec — word IDs are stored directly in the output vector using a marker value and scaling. Fallback path uses cosine similarity against known word embeddings.</p><p><strong>Query variants:</strong> When you ask “Where is the Eiffel Tower?”, the system generates multiple query variants by stripping stopwords and relation words: [&quot;eiffel tower&quot;, &quot;eiffel&quot;, &quot;tower&quot;, ...]. The bound memory is queried with each variant. Best scoring readout wins. Phrasing variations find the right answer.</p><h3>manas-ingest</h3><p>Input pipeline. Converts text, files, and folders into clean chunks. Supports .txt, .md, .rs, .toml, .json, .csv. Chunking with overlap. Source metadata preserved per chunk.</p><h3>manas-agent</h3><p>Internet refresh. When a neuron’s freshness category marks it as stale — Realtime after 1 day, Fast after 30 days, Slow after 365 days — manas refresh fetches updated information via DuckDuckGo, grows a new neuron for the fresh version, and the query system automatically prefers the newer answer. The old frozen neuron is never touched. This is the reconciliation system Decebal described.</p><h3>manas-language</h3><p>Language generation from learned knowledge. manas generate &quot;prompt&quot; produces fluent text based on what the brain has been taught.</p><h3>manas-benches</h3><p>Benchmark suite covering tokenization, embedding, forward pass, backprop, save, load, and full learn cycles.</p><h3>manas-cli</h3><p>The user interface. Thin layer over everything above. All business logic lives in the crates — the CLI only routes and formats.</p><pre>manas teach &quot;fact&quot;       # raw text<br>manas teach notes.md     # file<br>manas teach ./docs/      # folder<br>manas ask &quot;question&quot;     # answered from neural weights<br>manas generate &quot;prompt&quot;  # language generation<br>manas inspect            # full brain state<br>manas neurons            # list all neurons with importance scores<br>manas trace &quot;question&quot;   # trace the forward pass<br>manas forget             # compress low-importance neurons<br>manas refresh            # re-fetch stale facts<br>manas reset              # start fresh</pre><h3>The Neuron Lifecycle</h3><p>Every neuron moves through a lifecycle that directly controls the anti-forgetting system:</p><pre>Created (Open)<br>  → activation_count grows with use<br>  → importance_score = 0.40×freq + 0.30×recency + 0.20×magnitude + 0.10×age_grace<br>  → importance &gt; 0.50<br><br>Guarded<br>  → weight updates clamped to [-0.001, +0.001]<br>  → importance &gt; 0.85<br>Frozen<br>  → zero weight updates, ever<br>  → knowledge preserved structurally</pre><p>The age_grace component uses a smooth exponential decay: exp(-age_days / 7). A neuron born 3 days ago scores higher than one born 10 days ago, but both score above zero. No cliff functions.</p><h3>The Freshness System</h3><p>Different knowledge goes stale at different rates. Every neuron has a freshness category:</p><pre><br>| Category | Stale after | Examples |<br>|---|---|---|<br>| Timeless | Never | Mathematical theorems, laws of physics |<br>| Slow | 365 days | Historical facts, biographies |<br>| Fast | 30 days | Software versions, news |<br>| Realtime | 1 day | Stock prices, live scores |</pre><p>Detection is automatic. Teaching “Breaking news: the stock market fell today” assigns Realtime. Teaching “The Pythagorean theorem states that a²+b²=c²” assigns Timeless. When a stale answer is returned, a warning is appended:</p><pre>Answer<br>  Rust 1.70 was released in June 2023.<br><br>Note<br>  This knowledge may be outdated (Fast, 47 days old).</pre><h3>The Demo That Matters</h3><p>This is the test v1 failed. Here is the actual terminal output from today:</p><pre># teach 22 facts<br>./manas teach &quot;A cat is a small domesticated animal with fur and whiskers.&quot;<br>./manas teach &quot;The Eiffel Tower is located in Paris France and was built in 1889.&quot;<br># ... 20 more facts ...<br><br># delete ALL text files<br>rm -f brain.manas.sources brain.manas.sourceindex brain.manas.seq<br>rm -f brain.manas.transformer brain.manas.langmeta<br># ask - neural weights only<br>./manas ask &quot;What is a cat?&quot;</pre><pre>Answer<br>  small domesticated animal fur whiskers<br><br>Confidence<br>  1.00<br>Answered from<br>  neural weights</pre><pre>./manas ask &quot;Where is the Eiffel Tower?&quot;</pre><pre>Answer<br>  located paris france built 1889<br><br>Confidence<br>  1.00<br>Answered from<br>  neural weights</pre><pre>./manas ask &quot;What did Einstein develop?&quot;<br>./manas ask &quot;What is the mitochondria?&quot;<br>./manas ask &quot;When was Bitcoin created?&quot;</pre><p>All 5. Confidence 1.00. Neural weights only. Brain file: 82.8 KB.</p><p>No text files. No internet. No GPU. An i3 laptop.</p><h3>What Manas Is and Is Not</h3><p>I want to be direct about this because AI project descriptions have a tendency toward overclaiming.</p><p><strong>Manas v2.0.0 is:</strong></p><ul><li>An experimental research project exploring continual learning</li><li>A proof that a neural network can store and retrieve facts from weights alone</li><li>A local-first system that runs entirely on CPU</li><li>Built from scratch in Rust with no external ML framework</li><li>Something genuinely different from vector databases, RAG systems, or local LLM wrappers</li></ul><p><strong>Manas v2.0.0 is not:</strong></p><ul><li>A ChatGPT replacement</li><li>A production-ready AI system</li><li>A general-purpose language model</li><li>Something you should deploy in a real application today</li></ul><p>The honest claim: it works for the 22-fact demo. It answers from neural weights. The protection system prevents forgetting. The growth system scales with new knowledge. Whether it continues to work well at 10,000 facts or with more complex queries is still being explored.</p><p>That is what experimental means.</p><h3>Why This Matters Beyond the Project</h3><p>Modern AI has a training problem that nobody talks about honestly.</p><p>When a large language model needs to learn a new fact, the only option is retraining — on thousands of GPUs, consuming megawatts of power, requiring millions of liters of water for cooling, taking months of time. After all of that, you get a new model version. The old one is deprecated.</p><p>This is wasteful in a way that is hard to overstate.</p><p>If continual learning can be made to work reliably — even for small models — it changes what AI can be. A model that learns continuously from your own data, on your own machine, without ever needing a datacenter, is a fundamentally different kind of AI.</p><p>Manas v2.0.0 is one small experiment in that direction. It is nowhere near solving the full problem. But it proves the core piece: knowledge can live in weights, not files, and the network can grow around what it learns without forgetting what came before.</p><p>That is enough to build on.</p><h3>Getting Started</h3><p><strong>Install from release binary (Linux x86_64):</strong></p><pre>curl -fsSL https://github.com/AarambhDevHub/manas/releases/latest/download/manas-linux-x86_64.tar.gz \<br>  | tar -xz &amp;&amp; sudo mv manas-linux-x86_64 /usr/local/bin/manas<br><br>manas --help</pre><p><strong>Build from source:</strong></p><pre>git clone https://github.com/AarambhDevHub/manas.git<br>cd manas<br>cargo build --workspace --release<br>./target/release/manas --help</pre><p><strong>Requirements:</strong> Rust stable (1.75+), any CPU, no GPU, no ML library.</p><p><strong>Try the demo:</strong></p><pre>manas teach &quot;A cat is a small domesticated animal with fur and whiskers.&quot;<br>manas teach &quot;The Eiffel Tower is located in Paris France.&quot;<br>manas ask &quot;What is a cat?&quot;<br>manas inspect</pre><h3>What’s Next</h3><p>The immediate priorities for v2.x:</p><ul><li><strong>More facts, more stability</strong> — stress testing at 1,000+ facts across diverse domains</li><li><strong>Smarter growth decisions</strong> — distinguishing between “network lacks capacity” and “needs more training iterations”</li><li><strong>Improved generation</strong> — the manas generate output is functional but needs refinement</li><li><strong>macOS and Windows binaries</strong> — currently Linux x86_64 only</li><li><strong>crates.io publish</strong> — individual crates for use as libraries</li></ul><p>The longer-term question is whether the bound memory architecture scales to more complex associative reasoning — connecting facts that were learned separately to answer questions that require combining them.</p><p>That is the next hard problem.</p><h3>Links</h3><ul><li><strong>GitHub:</strong> <a href="https://github.com/AarambhDevHub/manas">github.com/AarambhDevHub/manas</a></li><li><strong>Release v2.0.0:</strong> <a href="https://github.com/AarambhDevHub/manas/releases/tag/v2.0.0">github.com/AarambhDevHub/manas/releases/tag/v2.0.0</a></li><li><strong>v1 archive:</strong> <a href="https://github.com/AarambhDevHub/manas-v1-archive">github.com/AarambhDevHub/manas-v1-archive</a></li><li><strong>ARCHITECTURE.md:</strong> full system design with binary format spec, data flow diagrams, and principle definitions</li><li><strong>ROADMAP.md:</strong> full stage-by-stage build plan with mandatory tests per stage</li><li><strong>SELF_LEARNING.md:</strong> the research concepts behind what Manas is exploring, with recommended reading</li></ul><p>LinkedIn — systems programming, Rust, and ML: <a href="https://www.linkedin.com/in/darshan-vichhi-rust-developer/">https://www.linkedin.com/in/darshan-vichhi-rust-developer/</a></p><p>YouTube — Aarambh Dev Hub (Rust tutorials and deep-dives): <a href="https://www.youtube.com/@AarambhDevHub">https://www.youtube.com/@AarambhDevHub</a></p><p>If this was useful, a star on the GitHub repo means more than you might think for an open-source project. It helps other developers find it. Thanks for reading and happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9df3d132fe80" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>