<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ekansh Jain on Medium]]></title>
        <description><![CDATA[Stories by Ekansh Jain on Medium]]></description>
        <link>https://medium.com/@ekansh.jain2011?source=rss-940d6d200e8c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*3rKDkd6MeXYI5fz6</url>
            <title>Stories by Ekansh Jain on Medium</title>
            <link>https://medium.com/@ekansh.jain2011?source=rss-940d6d200e8c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 12:07:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ekansh.jain2011/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[C++26 Is Changing How We Copy Objects]]></title>
            <link>https://medium.com/@ekansh.jain2011/c-26-is-changing-how-we-copy-objects-10935c3bd560?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/10935c3bd560</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Sat, 14 Nov 2026 15:47:50 GMT</pubDate>
            <atom:updated>2026-11-14T15:47:50.924Z</atom:updated>
            <content:encoded><![CDATA[<p>C++ has historically made copying objects pretty simple. You write a = b, and the members get copied. But introduce inheritance and start storing objects through base-class pointers, and suddenly you are responsible for building the whole thing yourself. Virtual functions work. Automatic destruction works. Copying the actual object? Now every class needs another function.</p><p>But C++26 is bringing a solution, std::polymorphic. An ownership wrapper that remembers how to copy the actual type inside it. And the reason this matters goes beyond removing a few clone() functions. It lets the classes containing those objects become ordinary copyable classes again.</p><h3>The Problem With Copying a Pointer</h3><p>Suppose you are writing a game. You have an Entity base class, and players, enemies, and other objects inherit from it. The scene stores them together, calls their virtual functions, and owns their memory. A pretty normal use of inheritance.</p><p>So you use std::unique_ptr&lt;Entity&gt;. That handles ownership and cleanup. But now you want to copy the scene. Maybe you are keeping a previous state for an undo operation, or making a separate version to simulate what happens next. Whatever the reason, you need a second set of entities that you can change independently.</p><p>And unique_ptr <a href="https://eel.is/c++draft/unique.ptr">cannot be copied</a>.</p><p>A simple solution would be to use shared_ptr instead. Now the scene can be copied. Except both scenes <a href="https://eel.is/c++draft/util.smartptr.shared">share the same entities</a>. Damage an enemy in the second scene, and the enemy in the first scene loses health too. You copied the pointers. The enemy is still the same enemy.</p><p>Storing the objects directly as Entity does not solve this either. A concrete base object only holds the base part; the extra state belonging to the enemy gets sliced away. And an abstract base cannot be instantiated at all. You need to preserve the actual derived objects, while continuing to access them through the common interface.</p><h3>Every Class Needs to Know How to Clone Itself</h3><p>The usual solution is a virtual clone() function. The base class declares it, and every concrete derived class implements its own version. For an enemy, that implementation might be as simple as:</p><pre>std::unique_ptr&lt;Entity&gt; clone() const override {<br>    return std::make_unique&lt;Enemy&gt;(*this);<br>}</pre><p>And that works. But now the player needs one too. Every other entity needs one too. The scene needs a copy constructor that goes through its collection and calls clone() on each entity. Copy assignment needs to handle the same ownership rules. You have taken an operation that normally happens automatically and spread its implementation across the hierarchy and the class owning it.</p><p>There are ways to reduce that repetition. Helpers, templates, existing libraries. But you still have to put that system together. Every new class has to participate in it, just to preserve the copying behavior you wanted from the beginning.</p><p>The frustrating part is that the concrete class often already knows how to copy itself. Enemy has a copy constructor. The missing piece is getting the owner to call that constructor when all the outside code sees is Entity.</p><h3>std::polymorphic Remembers the Actual Type</h3><p>This is what std::polymorphic&lt;Entity&gt; is going to provide. It owns an entity, but also retains the machinery needed to copy whichever concrete type it was given. Construct it with an enemy:</p><pre>std::polymorphic&lt;Entity&gt; original(<br>    std::in_place_type&lt;Enemy&gt;, 100);</pre><p>Here, 100 is passed to the enemy&#39;s constructor as its starting health. The wrapper exposes the Entity interface, while the object inside is an Enemy. Then copying becomes:</p><pre>auto copy = original;<br>copy-&gt;damage(30);</pre><p>The original enemy still has 100 health. The copy has 70. Two enemies, each with its own state. The <a href="https://eel.is/c++draft/polymorphic.ctor">copy constructor in the C++26 draft</a> preserves the concrete owned type, so the enemy’s extra data comes with it.</p><p>And the useful part is that the enemy no longer needs a virtual clone() function. Its normal copy constructor is enough. The wrapper carries the extra machinery that used to be repeated throughout the hierarchy.</p><h3>The Scene Gets Simpler Too</h3><p>Go back to the scene. Its collection can now be a std::vector&lt;std::polymorphic&lt;Entity&gt;&gt;. Copy the scene, and the vector copies its elements. Each element copies the entity it owns. The scene can use its automatically generated copy constructor again.</p><p>That is a much bigger change than saving one small function. The owning class no longer has to know how every entity gets duplicated. Add a new copyable entity type, put it in the wrapper, and the same scene-copying code continues to work. This is the <a href="https://github.com/jbcoe/value_types">composite-class design the proposal authors are targeting</a>: members that already have the behavior their containing class needs.</p><p>There is a useful detail here too. A const std::polymorphic&lt;Entity&gt; gives you <a href="https://eel.is/c++draft/polymorphic.obs">const access to its entity</a>. So when the scene is const, you cannot go through that wrapper and damage its enemy. The owned object behaves much more like a normal member of the scene.</p><p>The copying rules inside the enemy still matter. If it holds a shared texture, copying the enemy still shares that texture. If it points to another entity, copying that pointer does not automatically reconnect it to the corresponding entity in the new scene. Those relationships still need the game’s own rules. And the concrete entity itself has to support copying.</p><p>But the repeated work of preserving the derived type finally has a standard place to go.</p><p>C++ already lets us put a string or a vector inside a class and have copying work through all of its members. With std::polymorphic, that same idea is reaching objects owned through a base-class interface. An enemy can stay an enemy, the scene can own its own copy, and adding another derived class no longer means adding another clone() implementation. One less system to maintain just because you used inheritance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10935c3bd560" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[C++26 Got Rejected. Now the Three-Year Cycle Is at Risk.]]></title>
            <link>https://medium.com/@ekansh.jain2011/c-26-got-rejected-now-the-three-year-cycle-is-at-risk-20dddf33eadb?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/20dddf33eadb</guid>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[modern-cpp]]></category>
            <category><![CDATA[cpp-programming]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 06 Nov 2026 00:30:17 GMT</pubDate>
            <atom:updated>2026-11-06T00:30:17.294Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iEPJuV53aKx7fU_Et4LZrA.png" /></figure><p>C++ has been remarkably consistent about one thing since C++11. Every three years, there is another version. C++14, C++17, C++20, C++23. Some releases are bigger than others, some features take longer than expected, but the language keeps moving.</p><p>C++26 was supposed to continue that.</p><p>But in late October, the draft failed its ISO national-body ballot, for the first time in C++ history. Proposals get rejected constantly. This was the draft for the next version of the entire language.</p><p>And now there is more at stake than whether one feature makes it into C++26. There is another ballot to arrange, a possible cancellation, and a release schedule that has held together for over a decade.</p><p>The feature at the center of the public dispute? Contracts.</p><h3>A Simple Requirement Becomes a Complicated Feature</h3><p>Suppose you write a division function:</p><pre>int divide(int numerator, int denominator);</pre><p>The denominator cannot be zero. Everybody knows that, but the declaration does not say it. For a more complicated function, the requirement might be much less obvious. An index must be inside a buffer. Two arrays must have the same length. An object must have been initialized before a particular operation.</p><p>Historically, you put that in documentation, add an assertion, or check it yourself. C++26 contracts give the requirement a place in the language:</p><pre>int divide(int numerator, int denominator)<br>    pre(denominator != 0);</pre><p>Now it is right there. The pre keyword introduces a precondition, something expected to be true when the function is called. The draft also supports post-conditions and assertions inside function bodies.</p><p>That is useful. A function should be able to tell you what it needs.</p><p>But writing the condition is only the beginning. What happens when somebody violates it?</p><h3>The Contract Does Not Guarantee a Check</h3><p>The draft allows four evaluation semantics: ignore, observe, enforce, and quick-enforce.</p><p>Under ignore, the assertion has no effect. Under observe, a violation reaches a handler, but execution can continue if that handler returns normally. The enforcing forms terminate execution on a violation.</p><p>Which semantic applies is implementation-defined. The draft recommends enforcement by default, but that does not make every contract an unavoidable check.</p><p>So adding pre(denominator != 0) does not automatically make passing zero safe. If the assertion is ignored, the invalid division is still waiting inside the function.</p><p>And this distinction matters. A requirement that may be checked and a guarantee that invalid input will be stopped are different things. A programmer needs to know which one they are getting. The <a href="https://eel.is/c++draft/basic.contract">draft’s evaluation rules</a> explicitly allow that choice to depend on the implementation.</p><p>Obviously, configurable assertions are not a new idea. But standardizing them as a language feature means deciding exactly how that flexibility interacts with optimization, separately compiled code, error handling, and everything built on top of it.</p><p>That is where the disagreement becomes much bigger than the syntax.</p><h3>GCC Can Do Better With an Ordinary if</h3><p>There is already a concrete implementation problem in GCC 16.2. In specific examples, an enforced contract produces worse optimized code than a handwritten if checking the same condition.</p><p>Consider checking that x is nonnegative, then returning:</p><pre>return x &gt;= 0 ? x : -x;</pre><p>If the check is an ordinary if that calls std::terminate() for negative input, GCC understands that reaching the return means x is nonnegative. The absolute-value calculation disappears.</p><p>With an enforced contract, the published GCC 16.2 comparison found that the redundant calculation remained. The program pays for the check, but GCC still generates work that the check should have made unnecessary.</p><p>Why? GCC conservatively handles the possibility that separately compiled files use different contract evaluation modes. Its generated failure path can hide the fact that execution will never return. Disabling that conservative handling closed the optimization gap in the examples.</p><p>The <a href="https://hftuniversity.com/post/c-26-failed-its-iso-ballot-in-gcc-16-an-enforced-contract-optimizes-worse-than-an-if">investigation</a> compared assembly; it did not measure execution time. So this does not establish that every contract makes a program slower. It establishes something narrower and still important: GCC 16.2 can generate extra work for the contract version that it removes from the handwritten version.</p><p>For a language so concerned with avoiding unnecessary overhead, that deserves attention.</p><h3>The Opposition Includes Bjarne Stroustrup</h3><p>Bjarne Stroustrup and four coauthors argue that the current contracts design is too complicated, insufficiently proven in practical use, and does not provide the guarantees they want. Their objection is to this design becoming part of the standard, rather than to the general idea of expressing requirements.</p><p>Supporters disagree strongly. They argue that the concerns have already received extensive discussion, that some criticisms misrepresent the specification, and that the committee has made considered decisions about the tradeoffs. Their response reports that contracts entered the draft with 100 votes in favor and 14 against.</p><p>That is substantial support. This was not a feature that accidentally slipped into the language.</p><p>But it also had sustained opposition from people deeply involved in building C++. And when the draft reached the national bodies, it did not get the approval it needed.</p><p>Contracts are clearly central to the public dispute. The complete national-body comments were not available in the firsthand account checked for this article, so attributing every negative vote to exactly the same objection would go too far. What is clear from the <a href="https://wg21.link/p4433r0">papers on both sides</a> is that this argument was already serious before the ballot failed.</p><h3>November Is the Next Decision, Not the Next ISO Ballot</h3><p>The next WG21 meeting is scheduled for <strong>November 16–21, 2026, in Búzios, Brazil</strong>. That is the next major opportunity for the committee to agree on a response.</p><p>The committee could recommend another draft with contracts retained. It could pursue changes, including removing contracts. But removal itself requires work, particularly because other wording, including library hardening, depends on them.</p><p>A revised draft can go through another <strong>eight-week ISO ballot</strong>. As of November 5, there is no confirmed start or closing date for that ballot in the accounts checked.</p><p>And if another vote fails?</p><p>Cancellation is a possible outcome. But there is no simple rule saying that two failed votes automatically cancel C++26. The process also allows returning to an earlier draft stage or pursuing a Technical Specification. Those are decisions, not an automatic countdown triggered by a second rejection.</p><p>There is a separate deadline, though. The project was registered on <strong>January 2, 2025</strong>. Under the rules discussed in <a href="https://wg21.link/p4388r0"><em>Ship C++26</em></a>, it needs to reach the approval stage by the end of 2027 to avoid the automatic-cancellation limit, unless an extension is obtained. An extension of up to twelve months is possible.</p><p>So cancellation is real, but November is not an automatic cancellation deadline. Another failed ballot would consume more of the time available to find an acceptable way forward.</p><h3>The Three-Year Schedule Was Supposed to Prevent This</h3><p>The regular cycle exists for a reason. Pick a release date, ship what is ready, and let unfinished features wait for the next version.</p><p>That gives compiler developers, library authors, and users something predictable. A feature missing one release does not need to hold up everything else. There is another release three years later.</p><p>The committee’s <a href="https://wg21.link/p1000r6">schedule paper</a> makes this philosophy explicit. It even uses contracts being removed from C++20 as an example of letting a feature continue developing while the rest of the standard ships.</p><p>But now contracts are inside a draft that has failed approval.</p><p>If the next edition cannot be completed in 2026, the sequence established by C++11, C++14, C++17, C++20, and C++23 will lose its next expected three-year milestone. That would be the first break in that sequence.</p><p>There is an important distinction here: ISO publication can already happen after the year in the familiar name. C++23 was published as ISO/IEC 14882:2024. A late publication date alone does not prove the development cycle has broken.</p><p>The problem now is that approval of the next edition is unresolved. This is further work on what the standard will contain and whether national bodies will accept it, rather than simply waiting for publication paperwork.</p><p>Keeping the name C++26 would not make that delay disappear.</p><h3>C++ Has to Choose What It Is Willing to Delay</h3><p>Delaying the whole standard over one feature is painful. There is useful work outside contracts, and developers should reasonably expect that work to move forward.</p><p>But standardizing a disputed foundation has a cost too. Libraries build around it. Applications depend on it. People teach it. Eventually, changing the design becomes a compatibility problem for everybody who adopted it.</p><p>That is why this cannot be settled just by saying the release is late. A deadline does not prove a feature is ready. And continued opposition does not prove a feature should be removed.</p><p>The November meeting needs to produce a route that can actually get approved. Retain contracts and resolve the objections, change the design, or remove them and do the work needed to ship the rest.</p><p>C++ built its three-year cycle around the idea that no individual feature should make the entire language wait indefinitely. C++26 is now the test of whether that principle still holds when the feature has already made it into the draft.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20dddf33eadb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Got a 27B Model Running at 96K Context and 38 TPS on an 8GB Laptop GPU]]></title>
            <link>https://medium.com/@ekansh.jain2011/how-i-got-a-27b-model-running-at-96k-context-and-38-tps-on-an-8gb-laptop-gpu-f41c41281c91?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f41c41281c91</guid>
            <category><![CDATA[llm-inference]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[cuda]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 30 Oct 2026 02:13:07 GMT</pubDate>
            <atom:updated>2026-10-30T02:13:07.784Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*15k6gnKaK_lFTM0RHPre5g.png" /></figure><p>Running a 27B model on an 8GB GPU sounds like a memory problem. And initially, it is. The weights alone would take roughly 54GB in FP16, which makes the whole idea seem completely unreasonable. But Bonsai 2 27B makes that part possible through ternary quantization. The more interesting problem comes after the model fits: making it run fast with a large context. My initial setup generated about 18 tokens per second at 96K context. After working through the CUDA implementation, changing how projections and attention were executed, and testing a large number of different approaches, I eventually saw around 38 TPS on my RTX 5060 Laptop GPU.</p><p>That improvement did not come from one setting. There were changes to how the GPU read the weights, how it shared cached attention data, how it unpacked quantized values, and how the runtime selected the kernels. Some changes helped. Some made individual operations faster without noticeably improving the model. And some produced completely reasonable output while failing the numerical checks. The difficult part was figuring out which was which.</p><h3>First, the Model Has to Fit</h3><p>The starting point was Bonsai 2 27B in its PTQ1_0 format. Its main weight matrices use ternary values, meaning each weight is represented by −1, 0, or +1, together with a scale shared across a group. PTQ1_0 packs 128 weights and their scale into 28 bytes, giving those blocks a storage cost of 1.75 bits per weight. The language model is about 5.95GB, which is small enough to make GPU residency practical on my laptop. The GPU kernels consume the packed representation directly, so there is no requirement to expand the entire model back into FP16 before using it. <a href="https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf">Prism’s model card</a> describes the representation and memory requirements.</p><p>But the weights are not the only thing taking up memory. The model also needs intermediate buffers, recurrent state, and the keys and values retained by its full-attention layers. That last part becomes important when the context grows. A model fitting with a short prompt does not establish that it will fit with 96K tokens, and fitting at 96K does not establish that generating the next token will be fast.</p><p>The model I tested uses a hybrid architecture, with 48 Gated DeltaNet layers and 16 full-attention layers. This already reduces the amount of growing full-attention state compared with having full attention in every layer. I also used Q4_0 for the key and value caches while retaining FP32 recurrent state. These are separate choices: PTQ1_0 compresses the model weights, while Q4_0 compresses the attention cache. Both matter for this setup, but they introduce different work for the GPU.</p><h3>The GPU Was Doing Too Much General Work</h3><p>During single-sequence generation, the model produces one token at a time. A large part of the workload consists of multiplying weight matrices by a single activation vector. The inference engine supports many shapes, batch sizes, and devices, but my main workload was much more specific. That gave me an opportunity to specialize the projection kernels for the SM120 GPU architecture.</p><p>I added a native PTQ1_0 path for eligible single-column, two-dimensional projections, including fused up and gate projections with SwiGLU. Instead of routing this workload through the general implementation, the specialized kernel works directly with the packed ternary blocks and uses a work distribution suited to decoding one token. I also explored an eight-lane-per-row arrangement, where four rows share a 32-thread warp. The important detail was preserving the logical partial sums and reduction order while changing which threads performed the work.</p><p>There were several other experiments around this path: combining the packed tail and scale into one aligned load, staging weights asynchronously in shared memory, unrolling loops, changing thread-block geometry, and adjusting matrix-multiplication tile sizes. These are all specific changes present in the optimization work, but they should not all receive credit as successful speedups. Several remain optional because their benefit depends on the shape or because the measurements did not justify enabling them generally. Reducing the work in one kernel is useful only if it reduces the time needed to generate a token.</p><h3>At 96K, Attention Becomes Expensive</h3><p>Optimizing the weight projections was only part of the problem. With a long context, the full-attention layers have to read a large amount of cached information for each generated token. Even though the cache is quantized, those reads and the associated unpacking still cost time. The model uses grouped-query attention, where multiple query heads share the same key and value head. In this workload, six query heads use each KV head. That sharing exists in the model architecture, but the kernel has to take advantage of it.</p><p>I implemented a Q4 attention path that handles those six query heads together. It stages the packed keys and values into shared memory and reuses them across the queries, including sharing the value-unpacking work. It also uses asynchronous copies to bring in upcoming cache data while processing data already available. If six query heads need the same cached values, repeatedly loading and unpacking them separately is an expensive way to execute that sharing.</p><p>This produced one of the clearest intermediate improvements. At a 98,304-token prefix, the earlier validated six-head path moved whole-model decode from roughly 18.3–18.5 TPS to 28.7–29.0 TPS. Those were controlled decode measurements, excluding prompt processing. The later implementation also added an integer tensor-core path for the query-key dot products within the six-head kernel. The quantization scales and correction terms still have to be handled correctly, but part of the attention calculation can use the tensor cores instead of relying entirely on the previous instruction path.</p><p>There is a practical limit to how much reuse a kernel can exploit. Keeping more work together also consumes more registers and shared memory, which can reduce the number of blocks the GPU executes simultaneously. So grouping heads was not simply a matter of choosing the largest possible group. It had to be measured against the complete workload.</p><h3>Quantization Saves Memory, but Unpacking Costs Instructions</h3><p>A packed Q4 value cannot be used directly as an ordinary floating-point value. The kernel has to extract it, interpret it, and apply the scale. That sounds like a small amount of work, but it is repeated across the cache, across attention layers, for every generated token. At 96K, even a small reduction in that repeated work becomes worth investigating.</p><p>I changed the value-unpacking path to construct the required values using floating-point bit patterns, reducing integer-to-float conversions. In the examined 256-wide vector kernel, the generated code dropped from 4,848 to 4,512 static instructions and removed 256 integer-to-float conversions. That does not mean the whole model became proportionally faster; instruction counts do not translate directly into end-to-end throughput. But it removed work from an operation the model performs repeatedly.</p><p>The more important requirement was that the new unpacking produced the same values. Exhaustive checks covered more than 16 million finite-scale and nibble-pair combinations, including signed zero and subnormal values. The packed cache format stayed the same. I was optimizing how the existing representation was consumed, so changing its numerical meaning would have defeated the purpose.</p><h3>There Were Problems Outside the Main Kernels Too</h3><p>Bonsai’s runtime applies a Hadamard transform to activations before certain projections. When multiple projections use the same transformed activation, the graph can share that result. But sharing is correct only when the complete transform matches. I tightened the memoization key to include the input, rotation, signs, and feature ordering, rather than assuming that matching only part of the transform was sufficient. I also tracked logical feature width separately from padded storage width and added controls for testing different numbers of Gated DeltaNet state columns per warp.</p><p>One particularly annoying issue involved unified memory. The existing code checked whether GGML_CUDA_ENABLE_UNIFIED_MEMORY existed, so setting it to 0 still enabled the override. That affected both allocation and reported available memory. I changed the handling so zero actually disables it. When you are trying to establish that a workload fits in physical VRAM, having the runtime silently treat system memory as available changes what you are measuring.</p><p>Porting the optimizations into the newer upstream-llama tree exposed another integration problem. Its attention dispatcher selected a newer kernel before reaching the wrapper containing my optimized path. The model loaded and ran, but it was not executing the intended attention implementation. Moving the guarded Q4 path into the main dispatcher fixed that. This is why checking the source and successfully compiling it were not enough; I also needed to check which kernels actually ran.</p><h3>Correctness Was Part of the Optimization</h3><p>One of the more difficult failures came from floating-point rounding. An earlier attention candidate used a mathematically equivalent arrangement of multiplication and addition, but changed where rounding occurred. A fused multiply-add does not necessarily produce the same bits as separately rounded operations. With a quantized cache, a small difference can cross a quantization boundary, and later tokens then consume different cached values.</p><p>I preserved the reference contraction and accumulation order, built the validated native configuration with CUDA fast math disabled, and compared complete model logits. In the sustained port validation, three candidate runs each checked more than 127 million logits against the reference with zero bit mismatches. Relevant projection and attention kernels also passed memory, race, and synchronization checks. These checks establish correctness for the tested workloads; they do not mean every possible model and GPU configuration has been validated.</p><p>The final performance numbers also need that same precision. Around 38 TPS was the result I reached in use. The preserved controlled benchmark averaged about 34.8 TPS across three sustained runs from a saved 96K-token prefix, timing the final 480 of 512 decode steps. That measurement excludes model loading, checkpoint restoration, and logit copies. It is also separate from the time required to process a fresh 96K prompt. The newer-tree comparison used my already optimized implementation as its control, so its small additional gain measured preservation of performance through the port, rather than the entire improvement from 18 TPS.</p><h3>Making It Fit Was Only the Beginning</h3><p>The useful part of this work was seeing how the bottleneck changed. Once the projections became faster, attention mattered more. Once repeated cache reads were reduced, unpacking and arithmetic became more visible. Some experiments improved a narrow benchmark and then disappeared into the cost of the complete model. Others failed because the GPU spent more time coordinating the optimization than it saved.</p><p>Bonsai made a 27B model practical on my GPU. Getting useful speed at 96K required the runtime to use that compact representation efficiently, share work the architecture already allowed it to share, and avoid repeating expensive operations for every token. That is what took the setup beyond merely loading successfully. With a large codebase or a long conversation already in context, the speed of the next generated token is what I have to live with.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f41c41281c91" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Faster, Annotation Based Testing for C++: AnnoTest]]></title>
            <link>https://medium.com/@ekansh.jain2011/junit-for-c-annotest-908a947e9dab?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/908a947e9dab</guid>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[cpp26]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[modern-cpp]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 16 Oct 2026 16:45:16 GMT</pubDate>
            <atom:updated>2026-10-25T20:08:00.459Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BEE30LZMQuSAZMjo2yFEDA.png" /></figure><p>JUnit makes writing tests simple. You write a class, put methods inside it, and use annotations to describe which methods are tests, which prepare the fixture, and which clean it up. The test still looks like a method. The class still looks like a class. C++ testing has historically leaned heavily on macros to provide that same experience. But with C++26 reflection and annotations, we can finally express those instructions directly in the language.</p><p>That is why I built AnnoTest. It brings JUnit-style testing to C++, with annotated methods, parameterized cases, nested suites, lifecycle hooks, and readable configuration beside the test it affects. And it fits into the CMake and XML reporting workflows you would normally use with GoogleTest. The point is simple: writing cleaner tests should not require giving up the features or tooling that made your existing framework useful.</p><h3>A Test Can Just Be a Method</h3><p>Here is a complete AnnoTest executable:</p><pre>import annotest;<br>using namespace annotest;<br><br>struct ArithmeticTests {<br>    [[= test{}]]<br>    void addition() {<br>        assert_equal(4, 2 + 2);<br>    }<br>};<br><br>int main(int argc, char** argv) {<br>    return run_tests&lt;ArithmeticTests&gt;(argc, argv);<br>}</pre><p>ArithmeticTests is an ordinary struct. addition is an ordinary member function. The annotation marks it as a test, and assert_equal is a function call. When you pass the suite to run_tests, compile-time reflection discovers its annotated methods and prepares them for execution. Add another annotated method, and you have another test. There is no separate method registration to remember.</p><p>Obviously, a basic GoogleTest test is already short. Replacing a few lines of code with a few different lines would not be a massive improvement. The difference becomes useful when the suite grows. Your fixture state, helper methods, setup, and tests can all live together in a normal class. You do not need to inherit from a framework base class just to make that class a fixture, and you do not need macros to define the methods that belong to it.</p><h3>The JUnit Structure, Without Leaving C++</h3><p>If you have used JUnit, the organization should immediately feel familiar. AnnoTest has annotations for tests, setup before each test, cleanup after each test, and setup and cleanup around the suite. It supports readable display names, disabled tests, tags, parameterized cases, nested suites, runtime conditions, and dynamic test factories. These are the features that make annotation-based testing useful once you move beyond the first example.</p><pre>import annotest;<br>import std;<br><br>using namespace annotest;<br><br>struct VectorTests {<br>    std::vector&lt;int&gt; values;<br><br>    [[= before_each{}]]<br>    void setup() {<br>        values = {1, 2, 3};<br>    }<br><br>    [[= test&lt;&quot;appending preserves existing elements&quot;&gt;{}]]<br>    void append() {<br>        values.push_back(4);<br><br>        expect_equal(4u, values.size());<br>        expect_equal(1, values.front());<br>        expect_equal(4, values.back());<br>    }<br><br>    [[= test{}]]<br>    void initial_size() {<br>        assert_equal(3u, values.size());<br>    }<br>};</pre><p>The state and the behavior are right there. Each test gets a fresh fixture by default, so append does not change the starting conditions for initial_size. That also applies to individual parameterized cases. And because the fixture is still a C++ object, constructors, destructors, and RAII work normally. Use annotated hooks when they make the setup clearer, and use ordinary resource ownership where it already solves the problem. There is no reason to replace C++ resource management just because the object happens to contain tests.</p><p>AnnoTest also supports nested suites, letting related scenarios live inside the suite they belong to. Child suites inherit the enclosing name prefix and tags, so the structure you read in the source carries through into test selection and reporting. This is particularly useful when one component has several groups of behavior and putting every case into one flat collection starts making the tests harder to follow.</p><h3>Parameters Belong Beside the Test</h3><p>Parameterized tests are one of the clearest examples of why this approach is useful. You want to run one behavior with several inputs. But in GoogleTest, that normally means a parameterized fixture, a TEST_P definition, retrieving the parameter inside the body, and a separate instantiation step. It works, but the framework machinery spreads a simple idea across several declarations.</p><p>AnnoTest keeps the cases beside the method:</p><pre>constexpr int add(int a, int b) {<br>    return a + b;<br>}<br><br>struct ArithmeticTests {<br>    [[<br>        = test{},<br>        = parameterize{<br>            parameters(2, 3, 5),<br>            parameters(5, 7, 12),<br>            parameters(-1, -3, -4)<br>        }<br>    ]]<br>    void addition(int a, int b, int expected) {<br>        assert_equal(expected, add(a, b));<br>    }<br>};</pre><p>The method takes named arguments. The annotation supplies those arguments. The expected result is just another parameter. You can read the inputs, the signature, and the check together without jumping to another part of the file to find where the cases are instantiated. And each case is individually selectable, so debugging one failure does not require running every input again.</p><p>This extends beyond fixed integer examples. AnnoTest supports template parameterization, combined type and value parameters, cross products, runtime providers, and CSV or line-based data sources. Dynamic test factories can return named child tests when the cases need to be generated at runtime. So you get the straightforward annotation-based interface for small tests without running out of options when the inputs become more complicated.</p><h3>Configuration You Can Actually See</h3><p>Another advantage is that test configuration does not have to be hidden in names, runner scripts, or the beginning of the test body. AnnoTest lets you put a display name, tags, repetitions, conditions, and a deadline directly on the declaration.</p><pre>struct ServiceTests {<br>    [[<br>        = test&lt;&quot;service responds before its deadline&quot;&gt;{},<br>        = tag&lt;&quot;integration&quot;&gt;{},<br>        = timeout{5}<br>    ]]<br>    void response() {<br>        // ...<br>    }<br><br>    [[= test{.repetitions = 3}]]<br>    void repeated_check() {<br>        // ...<br>    }<br><br>    [[= test{.disabled = true}]]<br>    void unfinished_case() {<br>        // ...<br>    }<br>};</pre><p>Look at response, and you know that it is an integration test with a five-second subprocess deadline. Look at repeated_check, and you know it runs three times. A disabled test can keep its actual name. These are small things individually, but together they make a suite much easier to read. You should not have to reconstruct how a test behaves from a naming convention and three separate configuration files.</p><p>GoogleTest already has fixtures, typed tests, parameterized tests, exception checks, and death tests. AnnoTest’s distinction is how much it expresses directly through annotated declarations, together with features such as nested suites, explicit tags, per-method repetitions, and deadline annotations. Its runner also provides subprocess parallelism and hard per-case timeouts, and there is dedicated support for checking C++26 contract violations. You get more than a different spelling for TEST.</p><h3>It Fits Your Existing CMake Pipeline</h3><p>Cleaner syntax is only useful if you can actually adopt it. If changing the testing library means rebuilding your entire build and reporting setup, that becomes a completely different project. AnnoTest integrates directly with CMake and CTest, just like GoogleTest does. It provides an annotest::annotest target and an annotest_discover_tests helper that registers individual cases with CTest.</p><p>Once AnnoTest is available and the required C++26 module configuration is in place, the test target looks like this:</p><pre>enable_testing()<br><br>add_executable(my_tests test.cpp)<br>target_link_libraries(my_tests PRIVATE annotest::annotest)<br><br>annotest_discover_tests(my_tests)</pre><p>Build the target, then run ctest --test-dir build --output-on-failure. The discovery helper asks the executable for its cases and registers them individually, so your CMake workflow does not need to treat the entire executable as one opaque test. You can consume AnnoTest through a source checkout, FetchContent, or an installed package.</p><p>And the reporting side is familiar too. AnnoTest produces XML with suite and case names, timings, failures, skipped cases, and captured output. Its output follows the same general JUnit-style structure used by GoogleTest reports, so it can replace GoogleTest in pipelines that consume that structure. If your pipeline mainly builds an executable, checks its exit status, and publishes its XML report, most of that workflow can stay in place.</p><p>You still need to port the test source and update runner arguments, and a parser that depends on GoogleTest-specific details may need adjustments. But ordinary XML report publication does not have to become a migration project of its own.</p><h3>And It Compiles Faster Than GoogleTest</h3><p>The syntax is the main reason I built AnnoTest, but compilation time matters too. Tests are code you change repeatedly. Add a case, adjust an assertion, rebuild, and wait. If a framework makes that cycle faster, you benefit every time you work on the suite.</p><p>In the repository’s recorded benchmark, AnnoTest compiled faster than GoogleTest in all 24 comparisons across regular, value-parameterized, and output-producing workloads in Debug and Release. For 240 cases in Release, the compile times were 3x faster for regular tests and 1.5x faster for parameterized tests.</p><p>The regular-test file compiled in roughly a third of the time. These are medians of five samples using GCC 16.2.0 and GoogleTest 1.15.0, with the framework libraries and module interfaces already built. They measure test-file compilation, excluding linking and execution. Building AnnoTest and its dependencies from scratch has a substantial initial cost, so this is specifically an advantage in the measured compilation of test files after that setup.</p><p>AnnoTest requires GCC 16.1 or newer, CMake 4.3.3 or newer, and Ninja, with the required C++26 facilities enabled. It is open source under the MIT license, and the repository includes a development container, examples, and setup instructions.</p><p>If you have wanted JUnit-style tests in C++, this is the reason to try it. Keep the annotated methods, readable fixtures, and parameterized cases you wanted. Keep CMake and XML reporting in your pipeline. And when you add the next test, make the change where it belongs: beside the behavior you are testing.</p><p>Checkout AnnoTest <a href="https://github.com/EJainDev/AnnoTest">here</a> and the <a href="https://anno-test.readthedocs.io/en/latest/">docs</a> here. Version 1.0 coming soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=908a947e9dab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SerialXML 1.0: Stop Writing Your C++ Structs Twice]]></title>
            <link>https://medium.com/@ekansh.jain2011/serialxml-1-0-stop-writing-your-c-structs-twice-037dcbc1d40f?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/037dcbc1d40f</guid>
            <category><![CDATA[serialization]]></category>
            <category><![CDATA[cpp26]]></category>
            <category><![CDATA[xml]]></category>
            <category><![CDATA[cpp]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Sat, 10 Oct 2026 01:34:25 GMT</pubDate>
            <atom:updated>2026-10-10T01:34:25.828Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fwTUblfNGzXQNMpYMAg91A.png" /></figure><p>Serializing a C++ object often means describing the same object twice. First, you write the struct. Then you write the code telling a serialization library which members the struct has. Add another member, and now there are two places to update. Want to read the object back? Depending on your approach, there may be a third.</p><p>The compiler already knows those members exist. It knows their names and their types. So why are we telling it again?</p><p>That is the problem I built <strong>SerialXML</strong> to solve. And with version 1.0, it supports both XML serialization and deserialization using C++26 reflection.</p><p>Start with an ordinary struct. Add annotations where you want different XML behavior. Let the library handle the repetitive work.</p><h3>One Struct, One Function Call</h3><p>Here is a complete example:</p><pre>import std;<br>import serial_xml;<br><br>struct Person {<br>    int age;<br>    std::string favorite_food;<br>};<br><br>int main() {<br>    auto xml = serial_xml::to_xml(Person{3, &quot;pizza&quot;}, false);<br>    std::println(&quot;{}&quot;, xml);<br>}</pre><p>The output:</p><pre>&lt;Person&gt;&lt;age&gt;3&lt;/age&gt;&lt;favorite_food&gt;pizza&lt;/favorite_food&gt;&lt;/Person&gt;</pre><p>That is all the serialization code.</p><p>No registration macro. No list of member pointers. No serialization function inside Person. The second argument, false, just omits the XML declaration.</p><p>And reading it back is equally simple:</p><pre>auto person = serial_xml::from_xml&lt;Person&gt;(xml);</pre><p>Now person.age is 3, and person.favorite_food is &quot;pizza&quot;.</p><p>Obviously, this is a small example. Writing two fields manually would not be horrible. But now imagine an order with a customer, a shipping address, a list of products, and optional delivery instructions. Then imagine changing those types while keeping the serialization code synchronized.</p><p>That is where the repetition becomes a problem.</p><p>SerialXML uses compile-time reflection to inspect accessible members and build the mapping. Nested structs are unpacked automatically when they are not formattable, and common containers such as std::vector and std::array are iterated automatically.</p><p>You write the structure once.</p><h3>XML Still Needs Choices</h3><p>Automatically using member names is a useful default. But real XML formats have opinions.</p><p>Maybe the root must be lowercase. Maybe an identifier belongs in an attribute. Maybe favorite_food needs to become food because another application already expects that name.</p><p>SerialXML expresses those choices through C++26 annotations:</p><pre>struct [[= serial_xml::name{&quot;person&quot;}]] Person {<br>    [[= serial_xml::attribute]]<br>    int age;<br>    [[= serial_xml::name{&quot;food&quot;}]]<br>    std::string favorite_food;<br>};</pre><p>The same serialization call now produces:</p><pre>&lt;person age=&quot;3&quot;&gt;&lt;food&gt;pizza&lt;/food&gt;&lt;/person&gt;</pre><p>The annotation sits beside the member it affects. Look at age, and you immediately see that it becomes an XML attribute. Look at favorite_food, and you see its XML name.</p><p>And the same annotations work when reading the XML back.</p><p>There are annotations for skipping members, choosing range element names, emitting CDATA, controlling formatting, and omitting empty ranges. Invalid combinations are diagnosed at compile time. Asking for a member to be both an attribute and an unpacked object, for example, is rejected.</p><p>The defaults handle the ordinary case. The annotations describe the choices that are actually specific to your format.</p><h3>Reading XML Means Handling Missing Data</h3><p>Writing XML is only half the job. Reading it means dealing with someone else’s input.</p><p>What happens when a required field is missing? What happens when an integer contains &quot;hello&quot;? What happens when the tags do not match?</p><p>SerialXML reports malformed XML, missing required members, and invalid values through serial_xml::deserialization_error.</p><p>But sometimes a missing field is completely reasonable:</p><pre>struct Person {<br>    [[= serial_xml::attribute]]<br>    int age;<br><br>    std::string name;<br>    [[= serial_xml::optional]]<br>    std::string nickname = &quot;unknown&quot;;<br>};<br><br>auto person = serial_xml::from_xml&lt;Person&gt;(<br>    &quot;&lt;Person age=&#39;21&#39;&gt;&lt;name&gt;Ekansh&lt;/name&gt;&lt;/Person&gt;&quot;<br>);</pre><p>Here, nickname remains &quot;unknown&quot;. The XML does not contain it, and the annotation explicitly allows that.</p><p>There is also an in-place API for updating an existing object. Accessible getters can be paired with matching setters, and the setter annotation lets you select an input method directly. So reconstruction can go through a class’s public interface.</p><p>Custom text representations have an extension point too: serial_xml::from_string&lt;T&gt;.</p><p>But a custom formatter is not automatically reversible. If you write a product code as &quot;SKU-42&quot;, you need a matching conversion that understands &quot;SKU-42&quot;. Reflection can find your members. It cannot recover information your formatter discarded.</p><h3>What If You Cannot Change the Class?</h3><p>Annotations are convenient when you own the type. But what about a type from another library?</p><p>SerialXML supports external schemas, called mocks. These describe the accessible fields or const getters you want to serialize and supply their annotations.</p><p>For example:</p><pre>class mock_vector {<br>public:<br>    [[= serial_xml::attribute]]<br>    std::size_t size() const;<br>};<br><br>const std::vector&lt;int&gt; values{1, 2, 3};<br>auto xml = serial_xml::to_xml&lt;mock_vector&gt;(values, false);</pre><p>The output:</p><pre>&lt;vector size=&quot;3&quot;/&gt;</pre><p>The mock’s size() function does not need a definition. SerialXML matches the declaration to the real vector’s size() getter and calls that.</p><p>This is useful for more than third-party types. An external schema can select a particular view of your own class, control the output order, and keep XML choices outside the domain type. External schemas are supported for deserialization too, where the selected members can be reconstructed.</p><p>So now you have a choice. Put the XML configuration beside the data, or describe it separately when that makes more sense.</p><h3>Less Boilerplate, With Measured Performance</h3><p>Removing repetitive code was the reason for building SerialXML. But performance matters too.</p><p>The repository benchmark compares SerialXML with Boost.Serialization, cereal, and pugixml using the same logical order: customer details, a shipping address, three line items, and four integer tags.</p><p>On an Intel Core 7 240H with GCC 16.2.0 and an optimized release build, the median results across five runs were:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J-zXxVudhjO0ZS10Y8QtyA.png" /><figcaption>Benchmarks for SerialXML against other common libraries</figcaption></figure><p>LibrarySerializationDeserializationSerialXML830 ns/order849 ns/orderBoost.Serialization6,960 ns/order14,250 ns/ordercereal7,700 ns/order3,195 ns/orderpugixml1,523 ns/order936 ns/order</p><p>Deserialization includes XML parsing, value conversion, construction of an owning result object, and cleanup. The pugixml comparison uses handwritten field mapping, while SerialXML also checks required members.</p><p>These are measurements of this workload on this machine. Each library reads its own XML representation, and those representations differ. They are useful evidence, but they do not tell you which library will win on every possible XML document.</p><p>The <a href="https://github.com/EJainDev/SerialXML#benchmarks">repository</a> includes the benchmark and reproduction instructions. Try your own data. That is the comparison that matters for your application.</p><h3>Trying SerialXML 1.0</h3><p>SerialXML can be accessed <a href="https://github.com/EJainDev/SerialXML">here</a> and is open source under the MIT license. It is distributed as the serial_xml C++ module, with CMake integration and examples covering the main features.</p><p>The toolchain requirement is significant: the project currently supports GCC 16.1 or newer, CMake 4.3.3 or newer, and Ninja, with C++26 reflection and experimental import std support enabled. GCC documents reflection and reflection annotations among its <a href="https://gcc.gnu.org/gcc-16/changes.html">GCC 16 features</a>. The repository provides a matching development container and the complete build setup.</p><p>The XML reader supports UTF-8 XML 1.0, including attributes, comments, CDATA, and character references. DTDs and external entities are unsupported, and namespace prefixes are matched literally rather than resolved.</p><p>You can find the code, installation instructions, and examples on <a href="https://github.com/EJainDev/SerialXML">GitHub</a>.</p><p>C++ reflection is interesting because it removes work that exists only because a library previously could not see what the compiler already knew. Serialization is a very concrete example of that.</p><p>With SerialXML 1.0, adding an ordinary member to your struct can be enough to add it to your XML. You spend your time deciding what the format should look like, instead of repeatedly explaining what your object contains.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=037dcbc1d40f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[C++ Could Let You Generate Your Own Interfaces]]></title>
            <link>https://medium.com/@ekansh.jain2011/c-could-let-you-generate-your-own-interfaces-7e8772df32e1?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e8772df32e1</guid>
            <category><![CDATA[modern-cpp]]></category>
            <category><![CDATA[future]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[cpp]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Thu, 01 Oct 2026 23:30:43 GMT</pubDate>
            <atom:updated>2026-10-01T23:30:43.056Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FBUdizwgbGJuRg8tG-zBTA.png" /></figure><p>C++ makes you repeat a lot of code that contains almost no new information. You declare an interface, then write a wrapper with the same functions. Then a mock with the same functions. Then another wrapper that logs calls before forwarding them. The behavior changes, but the signatures keep getting copied. And every copy has to stay correct. Change one parameter in the original interface, and now several other declarations need updating, even though none of them made an independent decision about that parameter.</p><p>Token sequence injection could change that. The proposal would let C++ code generate declarations and function bodies during compilation, using information obtained through reflection. It is still proposed functionality, with examples implemented in an experimental Clang compiler. But the practical idea is straightforward: describe how the repeated code should be produced, then let the compiler produce it.</p><h3>Start With One Generated Method</h3><p>Before getting into interfaces, look at a much smaller example. Suppose a configuration object has a port number, and we want to generate a getter for it. Using the proposed syntax, the construction could look like this:</p><pre>#include &lt;meta&gt;<br>struct Config {<br>    int port;<br><br>    consteval {<br>        auto name = std::meta::id(&quot;get_port&quot;);<br>        std::meta::queue_injection(^^{<br>            int \(name)() const {<br>                return port;<br>            }<br>        });<br>    }<br>};</pre><p>This is illustrative proposal syntax, not code for an ordinary C++26 compiler. The consteval block runs during compilation. std::meta::id creates an identifier token, which we store in name. The ^^{ ... } expression creates a token sequence containing the method we want to add, and \(name) inserts the generated identifier into that sequence. Finally, std::meta::queue_injection queues the sequence for injection into the surrounding class.</p><p>The resulting method is equivalent to writing this inside Config:</p><pre>int get_port() const {<br>    return port;<br>}</pre><p>Obviously, this particular example saves nothing. We wrote more code to generate one getter than we would have written to declare it normally. But it shows the important difference: the method’s name and source can be assembled by compile-time code. They do not have to be fixed declarations written directly into the class. Once those pieces can come from a calculation, a loop, or reflected information, the same generator can produce many different methods.</p><h3>The Repetition Is Already Described</h3><p>Now take an interface for storing files:</p><pre>#include &lt;string&gt;<br>#include &lt;string_view&gt;<br><br>struct FileStore {<br>    virtual ~FileStore() = default;<br>    virtual std::string read(std::string_view path) const = 0;<br>    virtual void write(std::string_view path,<br>                       std::string_view contents) = 0;<br>};</pre><p>Suppose you want a wrapper that logs every operation before forwarding it to another FileStore. Today, the straightforward implementation repeats both function signatures. Its read method records the operation and returns the result of the underlying read. Its write method records the operation and forwards the arguments. Add another operation, and you write another forwarding method. Build a mock for testing, and you reproduce the interface again.</p><p>Most of that work is mechanical. Copy the function name, preserve the parameter types and qualifiers, forward the arguments, and handle the return value. The interface already describes those details. The programmer is translating them into another class by hand. For two functions, that is barely an inconvenience. For a large interface with several wrappers and testing implementations, it becomes a considerable amount of code whose main purpose is to stay synchronized with other code.</p><p>A generator could instead inspect the interface and produce the forwarding declarations from what it finds. The logging policy would provide the extra behavior. Reflection would provide the information about the original functions. Token injection would provide the means to create the new functions. Those are separate jobs, and bringing them together is what makes the proposal useful.</p><h3>Reflection Needs Somewhere to Put the Code</h3><p>C++26 reflection gives compile-time code access to information about program entities. It also includes limited construction facilities, such as completing an aggregate from descriptions of its data members. So reflection is not purely about reading existing declarations. But that aggregate construction facility does not give you a general mechanism for generating arbitrary methods and their bodies.</p><p>Token sequences provide a way to express that missing part using the shape of C++ itself. The tokens are assembled before being parsed at the point of injection. A generator can therefore construct a declaration in pieces, inserting types, names, and other token sequences where needed. Instead of requiring a separate construction API for every possible statement or declaration, the generated code still looks substantially like the code you would have written.</p><p>The simple getter used a fixed name and body to make the syntax visible. A reusable getter generator would obtain the member’s name and type through reflection, construct the appropriate method name, and generate access to that member. A forwarding generator would do more work, but the principle is the same. Write the transformation once, then apply it to the declarations that need it.</p><h3>Interfaces Without Changing the Original Classes</h3><p>There is another substantial consequence. Sometimes unrelated classes already provide the operations you need, but they do not inherit from the same base class. One might belong to a third-party library. Another might have existed years before your interface. Templates and concepts work well when the concrete types remain known, but using different concrete types through one runtime interface needs additional machinery.</p><p>The proposal demonstrates generating a wrapper with forwarding methods and a dispatch table from an interface description, checking that the supplied concrete types provide the required operations. The original classes do not need to inherit from that interface. The generated adapter connects the operations they already have to the interface the caller wants to use. The adapter still exists, but the application programmer does not have to reproduce its implementation for every combination.</p><p>That is useful because a class should not need advance knowledge of every interface someone might later want it to satisfy. When the necessary operations already exist, much of the remaining work is translation. Code injection lets that translation become reusable library machinery.</p><h3>The Generator Is Still Your Responsibility</h3><p>Generated code still needs to be correct. A forwarding method can mishandle references, lose qualifiers, or create a lifetime problem just as easily as a handwritten method. The difference is that a generator can spread the same mistake across many types. Its implementation needs careful review, and developers need good ways to inspect the result when something goes wrong. The proposal includes diagnostic examples and tools for examining generated tokens and compiler structures, but comfortable tooling would be essential to everyday use.</p><p>Still, reviewing one reusable generator is a different task from maintaining dozens of wrappers that gradually drift apart. The generator can capture the details that should remain consistent, leaving the application to supply the behavior that actually differs.</p><p>And that is where token sequence injection earns its complexity. C++ programmers should decide what a logging wrapper records, what a mock does, and what an interface requires. Rewriting the signatures is mostly bookkeeping. If the language can carry that information from the original declaration into the code built around it, changing an interface no longer has to mean changing every copy of that interface. The behavior would still be yours to write. The repetition could finally become the compiler’s job.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e8772df32e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pattern Matching is Coming to C++]]></title>
            <link>https://medium.com/@ekansh.jain2011/pattern-matching-is-coming-to-c-46f73638590e?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/46f73638590e</guid>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[modern-cpp]]></category>
            <category><![CDATA[low-level-programming]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 25 Sep 2026 02:17:30 GMT</pubDate>
            <atom:updated>2026-09-25T02:17:30.166Z</atom:updated>
            <content:encoded><![CDATA[<p>C++ has historically had a different solution for every way you might want to check a value. switch, if, std::visit, structured bindings, dynamic_cast, and a bunch of other features depending on exactly what you are checking. But pattern matching changes that. Instead of having one system for integers, another for variants and another for structured values, there is one way to describe what something should look like. And the reason this is so important is pretty simple, C++ already has almost all of the pieces. They just have never worked together.</p><h3>Checking Values — The Mess of C++</h3><p>Take a std::variant&lt;int, std::string&gt;. It can contain an integer or a string and the compiler already knows those are the only possibilities. But now what if you actually want to do something depending on which one is stored? You can check it with std::holds_alternative, then get the value using std::get. Obviously, doing that everywhere would be horrible, so C++ also has std::visit.</p><p>And std::visit works. That’s not really the problem. The problem is that now there is an entirely separate system just because the thing being checked happens to be a variant. If it was an integer or enum, you would probably use switch. If it was a structure, maybe structured bindings and then an if. If it was a polymorphic object, dynamic_cast. All of these features solve their own problem pretty well, but they are all solving some version of the same problem. You have something, you want to know what it is, and depending on that, you want to do something.</p><h3>The match Keyword</h3><p>Pattern matching gives this one system, match. Instead of checking the type first and then separately getting the value, you describe what you want directly.</p><pre>match (value) {<br>    case { int number } =&gt;<br>        handle_number(number);</pre><pre>    case { const std::string&amp; text } =&gt;<br>        handle_text(text);<br>}</pre><p>If the variant contains an integer, the first case matches and gives you the integer as number. If it contains a string, the second one matches and gives you the string. That’s it. There is no separate call to check what type is stored and another one to actually get it.</p><p>But if this was all match did, it really would not matter that much. std::visit already solves this problem. The much bigger part is that match is not specifically for variants.</p><p>Normal values can be matched too:</p><pre>match (value) {<br>    case 0 =&gt;<br>        handle_zero();</pre><pre>    case 1 =&gt;<br>        handle_one();</pre><pre>    case _ =&gt;<br>        handle_everything_else();<br>}</pre><p>Now it looks basically like switch. So the same feature can already do what normally takes 2 completely different systems depending on the type being checked. But even that is not really where pattern matching becomes interesting.</p><h3>Matching What Something Looks Like</h3><p>The much bigger part is that patterns can describe the structure of a value, not just compare one value against another. Say you have a point with an x and y, and you only care when y is 0. Normally, you pull the object apart, create variables for the values, and then add an if statement to check y. 2 separate steps for something that is really one condition.</p><p>With pattern matching, a pattern like [int x, 0] can mean that the first value should be stored into x, while the second has to be 0. The extraction and the check happen at the same time.</p><p>And well, now the reason for this feature starts becoming a lot clearer. switch checks values. Structured bindings pull values apart. std::visit checks which alternative a variant contains. Pattern matching can do those things as part of the same operation. You describe what the value should look like, and if it looks like that, the case runs.</p><p>This is also why calling pattern matching a better switch kind of misses the point. C++ does not desperately need another way to compare integers. switch already does that perfectly fine. The important part is that this same system keeps working when the value becomes more complicated.</p><h3>The Compiler Already Knows</h3><p>There is another massive advantage, the compiler already knows a lot of the possible cases. Go back to std::variant&lt;int, std::string&gt;. There are 2 possible alternatives. Not 3, not 20, 2. So if you handle the integer and completely forget about the string, the compiler already has enough information to know that.</p><p>This is exhaustiveness checking. If every possible case is known, the compiler can check that every possible case was actually handled. And this gets much more useful once code starts changing. Say that variant originally contains an integer and a string. Everything works. Then months later someone adds another type. There could be old code somewhere that still assumes there are only 2 possibilities. Normally, somebody has to remember that code exists. Eventually, somebody won’t.</p><p>But the compiler already knows that the variant changed. It knows the possible types and it knows what the match handles. There is no reason the developer should also have to manually keep track of the exact same information. You still decide what every case does, the compiler just makes sure you didn’t forget one.</p><h3>One Feature Instead of Five</h3><p>And this is really the important part of pattern matching. C++ already does almost everything here. It already has value matching. It already has type checks. It already has structured bindings. It already has variants. It already has visitors. Pattern matching does not suddenly make something possible that C++ could never do before.</p><p>It just stops making all of these things completely separate.</p><p>Instead of asking whether this should be a switch, a std::visit, a structured binding with an if, or some type check, you describe the thing you want to match. The type can change, the structure can change, the pattern can get more complicated, but the actual idea stays the same.</p><p>C++ has spent decades adding incredibly powerful features, but sometimes that also means there are 5 different ways to solve almost the same problem. Pattern matching is one of those rare features that does not really add a completely new ability. It takes a bunch of abilities C++ already has and finally makes them feel like they belong to the same language.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46f73638590e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why AI Subsidization is Here to Stay]]></title>
            <link>https://medium.com/@ekansh.jain2011/why-ai-subsidization-is-here-to-stay-b3dfde3297ac?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3dfde3297ac</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[money]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 23:22:52 GMT</pubDate>
            <atom:updated>2026-09-11T23:22:52.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vE3YIkVOO0y9hwypbShVcg.png" /></figure><p>Many have theorized that AI subsidization will go away. Fixed rate plans will permanently be replaced with token based billing, but that won’t happen and Anthropic is proof of that.</p><h3>AI Is Getting Cheaper</h3><p>The first problem with the idea that subsidization cannot last is that it assumes AI will always cost what it costs right now. That is just not true. The cost of serving models is dropping fast, and almost every part of the industry is pushing it down at the same time. Models are becoming more efficient, inference is becoming more optimized, hardware is getting faster, and competition is getting much stronger.</p><p>This is especially important because the newest model is not the only thing that matters. Frontier models might keep getting larger and more expensive, sure, but yesterday’s level of intelligence becomes cheaper over time. A model that takes a massive cluster to run today may be something much easier to serve a few years later. We have already seen this happen over and over. The cost of the capability falls, even if companies keep pushing the frontier forward.</p><p>And for subscriptions, that is what matters. If the same quality of output gets cheaper every year, then a plan that looks heavily subsidized today can become much more reasonable later without changing anything from the user’s perspective.</p><h3>Chinese Models Are Driving Prices Down</h3><p>One of the biggest forces here is Chinese open-source and open-weight models. DeepSeek, Qwen, and others have shown that you can get extremely strong performance without requiring the same amount of money and infrastructure people once assumed was necessary.</p><p>And that changes the entire market.</p><p>If one company can release a model that is close enough to a much more expensive proprietary model at a fraction of the cost, everyone else has to respond. OpenAI, Anthropic, Google, and everyone else cannot just ignore it. They have to make their own systems cheaper, more efficient, or significantly better.</p><p>That is why these models matter so much. Even if they are not always the absolute best model available, they put massive downward pressure on pricing. “Almost as good for dramatically cheaper” is one of the strongest competitive forces you can have.</p><p>The result is simple. AI companies are being forced to do more with less, and that makes subsidized plans easier to sustain over time.</p><h3>The GPUs Keep Getting Better</h3><p>Then there is hardware. NVIDIA is not just selling more GPUs, each generation is also getting significantly better for AI workloads. More compute, more memory bandwidth, better low precision support, better interconnects, and much higher inference throughput.</p><p>But it is not just the raw GPU either. The software around it is improving just as quickly. FP8, FP4, quantization, speculative decoding, better batching, KV cache optimizations, optimized kernels, and better request scheduling all mean that companies can squeeze more tokens out of the same hardware.</p><p>This is where the cost reductions really start stacking up. Better GPUs lower the cost. Better serving software lowers the cost again. Better model architectures lower it again. You do not need one massive breakthrough. You just need a dozen smaller improvements happening at once, and that is exactly what is happening.</p><p>So when people look at how expensive inference is today and assume that price will stay fixed forever, they are looking at a moving target.</p><h3>The Industry Is Building for This Demand</h3><p>There is also the fact that everyone knows AI demand is exploding. The industry is not sitting around hoping current datacenters will somehow be enough. Companies are spending enormous amounts of money building new datacenters, buying GPUs, expanding power capacity, improving networking, and building infrastructure specifically for AI.</p><p>Right now, demand is growing faster than supply. That is why these shortages keep happening.</p><p>But that does not mean the shortage is permanent.</p><p>If there is enough demand to justify building massive amounts of new compute, companies will keep building it. This is exactly what is happening right now. The industry is throwing capital at the problem because the demand clearly exists.</p><p>Over time, that means the supply of AI compute goes up dramatically. More capacity plus better hardware plus more efficient inference means the cost per user keeps moving down.</p><h3>Subscription Economics</h3><p>The other thing people forget is that not every subscriber uses anywhere near their maximum allowance. A plan can theoretically offer a ridiculous amount of usage without every single person actually consuming it.</p><p>This is how basically every high-limit subscription works.</p><p>Some people barely use it. Some people use it normally. A small group absolutely abuses it. The company prices around the average.</p><p>That means a power user might consume far more value than they pay for, while a lighter user consumes far less. As long as the average cost across the whole subscriber base works out, the plan works.</p><p>There is also a huge difference between API price and actual cost. If someone uses what would be $10,000 worth of API tokens in a month, that does not mean the company spent $10,000 serving them. API pricing includes margins, infrastructure costs, convenience, burst capacity, and a bunch of other factors. The internal cost is much lower.</p><p>So when a subscription gives someone thousands of dollars worth of theoretical API usage, the subsidy is not actually as insane as it first looks.</p><h3>The $200 OpenAI subscription</h3><p>OpenAI just blocked the $200 plan. No new subscriptions until they unblock it, and why? Because they, the AI company with the most compute, don’t have enough of it.</p><p>This is OpenAI’s Pro plan, or at least the other variant of it. The default variant is the $100 plan: 5x more usage than Plus with practically unlimited chat, but it also costs 5x. The other, more expensive plan, the $200 plan, offers 4x usage compared to the $100 plan at only 2x the price. This is ridiculously good offering up to $14,000 in tokens every month for a fixed rate of $200.</p><p>Now, you might say that this is a sign that subsidization will go away, but really, it isn’t. OpenAI pulled a similar move in 2023, temporarily disabling new Plus subscriptions because of lack of compute. Although they almost have 10x the compute now, the models are bigger, their user base is bigger, and the usage is bigger. So why do I think that this is irrelevant. Well, because Anthropic does the same thing with way less compute and has no problems running Fable 5.1. Yes yes, they do only limit to 50% of your weekly limit, but think about. The company has way less compute and resources than OpenAI. The only problem here is that OpenAI has to deal with more users and they were previously unprepared for such a surge. As they get more compute, these issues will be disappear and the subscriptions will remain.</p><h3>Anthropic Is the Proof</h3><p>Anthropic matters because they are profitable.</p><p>That is really the entire point.</p><p>If subsidized AI access was fundamentally unsustainable, then a company offering it should not be able to make money. But Anthropic does. They can offer huge amounts of expensive model usage at a fixed monthly price and still operate profitably.</p><p>And this is before all of the future cost reductions we just talked about.</p><p>Before the next generations of GPUs. Before all of the new datacenters come online. Before inference gets even more optimized. Before open-source competition pushes prices down further.</p><p>That is why Anthropic is such a strong example. They are showing that subsidization does not require burning money forever. It can exist inside an actual profitable business.</p><h3>The Important Difference</h3><p>There is also a difference between a capacity problem and a business model problem.</p><p>OpenAI blocking new subscriptions is a capacity problem. They do not have enough compute right now for the number of users they have and the amount those users are consuming.</p><p>That does not mean the plan itself is broken.</p><p>If a restaurant has a line out the door every night, you do not conclude that selling food is unsustainable. You conclude that they need more tables.</p><p>That is basically what is happening here. OpenAI has huge demand, and their infrastructure has not caught up yet. As they add more compute, improve serving efficiency, and deploy better hardware, that problem becomes easier to handle.</p><p>The pricing model does not suddenly stop working just because demand temporarily outruns supply.</p><h3>Where This Goes</h3><p>The long-term direction here is pretty obvious. Chinese open-source models keep pushing prices down. NVIDIA keeps making better GPUs. AI companies keep improving inference. Datacenters keep getting built. The supply of compute goes up and the cost per unit of intelligence goes down.</p><p>At the same time, companies have every reason to keep these plans around because they encourage people to use the product more. A fixed price makes people stop thinking about every request individually and just use the AI whenever they need it.</p><p>That is incredibly valuable if your goal is to make your product something people use for everything.</p><p>AI subsidization is not some temporary mistake that only works while companies burn money. Anthropic already proves that it can coexist with profitability, and every major trend in the industry is making the economics better, not worse. The exact prices and limits will change, but the basic idea of getting far more AI usage than the raw API equivalent suggests is probably here to stay.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3dfde3297ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Stopped Working Locally, You Should Too]]></title>
            <link>https://medium.com/@ekansh.jain2011/i-stopped-working-locally-you-should-too-645bbc68fa29?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/645bbc68fa29</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Fri, 04 Sep 2026 01:48:36 GMT</pubDate>
            <atom:updated>2026-09-04T01:48:36.592Z</atom:updated>
            <content:encoded><![CDATA[<p>I always used to work on my local Linux environment, Ubuntu 26.04 as of now. Now, I use Docker.</p><h3>Working Locally, Why I Switched</h3><p>Working locally is great. No need to reinstall packages, no bloat, no duplication, because it’s all shared. That worked fine, until Hermes Agent. Hermes Agent Desktop specifically. It needed a different version of npm, but I couldn’t install it because one of my applications needed the current version. Ok, let’s just use nvm. Well, now things like the desktop app shortcut need to be modified. Every time I launch hermes or my app, I need to first use nvm and set the right version for npm — a pain that I don’t want to deal with.</p><p>Another example: I’ve been working a lot recently with C++26 Reflection code, and there is only one compiler that can help me here, GCC 16. But more specifically, it was GCC 16.1 which wasn’t available on Ubuntu, even via the PPA. So, I built it from source and got the compiler working. The code only worked with 16.1 but the CMake presets all said GCC 16 because in my environment, I decided to call it gcc-16.1 to keep my 16 install. However, 16.1 obviously hadn’t fixed all bugs — especially in the case of reflection. Then 16.2 comes around with way less bugs, enough to get way more out of my code. Now, I could spend 8+ hours pulling the repo, setting it up, building it, etc., but this definitely isn’t feasible when I want others to contribute. I’d also rather use my laptop instead of having it sit there compile GCC and be unusable for a whole day. The solution: Docker images for GCC which were up to date and worked perfectly on any device. And I can’t expect developers with weaker devices to help if I myself don’t want to build from source.</p><h3>The Solution: Dev containers</h3><p>Dev containers are the perfect solution. I specify the VSCode extensions, the docker image that runs the workspace, everything in my control. No version conflicts between one repository and the other, it all works brilliantly. The price to pay in relation was small for me. Up to 25GB disk space for a single image, but that single image could be reused for different containers and different projects that had the same needs.</p><p>Now, another thing to consider is filesystem overhead, file sharing, and executable overhead. When I first got into dev containers, I hated them. I created a dev container in VSCode, wrote code, and the only way to see it was inside the dev container. Couldn’t access it easily, had to wait a few minutes for boot up, and it was a whole process. Well, the solution to this, as I have figured out, is just to create the .devcontainer and its files inside an existing repository. It could be dummy initialized, but what matters is that you control the path so you no longer have to work harder to find your files in your filesystem.</p><p>The next big thing was performance and overhead. Well, I’ve switched to Linux. On Linux, this overhead is near 0. Negligible overhead for a whole lot less pains, it’s amazing. But I’m not here to say that “Oh, the overhead is small and doesn’t make an impact” because that’s false. On Windows, it matters. Docker on Windows is SLOW because it requires emulation of Linux and sharing a kernel just isn’t possible for most images. There’s filesystem overhead, execution overhead, and your development experience ends up much slower. Well, what about WSL2? There’s still 2 layers of overhead, and WSL doesn’t make it much easier to do regular tasks with your files since Windows can’t natively access them without massive performance costs. So while dev containers are great, they almost need Linux to be successful, but as a serious developer, you already need Linux to begin with.</p><h3>One Final Note: Don’t Docker Everything</h3><p>I mentioned two big things that I though were problems but weren’t. Well, I’d like to mention a few things that actually are problems that are not too hard to solve but annoying at the least. I mentioned above that all of this began with Hermes, and so I put it in docker. But I encountered so many problems that I eventually just reverted. Git SSH forwarding is not the easiest and getting audio to work just didn’t happen. As for the Desktop app, it isn’t a standalone install on Linux meaning that to use it, I can’t use a remote gateway very easily. So my conclusion: code in dev containers, deploy (what you can) in dev containers, but some stuff can’t be dockered.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=645bbc68fa29" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Alibaba Prefired for NVIDIA’s RTX Spark]]></title>
            <link>https://medium.com/@ekansh.jain2011/alibaba-prefired-for-nvidias-rtx-spark-aa8bd183ed81?source=rss-940d6d200e8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa8bd183ed81</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[local]]></category>
            <category><![CDATA[qwen]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ekansh Jain]]></dc:creator>
            <pubDate>Tue, 25 Aug 2026 23:47:23 GMT</pubDate>
            <atom:updated>2026-08-25T23:47:23.553Z</atom:updated>
            <content:encoded><![CDATA[<p>NVIDIA’s RTX Spark releases this fall, within the next two months (precise date unconfirmed), and Alibaba is about to release the perfect model for it.</p><h3>The RTX Spark (And Friends)</h3><p>The RTX Spark is an Arm chip that supports <strong>up to</strong> (there are official, less capable versions)<strong> </strong>128 GB Unified LPDDR5, 20 core CPU, and 1 Petaflop AI performance with its RTX 5070-like core count. However, it has a big limitation, a maximum of 300 GB/s memory bandwidth. This makes it unusable for large, dense AI models, drastically reducing performance. It is a new, Windows capable version of NVIDIA’s DGX Spark.</p><p>However, the RTX Spark isn’t the first of its class, it’s the latest. The first were Apple’s M* Studios. However, that isn’t ideal for most developers that thrive in the CUDA ecosystem or for model training. Then, NVIDIA released the DGX Spark — a direct competitor with the exact same specs as this RTX Spark has, but it has slightly lower bandwidth at 273 GB/s. Next, and the most recently released, comes AMD’s Strix Halo, AMD’s x86 competitor to the DGX Spark with slower prefill and other specs being the same at a cheaper price.</p><p>These desktops all share one thing in common: Their memory bandwidth is a bottleneck. Although Apple is much better at maintaining a decent bandwidth, doubling that of the other NVIDIA and AMD chips, it is also much more expensive. The RTX Spark is also the first to fit into a laptop, without decreasing the specs!</p><h3>MoE LLMs</h3><p>The direct solution to this lack of memory bandwidth is MoE LLMs. Instead of running dense models with massive requirements for bandwidth, small MoE’s such as Qwen 3.6 35B-A3B and Qwen 3.5 122B-A10B that activate way less parameters but require much more RAM. Perfect fit, right? Well, 10B is still quite a lot. For a long time, Alibaba’s Qwen series of models has been the leading, shining star of open-weight models with the best intelligence. In fact, their recent 27B is at GPT 5.6 Luna level, surpassing Opus 4.6 and one point behind Deepseek Pro. Today, they just released the next level. Made for these chips, made for fast inference but high capability: Qwen 3.8 Flash Next.</p><h4>Qwen 3.8 Flash Next</h4><p>Contrary to the name, this model is a preview of Alibaba’s next generation of Qwen: Qwen 4. At 125 billion parameters + 51 billion N-gram embedding parameters but only 6B active, it is the perfect fit. At Q4, that’s roughly 83GB of RAM but only a very small portion of it is even required to generate a token. Moreover, for smaller RAM amounts, you can possibly offload the N-gram to SSD — due to how little parameters are required — and do other work on the system. Launching tomorrow with potential Day 0 support in llama.cpp and Unsloth, it marks a new step for local AI.</p><p>It fits perfectly, and on the RTX Spark, it will likely be the main driver of local agents. Likely to be extremely capable and close to the 27B or potentially surpassing it, it is the perfect match for anyone with these systems. Much faster inference, more knowledge, near 100% resource utilization, and the perfect power house.</p><p>Releases tomorrow on Hugging Face: <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next">https://huggingface.co/Qwen/Qwen3.8-Flash-Next</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa8bd183ed81" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>