<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Papers in 100 Lines of Code on Medium]]></title>
        <description><![CDATA[Stories by Papers in 100 Lines of Code on Medium]]></description>
        <link>https://medium.com/@papers-100-lines?source=rss-df93cc66596f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HFmaIJiZ7BehRryMgLDvxQ.jpeg</url>
            <title>Stories by Papers in 100 Lines of Code on Medium</title>
            <link>https://medium.com/@papers-100-lines?source=rss-df93cc66596f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 22:50:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@papers-100-lines/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why Qwen Mixes Gated DeltaNet and Attention at 3:1]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/why-qwen-mixes-gated-deltanet-and-attention-at-3-1-376860d2540b?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*Gy9HvS7v_SDWe75Fs_wixQ.png" width="1672"></a></p><p class="medium-feed-snippet">Inside Qwen&#x2019;s 3:1 hybrid architecture: why recurrent memory makes language models more efficient, but full attention remains essential for&#x2026;</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/why-qwen-mixes-gated-deltanet-and-attention-at-3-1-376860d2540b?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/why-qwen-mixes-gated-deltanet-and-attention-at-3-1-376860d2540b?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/376860d2540b</guid>
            <category><![CDATA[qwen]]></category>
            <category><![CDATA[gated-deltanet]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[attention-mechanism]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Tue, 20 Oct 2026 00:00:08 GMT</pubDate>
            <atom:updated>2026-10-24T18:01:01.567Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Qwen3.8–27B vs Claude Opus 4.6: How a 27B Model Competes]]></title>
            <link>https://papers-100-lines.medium.com/qwen3-8-27b-vs-claude-opus-4-6-how-a-27b-model-competes-078693ecb219?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/078693ecb219</guid>
            <category><![CDATA[qwen]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[claude]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 00:00:34 GMT</pubDate>
            <atom:updated>2026-10-14T18:01:01.536Z</atom:updated>
            <content:encoded><![CDATA[<h4>A single-GPU open model outscores Opus 4.6 on agentic coding and computer use</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rWDT1vy0eYdVXf8Wmx2sRQ.png" /><figcaption>Qwen3.8–27B vs Claude Opus 4.6 | Tutorial</figcaption></figure><p>In August 2026, Alibaba released Qwen3.8–27B, and the company’s benchmarks put the open model ahead of Claude Opus 4.6 Max at agentic coding (SWE-bench Pro, 61.7 to 53.4) and computer use (OSWorld-Verified, 84.3 to 72.7).</p><p>What makes this remarkable is its size. Most of today’s largest open models are mixture-of-experts (MoE) models: their feed-forward blocks are split into many “experts”, and a router sends each token to only a few of them. This lets a model hold hundreds of billions of parameters, sometimes more than a trillion, while each token uses only a small fraction of them. But all those parameters must stay in GPU memory, so serving one of these models usually takes several GPUs. Qwen3.8–27B is dense: every token uses all 27 billion of its parameters, and its weights fit on a single 80 GB GPU. So how does a 27B model, small by today’s standards, keep up with Claude Opus?</p><p>Part of the answer is architecture. Qwen redesigned its attention layers so the model can handle very long inputs cheaply, and these redesigned layers hold 27% of its parameters. The other part is training. Four months earlier, Qwen3.6–27B shipped with exactly the same architecture and scored far lower on the agent benchmarks.</p><h3>Where Qwen3.8–27B beats Opus 4.6 Max, and where it doesn’t</h3><p>Three of these benchmarks test agentic capabilities: the model works through long sessions of tool calls, code edits and screenshots. Qwen wins two of them, SWE-bench Pro and OSWorld-Verified, and beats Opus by 17 points on instruction following, a skill every agent depends on. Opus keeps the edge on the third, Terminal-Bench, and on the knowledge-heavy tests.</p><h3>Inside the 27B: 73% standard, 27% redesigned</h3><p>Most of Qwen3.8–27B follows the standard decoder recipe that nearly every open model builds on: RMSNorm and a SwiGLU feed-forward block in each of its 64 layers, with rotary position embeddings in its attention layers. The feed-forward blocks alone hold 63.6% of its 26.9 billion language-model parameters, and the embedding table and output head hold another 9.5%.</p><p>The remaining 27% is where Qwen breaks from the recipe. It sits in the token mixer, the part of each layer where a token reads from the tokens before it:</p><ul><li><strong>Gated DeltaNet in 48 layers (20.7% of parameters).</strong> Instead of looking back at every earlier token, it compresses everything it has read into a fixed-size memory. A learned decay lets that memory fade old information, and the delta rule updates the value stored under each key rather than piling new information on top. If that sounds like an LSTM, it is a close cousin: the decay plays the role of the forget gate. But the update is linear, so training can process a sequence in parallel chunks instead of one token at a time.</li><li><strong>Gated attention in the other 16 layers (6.2%).</strong> This is standard softmax attention with a sigmoid gate on each head’s output, so a head can stay almost silent when it has nothing to add. The gate costs 0.5 billion parameters, and in Qwen’s own paper it made training more stable and improved results on contexts longer than those seen in training. It also almost eliminated the “attention sink”, where heads park their attention on the first token.</li></ul><blockquote><strong>Don’t just read about Qwen3.8‑27B. Build it.</strong></blockquote><blockquote>Most engineers only call models like this through an API. In my course, <em>Implement Qwen3.8‑27B From Scratch in PyTorch</em>, you write every layer yourself, Gated DeltaNet and gated attention included, in under 1,000 lines of plain PyTorch. Then you load the official 27B weights into your own code and chat with the model. No toy version: the real model that beats Opus 4.6 Max at agentic coding.</blockquote><blockquote><a href="https://buildfrontierllms.com/?utm_source=blog&amp;utm_medium=organic&amp;utm_campaign=qwen_blog&amp;utm_content=qwen3-8-27b-vs-opus-mid">See the course →</a></blockquote><blockquote>If you want to test it first, Part 1 is a free 19-minute preview: <a href="https://buildfrontierllms.com/preview/?v=h1&amp;utm_source=blog&amp;utm_medium=organic&amp;utm_campaign=qwen_blog&amp;utm_content=qwen3-8-27b-vs-opus-mid-preview">get it here</a>.</blockquote><h3>Why Qwen redesigned the attention mechanism</h3><p>Qwen introduced this hybrid design in 2025 with Qwen3-Next, to handle very long contexts efficiently. Long sequences are exactly where standard attention gets expensive: to produce each new token, an attention layer compares it with every token before it, so each token costs more than the last. A feed-forward block, by contrast, does the same work at token 10 as at token 100,000.</p><p>In Qwen3.8–27B, this adds up quickly: at about 21,800 tokens of context, comparing a new token with all the earlier ones already takes as much compute as the feed-forward block. At the full 262,144-token window, it takes twelve times as much. A Gated DeltaNet layer, with its fixed-size memory, costs the same at any length.</p><p>Attention also takes more memory as the conversation grows: every attention layer stores the keys and values of all earlier tokens, in what is called the KV cache. By keeping attention in only 16 of its 64 layers, Qwen makes that cache four times smaller: 16 GiB instead of 64 GiB for one full-length conversation.</p><h3>Why agents need cheap long contexts</h3><p>An agent’s context fills up fast. Every file it opens, every command it runs and every screenshot it takes stays in the context, and the model reads all of it again at every step. That is exactly the workload Qwen3.8–27B’s hybrid design keeps affordable.</p><p>Cheap long contexts pay off at deployment, where agent sessions run long, and probably in training too, where reinforcement learning on agent tasks generates huge numbers of such sessions. But cheap context doesn’t make a model smarter. Training does.</p><h3>What training added: Qwen3.6–27B vs Qwen3.8–27B</h3><p>Qwen3.6–27B came out in April 2026 with exactly the same architecture as Qwen3.8–27B: the same 64 layers, the same three-to-one mix of Gated DeltaNet and attention layers, the same layer sizes. Alibaba’s model card benchmarks both under identical settings:</p><p>In four months, Qwen gained 20.4 points on OSWorld-Verified, 10.4 on IFBench, 9.6 on Terminal-Bench, 8.2 on SWE-bench Pro and 6.8 on Humanity’s Last Exam. On SWE-bench Pro, a dead heat with Opus became an eight-point lead; on OSWorld-Verified, a nine-point deficit became a lead of almost twelve. Only the weights changed, plus one chat-template default that keeps earlier reasoning in the context. So nearly all of these gains come from training: better data, supervised fine-tuning, reinforcement learning, or all three.</p><p>None of this makes the architecture secondary: it is what keeps long agent sessions cheap to run. Training then decides how well the model uses all that context.</p><h3>Where Opus still leads</h3><p>Opus still leads on coding in the terminal and on expert knowledge, but Qwen is closing the gap. In four months, it cut Opus’s lead on terminal coding from 15 points to 5, and on expert questions across fields from 16 points to 9. On graduate-level science, where every model already scores around 90%, it edged closer.</p><p>And remember: Qwen3.8–27B has 27 billion parameters and fits on a single GPU. Anthropic doesn’t disclose the size of Opus, but it very likely has close to a trillion parameters, or more. So a model dozens of times smaller beats a frontier giant at agentic coding and computer use. Size also explains what remains of the knowledge gap: a model stores its factual knowledge in its parameters, much of it in the feed-forward blocks, so a 27B model simply has less room for it.</p><h3>Architecture, size, training: what each one explains</h3><p>Architecture makes long contexts cheap, and agents live in long contexts. Size may still limit how much a 27B model can know. But training turned the same architecture, in just four months, into a model that beats Opus 4.6 Max at agentic coding and computer use.</p><p>Alibaba keeps its training recipe private, but the architecture is fully public and hasn’t changed since Qwen3.5–27B in February 2026. Learn it once, and you understand three releases.</p><blockquote><strong>Stop using frontier models as black boxes. Build one.</strong></blockquote><blockquote><em>Implement Qwen3.8‑27B From Scratch in PyTorch</em> takes you from LLM fundamentals to the full 64-layer model running on Alibaba’s official weights, with outputs that match the reference implementation. Then you rebuild DeepSeek 7B and 67B from the same blocks: proof that what you learn isn’t tied to one model.</blockquote><blockquote><a href="https://buildfrontierllms.com/?utm_source=blog&amp;utm_medium=organic&amp;utm_campaign=qwen_blog&amp;utm_content=qwen3-8-27b-vs-opus-end">See the course →</a></blockquote><blockquote>If you want to test it first, Part 1 is a free 19-minute preview: <a href="https://buildfrontierllms.com/preview/?v=h1&amp;utm_source=blog&amp;utm_medium=organic&amp;utm_campaign=qwen_blog&amp;utm_content=qwen3-8-27b-vs-opus-end-preview">get it here</a>.</blockquote><blockquote><strong>Consulting</strong></blockquote><blockquote>Weighing Qwen3.8–27B against a closed model? At Qubit Analytics, we deploy open models such as Qwen and DeepSeek, but our work goes deeper than integration: we work inside the model itself. We read the papers, reimplement architectures from scratch, adapt them to your problem and turn recent research into working prototypes.</blockquote><blockquote>Email <a href="/service/mailto:contact@qubitanalytics.be?subject=Consulting%20inquiry%20(blog%3A%20qwen3-8-27b-vs-opus)">contact@qubitanalytics.be</a> with what you’re building.</blockquote><h3>Sources</h3><ul><li>Qwen team, <a href="https://huggingface.co/Qwen/Qwen3.8-27B">Qwen3.8–27B model card</a> (2026).</li><li>Qwen team, <a href="https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/config.json">Qwen3.8–27B configuration file</a> (2026).</li><li>Qwen team, <a href="https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/config.json">Qwen3.6–27B configuration file</a> (2026).</li><li>Qwen team, <a href="https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/config.json">Qwen3.5–27B configuration file</a> (2026).</li><li>Qwen team, <a href="https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct">Qwen3-Next-80B-A3B-Instruct model card</a> (2025).</li><li>Songlin Yang, Jan Kautz and Ali Hatamizadeh, <a href="https://arxiv.org/abs/2412.06464">“Gated Delta Networks: Improving Mamba2 with Delta Rule”</a> (2024).</li><li>Zihan Qiu et al., <a href="https://arxiv.org/abs/2505.06708">“Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free”</a> (2025).</li></ul><p><em>Originally published at </em><a href="https://buildfrontierllms.com/blog/qwen3-8-27b-vs-opus/"><em>https://buildfrontierllms.com</em></a><em> on October 14, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=078693ecb219" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Llama, DeepSeek, Qwen and GPT-oss Look Almost Identical Inside]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/why-llama-deepseek-qwen-and-gpt-oss-look-almost-identical-inside-3f301d4eadaa?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*gAFcRgAFVD4uzTlptAT_Bw.png" width="1672"></a></p><p class="medium-feed-snippet">Four blocks from 2017&#x2013;2021 still power every major open model. Better data and compute changed, the architecture barely did.</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/why-llama-deepseek-qwen-and-gpt-oss-look-almost-identical-inside-3f301d4eadaa?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/why-llama-deepseek-qwen-and-gpt-oss-look-almost-identical-inside-3f301d4eadaa?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f301d4eadaa</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[qwen]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 00:00:15 GMT</pubDate>
            <atom:updated>2026-10-20T18:01:01.605Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[AnySplat Turns 32 Unposed Photos Into a 3D Scene in 1.4 Seconds]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/anysplat-turns-32-unposed-photos-into-a-3d-scene-in-1-4-seconds-732da3619c13?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*o39-MA72Zp6tYCd1fmyYOw.png" width="1672"></a></p><p class="medium-feed-snippet">Built on VGGT and trained without 3D ground truth, it predicts cameras, depth, and millions of 3D Gaussians in one forward pass &#x2014; and&#x2026;</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/anysplat-turns-32-unposed-photos-into-a-3d-scene-in-1-4-seconds-732da3619c13?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/anysplat-turns-32-unposed-photos-into-a-3d-scene-in-1-4-seconds-732da3619c13?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/732da3619c13</guid>
            <category><![CDATA[neural-rendering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 18:01:02 GMT</pubDate>
            <atom:updated>2026-09-21T18:01:02.152Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[DUSt3R Made 3D Reconstruction Work Without Camera Poses. MASt3R Made It Accurate.]]></title>
            <link>https://papers-100-lines.medium.com/dust3r-made-3d-reconstruction-work-without-camera-poses-mast3r-made-it-accurate-1c54a7d5b7f2?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c54a7d5b7f2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[multi-view-stereo]]></category>
            <category><![CDATA[depth-estimation]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Fri, 18 Sep 2026 17:16:01 GMT</pubDate>
            <atom:updated>2026-09-18T17:16:01.292Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Two unposed images in, dense 3D geometry out: how DUSt3R turned cameras into an output — and MASt3R added precise matching and metric scale.</strong></p><p>In <a href="https://www.3dgaussiansplattingcourse.com/pixelsplat-vs-mvsplat.html">a recent post</a> we looked at pixelSplat and MVSplat: feed-forward models that turn two photographs into a 3D Gaussian scene in a fraction of a second. Both share one requirement — the camera parameters of the input images must be known: positions, orientations, focal lengths. In practice they come from structure-from-motion — COLMAP, in most pipelines — which matches the images and then repeatedly runs an expensive optimization called bundle adjustment. That process takes minutes to hours, needs many overlapping images, and fails outright on scenes with little texture or few views. The “single forward pass” only begins after it has succeeded.</p><p><strong>DUSt3R</strong> (CVPR 2024, NAVER LABS Europe) removed that requirement. It takes two photographs with no camera information at all and directly regresses the 3D structure they show; the cameras are then estimated <em>from</em> that structure, by small optimizations that take seconds. Camera parameters stopped being an input to reconstruction; they are read off its result. <strong>MASt3R</strong> (ECCV 2024, an oral, from the same lab) kept the architecture and fixed its two weaknesses, adding precise image matching and metric scale. Together they are the foundation under nearly every pose-free method in <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html">feed-forward 3D Gaussian Splatting</a>. This post explains how they work, what they measurably changed, and where their limits are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ohcBetSJKlIUU5BiX0NCwQ.png" /><figcaption>DUSt3R and MASt3R Explained: 3D Reconstruction Without Camera Poses</figcaption></figure><h3>A 3D point cloud, one point per pixel</h3><p>Everything in DUSt3R follows from its output representation. For each input image, the network predicts a <strong>pointmap</strong>: for every pixel, the 3D coordinates of the surface point that pixel observes. An image of width W and height H becomes a W×H×3 array — a dense point cloud with one point per pixel, in one-to-one correspondence with the image.</p><p>The decisive design choice is the coordinate frame. Both pointmaps — the first image’s and the second’s — are expressed in the coordinate frame of the <em>first camera</em>. The network therefore does not just estimate each view’s geometry; it places both views’ geometry in a single shared 3D space. Where the two images see the same surface, their pointmaps must produce the same 3D points. The relative pose between the cameras is never predicted anywhere, yet it is implicitly contained in the output, because view two’s geometry is expressed in view one’s frame.</p><p>Alongside each pointmap, the network predicts a per-pixel confidence. Some pixels have no well-defined 3D point — the sky, or translucent surfaces — and the confidence lets the model say so instead of being forced to guess. Notice also what the network does <em>not</em> contain: no camera model. Classical reconstruction bakes the pinhole projection equations into every step; DUSt3R’s authors describe their formulation as relaxing the hard constraints of the usual projective camera models. The network just maps pixels to 3D points, and geometry is imposed only later, when cameras are extracted. Depth, for instance, is nothing special here — it is simply the z-coordinate of each point. Even the single-image case needs no separate treatment: feed the same photograph in twice, and the pairwise model becomes a monocular one.</p><h3>The architecture and the loss</h3><p>The architecture is deliberately plain. A ViT-Large encoder processes both images with shared weights. Two ViT-Base decoders, one per view, then exchange information at every block through cross-attention: each view’s tokens attend to the other view’s tokens, so each image is interpreted in the context of the other — which is what makes the shared coordinate frame learnable at all. A DPT head on each branch regresses the pointmap and its confidence. The network is initialized from CroCo pretraining, a self-supervised task in which a heavily masked image must be reconstructed with the help of a second view of the same scene — practice, in effect, for transferring geometry between viewpoints. The published model is trained on a mixture of eight datasets — indoor, outdoor, synthetic, driving — totalling 8.5 million image pairs, first at 224×224 resolution and then at 512 pixels.</p><p>The loss is a regression in 3D: the distance between each predicted point and its ground-truth counterpart. Two details matter. First, scale. The training scenes come with arbitrary scale — the same ambiguity <a href="https://www.3dgaussiansplattingcourse.com/pixelsplat-vs-mvsplat.html">pixelSplat had to engineer around</a> — so prediction and ground truth are each normalized by their own average point distance before comparison. The normalizer is computed over both views of the pair jointly, not per view: normalizing each view separately would destroy the relative scale between the two views, which is exactly the information the shared frame is supposed to carry. Second, confidence. Each pixel’s regression error is weighted by the predicted confidence, minus a penalty on low confidence, so the model can down-weight pixels whose geometry is unknowable but pays a price for doing so — it cannot simply declare everything uncertain.</p><h3>Cameras become an output</h3><p>Nothing about the cameras is predicted, and everything about them is recoverable. The focal length follows from the first view’s pointmap alone: assuming the principal point is near the image centre, the focal length that best projects the predicted 3D points back onto their own pixels is found by a fast iterative solver, in a few iterations. The relative pose between the cameras follows from comparing the two predictions of the shared geometry — either by directly aligning point clouds (Procrustes alignment), or, more robustly, by solving a standard pose problem with RANSAC on correspondences. Correspondences themselves come almost for free: two pixels match if their predicted 3D points are nearest neighbours in the shared space. One network output, and matching, depth estimation, focal estimation and pose estimation all reduce to small computations on top of it.</p><h3>More than two images</h3><p>The network only ever sees two images at a time. Reconstructing a collection works by running it on every useful pair — pairs of images that share visual content, organized as a graph — and then merging all the pairwise predictions. The merging step optimizes, for each image, a pointmap in a single world frame, together with one rigid transformation and one scale factor per pair, so that each pairwise prediction agrees with the world-frame geometry.</p><p>It is tempting to call this bundle adjustment, and the paper is explicit that it is not. Bundle adjustment minimizes 2D reprojection errors through the camera model; DUSt3R’s global alignment minimizes 3D distances between pointmaps directly. That makes it a much easier optimization problem: plain gradient descent converges in a few hundred steps, taking seconds on a GPU where structure-from-motion takes minutes to hours. The caveat is that this is still an optimization, not a forward pass — and the number of pairs grows quadratically with the collection. Both points define the follow-up research: NAVER’s own MUSt3R and Meta’s Fast3R rebuilt the network to ingest many views at once, and VGGT eventually removed the post-optimization entirely.</p><blockquote><strong>Want more posts like this?</strong><br> Subscribe to <a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">my newsletter</a> for future posts, updates, and practical guides on 3DGS, PyTorch, differentiable rendering, and recent splatting research.</blockquote><h3>What DUSt3R could do — and what it could not</h3><p>The results were remarkable for a single set of weights evaluated across tasks it was never tuned for. On multi-view pose estimation, the field’s standing benchmark numbers moved by a margin rarely seen in one paper: on CO3D, the fraction of image pairs with rotation error under 15 degrees rose from 80.5% for the best prior learned method, PoseDiffusion, to 96.2%. As a monocular depth estimator — again, the same weights, with the image simply fed in twice — it comes close behind supervised specialists trained for that single task (an absolute relative error of 6.50 versus 5.40 for DPT on NYUv2). On the DTU multi-view stereo benchmark it reaches 1.741 mm of overall error <em>zero-shot</em>, without ever training on DTU.</p><p>That last number also marks the boundary. Classical pipelines that triangulate with known cameras reach 0.578 mm on DTU, three times better, and the paper says plainly why: regression is less accurate than sub-pixel triangulation. Two further limits matter. The reconstruction is only defined up to an unknown global scale — DUSt3R cannot tell a large room from a small model of one. And while nearest-neighbour matching in pointmap space is strikingly robust to extreme viewpoint changes, it is not precise: the pointmaps are smooth regressed fields, and matches inherit their blur. Precision and scale are exactly what MASt3R adds.</p><h3>MASt3R: matching, grounded in 3D</h3><p>MASt3R starts from an observation about image matching, the task of finding which pixel in one image corresponds to which pixel in another — the raw material of localization and mapping systems. Matching had always been treated as a 2D problem, yet it is fundamentally a 3D one: two pixels correspond because they observe the same point of a 3D scene. So instead of building a better 2D matcher, MASt3R grafts matching onto DUSt3R’s 3D reconstruction — the paper’s title, “Grounding Image Matching in 3D,” is a literal description.</p><p>Concretely, MASt3R keeps the entire DUSt3R architecture and adds a second head that outputs a dense <strong>local feature map</strong>: a 24-dimensional descriptor per pixel, trained with a contrastive loss (InfoNCE) so that descriptors of truly corresponding pixels — known from the ground-truth geometry — match each other and nothing else. The pointmaps provide robustness, deciding roughly where in the scene a pixel belongs; the descriptors provide the precision the pointmaps lack.</p><p>The second change is a small modification of the loss that buys metric scale. DUSt3R normalizes the prediction by its own scale, so the network is never asked to know true size. MASt3R, whenever a training scene has metric ground truth, normalizes the prediction by the <em>ground truth’s</em> scale instead — and the only way to make that loss small is to predict distances at their true metric values. The released model predicts metric-scale pointmaps.</p><p>The third change makes dense matching affordable. Finding all mutual nearest-neighbour matches between two feature maps naively compares every pixel with every pixel — quadratic in image size, prohibitively slow at high resolution. MASt3R’s fast reciprocal matching starts from a sparse grid of seed pixels and repeatedly maps each one to its best match in the other image, then maps that match back — collecting, along the way, the pairs that map onto each other, which are kept as correspondences. This drops the complexity to linear in image size, and the paper reports a finding the authors themselves flag as surprising: subsampled matching is not just faster but <em>more accurate</em>, because points that survive the iterated reciprocal test are disproportionately correct matches — the procedure filters outliers as a side effect.</p><h3>What precision and scale bought</h3><p>The clearest test is a benchmark called map-free relocalization. It works like this: you are given a single photograph of a place. Someone then takes a new photograph of the same place, and the task is to estimate where that second camera was — its position in metres and its viewing direction — from the two images alone. Localization systems normally rely on a 3D map built beforehand from many photographs of the scene; here there is no map, and a single pair of images contains nothing that reveals the scene’s true size. The task therefore demands exactly the two things MASt3R added: matching that survives large viewpoint changes, and metric-scale geometry. The numbers reflect it: the median pose error drops from roughly a metre with DUSt3R to 0.36 m and 2.2°, and on the benchmark’s headline score — an area-under-curve measure between 0 and 1 — MASt3R reaches 0.933, against 0.697 for DUSt3R and 0.634 for the best method published before them.</p><p>The DTU boundary moves too. Triangulating MASt3R’s matches — precise enough, now, to triangulate — cuts DUSt3R’s 1.741 mm to <strong>0.374 mm</strong>, within reach of the best DTU-trained specialist (0.295 mm) while still never training on DTU. And on multi-view pose estimation on RealEstate10K, MASt3R adds almost nine points on top of DUSt3R’s already large improvement. The pattern across benchmarks is consistent: DUSt3R made pose-free reconstruction work; MASt3R made it accurate.</p><h3>Why splatting people should care</h3><p>Neither paper predicts a single Gaussian, and yet they sit at the root of the pose-free branch of feed-forward 3DGS. Splatt3R made the connection first and most literally: freeze MASt3R entirely, train a small head that turns each predicted 3D point into a Gaussian, and pose-free splatting from an uncalibrated pair follows. NoPoSplat took the deeper lesson — predict all geometry directly in the first camera’s frame — and designed a Gaussian prediction network around it, trained end to end rather than bolted onto a frozen backbone, with results strong enough to beat baselines that were given the ground-truth poses; that story continues in my <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html#stage-4">overview of feed-forward 3D Gaussian Splatting</a>, in the stage where camera poses stop being required. Around the pair of papers, an ecosystem formed: MASt3R-SfM and MASt3R-SLAM rebuilt structure-from-motion and real-time SLAM on top of the model, and MUSt3R and Fast3R removed the two-view bottleneck.</p><p>The lineage also shows where the design reached its limit. <a href="https://www.3dgaussiansplattingcourse.com/vggt-explained.html">VGGT</a>, CVPR 2025’s Best Paper, positions itself against exactly the two properties flagged above: in its authors’ words, DUSt3R and MASt3R “can only process two images at once and rely on post-processing to reconstruct more images.” VGGT processes hundreds of views in one forward pass with no global alignment afterwards, and outperforms both. That does not diminish what the pair of papers did — it confirms it. The representation VGGT regresses is the pointmap, the convention it uses is the first-camera frame, and the question it answers — can cameras be an output rather than an input? — was settled by DUSt3R.</p><h3>📘 Learn 3DGS Step-by-Step (PyTorch Only)</h3><p>The methods that turn DUSt3R’s point clouds into 3D Gaussians all train the same way: render the Gaussians with a differentiable rasterizer, compare against real photographs, backpropagate. My <strong>3D Gaussian Splatting Course</strong> builds that rasterizer from first principles in <strong>PyTorch only</strong> — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.</p><p><a href="https://www.3dgaussiansplattingcourse.com/">Explore the Course →</a></p><h3>📩 Join the Newsletter</h3><p>I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.</p><p><a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">Subscribe →</a></p><h3>Consulting</h3><h3>💼 Research &amp; Engineering Consulting</h3><p>At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.</p><p>For consulting inquiries:<br><strong>contact@qubitanalytics.be</strong></p><p><em>Originally published at </em><a href="https://www.3dgaussiansplattingcourse.com/dust3r-mast3r-explained.html"><em>https://www.3dgaussiansplattingcourse.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c54a7d5b7f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MVSplat Beat pixelSplat With 10× Fewer Parameters. Here’s How.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/mvsplat-beat-pixelsplat-with-10-fewer-parameters-heres-how-52beadcfe04c?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*EQKo3vfsKKKeb4-FQKnyyQ.png" width="1672"></a></p><p class="medium-feed-snippet">Same benchmark, opposite ideas: pixelSplat learned to predict depth, while MVSplat measured it by matching views &#x2014; and won on quality&#x2026;</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/mvsplat-beat-pixelsplat-with-10-fewer-parameters-heres-how-52beadcfe04c?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/mvsplat-beat-pixelsplat-with-10-fewer-parameters-heres-how-52beadcfe04c?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/52beadcfe04c</guid>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[multi-view-stereo]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Mon, 14 Sep 2026 18:01:02 GMT</pubDate>
            <atom:updated>2026-09-14T18:01:02.095Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Apple Turns One Photo Into 1.2 Million 3D Gaussians]]></title>
            <link>https://papers-100-lines.medium.com/apple-turns-one-photo-into-1-2-million-3d-gaussians-3b8a8e23597a?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b8a8e23597a</guid>
            <category><![CDATA[neural-rendering]]></category>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 18:01:01 GMT</pubDate>
            <atom:updated>2026-09-11T18:01:01.484Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>SHARP reconstructs a metric-scale 3D scene in under a second, using a single forward pass instead of diffusion or per-scene optimization.</strong></h4><p>In December 2025, Apple released <strong>SHARP</strong>, a model that turns a single photograph into a 3D Gaussian scene in under a second. The release was widely covered in the tech press, but the articles stop at the headline. This post covers what a practitioner actually needs: how the model works, the two training ideas that make it possible, what it deliberately does not attempt, and the licensing detail most articles got wrong. SHARP was accepted at ICLR 2026, and it closes <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html">my feed-forward 3DGS timeline</a> for a reason: it is the clearest demonstration of how far single-image reconstruction has come.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GO8dfShQSFp4rdWkQ2wgsg.png" /><figcaption>Apple SHARP Explained: How One Photo Becomes a 3D Scene in a Second</figcaption></figure><h3>What SHARP does — and deliberately does not do</h3><p>SHARP takes one photograph, at up to 1536×1536 resolution, and produces roughly 1.2 million 3D Gaussians in a single forward pass — about 0.9 seconds on an A100, with the resulting scene rendering above 100 frames per second. The scene is <strong>metric</strong>: distances are in real-world units, inherited from the metric depth model it is built on, so camera movements in it correspond to physical distances.</p><p>Just as important is what it does not attempt. The paper states the design target plainly: to “support natural posture shifts in AR/VR headsets, providing the experience of looking at a stable 3D scene from different perspectives,” while it “need not support substantial travel (‘walking around’) within the photograph.” SHARP is built for the view range of a person’s head, not a walkthrough. That constraint is what buys the quality: the only content the model ever has to invent is the thin regions at object edges that appear when the viewpoint shifts slightly, and it fills those deterministically from learned priors — it is regression, not a generative model. If you want the full taxonomy of that distinction, I cover it in <a href="https://www.3dgaussiansplattingcourse.com/world-models-vs-3d-reconstruction.html">world models versus 3D reconstruction</a>.</p><h3>How it works</h3><p>SHARP stands on <strong>Depth Pro</strong>, Apple’s own monocular metric depth model, which is what makes the output metric-scale. The backbone’s patch encoder stays frozen; the depth decoder and the low-resolution part of the encoder are unfrozen and fine-tuned for the new task. On top of this sit DPT-style decoders that output, for each cell of a 768×768 grid, <strong>two Gaussians</strong> — one for the visible surface, a second that can represent what that surface occludes or how its appearance shifts with viewing angle. Each Gaussian carries fourteen numbers: a 3D position, a 3D scale, a rotation quaternion, an RGB colour and an opacity. Notably, there are no spherical harmonics — plain RGB.</p><p>Two training-time ideas deserve more attention, because both address the same fundamental obstacle: a single photograph does not determine its own depth.</p><p>The first is a <strong>depth-adjustment module that exists only during training</strong>, and the problem it solves is the scale ambiguity of monocular depth. From a single photograph, scale cannot be determined: a large room far away and a small room close up can produce the same image, so many depth maps are consistent with what the network sees. Trained with an ordinary loss, a depth network handles this by predicting a compromise — in the paper’s words, outputs “at the mean scale of possible outcomes” — a depth map that matches none of the valid answers and produces artifacts when a scene is built on it. SHARP’s solution is to settle the scale question externally during training. A small side network — a U-Net of about two million parameters — receives both the predicted depth and the ground-truth depth, and outputs a per-pixel scale factor; multiplying the prediction by this factor brings it to the same scale as the ground truth. The scale map is regularized, so it cannot simply copy the ground truth into the prediction — it can align the scale, nothing more. With the scale aligned, the training loss no longer punishes the network for choosing a different but equally valid scale; it only punishes real errors. This is only possible during training, because the side network needs the ground truth as input. At inference it is simply removed, and the model predicts depth on its own. (The construction is adapted from conditional variational autoencoders, with the regularizer playing the role of the KL term.)</p><p>The second is <strong>view-swap self-supervision</strong>, which solves a data problem: real photographs come alone, with no second viewpoint to supervise novel views against. SHARP trains first on millions of synthetic multi-view renders, where perfect supervision exists. Then, on millions of real single photographs, it plays a swap: generate a pseudo-novel view from the photo, then treat that generated view as the <em>input</em> and the original photo as the <em>novel view target</em>. The real photograph — actual ground truth — ends up supervising novel-view synthesis, without a second camera ever existing.</p><p><strong>Want more posts like this?</strong><br> Subscribe to <a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">my newsletter</a> for future posts, updates, and practical guides on 3DGS, PyTorch, differentiable rendering, and recent splatting research.</p><h3>The results, in context</h3><p>The headline numbers from the paper: against the strongest prior method, Gen3C, SHARP reduces LPIPS by 25 to 34 percent and DISTS by 21 to 43 percent across six zero-shot benchmarks — none of which it was trained on. The speed comparison is the more striking one: Gen3C is a video-diffusion-based approach that takes on the order of <em>minutes</em> per scene, and ViewCrafter likewise; SHARP produces its scene roughly three orders of magnitude faster. Against <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html#flash3d">Flash3D</a>, the closest feed-forward relative, the quality gap is wide in SHARP’s favour. Being regression rather than diffusion is precisely what makes the sub-second time possible: one pass, one answer, no sampling loop.</p><h3>Running it yourself</h3><p>The <a href="https://github.com/apple/ml-sharp">code is on GitHub</a> and the practical path is short: create a Python environment, install the requirements, and run sharp predict -i your_images -o gaussians --render. The weights download automatically on first run. Prediction runs on CUDA, on CPU, and - notably - on Apple Silicon via MPS, so a Mac can turn photos into splats; only the video rendering path requires CUDA. The output is a standard 3DGS .ply file that loads in any of the usual splat viewers.</p><p>Now the licensing detail the news coverage flattened. Most articles called SHARP “open source.” The truth has two halves: the <em>code</em> ships under Apple’s permissive sample-code license, but the <em>weights</em> ship under the Apple Machine Learning Research Model License — research use only, with commercial exploitation explicitly excluded. You can study it, benchmark it, build research on it, and write blog posts about it. You cannot ship it in a product. If your plans involve revenue, read the license file before you architect around this model.</p><h3>Where it breaks</h3><p>The authors are specific about failure modes, and they are worth knowing before you feed it your photo library. Depth errors concentrate at object boundaries and in fine structures like foliage. Photographs with strong depth-of-field or macro shots mislead the depth backbone. A starry sky comes back as a curved surface. Water reflections are read as distant geometry — the classic non-Lambertian failure that, as the timeline article notes, no method in this field has solved. And by design, views far from the input camera are out of scope: that is the headbox constraint, not a bug.</p><h3>Where it sits in the bigger story</h3><p>SHARP follows the design that came to dominate feed-forward 3DGS in 2025: a largely frozen pretrained geometry backbone, a lightweight decoder on top, and the ingenuity invested in training rather than in architecture — the same pattern as <a href="https://www.3dgaussiansplattingcourse.com/depth-anything-3-explained.html">Depth Anything 3’s frozen-backbone Gaussian head</a>. SHARP is that pattern applied to a single photograph, and executed with product-level polish. Compare it with Splatter Image, which opened the same single-image problem two years earlier on 64×64 synthetic renders, and you have the full distance the field travelled — which is exactly how <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html">the timeline article</a> tells the story.</p><h3>📘 Learn 3DGS Step-by-Step (PyTorch Only)</h3><p>SHARP’s output renders through the same differentiable rasterizer as every method on this site. My <strong>3D Gaussian Splatting Course</strong> builds that pipeline from first principles in <strong>PyTorch only</strong> — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.</p><p><a href="https://www.3dgaussiansplattingcourse.com/">Explore the Course →</a></p><h3>📩 Join the Newsletter</h3><p>I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.</p><p><a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">Subscribe →</a></p><h3>💼 Research &amp; Engineering Consulting</h3><p>At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.</p><p>For consulting inquiries:<br><strong>contact@qubitanalytics.be</strong></p><p><em>Originally published at </em><a href="https://www.3dgaussiansplattingcourse.com/apple-sharp-explained.html"><em>https://www.3dgaussiansplattingcourse.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b8a8e23597a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Do World Models Replace 3D Reconstruction?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/do-world-models-replace-3d-reconstruction-dee4939e97b1?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*9QRMVaDsaWiX3SNoVu32Sg.png" width="1672"></a></p><p class="medium-feed-snippet">Genie 3, Marble, and SHARP can all turn images into explorable worlds &#x2014; but they disagree on something fundamental: whether a scene should&#x2026;</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/do-world-models-replace-3d-reconstruction-dee4939e97b1?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/do-world-models-replace-3d-reconstruction-dee4939e97b1?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/dee4939e97b1</guid>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[world-models]]></category>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 18:01:01 GMT</pubDate>
            <atom:updated>2026-09-08T18:01:01.083Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Depth Anything 3 Just Made VGGT Look Overengineered]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://papers-100-lines.medium.com/depth-anything-3-just-made-vggt-look-overengineered-2ecbfe4322a4?source=rss-df93cc66596f------2"><img src="https://cdn-images-1.medium.com/max/1672/1*BJNsq0CBiTJCPAIUnzPiQA.png" width="1672"></a></p><p class="medium-feed-snippet">A plain DINO transformer with one depth-and-ray prediction target beats VGGT on camera pose and geometry &#x2014; and can even produce 3D&#x2026;</p><p class="medium-feed-link"><a href="https://papers-100-lines.medium.com/depth-anything-3-just-made-vggt-look-overengineered-2ecbfe4322a4?source=rss-df93cc66596f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://papers-100-lines.medium.com/depth-anything-3-just-made-vggt-look-overengineered-2ecbfe4322a4?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ecbfe4322a4</guid>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[depth-estimation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Fri, 04 Sep 2026 18:01:00 GMT</pubDate>
            <atom:updated>2026-09-04T18:01:00.994Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[One Transformer Just Replaced the Classical 3D Vision Stack]]></title>
            <link>https://papers-100-lines.medium.com/one-transformer-just-replaced-the-classical-3d-vision-stack-f02a1397d5aa?source=rss-df93cc66596f------2</link>
            <guid isPermaLink="false">https://medium.com/p/f02a1397d5aa</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[3d-reconstruction]]></category>
            <category><![CDATA[3dgaussiansplatting]]></category>
            <dc:creator><![CDATA[Papers in 100 Lines of Code]]></dc:creator>
            <pubDate>Tue, 01 Sep 2026 18:01:01 GMT</pubDate>
            <atom:updated>2026-09-01T18:01:01.212Z</atom:updated>
            <content:encoded><![CDATA[<h4>VGGT predicts camera poses, depth maps, and dense 3D point clouds from hundreds of images in a single forward pass — no structure-from-motion, no global alignment, no optimization required.</h4><p>In March 2025, a single model made most of the specialized 3D reconstruction pipeline unnecessary. <strong>VGGT</strong> (Visual Geometry Grounded Transformer), from Oxford’s Visual Geometry Group and Meta AI, takes anywhere from one to hundreds of images and predicts, in a single forward pass, everything the classical pipeline used to produce: the camera parameters of every image, a depth map for every image, and a dense 3D point cloud of the scene. No structure-from-motion beforehand, no optimization afterwards. It won Best Paper at CVPR 2025, and within months it had become the backbone that newer reconstruction methods build on rather than compete with.</p><p>This post explains how it works, what it actually predicts, what the numbers say, and what it means for feed-forward 3D reconstruction. It is a companion to my longer overview, <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html">Feed-Forward 3D Gaussian Splatting: From Proof of Concept to Production</a>, where VGGT opens the foundation-model stage of the story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZFDT9FwpU33YuWDNbbuwIg.png" /><figcaption>Visual Geometry Grounded Transformer | VGGT tutorial</figcaption></figure><h3>The pipeline it replaces</h3><p>Before VGGT, recovering geometry from a set of photographs meant chaining specialized components. The classical route runs structure-from-motion (COLMAP) to estimate cameras, then multi-view stereo for dense geometry — minutes of processing, and failure on scenes with little texture. The 2024 alternative, DUSt3R, replaced much of that chain with a network that takes two images and directly predicts a 3D point for every pixel of both. That representation is sufficient on its own: once you know where every pixel lives in 3D, the depth maps follow immediately, and even the camera poses can be recovered from the predicted points. But the network only handles two images at a time, so larger collections require stitching the pairwise predictions together with a global alignment optimization. Either way, an optimization step sits somewhere in the pipeline.</p><p>VGGT’s claim is that none of this structure is necessary: one transformer, trained on enough data, predicts the cameras, the depth maps and the 3D point cloud at once, for any number of input images — and does it better than the pipelines it replaces.</p><h3>Architecture: a plain transformer with one twist</h3><p>Each input image is tokenized by a pretrained DINOv2 encoder. To each image’s tokens, VGGT appends one <strong>camera token</strong> and four register tokens — extra learnable tokens that will later carry that image’s camera information. The resulting sequence, all images together, passes through 24 transformer blocks totalling about 1.2 billion parameters.</p><p>The one architectural novelty is how attention is organized. The blocks alternate between <strong>frame-wise self-attention</strong>, where each image’s tokens attend only to tokens of the same image, and <strong>global self-attention</strong>, where every token attends to every token across all images. The frame-wise layers build a coherent understanding of each image on its own; the global layers exchange information between viewpoints — implicitly doing the job that epipolar constraints, cost volumes and pairwise matching did in earlier systems. There is no cross-attention and no geometric machinery anywhere in the network: it is self-attention all the way through.</p><p>The transformer itself only produces features; four small output heads turn those features into predictions. The <strong>camera head</strong> reads each image’s camera token and predicts that image’s camera: its orientation as a quaternion, its position, and its field of view — nine numbers in total, with the principal point assumed to be at the image centre. The <strong>depth head</strong> predicts a depth value for every pixel, and the <strong>point head</strong> predicts a 3D point for every pixel, which together form a dense point cloud of the scene; both are standard dense-prediction decoders (DPT). Finally, the <strong>tracking head</strong>, built on the CoTracker2 architecture, takes any pixel you select in one frame and predicts where that same physical point appears in every other frame.</p><h3>The coordinate frame trick</h3><p>Every prediction needs a coordinate system to be expressed in, and VGGT uses the same convention as DUSt3R: the first camera <em>is</em> the coordinate system. Its pose is not predicted — it is defined to sit at the origin, and everything else is expressed relative to it: the poses of the other cameras, the depth maps, the whole point cloud. The network therefore has to know which of its inputs is the first image, and the mechanism for that is remarkably simple: the first image is given its own set of learnable tokens, different from the set shared by all the other images. Nothing else distinguishes it. One practical consequence: the order of the remaining images does not matter — shuffle them, and the predictions are identical, just reordered.</p><h3>Training</h3><p>VGGT is trained on seventeen datasets spanning indoor scans, outdoor scenes, synthetic renders, driving footage and object-centric captures — among them Co3Dv2, ScanNet, MegaDepth, DL3DV, Kubric, Hypersim and Virtual KITTI. The loss is a weighted sum over the four tasks: a Huber loss on the cameras; depth and point losses weighted by the model’s own predicted uncertainty; and a tracking term. Training took 64 A100 GPUs for nine days at a maximum image dimension of 518 pixels. Large by academic standards; small by the standards of language models — and this scale of data, not any architectural insight, is what the specialized pipelines could not match.</p><p><strong>Following this series?</strong> I write one breakdown like this per paper worth covering, with the implementation details included — they go out on <a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">my newsletter</a> first.</p><h3>The results, and the two that matter most</h3><p>VGGT beats DUSt3R and MASt3R on every task the three share, usually by a wide margin. On camera pose estimation it scores 88.2 AUC@30 on Co3Dv2, against 81.8 for MASt3R and 76.7 for DUSt3R — and it produces that answer in 0.2 seconds, where the other two need an optimization stage on top of their forward pass. For depth estimation on DTU, it reaches the accuracy that MASt3R only achieves when given the ground-truth cameras, while receiving no camera information at all. Its point clouds are the most accurate of the three on ETH3D. And its features improve point tracking when plugged into an existing tracker.</p><p>Two results deserve more attention than they usually get. First, <strong>the forward pass and optimization are complementary</strong>. On its own, VGGT already outperforms the methods that rely on an optimization stage — DUSt3R and MASt3R with their global alignment, and VGGSfM, a learned structure-from-motion pipeline that uses bundle adjustment. But you can also run bundle adjustment <em>on top of</em> VGGT, using its predictions as the starting point, and it still helps: camera accuracy on Co3Dv2 rises from 88.2 to 91.8. The forward pass does not make optimization useless; it replaces the part that was expensive — starting from nothing.</p><p>Second, <strong>the dedicated point head turns out to be redundant</strong>. VGGT offers two ways to obtain the 3D point cloud: take the point head’s output directly, or combine two of its other outputs — use the predicted camera parameters to lift each pixel’s predicted depth into 3D. The second route is <em>more accurate</em> than the head that was built for the job (0.677 versus 0.709 Chamfer distance on ETH3D). Predicting depth and cameras separately and composing them beats predicting their combination directly.</p><p>Speed scales gracefully with the number of views. On an H100 with FlashAttention, one frame takes 0.04 seconds and 1.9 GB of memory; fifty frames take about a second and 11 GB; two hundred frames take under nine seconds and 41 GB.</p><h3>What VGGT does not do</h3><p>VGGT predicts geometry, not appearance. Its outputs contain no colour and no Gaussians, so there is no scene you can render from a new viewpoint. The paper does include a novel-view-synthesis experiment, but it takes a shortcut: the network is given the target viewpoint and generates that image directly, pixel by pixel, without ever building a 3D scene in between. It gets close to a specialized view-synthesis model this way, while training on roughly a fifth of the data. Rendering a scene you can move through freely still requires attaching a Gaussian head, which is exactly what <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html#anysplat">AnySplat</a> did ten weeks after release — using VGGT itself as the teacher that supplies geometry supervision.</p><p>The authors state the current limits plainly: no support for fisheye or panoramic images, degraded results under extreme camera rotations, and tolerance for only minor non-rigid motion — a scene with substantial deformation breaks the model.</p><h3>Using it in practice</h3><p>Code and weights are on <a href="https://github.com/facebookresearch/vggt">GitHub</a>. Licensing needs one minute of attention: the original VGGT-1B checkpoint is non-commercial, but since July 2025 the code is commercially usable and a separate <strong>VGGT-1B-Commercial</strong> checkpoint is available on request — it scores marginally better than the original. A memory optimization released in May 2026 roughly doubled the number of frames that fit on a GPU, and the same group has since released VGGT-Omega, a scaled successor extending the approach to dynamic scenes. If you build on this line of work, track the repository rather than the paper.</p><h3>Why this model matters beyond its numbers</h3><p>For two years, feed-forward 3D reconstruction meant designing geometric structure into networks: epipolar attention in pixelSplat, cost volumes in MVSplat, pointmap regression in DUSt3R. VGGT demonstrated that a plain transformer at sufficient scale learns whatever structure it needs — and generalizes better for it. The methods that came after treat reconstruction as a fine-tuning problem on top of a geometry backbone rather than an architecture design problem, and the strongest 2025 results in feed-forward 3DGS are built either on VGGT directly or on the paradigm it validated. That shift, from designing pipelines to attaching heads, is the subject of the second half of <a href="https://www.3dgaussiansplattingcourse.com/feed-forward-3d-gaussian-splatting.html">the full timeline</a>.</p><h3>📘 Learn 3DGS Step-by-Step (PyTorch Only)</h3><p>The Gaussian heads that get attached to backbones like VGGT all render through the same differentiable rasterizer. My <strong>3D Gaussian Splatting Course</strong> builds that pipeline from first principles in <strong>PyTorch only</strong> — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.</p><p><a href="https://www.3dgaussiansplattingcourse.com/">Explore the Course →</a></p><h3>📩 Join the Newsletter</h3><p>I write breakdowns like this one on 3DGS, NeRF, PyTorch implementations and recent papers, including the engineering details that usually get skipped.</p><p><a href="https://3dgs.teachable.com/p/newsletter-3d-gaussian-splatting">Subscribe →</a></p><p><em>Originally published at </em><a href="https://www.3dgaussiansplattingcourse.com/vggt-explained.html"><em>https://www.3dgaussiansplattingcourse.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f02a1397d5aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>