<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rippertnt on Medium]]></title>
        <description><![CDATA[Stories by Rippertnt on Medium]]></description>
        <link>https://medium.com/@rippertnt?source=rss-12f203d2f9c0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*0DJz1QTI2S5QVlpH.jpg</url>
            <title>Stories by Rippertnt on Medium</title>
            <link>https://medium.com/@rippertnt?source=rss-12f203d2f9c0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 21:50:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rippertnt/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Four Robot Cameras, One NPU, Every Frame: NPU-First Perception for Physical AI]]></title>
            <link>https://medium.com/@rippertnt/four-robot-cameras-one-npu-every-frame-npu-first-perception-for-physical-ai-5e95396349b8?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e95396349b8</guid>
            <category><![CDATA[ultralytics]]></category>
            <category><![CDATA[vision-ai]]></category>
            <category><![CDATA[intel]]></category>
            <category><![CDATA[npu]]></category>
            <category><![CDATA[yolo]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Mon, 26 Oct 2026 16:20:16 GMT</pubDate>
            <atom:updated>2026-10-26T16:35:01.916Z</atom:updated>
            <content:encoded><![CDATA[<p><em>onAIR-vision, the vision engine of the onAIR framework, measured on an Intel® Core™ Ultra robotics kit. General release in November 2026, after one month of stabilization on robot kits.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4eBJvudvrSnXS6oWhzktTQ.png" /></figure><p>A Physical AI robot does three things at once. It <strong>sees</strong> through its cameras, <strong>thinks</strong> with a policy model such as a VLA, and <strong>acts</strong> through planning, control loops and motors. Each of those jobs is a real-time workload, and on most robot computers they compete for the same chip.</p><p>A manipulation robot rarely has a single camera. The common setup has four: a top view over the workspace, a camera on each wrist and a front view. Run four detectors at 30 fps next to a VLA and something has to give. Either frames are dropped, or the policy loses GPU time, or the control loop jitters.</p><p><strong>onAIR-vision</strong> takes a different route. It puts the whole perception job on the NPU, the third engine of every Intel Core Ultra processor, which usually sits idle. It is the vision engine of Circulus’s onAIR framework: a C++ and Python SDK on OpenVINO™, specialized for the NPU.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B_mmdep7CNuDoE61v1Sg5w.png" /><figcaption>Physical AI on one processor: see, think, act</figcaption></figure><h3>What it means for the robot</h3><p>We measured onAIR-vision on an AAEON CEXD-INTRBL robotics kit (Intel Core Ultra X7 358H, NPU 5010, OpenVINO 2026.4.1).</p><ul><li><strong>Detectors:</strong> YOLO26m, YOLO26x, RT-DETRv4-S and RT-DETRv4-L.</li><li><strong>Engines:</strong> each detector ran on the NPU, the integrated GPU and the CPU.</li><li><strong>References:</strong> Ultralytics, ONNX Runtime and the plain OpenVINO Python API, with the same video, thresholds and weights.</li><li><strong>Four cameras:</strong> measured directly, not projected.</li></ul><p>These are the results that matter to a robot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PVRmulJTgdov2Hr87wPe3Q.png" /><figcaption>What it means for the robot</figcaption></figure><ul><li><strong>Every camera, every frame:</strong> four cameras × 30 fps with YOLO26x in <strong>61%</strong> of one NPU. Ultralytics would need 262%.</li><li><strong>Fast reaction:</strong> <strong>2.5 ms</strong> per frame with YOLO26m and <strong>5.0 ms</strong> with YOLO26x, pre- and post-processing included.</li><li><strong>See small, hard objects:</strong> the largest YOLO26 runs on all four cameras, and the larger the model, the larger the lead.</li><li><strong>The GPU for the brain:</strong> <strong>0%</strong> of the GPU is used for perception, so the VLA keeps all of it.</li><li><strong>Same input, same answer:</strong> repeat runs are identical on the NPU, and the lossless mode is identical to every-frame inference.</li><li><strong>Distance to objects:</strong> Depth Anything V2, Intel MiDaS and ZoeDepth are in the same SDK.</li><li><strong>Run for days:</strong> memory changed by about 2 MB or less in soak tests with camera reconnects, model swaps and packing.</li></ul><h3>Every frame: 4.3× Ultralytics on the NPU</h3><p>Inferring every frame is the honest baseline: no frame is skipped and nothing is predicted. It is also the number that holds in any scene.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ROiwJS2ZlEUSbzK4ZoA86w.png" /><figcaption>Every frame on the NPU, frames per second</figcaption></figure><p>NPU, every frame (onAIR-vision vs references):</p><ul><li><strong>YOLO26m:</strong> <strong>396 fps</strong> vs Ultralytics 101 and ONNX Runtime 115, which is <strong>3.9×</strong>.</li><li><strong>YOLO26x:</strong> <strong>199 fps</strong> vs Ultralytics 46 and ONNX Runtime 48, which is <strong>4.3×</strong>.</li><li><strong>RT-DETRv4-S:</strong> <strong>92 fps</strong> vs ONNX Runtime 80, which is 1.15×.</li><li><strong>RT-DETRv4-L:</strong> <strong>44 fps</strong> vs ONNX Runtime 41, which is 1.07×.</li></ul><p>Two notes on the comparison:</p><ul><li><strong>Camera-shaped input:</strong> robot cameras are 16:9, so onAIR-vision runs YOLO26 at 640×384. That is the same scale as 640×640 without the grey padding, and it agrees with 640×640 at F1 0.990–0.992. At the same 640×640 input the lead is 2.3–2.4×.</li><li><strong>References:</strong> Ultralytics is the reference for YOLO26 and ONNX Runtime for RT-DETRv4.</li></ul><p>For a robot, throughput is reaction time. With the largest YOLO26, onAIR-vision has seen each frame in 5 ms, where Ultralytics needs 22 ms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nL260bJlWINWV7cXFyK0aQ.png" /><figcaption>Reaction time: milliseconds per frame on the NPU</figcaption></figure><h3>No blind spots: four cameras × 30 fps on one NPU</h3><p>Four cameras at 30 fps is 120 frames every second. We ran four clips with offset still and motion phases on one NPU and measured how much of it they take.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6zGgLXFZnSmfgdCZecceWw.png" /><figcaption>Share of the NPU for four cameras at 30 fps</figcaption></figure><p>Share of the NPU, four cameras × 30 fps, every frame:</p><ul><li><strong>YOLO26x:</strong> <strong>61%</strong> with onAIR-vision; Ultralytics would need 262%, so it cannot keep up.</li><li><strong>YOLO26m:</strong> <strong>30%</strong> with onAIR-vision; Ultralytics would need 119%, so it cannot keep up.</li></ul><p>Turned around, that is the frame rate each camera can get when four share one NPU. With Ultralytics, every camera falls below its own 30 fps. With onAIR-vision, each camera has room to spare.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hx1Tx5W7OYLlYiwHFfKy-g.png" /><figcaption>With four cameras on one NPU: the most each camera can get</figcaption></figure><h3>On video: 6.9× lossless, 18× adaptive</h3><p>Neighbouring video frames look alike. <strong>AIR-X</strong>, onAIR-vision’s adaptive inference layer, runs the model less often while every frame still gets a result. It reports how much it saved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SvJtsUQPDYX7-ITllcP6Fg.png" /><figcaption>YOLO26x on the NPU: every frame, lossless and adaptive</figcaption></figure><p>YOLO26x on the NPU:</p><ul><li><strong>Ultralytics, every frame:</strong> 46 fps (1×).</li><li><strong>onAIR-vision, every frame:</strong> 199 fps (<strong>4.3×</strong>).</li><li><strong>onAIR-vision, lossless:</strong> 316 fps (<strong>6.9×</strong>), with the same detections as every frame.</li><li><strong>onAIR-vision, adaptive:</strong> 830 fps (<strong>18×</strong>), at agreement F1 0.970.</li></ul><p>These gains depend on the scene: a still workspace saves more than a moving wrist camera. Our test video is 40% still, then a slow pan. With four cameras moving out of step, the adaptive mode’s agreement falls to 0.90–0.97 while lossless stays at 0.998–1.000, so we recommend the lossless mode for multi-camera robots.</p><p>The larger the model, the larger the lead. A bigger detector sees smaller and harder objects, and it is exactly where a robot runs out of compute first.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ZI_dmu51AcQO0EQGXNamA.png" /><figcaption>The larger the model, the larger the lead</figcaption></figure><h3>The GPU stays with the policy</h3><p>Robot computers built around one large GPU, such as NVIDIA Jetson AGX Thor, run perception and the policy on the same GPU. The Thor series has no separate deep-learning accelerator. Multi-Instance GPU (MIG) can split the GPU into fixed partitions, but that isolates the two workloads without adding compute. A fixed perception slice is taken from the policy whether perception uses it or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*txNiwUZAHvO2n-tHeoW5Cg.png" /><figcaption>Where the perception load lands</figcaption></figure><p><em>Architecture comparison from public specifications; we did not measure Thor.</em></p><h3>When the CPU is free, and transformer detectors</h3><p>On a robot the CPU belongs to control, but when it is free the same SDK leads there too. Every runtime used all CPU cores.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lt8vREDEzV0trMFRqIuJng.png" /><figcaption>CPU every frame, frames per second</figcaption></figure><ul><li><strong>YOLO26x on the CPU:</strong> 4.7× Ultralytics on every frame and <strong>7.8×</strong> lossless. It is also 1.5× Ultralytics’ own INT8 export.</li><li><strong>YOLO26m on the CPU:</strong> 4.0× on every frame.</li><li><strong>RT-DETRv4-S · L on the NPU:</strong> 1.07–1.15× ONNX Runtime on every frame and 1.8–1.9× lossless.</li></ul><h3>Road to release: November 2026, then Nova Lake and Iris-1</h3><p>The benchmarks and the white paper are done, and the release check found no memory leaks. Next is one month of stabilization on robot kits. General release follows in November 2026.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2MC2yWx4WSt_4TLJx8Dvtg.png" /><figcaption>Roadmap: release in November 2026, then Nova Lake and Iris-1</figcaption></figure><ul><li><strong>October 2026 (done):</strong> benchmarks on NPU, GPU and CPU, the white paper (2nd edition) and a release check (code review, sanitizers, valgrind, soak tests).</li><li><strong>Stabilization, one month:</strong> long-run soak tests on the NPU of robot kits, recordings from real robots, accuracy on labelled robot data, and C++ and Python parity.</li><li><strong>November 2026:</strong> general release of the onAIR-vision SDK, with C++ and Python APIs and the model hub.</li><li><strong>Next, with Nova Lake:</strong> onAIR-vision will be optimized for Intel’s next-generation Core Ultra processors, codenamed Nova Lake. Timed with that launch comes <strong>Iris-1</strong>, our own detector, designed to run faster and more accurately than YOLO26 on onAIR-vision.</li></ul><h3>Why Iris-1</h3><p>YOLO26 is an excellent detector and the reference point of this benchmark. Ultralytics releases it under the AGPL-3.0 license, so a closed commercial robot product needs a separate enterprise license. Iris-1 is built for onAIR-vision and the NPU from the start, so robot makers get a detector they can ship in commercial products with the same .infer() call. We will publish its benchmarks at release.</p><h3>How we measured</h3><ul><li><strong>Same inputs:</strong> the same 1280×720 video frames, score threshold and model weights for every runtime. Frames per second include pre- and post-processing.</li><li><strong>Agreement, not accuracy:</strong> F1 is agreement with onAIR-vision inferring every frame (same class, IoU ≥ 0.5), not COCO mAP.</li><li><strong>References:</strong> Ultralytics 8.4 with its own OpenVINO and INT8 export. ONNX Runtime 1.30, using whichever was faster of its OpenVINO execution provider and its default provider.</li><li><strong>Scope:</strong> one processor, one driver and one OpenVINO version. Accuracy on labelled robot footage is part of the stabilization month.</li></ul><h3>What onAIR-vision is</h3><ul><li><strong>One API, any engine:</strong> the same call and result type on the NPU, CPU and GPU for YOLO26, RT-DETRv4, RF-DETR, D-FINE, open-vocabulary and depth models.</li><li><strong>Built for the NPU:</strong> INT8 and graphs shaped for the NPU, camera-shaped input, a header-only C++ SDK and Python.</li><li><strong>AIR-X adaptive inference:</strong> lossless and adaptive modes for video (patent pending).</li><li><strong>Iris-1 (coming with Nova Lake):</strong> our own detector for commercial products, on the same API.</li><li><strong>Runs for days:</strong> bounded memory across long runs, camera reconnects and model swaps.</li></ul><p>A robot’s eyes deserve their own engine. NPU-first, the largest YOLO26 runs 4.3× faster than Ultralytics on every frame in any scene, and four cameras fit in 61% of one NPU. The GPU keeps all of its compute for the policy, and the CPU keeps the control loop.</p><p><em>Test system: AAEON CEXD-INTRBL robotics kit, Intel Core Ultra X7 358H, Arc B390 integrated GPU, NPU 5010, Ubuntu 26.04, OpenVINO 2026.4.1, NPU driver 1.38; measured October 2026. Release timing, Nova Lake optimization and Iris-1 are plans and may change; Iris-1 performance will be measured and published at its release. YOLO26 is licensed by Ultralytics under AGPL-3.0, with enterprise licenses available from Ultralytics. Performance varies by use, configuration and other factors. Published by Circulus Inc.; not an Intel publication and does not imply endorsement by Intel, AAEON, NVIDIA or Ultralytics. Intel, Core, Arc and OpenVINO are trademarks of Intel Corporation or its subsidiaries; Ultralytics and YOLO are trademarks of Ultralytics; other names may be claimed as the property of others. onAIR-vision’s adaptive inference technology (AIR-X) is patent pending.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e95396349b8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Instant Responses for Physical AI on Intel GPUs]]></title>
            <link>https://medium.com/@rippertnt/instant-responses-for-physical-ai-on-intel-gpus-d36815d9876b?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d36815d9876b</guid>
            <category><![CDATA[openvino]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[vllm]]></category>
            <category><![CDATA[physical-ai]]></category>
            <category><![CDATA[edge-ai]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 11:34:43 GMT</pubDate>
            <atom:updated>2026-10-15T11:38:07.614Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TptqQS2oEC64UG6PH0RL3Q.png" /></figure><p><em>Introducing onAIR-infer, the inference engine of Circulus’s onAIR framework: a C++ LLM and VLM server for robot fleets and for the robots themselves. Unveiled and raced live at the Intel Robotics Workshop in Korea, with release planned for November after one month of stabilization.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sUbN1RRXE_DApNfugM0UmQ.png" /><figcaption>Physical AI runs in two places: a site server and the robot itself. onAIR-infer covers both on Intel hardware.</figcaption></figure><p>A robot that talks, plans and looks at the world can’t wait for a data center. When a care robot hears “Where did I leave my glasses?”, or a warehouse robot has to re-plan around a blocked aisle, the first word of the answer matters more than the hundredth. And unlike a chatbot, a robot keeps asking similar things: the same status checks, the same routine plans, the same scene described with small changes.</p><p>We built onAIR-infer at Circulus for exactly that workload. It is the inference engine of our onAIR framework: a C++ server on OpenVINO™ that serves large language and vision-language models through an OpenAI-compatible API, with no cloud, no Python at inference time and no per-token fees. This week at the Intel Robotics Workshop in Korea, we showed it for the first time and ran it live next to another engine.</p><h3>Two places a Physical AI model runs</h3><p><strong>The site server.</strong> One on-premise GPU serves a whole site: the robots, the operators’ tablets and the camera agents. Our test system is a single Intel® Arc™ Pro B70 (32 GB) running Qwen3.8–27B in int4.</p><p><strong>The robot itself.</strong> The model runs on the processor inside the robot, with no network dependency. Our test system is an Intel® Core™ Ultra X7 358H (Panther Lake) with its integrated Arc™ B390 GPU and 64 GB of memory. It runs the Qwen3.6–35B-A3B vision-language model in int4.</p><p>We compared onAIR-infer with the OpenVINO GenAI reference pipeline and with vLLM for Intel GPUs. Every engine ran on the same machine, with the same open benchmark client, the same prompts and the same rules: greedy decoding, fixed 256-token answers, thinking off.</p><h3>What it means for a robot</h3><h3>The first word arrives at conversation speed</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ReXlc_q2Ny4IW9sO0Y4Hkg.png" /><figcaption>Time to first token. One robot, and the whole fleet at once.</figcaption></figure><p>When one robot asks, the first token arrives in <strong>0.06 seconds</strong> on the site server, five times sooner than GenAI. Inside the robot it arrives in <strong>0.20 seconds</strong>, about half of GenAI’s time. When 32 users ask the site server at the same moment, the median first token is still <strong>0.58 s</strong>. The fleet doesn’t wait in line.</p><h3>Routine work finishes before the robot takes its next step</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPjsfqXHJWb9gKEwmAjdZw.png" /><figcaption>Robots repeat themselves. Tokens per second for one user, new and routine questions.</figcaption></figure><p>Most of what a robot asks has been asked before. On those routine requests onAIR-infer runs <strong>12× faster than GenAI</strong> on the B70 (769 tokens/s) and <strong>5.8× faster</strong> inside the robot (334 tokens/s). New questions are faster too: <strong>+28% over GenAI and 1.5× vLLM</strong> on the B70, and <strong>+23% over GenAI</strong> on the robot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aVXRKvKauqYvNkbdEy_1cQ.png" /><figcaption>A full 256-token answer for one robot on one Arc Pro B70.</figcaption></figure><p>For a robot, that means a full 256-token routine plan is ready in about <strong>0.4 seconds</strong> instead of more than four. A new question takes about 3.2 seconds from start to finish.</p><h3>One GPU serves a whole site</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7oIwevZy_G8zdrTxz5dytw.png" /><figcaption>Total tokens per second: 32 users on one Arc Pro B70, and 8 agents on the robot’s own chip.</figcaption></figure><p>With 32 users on one Arc Pro B70, onAIR-infer serves <strong>525 tokens/s on new questions and 1,205 on routine ones</strong>, and peaks at <strong>1,390 tokens/s</strong>, 3.1× GenAI’s best. Inside the robot, eight agents share one chip at <strong>765 tokens/s</strong> on routine requests and 171 on new ones. Several agents, such as a planner, a conversation agent and a scene describer, can share the robot’s own processor.</p><h3>Live, side by side</h3><p>Benchmark tables are easy to doubt, so at the workshop we let people watch instead.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*snPsRFuPFQYBJQpJyx9OoQ.png" /><figcaption>The live race: one PC, two Arc Pro B70 cards, 24 robots per side.</figcaption></figure><p>One PC held two Intel Arc Pro B70 cards. onAIR-infer ran on one and vLLM on the other, both serving Qwen3.8–27B in int4, with vLLM set up following public Arc Pro B70 guides. Each side got 24 simulated robots asking the same fleet questions in the same order at the same moment. Half were routine questions asked again, a quarter were reworded and a quarter were brand new. All 48 conversations streamed on one screen, with tokens per second, time to first token and running totals for both engines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*CZbw3YSEYkcvG5GF" /><figcaption>onAIR-infer (blue) and vLLM (red) at the Intel Robotics Workshop in Korea.</figcaption></figure><h3>Measured cell by cell</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DkUANA-orCbUNcjE2lSKow.png" /><figcaption>Multi-user benchmark cells where onAIR-infer is even with or ahead of each engine.</figcaption></figure><p>Across the full grid of multi-user cells on the B70 (2 to 32 users; chat, code, document Q&amp;A and vision), onAIR-infer is even with or ahead of GenAI’s best in 54 of 63 cells, and ahead of vLLM in 90 of 93. On the Panther Lake system it is even with or ahead of vLLM in all 27 throughput cells, and ahead of GenAI in all 25 cells for both throughput and first-token time.</p><p>We also list where other engines still lead, because you will find these if you rerun the benchmark. On the B70, OpenVINO GenAI is faster on similar code questions at 16 to 32 users, and on new questions about a fresh 2,000-token document at two or more users. On Panther Lake, vLLM reads a 2,000-token document faster (its first token arrives 1.1 to 1.5× sooner) and starts sooner on images with four to eight users. These are the cells we’re working on next.</p><h3>The road to November</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cnFfftxoK0c3eXA-DF-NQw.png" /><figcaption>From unveiling to release: one month of stabilization, release in November 2026.</figcaption></figure><p>A robot fleet runs all day, so before we hand the runtime to anyone, we’re spending one month on stabilization. That means long continuous runs at full load, checking the tested presets on more Intel systems, and finishing the packaging and documentation. The release is planned for November 2026.</p><p>The November release has two packages:</p><ul><li><strong>Runtime</strong> (Linux x86_64): the server binaries with the OpenVINO 2026.4 runtime bundled, and the live race program. It detects the CPU, GPU and NPU and picks a tested preset for the hardware it finds. One command serves a model, either for one user at the lowest latency or for many users. It also includes what a fleet operator needs: node health and energy per token, protection against overheating and overload, and remote drain and restart.</li><li><strong>Developer kit</strong> (any OS with Node.js): the open-source benchmark client (Apache-2.0) with the raw results behind both white papers, the two white papers, API and operations docs, and examples (curl, Python with the OpenAI SDK, vision, a systemd unit).</li></ul><p>The benchmark client lets you check our numbers yourself. Start any engine, point the client at it, and it writes your run next to ours. It works the same with onAIR-infer, OpenVINO GenAI, vLLM or any OpenAI-compatible server.</p><p>The engine source is closed, and its acceleration technology is patent pending. The runtime will be free for evaluation and research; production use needs a commercial license.</p><h3>What’s next</h3><p>Robots don’t all carry the same computer, so after the release we’ll widen the model range on both the Arc Pro B70 and Panther Lake. That covers small Qwen3.5 models (2B, 4B, 9B) for robot controllers, and the Gemma 4 family from E2B and E4B up to 12B, 31B and the 26B mixture-of-experts model. Each will ship with tested presets and the same open benchmark.</p><h3>Looking further: toward the dark factory</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ne77iYJoRx8Nwio5GBnuxw.png" /><figcaption>Three tiers of Physical AI on Intel. The factory tier on Intel GPU D270 is the next step.</figcaption></figure><p>This week Intel announced the GPU D270, an inference GPU with 480 GB of memory on a single card. That’s fifteen times the memory of the Arc Pro B70 we tested here.</p><p>After the November release, we plan to bring onAIR-infer to it. The goal is the same as on the B70, at factory scale: many more robots on one card and much larger models, still answering at conversation speed.</p><p>That’s where Physical AI is heading. In the dark factory, production can run with the lights off, but people stay in charge. A person says what needs to happen, in plain language. The factory’s AI turns that into plans, and the robots carry them out. We want onAIR-infer to be the inference layer for all of it: the processor inside each robot, the site server, and the factory’s own AI.</p><h3>Why Intel</h3><p>For Physical AI, the useful question isn’t which accelerator is fastest in a data center. It’s which hardware a robot company can actually deploy: a workstation-class discrete GPU in a site server, and a processor inside the robot that can run a 35B vision-language model by itself. Intel covers both ends with one software stack, OpenVINO. That’s why onAIR-infer is built on it, and why we tested on both.</p><h3>Get in touch</h3><ul><li>Release: November 2026</li><li>Early access, commercial licensing and partnerships: info@circul.us</li></ul><p>If you build robots or edge systems on Intel hardware, we’d like to hear which models and workloads you need next.</p><p><em>Circulus is a Physical AI company in Korea building the ARCos robot platform. Intel, Arc, Core and OpenVINO are trademarks of Intel Corporation or its subsidiaries. onAIR-infer is published by Circulus and is not an Intel product. Performance varies by use, configuration and other factors. Release scope and timing may change with the stabilization results. Intel GPU D270 support is a plan; nothing has been measured on it yet.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d36815d9876b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Priced Out of NVIDIA? We Benchmarked Five LLM Engines on Intel’s 32 GB Arc Pro B70]]></title>
            <link>https://medium.com/@rippertnt/priced-out-of-nvidia-we-benchmarked-five-llm-engines-on-intels-32-gb-arc-pro-b70-da6c7588b79a?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/da6c7588b79a</guid>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[edge-ai]]></category>
            <category><![CDATA[openvino]]></category>
            <category><![CDATA[intel]]></category>
            <category><![CDATA[local-llm]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Tue, 22 Sep 2026 08:01:53 GMT</pubDate>
            <atom:updated>2026-09-22T08:14:09.372Z</atom:updated>
            <content:encoded><![CDATA[<h3>The card wasn’t the bottleneck. Five serving engines on identical silicon came out 3.9× apart — and running it five times found bugs in our engine and in our benchmark.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WuPrGnhi0HZ_dqBJFIi35w.png" /></figure><h3>The 32 GB problem</h3><p>If you are trying to run a serious model locally right now, you have probably already had this conversation with your finance team.</p><p>Memory prices have climbed, and NVIDIA’s RTX PRO-class cards have climbed with them. For a lot of teams, the card everyone actually wants is simply not a line item that gets approved this quarter.</p><p>Which leaves a short list. At the 32 GB tier — enough to hold a 27B model at 4-bit with real KV-cache headroom — the practical alternatives are <strong>Intel’s Arc Pro B70</strong> and <strong>AMD’s Radeon AI PRO R9700</strong>. Both have the memory. Both cost a fraction of the NVIDIA option.</p><p>And both come with the same asterisk: <strong>the software ecosystem is not CUDA.</strong></p><p>This is not a niche problem. Outside the hyperscalers, most teams running models locally are now making exactly this decision, on exactly these two cards, with almost no published data to make it on.</p><p>That asterisk is the actual decision. VRAM is a spec sheet number and it is easy to compare. What is hard to compare, and what nobody publishes, is whether the serving stack on those cards <em>delivers</em> the hardware you paid for. A 32 GB card that runs your model at a third of its potential is not a bargain, it is a slower way to spend money.</p><p>So we measured it. Five serving engines, one Intel Arc Pro B70, one 27B model, the same workload for all of them, run five times.</p><p><strong>One thing up front, so nobody is misled:</strong> we tested Intel, not AMD. We have no measurements for the R9700 and nothing in this post should be read as a comparison between the two cards. What this post can tell you is how much of the gap on a non-NVIDIA card is the <em>software</em> — and the answer turned out to be: almost all of it.</p><h3>The setup</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ho1nbbk04mSLYPTOqzTs7g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*cpmTLvIQwX9_JWeEgLl1lg.png" /></figure><p>Each engine runs its own native optimal quantization — INT4-OV, AWQ, Q4_K_M — which is the fair comparison for serving speed. Output quality equivalence was verified separately and is not the subject of this post.</p><p>We build inference software for robots, which means we care about a number most LLM benchmarks treat as secondary: <strong>time to first token</strong>. Throughput is a cost line. Latency is whether the product exists at all — a robot that answers in 1.1 seconds is not conversational, it is waiting. Both numbers are reported throughout.</p><p>The workload also splits every request into three cache classes, and this turned out to matter more than anything else we measured:</p><ul><li><strong>new</strong> — a task unseen in this process</li><li><strong>similar</strong> — the same task with changed requirements or data</li><li><strong>repeat</strong> — an identical request replayed after at least two intervening waves</li></ul><p>Real workloads are full of repeat. A production line, a support desk, an agent loop with a fixed system prompt — all of them send the same prefix thousands of times a day. Most benchmarks never separate this out, which means most benchmarks measure a condition that barely occurs in practice.</p><h3>Result 1: the software gap is bigger than any hardware gap</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*zk7WLwUtdZvqVTBNU29Geg.png" /></figure><p>Same card. Same weights. <strong>A 3.9× spread.</strong></p><p>Sit with that for a second in the context of the buying decision above. The difference between the best and worst serving stack on this one card is larger than the difference most people are arguing about when they compare cards. If you pick the 32 GB Intel card and then run the wrong engine on it, you have thrown away more performance than you saved by not buying NVIDIA.</p><p>The most instructive pair is ARCinfer against OVMS. <strong>Both run on OpenVINO.</strong> Same runtime, same silicon, same model — and a 2.04× difference in throughput. Whatever is happening here, it is not the hardware.</p><p>That is the thesis this whole exercise kept confirming: <strong>the practical ceiling on non-NVIDIA cards has been set by serving software, not by the chip.</strong></p><h3>A note for readers of our earlier post</h3><p>We published numbers for this engine before, under the name <strong>AERO.X</strong>, alongside XPU-TOP — the time-aligned measurement tool we open-sourced first, because we couldn’t optimize what we couldn’t see. Same engine, same team; the name in our repos and docs is ARCinfer and that is what we will use from here.</p><p>Those numbers and these numbers are not the same measurement, and it is worth being explicit about why before someone puts them side by side:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/855/1*3HXxk7NrV7SSI3VIMNuK2Q.png" /></figure><p>Both are true. The earlier figure is a short-context best case against one baseline; this one is an average across a realistic task mix measured five times. If anything, the more durable result from that post is the one that wasn’t the headline: <strong>throughput held at 4.17× when context grew 512× to 65,536 tokens.</strong> A 10% fall across that range is the difference between a benchmark and a product — and long-context behavior is not something this post measured at all.</p><h3>Result 2: where the gap actually opens</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OU6yPYD8zGhMyWUG55U7kQ.png" /></figure><p>On unseen tasks, the top three engines are within 8% of each other — 57.1, 53.1, 53.0. Nothing remarkable.</p><p>Then you replay an identical request:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*tXI7lBTiyQHXTCqWahgT2w.png" /></figure><p>That last row deserves a paragraph of its own, because it is the clearest measurement of ecosystem immaturity in the whole set.</p><p><strong>SGLang’s headline feature is prefix caching.</strong> RadixAttention is the reason a lot of people reach for it. On this platform, in the configuration that actually runs, it shows <em>no</em> benefit from a repeated request — 28.9 new, 28.8 similar, 28.8 repeat. Three conditions, one number.</p><p>We want to be careful here, because the honest reading is not the flattering one. The SGLang XPU build does not support MTP and ran in normal mode, and we did not confirm whether the radix cache is active on the XPU backend at all. So this is <strong>not</strong> evidence that our cache design beats RadixAttention as an architecture.</p><p>It is evidence of the thing the opening section was about. <strong>A leading engine’s flagship capability does not land on this platform.</strong> If you are evaluating a non-NVIDIA card by reading feature lists, this is the failure mode to worry about: the feature exists, the build installs, the server starts, and the capability silently isn’t there.</p><p>That gap is exactly where we build.</p><h3>Result 3: concurrency, where we stop winning cleanly</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ovow772M38B42OqxVuNqaw.png" /></figure><p>One session is not a realistic load. A robot runs speech, scene understanding and task planning at the same time; a dev team shares one box. “2 to 4 concurrent” is the normal case, not the stress case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*tBEnwdWpIHK7o_KyUC1M5w.png" /></figure><p>Two things are true at once here, and we would rather say both.</p><p><strong>We lead on absolutes.</strong> Highest throughput, lowest latency, and the only engine besides vLLM to keep TTFT under half a second with four streams attached. OVMS at 6.6 seconds and llama.cpp at 5.3 seconds are outside interactive range entirely — and in one configuration OVMS didn’t just slow down, it returned HTTP 400 and terminated the run on four simultaneous vision requests.</p><p>A note on that, because it is not a fair fight otherwise: <strong>OVMS does have a continuous batching mode, and it is not what we measured.</strong> We tried it and it failed with OUT_OF_RESOURCE on this workload, so it was excluded rather than reported as a broken number. The figures above are OVMS in the mode that completed. That exclusion is itself a data point about the ecosystem — the vendor&#39;s own stack has the right feature, and on this hardware and this workload we could not keep it alive.</p><p><strong>We lose on slope.</strong> SGLang gains 3.61× going from one stream to four. We gain 1.84×. We start from more than three times the base, so we stay ahead in absolute terms across everything we measured — but the trend line is not ours, and at 8 or 16 streams it would stop being ours. We haven’t measured that range, and we’re not going to claim it.</p><h3>Result 4: what running it five times found</h3><p>Here is the part we think is worth more than the numbers above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E7VVdcguONFLl4tds-guKA.png" /></figure><p>We ran the concurrency benchmark five times. Latency reproduced beautifully. Throughput did not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*JwrztTr7zwQs95xU2CMPFg.png" /></figure><p>The variance is not spread out. It sits <strong>entirely in the repeat class</strong>. Across three runs, the new and similar waves land within a couple of percent of each other every single time. The repeat waves swing by up to 52%.</p><p>Then we looked inside a single wave, at the four streams running in parallel:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJQrZGPDVjvP9qgyuwy88A.png" /></figure><p>Class (4 streams)RUN 1RUN 2RUN 3new1.211.181.18similar1.241.211.19<strong>repeat1.381.761.91</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*Vr68rOPQmxyGBeR0pNjtVg.png" /></figure><p><em>(Metric: fastest stream ÷ slowest stream within one wave. 1.00 means perfectly even.)</em></p><p>Worst individual wave: <strong>3.01×</strong>. In one run, four parallel streams on the same request split cleanly into two groups — 133 and 137 tok/s, against 63 and 63.</p><p>So: <strong>some streams get the cache and speculative-decoding benefit and some don’t, and which ones changes between runs.</strong></p><p>Meanwhile TTFT’s internal structure repeats perfectly. In every 4-stream wave, exactly one stream answers in 132–138 ms while the other three take 362–376 ms, at a ratio of 2.63 / 2.78 / 2.86 across the three runs. Admission and queueing are deterministic. Only the <em>decode-stage resource allocation</em> is not.</p><p>That is a much smaller bug than “throughput is unstable,” and a much more actionable one.</p><p>Our current hypothesis is that this is less about cache eviction than about <strong>speculative decoding interacting badly with continuous batching</strong>. The draft step runs batch-synchronously, but acceptance rate is per-sequence. Sequences whose drafts get accepted fly; sequences whose drafts get rejected fall back to plain decode speed. 63 against 135 is roughly the shape you’d expect. It’s testable — log accepted-draft-token ratio per request alongside the cache-hit flag — and it’s the first thing we’re doing.</p><h3>And then we broke our own harness</h3><p>Runs 4 and 5 came back at 331 and 319 tok/s at four streams — a 75% jump.</p><p>They were wrong.</p><p>Runs 3, 4 and 5 executed back to back at 12:12, 12:26 and 12:31, and <strong>the server process was never restarted between them.</strong> The workload defines new as a task unseen <em>in this process</em>. Once run 3 had served those tasks, runs 4 and 5 weren&#39;t measuring new tasks at all.</p><p>The evidence is unambiguous once you look for it:</p><p>new ÷ repeat ratioRUN 1RUN 2RUN 3RUN 4RUN 5code0.270.260.27<strong>1.021.01</strong>ko0.240.250.25<strong>1.030.99</strong>vlm0.410.330.37<strong>1.011.02</strong></p><p>In runs 4 and 5, new performs identically to repeat. TTFT agrees — new-class TTFT p95 drops from 401–416 ms to 243–248 ms, landing exactly on the repeat values.</p><p>So 331 tok/s is not our general performance number, and we won’t quote it as one. What it <em>is</em> is a legitimate separate observation: <strong>the ceiling with a fully warm cache</strong>, at 272 ms TTFT and 103–106 tok/s per stream. For a workload that repeats the same prefix all day, that’s not an artifact — that’s the operating condition. It just has to be labeled.</p><p>The harness lesson is cheaper and more embarrassing: <strong>enforce a process restart between runs.</strong> We also found that only 92% of requests produced an identical completion-token count across runs, because generation wasn’t pinned. Since throughput is tokens over wall time, that variance goes straight into the result.</p><p>Both are half-day fixes, and both are now prerequisites for everything else — you cannot verify a scheduler improvement while your measurement floor is moving.</p><h3>If you’re making the 32 GB decision</h3><p>We can’t tell you whether to buy Intel or AMD; we only measured one of them. But the data does support three things worth carrying into that decision.</p><p><strong>1. Budget more attention to the serving stack than to the card.</strong> On one card, engine choice moved throughput by 3.9× and TTFT by 10×. That is a larger effect than any hardware comparison you are likely to be weighing.</p><p><strong>2. Don’t evaluate by feature list.</strong> The single most surprising result here is a flagship caching feature returning exactly 1.00× on a platform where it nominally exists. Features that work on CUDA do not automatically work elsewhere, and they fail quietly — the server starts, requests succeed, and the capability simply isn’t there. Measure the capability you’re buying, on the card you’re buying.</p><p><strong>3. Benchmark your own cache pattern.</strong> The engines cluster tightly on unseen tasks and separate violently on repeated ones. If your workload has a stable system prompt or a repeated instruction set — and most production workloads do — a benchmark that only measures cold requests will rank engines in an order that has nothing to do with your bill.</p><h3>Is any of this Intel-specific?</h3><p>Fair question, and the answer matters if you’re reading this from a different platform.</p><p>The Intel-specific part of our work is the smaller part: OpenVINO backend plumbing, kernel selection, the quantization path. Real work, but replaceable work.</p><p>The part that produced the numbers above is not vendor-specific at all. Every gap we found, and every fix we are shipping, lives at the same layer:</p><ul><li><strong>Prefix-cache admission and eviction under concurrency.</strong> The bug we diagnosed in our own engine — some streams getting the cache benefit and others not, non-deterministically — is a scheduling problem, not a kernel problem. Every serving stack on every vendor has to solve it, and most solve it approximately.</li><li><strong>Speculative decoding under continuous batching.</strong> The draft step is batch-synchronous; acceptance is per-sequence. That contradiction exists identically on CUDA. It is arguably worse there, because higher batch sizes make the divergence wider.</li><li><strong>Admission control and fairness.</strong> OVMS dying at OUT_OF_RESOURCE and SGLang holding a perfect 1.00 fairness ratio are two ends of the same design axis, and the axis has nothing to do with silicon.</li></ul><p>Which is also why the SGLang result was the most interesting one to us. A capability that works on one backend and silently doesn’t on another is not a hardware limitation. It is an integration gap, and someone has to close it per platform, by hand, with measurements like these.</p><p>We have been closing them on Intel because that is the hardware in front of us. <strong>The techniques don’t know that.</strong></p><p>And the list of platforms with this problem is not short. Every non-CUDA backend — ROCm, SYCL, Vulkan, the NPU runtimes on mobile-class SoCs — inherits the same three problems above, plus a tighter memory budget and, on battery-powered hardware, a thermal ceiling that makes wasted compute expensive twice over. That last constraint is the one we live with anyway: the other two layers of our stack, vision and robot control, target NPUs rather than a GPU, because a robot that pulls a workstation card’s power budget is not a robot you ship. Those NPU measurements aren’t finished and we’re not claiming them here, but it is the same team solving the same class of problem one tier down.</p><h3>Three questions we’d genuinely like answered</h3><p>Not rhetorical. If you work on any of these, we’d rather be corrected than right:</p><ol><li><strong>Is the radix cache actually active on SGLang’s XPU backend?</strong> A 1.00× repeat ratio is consistent with “not enabled” and with “enabled but evicting,” and we couldn’t distinguish the two from the outside.</li><li><strong>Is there an OVMS continuous-batching configuration that survives four concurrent vision requests on 32 GB?</strong> Ours hit OUT_OF_RESOURCE and we&#39;d like to re-run it properly rather than leave the mode unmeasured.</li><li><strong>Has anyone characterized speculative-decode acceptance divergence under continuous batching on their backend?</strong> We suspect it’s the mechanism behind our own fairness bug, and we’d expect it to be worse, not better, at larger batch sizes.</li></ol><p>Send us a config and we’ll re-run it with the same hash-pinned workload and publish whatever comes out — including if it makes us look worse.</p><h3>What we’re fixing</h3><p>The gap between peak and mean is the work. Best cold run at four streams: 205.7 tok/s. Three-run mean: 186.1. With a warm cache, a single wave hit 494.2.</p><p><strong>The ceiling is already demonstrated. What remains is making the mean meet it.</strong> That’s not a research problem, it’s an engineering schedule.</p><ol><li><strong>Harness hygiene</strong> — forced process restart between runs, temperature 0 / pinned seed. Prerequisite for everything below.</li><li><strong>Per-request logging</strong> of cache-hit and accepted-draft-token ratio. Right now we’re inferring the mechanism from a throughput split; we should be reading it directly.</li><li><strong>Concurrency fairness</strong> — eliminate the intra-wave split. Target: 1.00. SGLang already does this at every class and every concurrency level. It is slower than us by a wide margin and still the reference implementation for how a scheduler should behave.</li><li><strong>Cache allocation</strong> — make admission and eviction deterministic across concurrent sessions.</li><li><strong>Batch scaling slope</strong> — 1.84× against SGLang’s 3.61×.</li></ol><h3>Where this goes: from a workstation card to an AI PC</h3><p>Everything above was measured on a discrete 32 GB card, because that is what fits a 27B model today. But the logic of the opening section does not stop there, and it is the reason we think this work matters beyond our own product.</p><p>If the ceiling on non-CUDA hardware is set by serving software rather than silicon, then <strong>lifting that ceiling makes smaller, cheaper silicon sufficient.</strong> That is a different claim from “our engine is fast.” It says the hardware tier you need for a given experience is partly a software choice — and right now most of the industry is buying its way past a software problem.</p><p>So the next measurements are deliberately going down a tier, not up:</p><ul><li><strong>Panther Lake iGPU</strong> — an integrated GPU in a client SoC, with a fraction of the memory bandwidth and power budget of the card above. This is the real test of whether the techniques survive when the hardware stops being generous.</li><li><strong>Nova Lake full-chip iGPU</strong>, if we can get one — the point at which an integrated part stops being a compromise tier.</li></ul><p>We are not claiming results we do not have. We have measured none of this yet, and integrated parts may well expose problems a 32 GB discrete card papers over. That is precisely why it is worth measuring.</p><p>But the target is clear enough to state. <strong>An AI PC is only real when the machine people already own runs the model well enough that they stop thinking about it</strong> — not when a tier of hardware is named after the capability. Getting there on integrated silicon is an economics problem before it is a performance problem, and the serving layer is where that economics is decided.</p><p>The same argument applies one tier further down, which is where the rest of our stack lives: a robot cannot carry a workstation card’s power budget, so vision and control run on NPUs. Different silicon, identical problem.</p><h3>The part we keep coming back to</h3><p>The headline finding is a 3.9× spread between engines on identical hardware. But the finding we’d defend hardest is the boring one: <strong>five runs of the same benchmark told us things one run could not.</strong></p><p>One run gives you a number. Three runs told us the number moves, and where. Five runs told us our harness had a hole in it. If we had shipped the first run’s 205.7 tok/s as “the” figure, we’d have been quoting the top of a distribution we hadn’t measured yet — and we’d still have a scheduler bug we didn’t know about.</p><p>A benchmark is worth what it reproduces, not what it peaks at.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OZX__cnIdq-Tl1YSdeZMOQ.png" /></figure><p><em>ARCinfer is the deliberation layer of the ARCos Physical AI stack at Circulus. It sits alongside OVeye, our vision framework, and ARCforge, our robot control stack — three inference layers running on three different compute units of the same machine. Measurements in this post cover the deliberation layer only.</em></p><p><em>If you maintain a serving stack or a backend and want to know what these numbers look like on your platform, we would like to find out too. The harness is workload-hash-pinned and the methodology transfers.</em></p><p><em>Full data — all five runs, every wave, both cold and warm cache, with the engine-by-engine breakdowns — is published as an interactive report. Raw </em><em>waves.csv and </em><em>requests.csv per engine, workload hash-pinned for reproduction.</em></p><h3>Method notes</h3><ul><li>Every engine ran its native optimal quantization format. This is the fair comparison for serving throughput, not for model quality; accuracy equivalence was verified separately.</li><li>TTFT is p95 unless stated. Throughput is wave-level: total completion tokens divided by wall-clock time for the wave.</li><li>“Intra-wave imbalance” is the fastest stream’s tok/s divided by the slowest stream’s, within the same wave.</li><li>All waves included here passed integrity validation. One request failed across the entire set (OVMS, 4 concurrent vision requests, HTTP 400) and is reported rather than dropped.</li><li>OVMS was run under two different served-model configurations across the set. The figures quoted here use the first, which is the comparable one; the second is reported separately in the full data.</li><li><strong>OVMS continuous batching was excluded from this test.</strong> OVMS ships a continuous batching mode; on this workload it failed with OUT_OF_RESOURCE and could not complete a run, so it was left out rather than reported as a degraded result. The OVMS figures here are from the mode that completed. We would re-test it against a configuration that survives the workload.</li><li>We did not test AMD hardware. The Radeon AI PRO R9700 is referenced only as the other card in the same memory tier, not as a measured comparison.</li><li>The reflex-layer latency in Chart 6 is a design target. NPU measurements for the perception and reflex layers are not yet complete and are not claimed here.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da6c7588b79a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Physical AI Doesn’t Get to Pick a Vendor]]></title>
            <link>https://medium.com/@rippertnt/physical-ai-doesnt-get-to-pick-a-vendor-b3a8337e95d1?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3a8337e95d1</guid>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[amd]]></category>
            <category><![CDATA[intel]]></category>
            <category><![CDATA[physical-ai]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Thu, 27 Aug 2026 07:02:33 GMT</pubDate>
            <atom:updated>2026-08-27T07:02:33.542Z</atom:updated>
            <content:encoded><![CDATA[<h3>We build robots on Intel, AMD and NVIDIA silicon at the same time. So we built the instrument that watches all of it — and open-sourced it. Here is what it found.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*12TH1RF5qeGsRDLSD9wmgQ.png" /></figure><h3>The 45-watt problem</h3><p>A robot is standing in front of a person. It has to see them, understand what they said, decide, and answer.</p><p>Right now, inside that machine: a vision model is running continuously on the NPU. A language model is generating a reply on the GPU. Motion control is holding a real-time loop on the CPU. All three are drawing from one power envelope, dumping heat into one chassis, and racing one clock — the roughly 300 milliseconds before a human being decides the robot is slow.</p><p>That is the actual engineering problem in Physical AI. Not model accuracy. <strong>Scheduling.</strong> Which engine runs what, at what precision, right now, under this thermal budget.</p><p>And for a long time we could not see it.</p><h3>Our silicon is heterogeneous on purpose</h3><p>Circulus builds the brain of robots — on-device AI stacks that let machines see, hear and decide without a cloud round-trip. That runs on <a href="https://pibo.circul.us/">pibo</a>, our companion robot; on Seoul city care robots and food-manufacturing robots; and on <strong>Manserro</strong>, a K-humanoid we build with Mandaro and Roboworks.</p><p>Here is what that actually runs on today:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q0WgUtN3J9tH7aDwp2vy1Q.png" /></figure><p>Where Silicon What it does <strong>Ship</strong> Intel Panther Lake (Core Ultra 358H) CPU + GPU + NPU in one package — the robot’s brain <strong>Ship</strong> Intel Arc Pro B70 Heavier on-device inference where the envelope allows <strong>Develop</strong> NVIDIA Jetson Thor Edge development and validation <strong>Develop</strong> NVIDIA RTX PRO 6000 Workstation test and iteration <strong>Train</strong> NVIDIA H100 Model fine-tuning <strong>Alternate</strong> AMD Ryzen AI Max+ 395 + Radeon AI PRO R9700 A second viable platform path</p><p>People sometimes read a list like that as indecision. It isn’t. It’s the shape of the problem.</p><p>Cloud AI got to converge on one vendor because it optimizes for one thing: throughput per dollar in a rack with effectively unlimited power. Physical AI optimizes for something else entirely — <strong>latency per watt inside a moving object</strong> — and that has produced genuinely different silicon. The NPU exists precisely because the CPU and the GPU are the wrong answer for always-on perception. Unified memory on Jetson Thor and the AI Max exists because copying a frame across PCIe is a luxury a robot doesn’t have.</p><p>So heterogeneity in this field is not a transitional phase on the way to a winner. It is a permanent property of the domain. Which means the tooling gap around it is permanent too — unless somebody closes it.</p><h3>Five tools, five unit conventions, no shared clock</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EdogLdnDMJip8t4a1-Gg_g.png" /></figure><p>When we went to profile a robot workload end to end, this is what we had:</p><ul><li>top / htop — CPU only, and blind to whether the thread landed on a performance core or a low-power one</li><li>intel_gpu_top — GPU engines only, Linux only</li><li>xpu-smi — built for data-centre GPUs; on client Arc, several fields simply return N/A</li><li>nvidia-smi — excellent, and only for NVIDIA</li><li>rocm-smi — excellent, and only for AMD</li><li>The NPU — no official CLI at all</li><li>Windows Task Manager — shows everything, exports nothing</li></ul><p>Five tools. Five unit conventions. No shared schema. No shared clock. And no way to answer the only question that matters: <strong>when the robot got slow, which engine was the bottleneck?</strong></p><p>You cannot answer that by alt-tabbing between terminals. A thermal event is 200 milliseconds wide. If your CPU sample and your GPU sample are not from the same instant, you are not looking at a system — you are looking at two unrelated graphs and guessing.</p><h3>So we built one table</h3><p><strong>XPU-TOP</strong> is a single binary that reports CPU, integrated GPU, discrete GPU and NPU with the same columns, the same units and the same timestamp — on Linux and Windows, on x86 and ARM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I6AnvPZ-LnUScT7CQQTs0g.png" /></figure><p>It is deliberately boring in shape: nvidia-smi-like, because that&#39;s the vocabulary everyone already has. And it is deliberately austere in engineering: one C++17 file, zero build dependencies, no vendor SDK linked. Utilization, clocks, memory, power, temperature, per-core-class breakdown, driver and firmware versions — streamed to CSV or JSON so a benchmark produces a time series instead of a screenshot.</p><p><strong>In our first release we opened Intel support only.</strong> That was honest scoping — Intel was what we had validated on real hardware, and shipping a monitoring tool with guessed numbers is worse than shipping nothing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*ohdl2ClD2Goe6QL8GzsmVw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4sfpMvkiYjw1y13muhLcmQ.png" /></figure><p><strong>Today we’re opening AMD and NVIDIA.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R8694SaqgsHmqUd5vGPudw.png" /></figure><p>That means Ryzen and EPYC with Zen/Zen-c core-class breakdown, Radeon and Radeon AI PRO, Ryzen AI NPUs; and GeForce, RTX PRO, datacenter cards, <strong>DGX Spark and Jetson</strong> — including the ARM64 platforms, which most x86-born tools simply don’t compile for.</p><p>One rule shaped every line of it: <strong>never invent a number.</strong> Windows has no public API for per-adapter GPU watts on Intel and AMD, so that column reads N/A rather than scaling a TDP percentage into something plausible. DGX Spark’s GB10 has no framebuffer, so its memory row is the unified pool the CPU shares — the same figure, from the same read, because that is what is true. An N/A you can trust is worth more than a number you can’t.</p><p>That rule is also why this is useful to people other than us. A measurement tool is only worth anything to a silicon vendor if it doesn’t flatter anybody.</p><h3>What we found once we could see the whole machine</h3><p>Here is the part that matters.</p><p>We are building <strong>AERO.X</strong> — an inference engine for edge deployment, the runtime under our robots’ language and vision stack. It is not released yet. And the speedups in it did not come from a clever kernel. <strong>They came from finally being able to see where the time went across engines, in one time-aligned view.</strong></p><p>XPU-TOP is the instrument. AERO.X is what the instrument found.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FZQgbwTi73WItcv8l7iNAw.png" /></figure><p>Measured on an <strong>Intel Arc Pro B70</strong>, against openvino_genai continuous-batching greedy as the baseline:</p><p>Configuration tok/s vs. baseline openvino_genai CB greedy (baseline) 31.0 1.00× Pure greedy decode 24–26 0.8× <strong>AERO.X</strong> — no draft model <strong>43.6</strong> <strong>1.41×</strong> <strong>AERO.X</strong> — 128-token context <strong>143.1</strong> <strong>4.62×</strong> <strong>AERO.X</strong> — 16,384-token context <strong>141.3</strong> <strong>4.56×</strong> <strong>AERO.X</strong> — 65,536-token context <strong>129.3</strong> <strong>4.17×</strong></p><p>Two things are worth pulling out.</p><p><strong>1.41× with no draft model at all.</strong> That’s not speculative decoding doing the work — that’s scheduling and memory movement, found by watching engine occupancy and power draw together rather than one at a time.</p><p><strong>And it holds at long context.</strong> 4.62× at 128 tokens is a nice benchmark number. 4.17× at 65,536 tokens is a <em>product</em>. Context grew 512× and throughput fell 10%. For a robot that has been in a room with a person for an hour, that difference is the entire difference between a demo and a deployment.</p><p>We’ll publish AERO.X, the methodology and the reproduction steps when it’s ready. We’re putting the numbers out now because they are the argument for why the measurement layer mattered enough to open-source first.</p><h3>Why we’re open-sourcing the instrument and not just the result</h3><p>Because the instrument is worth more to the field than to us.</p><p>Every silicon vendor building for the edge is currently being evaluated with tools written by that same vendor. That is not a conspiracy — it’s just how it happened. But it means cross-vendor comparison in Physical AI is mostly vibes, screenshots and mismatched units. Nobody benefits from that. Least of all the vendor whose part is actually better on the metric that matters.</p><p>XPU-TOP is Apache-2.0, has no vendor SDK in it, and reports every vendor with the same schema and the same honesty about what it cannot measure. If your part wins on latency per watt, this makes that legible. If it doesn’t, you’ll find out from your own engineers before you find out from a customer.</p><h3>What we’d like</h3><p>We’re a robotics company that ships on Intel, develops on NVIDIA, and keeps AMD as a live second path. We are not neutral because we’re diplomatic — we’re neutral because our bill of materials is.</p><p><strong>If you build silicon, a runtime or a driver for the edge:</strong> we will test on your hardware, we will publish what we find with the same schema we publish everyone else’s, and we will tell you plainly what our robots need that your platform doesn’t do yet. We already maintain that list internally for three vendors. It’s a more useful document than most benchmark decks. If you want AERO.X brought up on your platform ahead of release, that conversation is open too.</p><p><strong>If you’re building Physical AI:</strong> the tool is out. Reports from Panther Lake, Battlemage, Strix Point, Radeon RX 9000, RTX 50-series, DGX Spark and Jetson Thor systems are especially welcome — that’s hardware we either can’t reach or haven’t stressed the way you have.</p><p><strong>→ </strong><a href="https://github.com/circuluspibo/xpu-top"><strong>github.com/circuluspibo/xpu-top</strong></a></p><p>Apache-2.0. One C++17 file. No dependencies. Intel, AMD, NVIDIA. Linux and Windows. x86 and ARM.</p><p><strong>AERO.X</strong> — the edge inference engine those numbers came from — follows.</p><p><em>Circulus builds on-device AI for robots. When your product is a robot, the CPU/GPU/NPU split is not a benchmark — it is thermal budget, battery life, and whether a person waits for an answer.</em></p><p><em>Copyright 2026 Circulus Inc.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3a8337e95d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intel Has CPU, GPU, and NPU. Why Doesn’t It Have an NVIDIA-SMI?]]></title>
            <link>https://medium.com/@rippertnt/intel-has-cpu-gpu-and-npu-why-doesnt-it-have-an-nvidia-smi-126be766c671?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/126be766c671</guid>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Thu, 20 Aug 2026 04:02:59 GMT</pubDate>
            <atom:updated>2026-08-20T04:02:59.674Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building XPU-TOP: one table for CPU, GPU and NPU, on Linux and Windows</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6uhPInrM8miWhWYGoaVaoA.png" /></figure><p>Intel’s biggest advantage in on-device AI is also its most annoying operational problem.</p><p>With OpenVINO you can take one model and run it on the CPU, the integrated GPU, a discrete Arc card, or the NPU. Not “port it to” — <em>run it on</em>. The AUTO plugin picks a device for you. HETERO splits a single inference graph across several. MULTI runs them in parallel. One compiled model, four kinds of silicon.</p><p>At <a href="https://circul.us/">Circulus</a> we lean on that hard. Our on-device stack puts <strong>vision and real-time perception on the NPU and language models on the GPU</strong>, with the CPU handling control and orchestration. That split is not an academic preference — it is the difference between a robot that responds in 200 ms on battery and one that doesn’t.</p><p>So we tune it constantly. And for a long time we tuned it nearly blind.</p><h3>The problem: four engines, four tools, zero overlap</h3><p>Here is what monitoring an Intel AI workload actually looked like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2oc9E7QfeDfPZzVok2M73Q.png" /></figure><p><strong>top / </strong><strong>htop</strong> gives you the CPU. On a Core Ultra that is already a half-truth: Meteor Lake has performance cores, efficiency cores, <em>and</em> a two-core low-power island on the SoC tile. &quot;CPU 12%&quot; tells you nothing about whether your inference thread landed on a 4.5 GHz P-core or a 2.5 GHz LP-E core. Those are wildly different latency stories.</p><p><strong>intel_gpu_top</strong> covers the GPU. Only the GPU. Only on Linux.</p><p><strong>xpu-smi</strong> — Intel&#39;s XPU Manager — is the closest thing to an official answer, and it is genuinely good on the hardware it targets. It targets data-centre GPUs. Point it at client Arc and you get this:</p><pre>+--------------------------------------------+--------------------+--------------------+<br>| GPU  Name                  Persistence-M   | Bus-Id      Disp.A |  Volatile Uncorr. ECC |<br>| Fan            Temp  Pwr:Usage/Cap         | Memory-Usage       |  GPU-Util  Compute M. |<br>+--------------------------------------------+--------------------+--------------------+<br>|   0  Intel(R) Arc(TM) Pro B  Off           | 0000:03:00.0  Off  |            Disabled |<br>| N/A             N/A  46W / 230W            | 25MiB / 32656MiB   |   N/A       Default |<br>+--------------------------------------------+--------------------+--------------------+<br>[Error] Failed to get process count: 0x78000003 (ZE_RESULT_ERROR_UNSUPPORTED_FEATURE)</pre><p>Power works. GPU-Util is N/A. Temperature is N/A. Fan is N/A. Process enumeration throws ZE_RESULT_ERROR_UNSUPPORTED_FEATURE. And of course there is no CPU row and no NPU row, because it is a GPU tool.</p><p><strong>The NPU</strong> had no official CLI at all. The state of the art was a community Python script reading a single sysfs counter — genuinely useful, and about 200 lines that do one thing.</p><p><strong>Windows Task Manager</strong> shows all four engines, which is more than any Linux tool managed. It also has no CLI, no export, no logging, and no way to correlate a spike with the line of code that caused it.</p><p>Four tools. Four unit conventions. Four output formats. Nothing you can tail -f into a CSV while a benchmark runs.</p><p>The gap is a little absurd when you say it out loud: <strong>the platform’s headline feature is that AI runs across CPU, GPU and NPU — and there was no way to watch AI run across CPU, GPU and NPU.</strong></p><p>So we wrote one.</p><h3>XPU-TOP</h3><p><em>[terminal.png]</em></p><p>One binary. One table. CPU, integrated GPU, discrete GPU and NPU, with the same columns and the same units for every device, on Linux and on Windows.</p><pre>+---------------------------------------------------------------------------------------------+<br>| XPU-TOP 0.2.0         Devices : 5                                       Level Zero : 1.28.6 |<br>| Linux / Ubuntu 25.10 / kernel 6.17.0-41-generic                 Copyright 2026 Circulus Inc |<br>|---------------------------------------------+-----------------------+-----------------------|<br>| Idx  Name                               Type|Clk Cur /  Clk Max MHz |Mem Use /  Mem Tot MiB |<br>|      Temp        Power:Use /          Cap   |      Utilization      |  Util%   Driver       |<br>|=============================================+=======================+=======================|<br>|   0  Intel Core Ultra 7 265K            CPU |   2470 /     5600 MHz |  12095 /    62713 MiB |<br>|       56C            30.8W /       250.0W   | [###----------------] |    14%   intel_pstate |<br>|   P-core     x8    util    16%    clock   2715 /   5600 MHz                                 |<br>|   E-core     x12   util     6%    clock   2508 /   4600 MHz                                 |<br>+---------------------------------------------+-----------------------+-----------------------+<br>|   0  Intel Arc Pro B70                  GPU |    400 /     2800 MHz |     25 /    32656 MiB |<br>|       56C            45.9W /       230.0W   | [-------------------] |     0%   xe           |<br>+---------------------------------------------+-----------------------+-----------------------+<br>|   1  Intel Arrow Lake-S Graphics        GPU |    525 /     2000 MHz |    N/A /      N/A MiB |<br>|       56C             1.1W /        15.0W   | [######-------------] |    32%   i915         |<br>+---------------------------------------------+-----------------------+-----------------------+<br>|   2  Intel Arc Pro B70                  GPU |    400 /     2800 MHz |     25 /    32656 MiB |<br>|       57C            46.7W /       230.0W   | [-------------------] |     0%   xe           |<br>+---------------------------------------------+-----------------------+-----------------------+<br>|   0  Intel Arrow Lake NPU               NPU |   1533 /     1600 MHz |    412 /      N/A MiB |<br>|       N/A              N/A /          N/A   | [########-----------] |    43%   intel_vpu    |<br>+---------------------------------------------+-----------------------+-----------------------+</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tY1AhQLeeNke8885THznbg.png" /></figure><p>The layout is deliberately nvidia-smi-shaped, because that is the vocabulary everyone already has. If you have ever read an nvidia-smi header, you can read this one.</p><p>It has no dependencies. Not “few” — none. No intel_gpu_top, no xpu-smi, no Level Zero runtime, no vendor SDK, no submodules. One C++17 file, about 3,300 lines, built by a shell script.</p><h3>Design rule: never invent a number</h3><p>This turned out to be the decision that shaped everything else.</p><p>Cross-platform monitoring tools have a strong temptation to paper over gaps — substitute a nearby sensor, scale a percentage into a fake wattage, silently show 0 instead of “unknown.” That produces a table that always looks complete and is occasionally lying.</p><p>XPU-TOP reports N/A (and null in JSON) for anything the platform genuinely cannot supply, and the schema never changes shape between operating systems. Three places where that rule cost us something:</p><p><strong>Windows GPU power.</strong> D3DKMT_ADAPTER_PERFDATA has a Power field. It is tenths of a percent of TDP — not watts. We could have multiplied it by a TDP guess and printed a number. Instead the power column stays N/A on Windows and the percentage shows up as a labelled sub-unit row.</p><p><strong>Integrated GPU temperature.</strong> An iGPU has no sensor of its own. The CPU package temperature is <em>right there</em>, one file away, and on a monolithic die it would be a defensible proxy. On Meteor Lake it isn’t — the GPU tile and the compute tile are different pieces of silicon. So the temperature stays N/A, with a sub-unit row explaining why rather than an unexplained blank.</p><p><strong>Linux GPU driver version.</strong> DRM_IOCTL_VERSION returns xe 1.1.0. That is the DRM uAPI version, and it is constant for the life of the driver — it is not a build number. In-tree Linux drivers simply do not have a vendor version string the way a Windows UMD does. We report it, and we say what it is.</p><p><strong>NPU power.</strong> This one we chased hard before accepting it. The ivpu driver registers no hwmon and exposes no energy attribute — not in sysfs, not in debugfs, not as an ioctl parameter, in any kernel from 6.12 to master. RAPL has no NPU domain either; its enum has been package / core / uncore / dram / psys since Sandy Bridge. The package and psys domains genuinely <em>do</em> include NPU activity, and it would have been easy to show one of them on the NPU row. But they are SoC-wide and cannot be decomposed, so that number would have been an attribution, not a measurement. Windows has nothing either. So: N/A, with a sub-unit row that says <em>why</em>.</p><p>An N/A you can trust is worth more than a number you can&#39;t.</p><h3>Three things that were harder than expected</h3><h3>1. Hybrid cores are readable — if you stop guessing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ybx0HmKIP1FuuQAFfT-9CA.png" /></figure><p>The obvious way to find P-cores versus E-cores is to cluster by maximum frequency. It works until it doesn’t: Turbo Boost Max 3.0 gives certain P-cores a higher ceiling than their siblings, and you end up inventing a core class that doesn’t exist.</p><p>The kernel already knows. Linux exposes the hybrid PMU topology directly:</p><pre>cat /sys/devices/cpu_core/cpus   # 0-15   -&gt; P-cores<br>cat /sys/devices/cpu_atom/cpus   # 16-21  -&gt; E-cores</pre><p>Windows has an equivalent in GetLogicalProcessorInformationEx, whose PROCESSOR_RELATIONSHIP carries an EfficiencyClass byte — highest class is P.</p><p>That leaves the third tier. Meteor Lake, Arrow Lake and Lunar Lake put two extra E-cores on the SoC tile, outside the LLC ring, and neither interface labels them separately. The tell is architectural: <strong>the low-power island has no L3 cache</strong>. So an Atom core with no level-3 entry in /sys/devices/system/cpu/cpuN/cache/ is LP-E. It&#39;s a structural property, not a heuristic threshold.</p><h3>2. Windows will give you the GPU clock — through a door nobody uses</h3><p>Every “you can’t get GPU clocks on Windows” answer online is <em>almost</em> right. There is no performance counter for it. But D3DKMT — the kernel-mode thunk layer behind DXGI — has KMTQAITYPE_NODEPERFDATA (engine clock, current and max) and KMTQAITYPE_ADAPTERPERFDATA (temperature in tenths of a degree, fan RPM, memory clock).</p><p>They are exported from gdi32.dll, documented for driver developers, and reachable from user mode with GetProcAddress. MinGW ships no d3dkmthk.h, so the structs get restated by hand — but that is the whole cost. It closed the largest remaining gap between our Linux and Windows output.</p><p>Vendors implement these DDIs optionally, so a field can still come back empty. Which is fine: N/A is a legitimate answer.</p><h3>3. The bug that ate CPU power</h3><p>Our first Linux run showed N/A for CPU power, and no RAPL rows at all — even though the canonical one-liner worked perfectly:</p><pre>E1=$(cat /sys/class/powercap/intel-rapl:0/energy_uj); sleep 1<br>E2=$(cat /sys/class/powercap/intel-rapl:0/energy_uj)<br>echo &quot;scale=2; ($E2 - $E1) / 1000000&quot; | bc      # -&gt; watts</pre><p>Our arithmetic was identical. The bug was one line earlier, in discovery:</p><pre>if (!file_exists(p + &quot;/energy_uj&quot;)) continue;   // ← discards the whole domain</pre><p>energy_uj has been mode 0400 since <a href="https://nvd.nist.gov/vuln/detail/CVE-2020-8694">CVE-2020-8694</a>. Running unprivileged, that check failed and threw away the <em>entire</em> domain — including constraint_0_power_limit_uw, the power limit, which is world-readable. So a non-root user lost the power cap for no reason and got no explanation for the missing watts.</p><p>Now the domain survives, the limit shows without root, and the header says exactly what is wrong and how to fix it.</p><p>Hunting that also surfaced a quieter bug: on machines where both the MSR and MMIO RAPL backends are registered, /sys/class/powercap/intel-rapl:0 and intel-rapl-mmio:0 <strong>both</strong> publish a domain named package-0. Summing every domain whose name starts with package reported double the real package power. Deduplicating by domain name fixed it.</p><h3>The bugs a real machine finds</h3><p>Everything above was written against a laptop. Then a colleague ran it on a workstation — Core Ultra 7 265K, two Arc Pro B70 32 GB cards, plus the integrated GPU — and three things were wrong at once.</p><p><strong>Every GPU was called “Intel Graphics.”</strong> Both B70s and the iGPU, identical strings. The name comes from /usr/share/hwdata/pci.ids, and that database lags months behind new silicon — Battlemage was simply not in the installed copy. xpu-smi gets the real name from the Level Zero runtime, which we deliberately do not depend on. The fix was a small curated table of recent Intel GPU device IDs, consulted only when pci.ids returns something generic. Arc Pro B70 is 8086:e223, if you are curious. (Arc Pro B60 is 8086:e211 — and they are different dies, not two bins of one.)</p><p><strong>VRAM was </strong><strong>N/A on 32 GB cards.</strong> Our code looked for /sys/class/drm/cardN/device/tile0/physical_vram_size_bytes. That path does not exist in mainline Linux — not in 6.17, not anywhere. Reading the kernel source is unambiguous: xe_tile_sysfs.c creates the tile0 kobject and nothing else, and i915 has never had mem_info_vram_* either (those are amdgpu attributes). <strong>There is no sysfs interface for Intel VRAM.</strong></p><p>The supported route is a query ioctl on the render node — DRM_IOCTL_XE_DEVICE_QUERY with DRM_XE_DEVICE_QUERY_MEM_REGIONS, or DRM_IOCTL_I915_QUERY with DRM_I915_QUERY_MEMORY_REGIONS. Both are DRM_RENDER_ALLOW, so no root needed for the total. Both are two-pass: ask with a zero size, get told how big the answer is, allocate, ask again. xe is stricter than i915 — it rejects the second call unless the size matches <em>exactly</em>, and it rejects any nonzero reserved[] field.</p><p>There is a catch worth knowing about. <strong>used is privileged in both drivers, and they fail differently.</strong> xe zeroes the field for unprivileged callers; i915 returns unallocated_size == probed_size. So without CAP_PERFMON you cannot distinguish &quot;the GPU is idle&quot; from &quot;you are not allowed to know.&quot; We report the total, leave used as N/A, and print a sub-unit row saying which it is — rather than showing a confident 0 MiB.</p><p><strong>The power cap was exactly double.</strong> We showed 460 W where xpu-smi showed 230 W. Not a unit error — a semantics error:</p><pre>if (power1_max &gt; 0)        cap = power1_max;<br>else if (power1_crit &gt; 0)  cap = power1_crit;   // ← wrong fallback</pre><p>In xe_hwmon.c, power1_max is the card&#39;s sustained PL1 — the number everyone means by &quot;TDP&quot;. But <strong>a read of 0 does not mean zero watts; it means PL1 is disabled</strong>, and the driver returns PL_DISABLE (which is literally 0) in that case. Our fallback then grabbed power1_crit, which is not a power limit at all — it is the I1 overcurrent protection point, read from a completely different pcode mailbox, and on this card it happens to sit at roughly twice PL1.</p><p>Two numbers, both real, one of them meaningless as a cap. The fix orders the fallbacks by what they actually mean — power1_max, then power1_rated_max, then power1_cap (the PL2 burst limit) — and never power1_crit, which now appears as its own labelled sub-unit row so the information is still there without pretending to be a TDP.</p><p>That is the whole lesson of this project in one bug: <strong>the file existed, the read succeeded, the units were right, and the number was still wrong.</strong> Hardware monitoring is mostly semantics.</p><h3>Streams, because a screenshot is not data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hZisNH83Iqh1dXvKkaRryg.png" /></figure><p>A live table is for humans. Tuning a device-placement policy needs a log.</p><pre>xpu-top -f csv -o run.csv          # 31 columns, flushed every sample<br>xpu-top -f json | your_collector   # NDJSON, one object per sample</pre><p>Same 31 columns on both platforms, N/A where a platform can&#39;t help, flushed per sample so it works through a pipe. Start it, run your benchmark, and you have a time series you can line up against your own traces — including per-core-class utilization, so you can see the scheduler move work between P and E cores while your model runs.</p><h3>Where the numbers come from</h3><p><em>[architecture.png]</em></p><p>Everything is read from interfaces the OS already ships — sysfs, procfs, DRM ioctls and RAPL on Linux; PDH, DXGI and D3DKMT on Windows. Full mapping in the repo.</p><h3>Try it!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k4FsbG2WKQvqMsJ7Txv_Ow.png" /></figure><pre>git clone https://github.com/circuluspibo/xpu-top.git<br>cd xpu-top &amp;&amp; ./build.sh<br>sudo ./xpu-top          # sudo unlocks RAPL watts</pre><p>Any Linux with kernel 6.2+ (6.10+ for NPU utilization) and Windows 10/11. Apache-2.0.</p><p><a href="https://github.com/circuluspibo/xpu-top/releases/tag/0.2.0">Release First release for Linux &amp; Windows · circuluspibo/xpu-top</a></p><p><strong>→ </strong><a href="https://github.com/circuluspibo/xpu-top"><strong>github.com/circuluspibo/xpu-top</strong></a></p><h3>Why a robotics company wrote a monitoring tool</h3><p>Circulus builds the brain of robots — on-device AI stacks that let machines see, hear and decide without a cloud round-trip. That runs on <a href="https://pibo.circul.us/">pibo</a>, our companion robot; on Seoul city care robots and food-manufacturing robots, a K-humanoid we build with Mand.ro and Roboworks. We work with Intel on next-generation physical-AI platforms.</p><p>When your product is a robot, the NPU/GPU/CPU split is not a benchmark — it is thermal budget, battery life, and whether a person waits for an answer. We needed to see it, so we built the instrument. It was useful enough internally that keeping it to ourselves seemed silly.</p><p>It is early — version 0.2.0, and there is hardware we have not touched. If you are building on-device or physical AI on Intel silicon, issues and PRs are very welcome. We would especially like reports from Lunar Lake, Arrow Lake and Battlemage systems.</p><p><em>XPU-TOP is Apache-2.0 licensed. Copyright 2026 Circulus Inc.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=126be766c671" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI That Nobody Has to Ask Permission to Use]]></title>
            <link>https://medium.com/@rippertnt/ai-that-nobody-has-to-ask-permission-to-use-3b8087dc6ebb?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b8087dc6ebb</guid>
            <category><![CDATA[on-device-ai-pc]]></category>
            <category><![CDATA[edge-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[intel-core-ultra]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 10:21:45 GMT</pubDate>
            <atom:updated>2026-08-16T02:57:37.884Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DFihb9uNvOMYRRF_nuW0hg.png" /></figure><h3>On Korea’s 81st Liberation Day, we are open-sourcing CanversON — a full generative AI studio that runs on an ordinary laptop, with the network cable pulled out.</h3><h3>⬇️ <a href="https://github.com/circuluspibo/CanversON-personal/releases/tag/20260815-0001">Download CanversON v0.1.0 — first alpha release</a></h3><blockquote><em>Windows 10/11 · Intel Core Ultra or Arc · MIT licensed · </em><a href="https://github.com/circuluspibo/CanversON-personal"><em>source on GitHub</em></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8TBYuSH917wy_0_nSIpQaA.png" /></figure><blockquote><em>[PHOTO: PIBO and a Core Ultra laptop running CanversON on a dune outside Dubai, GITEX. Caption: “No signal, no account, no subscription. The laptop is doing all of it.”]</em></blockquote><p>Today is <strong>Gwangbokjeol</strong> — Korea’s Liberation Day, the 81st. This year’s official theme is <em>“The Irreplaceable Republic of Korea Where Tomorrow Shines Brighter.”</em></p><p>Liberation is a word about <strong>access</strong>: who gets to participate in their own future, and who is locked out by circumstances they did not choose. That is not a metaphor we reached for. It is the problem we have been working on for two years, and it is why <strong>CanversON</strong> is free and open source as of today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ppUUmb3AoPIlkr_rrEBT8w.jpeg" /></figure><h3>The desert taught us the wrong lesson first</h3><p>We took the photo above at <strong>GITEX in Dubai</strong>, out on the dunes, running a proof of concept built with <strong>LG Electronics</strong> and <strong>Intel</strong> on the Meteor Lake platform. Image generation, conversation, speech — all of it on a laptop, with nothing upstream.</p><p>We went in treating “offline” as a <strong>constraint</strong>: the degraded mode, the unhappy path you engineer around. The system ran, and the lesson we took home was narrow — <em>good, we survived the bad case.</em></p><p>That was the wrong lesson. It took another two years to see why.</p><h3>Robotics made it impossible to miss</h3><p>We went deep into robotics after that — <strong>ARCos</strong>, our platform for humanoid and quadruped systems. Robots are unsentimental about inference. A robot cannot wait 300ms for a datacenter round trip to decide whether the thing ahead is a stair or a ledge.</p><p>That work forced a specific discipline: <strong>put each workload on the engine built for it.</strong> We wrote about this in detail when we <a href="https://medium.com/@rippertnt/intel-panther-lake-vs-nvidia-jetson-thor-which-brain-should-power-your-humanoid-robot-2f279fec7c74">benchmarked Intel Panther Lake against NVIDIA Jetson Thor</a> for humanoid use. The headline finding was blunt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kvye9sUy5upKw_bSjh8r3w.png" /></figure><p><strong>CanversON inherits that scheduling discipline directly.</strong> Image generation and the vision-language model run on the GPU; speech synthesis is pinned to the <strong>CPU on purpose</strong> so it never contends for GPU memory; the NPU is available for low-power always-on work. The heterogeneous-compute optimization we built for robots is the same optimization that makes a 4B model behave predictably on a school laptop.</p><p>And somewhere in that work, the thing we had misread in the desert finally landed:</p><blockquote><strong><em>On-device AI is not a fallback for people with bad internet. It is the only delivery model where the right to use AI does not depend on someone else’s permission.</em></strong></blockquote><p>No API key to revoke. No subscription to lapse. No region lock, no rate limit, no terms change. The capability is <em>in the building</em>.</p><h3>Who is actually locked out</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bXKrbm72K-VZHOUgKYsIQ.png" /></figure><p>The ITU counts <strong>6 billion people online and 2.2 billion still offline</strong>, with <strong>96% of the offline in low- and middle-income countries</strong>. Internet use in low-income countries: <strong>23%</strong>. And where there <em>is</em> a signal, a per-seat AI subscription priced for a developed market can exceed a teacher’s monthly wage.</p><p><strong>Korea proves connectivity alone does not fix it.</strong> The AI digital textbook program — 533.3 billion won in a single year, with industry-wide investment near 800 billion won — launched in March 2025 and was effectively unwound within months. Adoption fell from 37% to 19%, and the National Assembly stripped the textbooks of official status that August. The post-mortems cite a rushed 12-month build, technical failures that ate class time, teacher workload, and unresolved questions about children’s data leaving the classroom.</p><p>Read that list again. Most of those failures are <strong>architectural</strong>. A tool that phones home is a tool that can stall mid-lesson, that creates a data-protection problem by construction, and that costs money every time a child is curious.</p><p>The UN has been direct about the trajectory. In July 2026 Secretary-General António Guterres warned that AI “could widen global inequality instead of advancing sustainable development,” noting that <strong>“one-third of humanity is still offline”</strong> while compute and expertise concentrate in a few countries and companies.</p><p><strong>And sovereign models do not close this particular gap.</strong> Korea is investing seriously in AI independence — the Sovereign AI Foundation Model project advanced LG AI Research, SK Telecom and Upstage in its January 2026 first round. That matters. But a sovereign frontier model is a <em>datacenter</em> artifact: it needs GPUs, power and a serving stack. Korea’s own <strong>2026 AI Action Plan</strong> names the destination — an <strong>“AI basic society” (인공지능 기본사회)</strong> where every citizen shares in AI’s benefits, explicitly including <strong>reducing the gap between the capital region and the provinces</strong>.</p><p>Sovereign models answer <em>whose</em> AI it is. They do not answer <em>whether the kid in the room can reach it today.</em></p><p>A 4B model on an integrated GPU answers that one.</p><h3>What CanversON does</h3><p>Everything below runs locally on an Intel Core Ultra laptop or an Arc GPU. No datacenter, no account, no network.</p><p><strong>Image : </strong>Text-to-image, sketch-to-image, camera-to-portrait. 28 styles, 1:1 / 16:9 / 9:16. (Including sketch to image, face to image)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rc7bUauP7DxMOM_bxHONbQ.png" /></figure><p><strong>Chat : </strong>Vision-language model with document and image attachments. 35 personas — professional roles, historical figures, character chat. Reasoning mode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6kVFVkJ_tQWG172HtiE9eA.jpeg" /></figure><p><strong>Voice : </strong>Speech synthesis in <strong>32 languages</strong> (66 voice packs) and speech-to-text via Whisper large-v3-turbo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*945Sdcm6ns6U5WYcsfijPg.jpeg" /></figure><p><strong>Gallery / Stats : </strong>Local history, live CPU / RAM / GPU telemetry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tU5HMG5wP_pySg0zj0gJvQ.jpeg" /></figure><p>The interface itself ships in 8 languages: Korean, English, Japanese, Chinese, Spanish, French, German, Vietnamese.</p><h3>The 32 speech languages</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*njv7LoeIAEMae5xdtIOGkw.png" /></figure><p>This list is the argument, not a feature bullet. Look at where these languages are spoken:</p><p>Khmer, Sinhala, Tamil, Telugu, Urdu, Mongolian, Filipino — these are not the languages a cloud AI product prioritizes. They are spoken by hundreds of millions of people, disproportionately in exactly the regions on the wrong side of the chart above. Speech recognition covers roughly 99 languages via Whisper.</p><h3>How it differs from the AI PC apps you already have</h3><p><strong>The hardware floor is the whole argument.</strong> NVIDIA’s Project G-Assist requires an RTX 30/40/50 desktop GPU with <strong>12GB+ of VRAM</strong>, is English-only, and is scoped mostly to PC and game control. Intel’s AI Playground wants an <strong>8GB</strong> Arc card or a Core Ultra. CanversON runs on the <strong>integrated GPU</strong> — the machine a school already has.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gu6jsiTcGaO6MlqG_zzhkw.png" /></figure><p><strong>One runtime, not four.</strong> Intel’s AI Playground is the closest comparison and a genuinely good piece of work, but look at its architecture: <strong>llama.cpp (Vulkan)</strong> for language, <strong>ComfyUI</strong> for images, <strong>OpenVINO</strong> alongside. Three inference stacks, three memory allocators, three sets of failure modes.</p><p>CanversON runs <strong>every model through OpenVINO GenAI</strong>. That buys three concrete things: <strong>one memory pool</strong> (so the model manager knows exactly what is resident and what to evict — it keeps everything loaded above 10% free RAM and sheds the inactive group below it), <strong>one compile cache</strong>, and <strong>one precision story</strong> (f16 inference hint so nothing silently falls back to fp32, u4 KV-cache quantization on the chat model, LATENCY performance hint because a classroom is one user waiting, not a batch server).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*V8G5J5N6-0SW3QJAa91Plw.png" /></figure><p>AMD’s GAIA is an honest open-source effort but is <strong>text-only</strong> — no image generation, no speech — and wants a Ryzen AI 300-series NPU to be worth running.</p><p><strong>Nothing leaves the device.</strong> No API keys, no telemetry, no uploads. Output lives in %LOCALAPPDATA%\CanversON. For a school that is not a feature, it is a <strong>precondition</strong> — and precisely the box the Korean textbook rollout could not tick.</p><h3>This is not a new experiment</h3><p>The version we are open-sourcing today is an alpha. The thing itself is not.</p><ul><li><strong>October 2024</strong> — Circulus presented an on-device AI notebook built on language, image and speech models at the <strong>2024 On-Device AI Challenge</strong> results event, held at LG Science Park in Seoul.</li><li><strong>October 2024</strong> — CanversON was <a href="https://www.mt.co.kr/industry/2024/10/17/2024101710430729469">deployed with Gwangju City</a> at the <strong>National Asian Culture Center’s Children’s Culture Division</strong>, running multilingual guidance, AI multilingual picture-book creation for children, and role-play with our PIBO robot.</li><li><strong>Today</strong> — CanversON is listed in <a href="https://builders.intel.com/ecosystem-engagement/solution-hub/edge-ai-catalog/partner-spotlight/canverson-216">Intel’s Edge AI partner catalog</a>, described by Intel as turning “Core Ultra-powered AI PCs into fully capable, on-device generative AI workstations for education and enterprise environments,” using CPU, GPU and NPU acceleration through OpenVINO.</li></ul><p>Two years of classrooms and a desert, in other words — then the source code.</p><h3>What is free, and what is not</h3><p><strong>CanversON Personal is free and open source. Permanently. MIT licensed.</strong> Individuals, students, teachers, schools, NGOs — no cost, no account, no telemetry, no seat count.</p><p><strong>The commercial line is a different product for a different machine.</strong> On workstation-class GPUs, the same engine becomes an <strong>AI Factory</strong>: synthesizing robot training data — images, video, and action sequences — and running <strong>manager agents that supervise fleets of robots on a factory floor</strong>. That is the Physical AI path out of ARCos, and it is where Circulus makes its value.</p><p>It is a fair trade. Enterprises with GPU budgets fund the version that runs on a school laptop.</p><h3>Roadmap</h3><p><strong>v0.1.0 — alpha.</strong> It works, it is in use, and it will have rough edges.</p><ul><li><strong>September 2026</strong> — Agentic AI: multi-step tasks, tool use, workflows that run to completion on-device.</li><li><strong>October 2026</strong> — Creation tools for real work: document, presentation and media production for individuals, beyond classroom exercises.</li></ul><h3>Why today</h3><p>Liberation Day marks the end of a period when Koreans did not get to decide their own future. Shipping an offline AI studio is not remotely comparable — but the shape rhymes, and it is worth naming: <strong>a technology this consequential should not require anyone’s permission to use.</strong></p><p>2.2 billion people are offline. 96% of them are in low- and middle-income countries. For every one of them the current delivery model is a closed door, and the sovereign-model race does not open it either.</p><p>An ordinary laptop, running a 4B model, with the cable pulled out, opens it today.</p><p><strong>⬇️ </strong><a href="https://github.com/circuluspibo/CanversON-personal/releases/tag/20260815-0001"><strong>Download the first alpha release</strong></a> · <a href="https://github.com/circuluspibo/CanversON-personal"><strong>Source on GitHub</strong></a> — MIT licensed.</p><p>Run it in your classroom, fork it, translate it, ship it somewhere we have never been.</p><p><em>CanversON is built by </em><a href="https://circul.us/"><em>Circulus Inc.</em></a><em> — the robotics and AI company behind the companion robot </em><a href="https://pibo.circul.us/"><em>PIBO</em></a><em> and the ARCos platform. We especially welcome language packs, hardware reports from machines we have never touched, and bug reports from environments harsher than a desert.</em></p><h3>Appendix — Sources</h3><p><strong>Liberation Day</strong></p><ol><li>SBS — <a href="https://news.sbs.co.kr/english/article.do?news_id=N1008706747">South Korea Marks 81st Liberation Day</a></li></ol><p><strong>Access and the AI divide</strong> 2. ITU — <a href="https://www.itu.int/en/mediacentre/Pages/PR-2025-11-17-Facts-and-Figures.aspx">Facts and Figures 2025, press release</a> · <a href="https://www.itu.int/itu-d/reports/statistics/facts-figures-2025/">full report</a> — 6B online / 2.2B offline; 96% of offline in LMICs; 23% use in low-income countries; 5G 84% vs 4% 3. UN News, 17 Jul 2026 — <a href="https://news.un.org/en/story/2026/07/1167965">Guterres: AI must be shaped by ‘all of humanity’</a> 4. UN — <a href="https://www.un.org/digital-emerging-technologies/sites/www.un.org.digital-emerging-technologies/files/A_79_966-EN.pdf">Global Digital Compact</a> 5. EJIL:Talk! — <a href="https://www.ejiltalk.org/the-first-united-nations-general-assembly-resolution-on-artificial-intelligence/">The First UN General Assembly Resolution on AI</a></p><p><strong>Korean policy</strong> 6. Government of Korea — <a href="https://www.aikorea.go.kr/web/board/brdDetail.do?menu_cd=000011&amp;num=328">AI Action Plan 2026 / 대한민국 AI 행동계획</a> — “AI basic society”, regional AI hubs 7. Korea Herald — <a href="https://www.koreaherald.com/article/10656367">LG, SKT, Upstage advance in Korea’s sovereign AI project</a> (Jan 2026) 8. Korea.net / MSIT — <a href="https://www.korea.net/Government/Briefing-Room/Press-Releases/view?articleId=8191&amp;type=O&amp;insttCode=A110439">Five teams selected for the Sovereign AI Foundation Model project</a> 9. Cooley — <a href="https://www.cooley.com/news/insight/2026/2026-01-27-south-koreas-ai-basic-act-overview-and-key-takeaways">South Korea’s AI Basic Act</a> — in force 22 Jan 2026</p><p><strong>AI in Korean classrooms</strong> 10. Rest of World — <a href="https://restofworld.org/2025/south-korea-ai-textbook/">South Korea’s AI textbooks fail after rushed rollout</a> — 37% → 19% adoption 11. Korea Herald — <a href="https://www.koreaherald.com/article/10546695">South Korea pulls plug on AI textbooks</a> — 533.3B won in one year; official status revoked 4 Aug 2025 12. Korea Times — <a href="https://www.koreatimes.co.kr/southkorea/society/20250610/ai-textbooks-targeted-for-phaseout-under-south-koreas-new-administration">AI textbooks targeted for phaseout</a></p><p><strong>CanversON track record</strong> 13. 전자신문 / ETNews — <a href="https://www.etnews.com/20241023000327">2024 온디바이스 AI 챌린지 성과발표회</a> (23 Oct 2024, LG Science Park) 14. 머니투데이 / MoneyToday — <a href="https://www.mt.co.kr/industry/2024/10/17/2024101710430729469">서큘러스, AI PC 탑재 가능 솔루션 ‘캔버스-온’ 광주시에 공급</a> (17 Oct 2024) 15. Intel — <a href="https://builders.intel.com/ecosystem-engagement/solution-hub/edge-ai-catalog/partner-spotlight/canverson-216">CanversON, Edge AI Partner Spotlight</a></p><p><strong>Comparable local AI applications</strong> 16. Intel — <a href="https://github.com/intel/ai-playground">AI Playground (GitHub)</a> — llama.cpp (Vulkan) + ComfyUI + OpenVINO 17. AMD — <a href="https://www.amd.com/en/developer/resources/technical-articles/gaia-an-open-source-project-from-amd-for-running-local-llms-on-ryzen-ai.html">GAIA</a> — text-only, Ryzen AI 300 series 18. NVIDIA — <a href="https://www.nvidia.com/en-us/geforce/news/g-assist-ai-companion-for-rtx-ai-pcs/">Project G-Assist</a> — RTX 30/40/50 desktop, 12GB+ VRAM, English only</p><p><strong>Heterogeneous compute</strong> 19. Circulus — <a href="https://medium.com/@rippertnt/intel-panther-lake-vs-nvidia-jetson-thor-which-brain-should-power-your-humanoid-robot-2f279fec7c74">Intel Panther Lake vs. NVIDIA Jetson Thor: Which “Brain” Should Power Your Humanoid Robot?</a></p><p><strong>Runtime and models</strong> 20. <a href="https://github.com/openvinotoolkit/openvino.genai">OpenVINO GenAI</a> · <a href="https://huggingface.co/black-forest-labs/FLUX.2-klein-4B">FLUX.2 klein 4B</a> · <a href="https://github.com/openai/whisper">OpenAI Whisper</a></p><p><em>The LG Electronics / Intel proof of concept and its GITEX desert validation are described from our own first-hand involvement; that engagement was not publicly documented.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b8087dc6ebb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The $949 GPU That Embarrassed a $3,099 Rival — Intel ARC Pro B70 32GB]]></title>
            <link>https://medium.com/@rippertnt/the-949-gpu-that-embarrassed-a-3-099-rival-intel-arc-pro-b70-32gb-d02a6ea2e8d7?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d02a6ea2e8d7</guid>
            <category><![CDATA[amd]]></category>
            <category><![CDATA[intel]]></category>
            <category><![CDATA[openvino]]></category>
            <category><![CDATA[intel-arc]]></category>
            <category><![CDATA[nvidia]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 23:26:35 GMT</pubDate>
            <atom:updated>2026-05-01T11:32:24.787Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dx-k-mIR3NEl_kfFZAOCVg.png" /></figure><p><em>How Intel’s budget pro GPU beat AMD’s $1,349 card — and embarrassed a $3,099 rival on 32B AI inference</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ROmJN8UFEScfkreF4aDO8g.png" /></figure><p>The AI infrastructure investment wave of 2026 is unlike anything the semiconductor industry has seen. <strong>Hyperscalers are on track to spend $720 billion in capital expenditures this year</strong>, and — for the first time in history — inference workloads now account for more than half of all AI compute spend, overtaking training.</p><p>Meanwhile, Intel’s stock has surged <strong>+124% year-to-date</strong>, closing at $93.29 on April 29 after a blowout Q1 2026 earnings beat ($13.6B revenue, +7% YoY). CEO Lip-Bu Tan called CPUs <em>“the indispensable foundation of the AI era.”</em> Intel’s data center division grew 22% year-over-year to $5.1B.</p><p>I ran a full benchmark of three professional-grade GPUs on <strong>Qwen3-VL-32B-Instruct</strong> — a 32-billion parameter vision-language model — with all power figures measured directly from hardware sensors. No TDP assumptions. No synthetic tests.</p><p>The result? Intel’s most affordable pro GPU <strong>outperforms AMD on throughput despite having lower bandwidth</strong>, beats NVIDIA on every value metric, and at scale — three cards for ~$3,000 — delivers <strong>131% more throughput</strong> than a single NVIDIA RTX Pro 4500.</p><blockquote>“The CPU is reinserting itself as the indispensable foundation of the AI era.”<em> — Lip-Bu Tan, Intel CEO, Q1 2026 Earnings Call</em></blockquote><h3>The Three Contenders</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*avjZt0h7kXVz6_XHK-lKeg.png" /></figure><p>Each card runs its optimal inference stack on the same model:</p><ul><li><strong>Intel Arc Pro B70</strong> ($949) — OpenVINO.genai / AWQ — 11 measured runs</li><li><strong>AMD Radeon AI PRO R9700</strong> ($1,349) — llama.cpp ROCm / GPTQ — 10 measured runs</li><li><strong>NVIDIA RTX Pro 4500 Blackwell</strong> ($3,099) — vLLM / NVFP4–9 steady runs (iter 1 warmup excluded)</li></ul><p>All power figures are measured by hardware GPU sensor under inference load — not TDP spec.</p><h3>Finding #1: The Numbers Don’t Lie</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KLs4QmTKGB-Qjo4CygYgPQ.png" /></figure><p>The Intel Arc Pro B70 — the cheapest card at $949 — delivers <strong>27.46 TPS</strong>, beating the $1,349 AMD R9700 (23.90 TPS) by nearly 15%.</p><p>Here’s the paradox: <strong>AMD has 34.7% more memory bandwidth</strong> (819 GB/s vs 608 GB/s). For 32B dense model inference, which is almost entirely memory-bandwidth-bound, this should mean AMD wins. It doesn’t.</p><p>Intel’s OpenVINO.genai + AWQ combination extracts significantly more throughput per GB/s than AMD’s llama.cpp ROCm + GPTQ on RDNA 4. The hardware advantage is completely negated — and reversed — by a roughly 50 percentage point software stack efficiency gap.</p><p>On power: AMD draws <strong>290W</strong> for the <em>lowest</em> TPS. RTX Pro 4500 runs at a very efficient 198.7W. B70 sits at 228.6W, within its 230W TDP. The B70 vs RTX Pro 4500 electricity gap is only <strong>$31/year</strong> — meaning it would take <strong>71 years</strong> of power savings to recover the $2,150 price premium.</p><h3>Finding #2: Value &amp; Efficiency Ratios</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VLSnWsL20yWN2jZRczOSyA.png" /></figure><p><strong>Price value (TPS per $100 invested):</strong></p><ul><li>Intel B70: <strong>2.89</strong> — benchmark leader</li><li>AMD R9700: 1.77–39% worse than Intel</li><li>NVIDIA RTX Pro 4500: 1.15<strong>–3.74× worse than Intel</strong></li></ul><p>The $3,099 NVIDIA card posts the worst price efficiency despite the highest absolute TPS. Paying 3.26× more for 1.30× the throughput is a losing trade on this metric.</p><p><strong>Power efficiency (TPS per 100W):</strong> NVIDIA leads at 17.91 — NVFP4 on Blackwell is genuinely impressive. The B70 at 12.01 is 45.8% more efficient than the AMD R9700 (8.24), which draws 290W for the lowest throughput in the group.</p><p><strong>Stability:</strong> All three are production-grade. RTX Pro 4500: ±0.047 TPS. AMD: ±0.057. B70: ±0.161.</p><h3>The Radar: Full Competitive Profile</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-RfP5vxoWkNHHM_RCaMSNA.png" /></figure><p>The radar normalizes every metric to best-in-group (100) across six dimensions:</p><ul><li><strong>Intel B70</strong> scores <strong>100 on Price Value</strong> and leads on TCO efficiency. Software maturity (55/100) is the only genuine weakness — OpenVINO.genai covers fewer models than CUDA/vLLM.</li><li><strong>AMD R9700</strong> is penalized by its software ceiling. The bandwidth advantage doesn’t translate, and 290W for the lowest TPS gives the worst power efficiency score (46/100).</li><li><strong>NVIDIA RTX Pro 4500</strong> scores 100 on Power Efficiency, Token Cost, and Software Maturity — but its $3,099 price drops Price Value to 40/100, the lowest in the group.</li></ul><h3>The TCO Story: 1, 3, and 5 Years</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_2_o7xAMf2SXiMPesPwwBw.png" /></figure><p>The B70 saves <strong>$1,993 vs NVIDIA over 5 years</strong>, while delivering higher throughput for the full operating life. The RTX Pro 4500’s electricity advantage ($31/year vs B70) is real — but mathematically inconsequential at these price levels.</p><h3>The Scale-Out Case: ~$3,000 Budget</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b4UVL_0jusVgoOAIl-NZ7w.png" /></figure><p>With a ~$3,000 budget and 3 PCIe ×16 slots:</p><p><strong>3× Arc Pro B70 at $2,847 = 82.4 combined TPS</strong> — 131% more than a single RTX Pro 4500 at essentially the same price.</p><p>This is the decisive argument for inference cluster builders on a budget. The tradeoffs are real: 3× B70 requires 3 PCIe ×16 slots, a multi-GPU capable motherboard, and a 750W+ PSU. For single-card deployments, RTX Pro 4500’s 199W footprint wins. But for maximum throughput per dollar, the math strongly favors Intel’s scale-out strategy.</p><h3>The Investment Context</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zhtrXOK9zaDQtXdOfxe0VQ.png" /></figure><p>Intel’s stock has rallied from ~$18 (April 2025) to <strong>$93.29</strong> — the best single-day performance since 1987 (+24%) on Q1 2026 earnings. Multiple analyst price targets now sit at $95–$111.</p><p>The structural market shift is equally important: <strong>AI inference spend ($20.6B) surpasses training spend ($9.0B)</strong> in cloud infrastructure for the first time. Companies are done experimenting and running AI in production at scale — which favors stacks optimized for cost-efficient inference.</p><p>The Arc Pro B70 achieving 27.46 TPS on a 32B VLM — outperforming AMD’s higher-bandwidth hardware — is evidence that Intel’s OpenVINO software investment is beginning to pay off at the hardware level.</p><h3>Summary</h3><p><strong>For value-maximizing Linux inference:</strong> ✅ <strong>Intel Arc Pro B70 ($949)</strong> — 2.89 TPS/$100 · $1,189 yr-1 TCO · 81.6ms TTFT</p><p><strong>For scale-out at ~$3,000:</strong> ✅ <strong>3× Arc Pro B70 ($2,847)</strong> — 82.4 combined TPS · 131% more than 1× RTX Pro 4500</p><p><strong>For CUDA / ECC / ISV requirements:</strong> ⚡ <strong>NVIDIA RTX Pro 4500 ($3,099)</strong> — 35.59 TPS · 17.91 TPS/100W · $0.186/1M tokens</p><p><strong>Avoid for single-card 32B inference:</strong> ❌ <strong>AMD R9700 ($1,349)</strong> — 290W for lowest TPS · GPTQ underutilizes RDNA 4</p><p><em>Methodology: All benchmark data hardware-measured. Sources: intel_arc_b70_ov.csv (11 iter) · r9700_rocm.csv (10 iter) · nvfp4_rtxpro4500_metrics.csv (10 iter, warmup excl.). TCO: $0.12/kWh × 8,760h/yr. INTC data: CNBC, StocksToTrade April 2026. Inference market: Market Clarity, Deloitte 2026. Not financial advice.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d02a6ea2e8d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intel Panther Lake vs. NVIDIA Jetson Thor: Which “Brain” Should Power Your Humanoid Robot?]]></title>
            <link>https://medium.com/@rippertnt/intel-panther-lake-vs-nvidia-jetson-thor-which-brain-should-power-your-humanoid-robot-2f279fec7c74?source=rss-12f203d2f9c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f279fec7c74</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[panther-lake]]></category>
            <category><![CDATA[physical-ai]]></category>
            <category><![CDATA[humanoid-robot]]></category>
            <category><![CDATA[jetson-thor]]></category>
            <dc:creator><![CDATA[Rippertnt]]></dc:creator>
            <pubDate>Fri, 20 Feb 2026 07:16:10 GMT</pubDate>
            <atom:updated>2026-02-20T12:19:10.528Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Real-World Concurrent Benchmark for Physical AI — Performance, Power, Safety, and the “Robotic Reflex” Problem</h3><p><em>Analyzed by Circulus · February 2026</em></p><p>Humanoid robots are no longer science fiction. From factory floors to hospitals, they’re beginning to work alongside humans — carrying, sorting, assisting, and communicating. But every humanoid needs a “brain”: an on-device computing platform that can simultaneously see the world, think about it, and speak — all while running on battery power.</p><p>We put two leading candidates head-to-head in a real-world concurrent workload benchmark that simulates actual humanoid robot operation:</p><ul><li><strong>Intel Panther Lake</strong> (Core Ultra X7 358H) — Intel’s latest AI PC processor built on the 18A process (~2nm class), featuring a dedicated NPU</li><li><strong>NVIDIA Jetson Thor</strong> (T6000 Full Chip) — NVIDIA’s Blackwell-powered robotics supercomputer, the platform chosen by Boston Dynamics, Figure, and Amazon Robotics</li></ul><p>This isn’t a synthetic benchmark. We ran three AI workloads simultaneously — just like a real humanoid robot would — and measured performance, power, consistency, and what we call the “robotic reflex”: the ability to react to sudden hazards even while thinking hard.</p><p>The results surprised us.</p><h3>The Test: Simulating a Humanoid Robot’s Brain</h3><p>A humanoid robot in operation must handle three concurrent tasks:</p><p><strong>👁️ Vision (NPU/GPU):</strong> Three simultaneous YOLOv11s segmentation streams for 360° environmental perception — detecting objects, people, and hazards in real-time.</p><p><strong>🧠 Reasoning (GPU):</strong> Google Gemma 3 12B multimodal LLM for understanding context, making decisions, and generating responses.</p><p><strong>🗣️ Speech (CPU):</strong> Text-to-Speech synthesis for verbal communication with humans.</p><p>We tested three operational scenarios that mirror real humanoid deployment:</p><ul><li><strong>Full Load</strong> — All active: see → think → act → speak (normal operation)</li><li><strong>React Mode</strong> — CPU + Vision only: vision-driven reflexive safety response</li><li><strong>Standby</strong> — Vision only: passive environmental monitoring</li></ul><p><strong>Important quantization note:</strong> Vision uses INT8 on PTL’s dedicated NPU vs. FP16 TensorRT on Thor’s GPU. For LLM, PTL uses AWQ 4-bit quantization while Thor uses bitsandbytes 4-bit with Flash Attention 2. These differences are factored into our analysis.</p><h3>The Numbers: Power Efficiency Dominance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mQa1hoXrIfRFc5RhxVeEQw.png" /></figure><p>The first thing that jumps out is the power gap. These are <strong>total system power</strong> measurements — not per-component, but what the entire platform draws from the battery:</p><ul><li><strong>Full Load:</strong> PTL ~52W vs. Thor ~77W (PTL 33% less)</li><li><strong>React:</strong> PTL ~44W vs. Thor ~65W (PTL 32% less)</li><li><strong>Standby:</strong> PTL ~18W vs. Thor ~51W (PTL 64% less)</li></ul><p>That standby number deserves special attention: <strong>18 watts vs. 51 watts</strong> just to keep the robot’s eyes open. For a robot that spends most of its day monitoring and waiting, this is the difference between lasting one shift and lasting three.</p><h3>Performance Under Full Load</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EQO8sPKfjN2dPUvwbY2aQQ.png" /></figure><p>With all three tasks running simultaneously:</p><ul><li><strong>CPU (TTS):</strong> PTL 5.79× real-time factor vs. Thor 4.49× — <strong>PTL +29% faster</strong></li><li><strong>GPU (LLM):</strong> PTL 11.28 tok/s vs. Thor 11.06 — <strong>essentially even</strong>, but PTL is <strong>8× more consistent</strong> (σ=0.51 vs. 4.07)</li><li><strong>Vision (YOLO ×3):</strong> PTL 16.94 FPS vs. Thor 13.19 — <strong>PTL +28% faster</strong> (INT8 vs. FP16 partially accounts for this)</li></ul><p>Note the standard deviations. PTL doesn’t just match or beat Thor — it does so with dramatically better consistency. The GPU LLM throughput is nearly identical, but Thor’s variance is 8× larger, meaning its token generation stutters unpredictably. For a robot that needs to deliver smooth, real-time responses, consistency matters as much as raw speed.</p><h3>The “Robotic Reflex” Problem: This Is Where It Gets Serious</h3><p>Here’s the finding that matters most for anyone deploying humanoid robots around humans.</p><h3>The Human Reflex Analogy</h3><p>The human nervous system has a brilliant safety architecture: <strong>reflexes bypass the brain entirely.</strong> Touch a hot surface, and your spinal cord fires a motor response in ~50ms — your hand pulls away <em>before</em> your brain even registers pain. The cerebral cortex (reasoning) never interferes with the reflex arc (survival response).</p><p>A humanoid robot needs the same architectural guarantee: <strong>vision must never go blind while the brain is thinking.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y17SaGShgxF7wqMwPfzvSA.png" /></figure><h3>The Architecture Gap</h3><p><strong>Intel Panther Lake</strong> has three separate processing units — CPU, GPU, and a <strong>dedicated NPU</strong> — each running independently on isolated silicon. Vision runs on the NPU, LLM on the GPU, speech on the CPU. They never compete for resources. This is like the human spinal reflex arc: perception operates independently of cognition.</p><p><strong>NVIDIA Jetson Thor</strong> runs both vision (TensorRT) and reasoning (LLM) on the <strong>same GPU</strong>. They must time-share GPU cycles. When the robot thinks harder, its eyes go dimmer. This is like a human whose eyesight fades every time they concentrate.</p><h3>What Actually Happens Under Load</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NnOhTOYEsof0BdT9scxNIg.png" /></figure><p>The numbers tell the story clearly:</p><ul><li>Thor’s vision FPS drops <strong>56%</strong> when the LLM is running (29.95 → 13.19 FPS)</li><li>PTL’s vision FPS drops only <strong>17%</strong> (20.42 → 16.94 FPS)</li><li>Thor’s worst-case drops to <strong>9.69 FPS</strong> — below the 15 FPS safety threshold</li><li>PTL’s worst-case holds at <strong>15.81 FPS</strong> — maintaining safe operation</li></ul><h3>The Physical Meaning: A Falling Object</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*47Uj7K6jN0WZrT14UrkmyA.png" /></figure><p>Imagine a humanoid robot is having a conversation with a worker (Full Load) when a heavy tool falls from a shelf toward the worker. At 5 m/s falling speed:</p><ul><li><strong>PTL detects it at 31 cm</strong> of fall distance (~62ms latency)</li><li><strong>Thor detects it at 41 cm</strong> average, <strong>up to 55 cm</strong> worst-case (~82–110ms latency)</li></ul><p>PTL detects the falling object <strong>10–24 cm earlier</strong> than Thor. In robotics safety, that margin is the difference between a successful protective action and a collision.</p><blockquote><em>PTL’s dedicated NPU functions like the human spinal reflex arc — it sees and reacts on its own hardware, regardless of what the “brain” (GPU) is doing. Thor’s architecture is like a human whose eyesight dims by half every time they think hard.</em></blockquote><p>For ISO 13482 (personal care robots) and ISO 10218 (industrial robots), PTL’s deterministic, hardware-isolated perception is fundamentally more certifiable.</p><h3>Performance-per-Watt: PTL Sweeps All Six Categories</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ErVgwqqjAfJidLJJzuFI4A.png" /></figure><p>For battery-powered humanoid robots, performance-per-watt is the ultimate metric. PTL wins every single category:</p><ul><li>CPU RTF/W (Full Load): <strong>1.93×</strong> advantage</li><li>CPU RTF/W (React): <strong>3.02×</strong> advantage</li><li>GPU Tok/s/W: <strong>1.53×</strong> advantage</li><li>Vision FPS/W (Full): <strong>1.92×</strong> advantage</li><li>Vision FPS/W (React): <strong>1.57×</strong> advantage</li><li>Vision FPS/W (Standby): <strong>1.90×</strong> advantage</li></ul><p><strong>Average efficiency advantage: 1.98× (98% more efficient)</strong></p><p>Even accounting for PTL’s INT8 vs. Thor’s FP16 in vision tasks, the system-level power efficiency gap is substantial and driven by Intel’s dedicated NPU architecture and 18A process advantage.</p><h3>Battery Life: The Fleet-Scale Dealbreaker</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hy2istTwuyUuVj2-qWozzA.png" /></figure><p>We modeled battery life on a 960Wh pack (48V / 20Ah) — typical for medium-sized humanoids — across three deployment scenarios with realistic duty cycles:</p><ul><li><strong>Factory (60% standby / 30% full / 10% react):</strong> PTL <strong>31.1h</strong> vs. Thor <strong>16.0h</strong> — <strong>1.95× longer</strong></li><li><strong>Warehouse (70/20/10):</strong> PTL <strong>34.8h</strong> vs. Thor <strong>16.7h</strong> — <strong>2.09× longer</strong></li><li><strong>Service (50/30/20):</strong> PTL <strong>28.7h</strong> vs. Thor <strong>15.6h</strong> — <strong>1.84× longer</strong></li></ul><p>A PTL-powered humanoid can run <strong>3+ full 8-hour shifts</strong> in a warehouse without charging. Thor requires charging after every 2 shifts. For 24/7 factory operations, this means PTL needs half the charging infrastructure and experiences significantly less downtime.</p><h3>The Software Ecosystem Reality Check</h3><p>NVIDIA’s marketing would have you believe that CUDA equals frictionless development. The reality for Jetson edge deployment is more nuanced.</p><p><strong>The x86→ARM porting tax is real.</strong> Jetson Thor runs ARM (Neoverse-V3AE), not x86. Every Docker container, custom C++ module, and system library must be cross-compiled or rebuilt. JetPack 7.x doesn’t support all desktop CUDA features. TensorRT engine files aren’t portable across GPU architectures — a model optimized for an RTX 4090 won’t run on Thor without reconversion. This adds weeks to months of engineering effort that is often underestimated.</p><p><strong>PTL runs the same x86 architecture as virtually every AI development environment on Earth.</strong> Models developed on any workstation, cloud server, or laptop deploy with minimal modification. OpenVINO provides mature model optimization. Docker containers run unmodified.</p><p>That said, <strong>NVIDIA’s robotics-specific tooling is genuinely superior</strong>: Isaac Sim, GR00T foundation models, and Holoscan sensor bridge have no Intel equivalent today. For teams building new robot capabilities from scratch, this matters.</p><p><strong>Our assessment:</strong> NVIDIA’s edge ecosystem advantage is real but narrower than marketing suggests. The porting friction from desktop CUDA to Jetson edge partially offsets the Isaac/GR00T advantage.</p><h3>Platform Specifications at a Glance</h3><p><strong>Intel Panther Lake (Core Ultra X7 358H)</strong></p><ul><li>Process: Intel 18A (~2nm class)</li><li>CPU: 4P + 8E + 4LPE (16 cores)</li><li>GPU: Intel Arc B390 (12 Xe3 cores, integrated)</li><li>AI Accelerator: <strong>NPU 5 (50 TOPS INT8)</strong> — dedicated, hardware-isolated</li><li>Memory: 64 GB DDR5–7200 (2×32 GB SO-DIMM)</li><li>TDP: 25W base / 80W max turbo</li></ul><p><strong>NVIDIA Jetson Thor (T6000 Full Chip)</strong></p><ul><li>Process: TSMC 4NP (Blackwell)</li><li>CPU: 14× Arm Neoverse-V3AE @ 2.6GHz</li><li>GPU: Blackwell (2,560 CUDA / 96 Tensor Cores)</li><li>AI Accelerator: None — <strong>GPU shared between vision and LLM</strong></li><li>Memory: 128 GB LPDDR5X</li><li>TDP: 40–130W configurable</li></ul><h3>The Composite Scorecard</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O2hk260sVaJfflUhmL8CpQ.png" /></figure><p>Dimension PTL Thor Performance/Watt <strong>9.5</strong> 5.0 Raw Performance 7.5 7.5 Safety &amp; Reflex <strong>9.0</strong> 5.5 Cost Efficiency <strong>9.0</strong> 4.0 SW Ecosystem 7.0 <strong>8.0</strong> Battery Life <strong>9.5</strong> 5.0</p><p><strong>Weighted composite: Intel Panther Lake 8.6 / 10 vs. NVIDIA Jetson Thor 6.5 / 10</strong></p><h3>Our Verdict</h3><h3>Intel Panther Lake → Production Fleet Deployment</h3><p>PTL delivers comparable or superior performance at 33–64% less power, 3–6× lower cost, with a fundamentally safer architecture for human-adjacent operation. Its dedicated NPU guarantees uninterrupted vision — the “robotic reflex” — regardless of cognitive load. For any team deploying humanoid robots at scale, PTL is the clear choice.</p><h3>NVIDIA Jetson Thor → R&amp;D, Simulation, Premium Flagships</h3><p>Thor’s 128 GB memory, 2,070 TFLOPS of raw compute, and the Isaac/GR00T ecosystem make it the natural choice for research labs, university programs, and flagship demonstration units where development velocity and model headroom matter most.</p><h3>The Dual-Platform Strategy</h3><p><strong>Develop on Thor</strong> (leveraging Isaac Sim, GR00T, and the CUDA research ecosystem) → <strong>Deploy on PTL</strong> (leveraging efficiency, cost, safety, and x86 deployment simplicity). This mirrors the common pattern in automotive where development and production platforms differ.</p><h3>What This Means for the Industry</h3><p>The Physical AI revolution is increasingly defined not by raw compute, but by <strong>compute-per-watt, compute-per-dollar, and architectural safety guarantees.</strong> Humanoid robots must operate untethered for entire shifts, work safely alongside humans, and be economically viable at fleet scale.</p><p>Intel’s Panther Lake — with its dedicated NPU acting as an always-on “reflex arc” — represents a fundamentally different architectural approach than NVIDIA’s GPU-centric design. For the first time, a sub-$1,000 platform delivers competitive AI performance in a power envelope that makes battery-powered humanoid deployment practical.</p><p>The future of humanoid robotics may not be decided by who has the most TFLOPS — but by who can deliver the most intelligence per watt, per dollar, per hour of battery life, while guaranteeing that the robot never goes blind while thinking.</p><p>On that measure, Intel Panther Lake makes a compelling case.</p><p><em>This analysis was conducted by Circulus using concurrent multi-workload benchmarks collected on February 17, 2026. All workloads ran simultaneously to simulate real humanoid operational conditions. Power measurements represent total system-level draw. Vision comparison accounts for INT8 (PTL NPU) vs. FP16 (Thor GPU TensorRT) quantization. LLM comparison: AWQ 4-bit (PTL) vs. bitsandbytes 4-bit + Flash Attention 2 (Thor). Battery projections assume 960Wh with 100% DoD. For the full interactive report with real-time charts and detailed data, contact Circulus.</em></p><p><em>#PhysicalAI #HumanoidRobots #IntelPantherLake #NVIDIAJetson #JetsonThor #EdgeAI #Robotics #AIBenchmark #PerformancePerWatt #OnDeviceAI #Circulus</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f279fec7c74" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>