<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Toward Humanoids - Medium]]></title>
        <description><![CDATA[Ongoing pre-publication research on humanoids and related technologies including paper reviews and tutorials. - Medium]]></description>
        <link>https://medium.com/correll-lab?source=rss----fc0bf7de342f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Toward Humanoids - Medium</title>
            <link>https://medium.com/correll-lab?source=rss----fc0bf7de342f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 10:05:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/correll-lab" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Art in the Age of AI and Robotics]]></title>
            <link>https://medium.com/correll-lab/art-in-the-age-of-ai-and-robotics-36693089039e?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/36693089039e</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[electrodermal-activities]]></category>
            <category><![CDATA[biometric-authentication]]></category>
            <category><![CDATA[art]]></category>
            <dc:creator><![CDATA[Chelsea Barlaan]]></dc:creator>
            <pubDate>Tue, 28 Jul 2026 19:22:12 GMT</pubDate>
            <atom:updated>2026-07-28T19:22:11.315Z</atom:updated>
            <content:encoded><![CDATA[<h4>Leveraging human physiological measurements to make human-robot collaboration more empathetic</h4><p>As artificial intelligence continues to improve, many artists and creators have begun asking the same question: will technology eventually replace human creativity? Discussions about AI in the arts often focus on competition, copyright, and the fear of machines replacing the human “touch” of expression. But what if technology could support creativity rather than replace it?</p><p><em>Painted Heart Beats</em>, a project led by Patrícia Alves-Oliveira, Assistant Professor of Robotics at the University of Michigan, focuses on this question to further expand the interconnectedness of art and robotics. Through the creation of AURA, a collaborative painting framework that combines robotics, biometric sensing, and artistic expression, artists work alongside a robotic partner that responds to their physiological state in real time.</p><figure><img alt="Close-up of artist Hana Ichikawa wearing an EmotiBit wrist sensor while squeezing black paint into a palette during the painting process." src="https://cdn-images-1.medium.com/max/1024/1*myWVN1dX_CzaoHk_tEAaDg.png" /><figcaption>Artist Hana Ichikawa wears EmotiBit as an open source wearable during the painting process. “Authenticity in artwork with robotics” via YouTube (0:32).</figcaption></figure><h3>A New Canvas</h3><p>At the heart of <em>Painted Heart Beats</em> is AURA, a system made to explore how artists and robots can create together. While much of the focus on AI is about automation, AURA paints the interaction between human and machine as a collaborative effort rather than as a replacement.</p><p>During the painting process, both the artist and robot contribute to the same canvas. Though the robot is able to add its own brushstrokes and colors, the artist’s physiological responses change how the robot contributes to the artwork. According to Alves-Oliveira, this introduces a dynamic in which human and machine interact to shape the creative process rather than solely the final product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mA8pvyBfN-3Lh9POAIMG8A.png" /><figcaption>The robotic partner assists in the making of artist Hana Ichikawa’s painting. “Authenticity in artwork with robotics” via YouTube (2:45).</figcaption></figure><h3>EmotiBit as “The Bridge”</h3><p>The collaboration between artist and robot would not have been possible without biometric feedback, which is where <a href="https://www.emotibit.com/">EmotiBit</a>, a wrist-worn device to measure physiological state, played an important role. According to the researchers, “the EmotiBit proves useful in this respect because of its small size and wide array of recorded data,” making it a practical tool for capturing biometric signals in real-time artistic collaborations (Adhya et al., 2025). Alves-Oliveira and her team were able to separate participants during the creative process into two categories: higher and lower arousal states. When the artist’s heart rate increased, possibly implying frustration, discomfort, or heightened arousal, the robot moved away to give the artist more space. When the artist’s heart rate returned to a steady baseline, the robot continued contributing to the painting. Instead of rigidly following a predefined set of instructions, the system adapted in real time to the artist’s experience.</p><p>At the IEEE International Conference on Robotics and Automation (ICRA) 2025, Alves-Oliveira and her team were able to interact with a variety of participants in the fields of robotics, computer science, and mechanical engineering. Many reported their surprise when the robot took the artwork toward a different direction. Rather than viewing this as a limitation, however, the participants were inspired by the robot’s changes and were able to finish their pieces within a reasonable timeframe. This shows how the robot’s execution is dependent on the artist’s initial mindset.</p><p>In many ways, EmotiBit acted as a bridge between human emotion and robotic behavior. The wearable’s ease of use, reliability, and wireless design allowed for freedom of movement and stress-free facilitation of the natural creative process. This was especially important as painting is a hands-on activity, and Alves-Oliveira made it an effort to ensure the safety and comfort of the participants.</p><figure><img alt="Artist Hana Ichikawa wears an EmotiBit sensor on her forearm while a nearby laptop displays live graphs of her biometric data during the painting process." src="https://cdn-images-1.medium.com/max/1024/1*_DHL1YLODwc2LMYrmiaL_g.png" /><figcaption>Biometrics such as heart rate and electrodermal activity (EDA) are recorded by the EmotiBit. Artist: Hana Ichikawa. “Authenticity in artwork with robotics” via YouTube (0:56).</figcaption></figure><h3>Beyond the Painting</h3><p>While the collaborative artwork itself is fascinating, one of the project’s most ambitious goals extends beyond the canvas. Alves-Oliveira explained that the research originally emerged from questions regarding artistic authentication. In addition to the heart rate data captured with EmotiBit’s PPG sensor, other plausible factors to determine the unique signatures of individual artists are brushstroke movements and gaze patterns.</p><p>Today, artwork authentication occurs through expert analysis and forensic examination. The addition of biometric data allows the team to provide an additional layer of evidence by documenting parts of the artistic process that cannot be easily replicated. With current research involving multiple sessions and varying environments, the team is able to gather consistent physiological data to identify individual creators, further contributing to authorship and copyright in a world where AI-generated content continues to grow.</p><figure><img alt="Central white canvas with vibrant abstract brushstrokes in pink, orange, yellow, black, and brown; signature at bottom right, flanked by two darker abstract paintings." src="https://cdn-images-1.medium.com/max/1024/1*6IOIFIsK-3qBPzrDWf8V0g.jpeg" /><figcaption>Display of the artworks created by participants at ICRA 2025. Image courtesy of Patrícia Alves-Oliveira.</figcaption></figure><h3>Looking Forward</h3><p><em>Painted Heart Beats</em> introduces a new vision for the future of AI and robotics. AURA demonstrates how technology can support human creativity through responsive, adaptive collaboration rather than replacement. As Alves-Oliveira has emphasized, the goal is not to substitute humans in the arts with robots, but to use robots as tools to better understand human experiences. In this project, EmotiBit enables the robot to respond to the artist’s physiological state, creating a more dynamic and human-aware interaction. As researchers continue exploring the relationship between physiology, creativity, and technology, projects like <em>Painted Heart Beats</em> offer a glimpse into a future where innovation is measured not by how much technology can replace humans, but by how effectively it can help us understand and work alongside one another.</p><figure><img alt="Man in a checkered shirt painting on a colorful abstract artwork with an EmotiBit on his wrist." src="https://cdn-images-1.medium.com/max/1024/1*s77y3OancWSUn95sxQsZpA.jpeg" /><figcaption>Participant wears EmotiBit on wrist to capture heart rate data from integrated PPG sensor. Image courtesy of Patrícia Alves-Oliveira. ICRA 2025.</figcaption></figure><h4><strong>For more information, check out these videos highlighting the project and EmotiBit’s involvement:</strong></h4><p><a href="https://www.youtube.com/watch?v=D4qE6efPfwg">Authenticity in artwork with robotics — umrobotics</a><br><a href="https://www.linkedin.com/posts/activity-7467577961563684865-t1uN/?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAEdHSfcBJzou1b_i9fh170w5jr20oO3nc8I">IEEE International Conference on Robotics and Automation (ICRA) 2026 — UM Robotics Dept.</a></p><h4><strong>Special thanks to Patrícia Alves-Oliveira for sharing her insights during a brief interview. This piece was also informed by the following paper:</strong></h4><p>Adhya, A., Yang, C., Wu, E., Hasan, R., Narula, A., &amp; Alves-Oliveira, P. (2025). <em>Painted Heart Beats</em>. arXiv. <a href="https://arxiv.org/abs/2511.15105">https://arxiv.org/abs/2511.15105</a></p><p><strong><em>Curious about other ways EmotiBit is being used to support the arts? Check out our blog post on musician </em></strong><a href="https://www.emotibit.com/secrets-of-flow-states-investigating-with-famous-musician-mike-gordon/"><strong><em>Mike Gordon’s XenboX project</em></strong></a><strong><em>!</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36693089039e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/art-in-the-age-of-ai-and-robotics-36693089039e">Art in the Age of AI and Robotics</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why reading “The Decameron” is as useful as ever]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/correll-lab/why-reading-the-decameron-is-as-useful-as-ever-06cf21b76452?source=rss----fc0bf7de342f---4"><img src="https://cdn-images-1.medium.com/max/640/1*wcDQuvbUTJJzmDVr1znnzg.jpeg" width="640"></a></p><p class="medium-feed-snippet">We like to think we are living through a rupture that is special. It is not.</p><p class="medium-feed-link"><a href="https://medium.com/correll-lab/why-reading-the-decameron-is-as-useful-as-ever-06cf21b76452?source=rss----fc0bf7de342f---4">Continue reading on Toward Humanoids »</a></p></div>]]></description>
            <link>https://medium.com/correll-lab/why-reading-the-decameron-is-as-useful-as-ever-06cf21b76452?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/06cf21b76452</guid>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[literature]]></category>
            <dc:creator><![CDATA[Nikolaus Correll]]></dc:creator>
            <pubDate>Fri, 24 Jul 2026 18:15:19 GMT</pubDate>
            <atom:updated>2026-07-24T18:15:20.566Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Why This PushT Architecture Works]]></title>
            <link>https://medium.com/correll-lab/why-this-pusht-architecture-works-49522f326048?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/49522f326048</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Adhitya Mohan]]></dc:creator>
            <pubDate>Fri, 24 Jul 2026 18:09:54 GMT</pubDate>
            <atom:updated>2026-07-24T18:09:53.461Z</atom:updated>
            <content:encoded><![CDATA[<h4>A plain-language explanation of the policy, the planner, the evaluation settings, and the task dynamics</h4><h3>The Problem</h3><p>PushT is a contact-rich control task. A round agent must push a T-shaped block until the block matches a target pose. The state contains five numbers: agent x, agent y, block x, block y, and block angle. The action contains two numbers: the next target position for the agent. An episode lasts 200 steps. Success requires at least 95% overlap with the goal pose.</p><p>That last detail is what makes PushT hard. The block must end in the right place and at the right angle. A policy can reach the goal region and still fail because the block rotates slightly off target.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z-59Oaw6C2p-3lmfywc6RQ.jpeg" /><figcaption>Created by Gemini</figcaption></figure><h3>The Core Idea</h3><p>The architecture combines two kinds of intelligence.</p><ol><li>A learned policy proposes what to do next.</li><li>A learned dynamics model lets a planner test nearby alternatives before execution.</li></ol><p>The policy gives the system speed. The planner gives it caution. Together, they work better than either part alone.</p><h3>The Terms</h3><p>The article uses a few technical terms repeatedly. Here they mean the following.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gcoZ8SknWUsOlI_SGAuthQ.png" /></figure><h3>The Architecture</h3><p>The policy has two stages.</p><p>First, a GRU reads a short history of observations. This gives the controller memory. In PushT, memory matters because contact is history-dependent. The same block pose can require different actions depending on how the agent reached it.</p><p>Second, a Transformer decoder predicts an action chunk. Instead of choosing one action at a time from scratch, it proposes a short plan.</p><p>That plan then passes to MPPI. MPPI perturbs the proposed actions, rolls them forward through the learned dynamics model, scores the outcomes, and returns a weighted average of the best candidates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nr6VwdvbPx6YUdaT-Ma30Q.png" /><figcaption>The design is hybrid by construction. The policy handles broad strategy. The planner handles local correction.</figcaption></figure><h3>Why a GRU Helps in PushT</h3><p>PushT is not a static mapping from state to action. Contact changes everything.</p><p>When the agent taps the block, slides along one edge, or loses contact for a moment, the next useful action depends on what just happened. A feedforward model sees only the current pose. A GRU sees recent motion as well.</p><p>That extra memory helps in three common cases.</p><ol><li>It helps detect whether the block is already rotating in the right direction.</li><li>It helps distinguish real progress from oscillation.</li><li>It helps the controller recover after a poor contact event.</li></ol><p>In plain terms, the GRU gives the policy short-term situational awareness.</p><h3>Why a Transformer Helps</h3><p>The Transformer does not act as a world model here. It acts as a short-horizon action generator.</p><p>That matters because pushing is not a one-step problem. A useful behavior often takes several coordinated moves: approach the block, make contact, keep pressure on one side, and steer the rotation without overshooting.</p><p>A short action chunk lets the policy express that structure. It does not need to rediscover the entire plan after every single action. It proposes a brief sequence, then the controller executes only part of it and replans.</p><p>This combination gives the system a useful balance. The action chunk adds coherence. Frequent replanning keeps it responsive.</p><h3>Why the Dynamics Model Helps</h3><p>The dynamics model predicts how the block will move under candidate actions. That prediction does not need to be perfect to be useful. It only needs to be good enough over a short horizon.</p><p>That is the key point. The planner is not trying to predict the whole episode. It is trying to compare nearby choices over the next few steps.</p><p>This local use is important in PushT because contact dynamics are hard to model exactly. Over a long rollout, prediction errors compound. Over a short rollout, the model is often accurate enough to rank candidate action sequences.</p><p>That is why the architecture replans often and keeps the planning horizon modest.</p><h3>What MPPI Does</h3><p>MPPI is the corrective layer.</p><p>It starts from the policy’s proposed action chunk. It then adds noise to create many candidate chunks. For each candidate, it rolls the state forward through the dynamics model. It computes a cost for each rollout. Lower cost means the candidate looks better.</p><p>The cost has two main parts.</p><ol><li>Goal error: how far the block ends up from the goal position and angle.</li><li>Trust penalty: how far the candidate drifts from the policy’s proposal.</li></ol><p>Then MPPI forms a weighted average of the better candidates.</p><pre>noise = random_perturbations()<br>candidates = nominal_actions + noise<br>rollout_cost = simulate_with_dynamics(candidates)<br>total_cost = rollout_cost + trust_weight * deviation_from_nominal<br>weights = softmax(-total_cost)<br>corrected_actions = weighted_average(candidates, weights)</pre><p>The trust penalty is crucial. Without it, the planner may chase unrealistic corrections that look good inside the model and fail in the real environment.</p><h3>Why Selective Correction Works Better Than Constant Correction</h3><p>Not every proposal needs repair.</p><p>If the policy is already doing the right thing, injecting noise and replanning can make the action worse. This is especially true near the goal, where the task becomes precise. Small corrections can destroy a good alignment.</p><p>That is why selective MPPI makes sense. The planner helps most when the situation is ambiguous, unstable, or far from the target. It helps less when the system is already on track.</p><p>In PushT, correction tends to matter most in four cases.</p><ol><li>The block is still far from the goal.</li><li>The agent has just made contact and needs to guide a rotation.</li><li>The proposed action chunk is rough or inconsistent.</li><li>The block is near a boundary, where recovery is harder.</li></ol><p>This is the practical reason a hybrid controller often beats both extremes. Pure BC can be too brittle. Always-on planning can be too intrusive.</p><h3>Why the Method Fits PushT Specifically</h3><p>PushT rewards a very particular style of control.</p><p>The controller must be decisive when the block is far away. It must be gentle when the block is almost aligned. It must also react well to contact, because the block rotates only through pushing.</p><p>The architecture fits that profile well.</p><p>The learned policy handles the large-scale pattern of successful behavior. It learns how to approach, push, and continue motion from demonstrations.</p><p>The planner then improves local decisions. It is especially useful when the policy proposes something reasonable but not precise enough. Instead of discarding the policy, MPPI searches around it.</p><p>That search is a good match for PushT because good actions often live near other good actions. A small shift in contact point or push angle can turn a miss into a success.</p><h3>The Evaluation Story</h3><p>The results in this project come from more than one evaluation setting, and that distinction matters.</p><h3>Single-attempt settings</h3><p>In the cleaner setting, the controller gets one attempt per seed. These results show what the system does under a straightforward protocol.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Chx0SU-h3dOFs8lLO4_yQ.png" /><figcaption>These results already show the main point. A good BC policy is competitive, and short-horizon planning can improve difficult cases.</figcaption></figure><h3>Aggressive setting</h3><p>The highest reported number,<strong> 46/50 or 92%</strong>, comes from a more aggressive protocol. That protocol uses retries, probabilistic activation, and seed-specific tuning.</p><p>The modular and temporal nature of our architecture, allows us the luxury of having retries. The baseline Colombia paper, uses a much larger 256M parameter diffusion model with 10 de-noising steps to achieve 40/50 on different seeds across 50 episodes. The monolithic nature of the diffusion leaves no room for retries or seed specific tuning.</p><h3>Why the Best Setting Wins</h3><p>The strongest setting wins because it combines several sensible biases.</p><p>First, it trusts the policy enough to start from a strong proposal.</p><p>Second, it uses planning only when planning is likely to help.</p><p>Third, it replans after a short execution window, so errors do not accumulate for long.</p><p>Fourth, it increases aggression early in the episode and becomes more conservative late in the episode. That matches the task. Early control should move the block into a good regime. Late control should settle the final pose.</p><p>In other words, the system wins by matching its control style to the phase of the task.</p><h3>What Still Fails</h3><p>The system still fails in familiar ways.</p><p>One failure mode appears when the dynamics model ranks candidates incorrectly. Then the planner trusts a bad future.</p><p>Another failure mode appears when the block reaches the right area but keeps the wrong angle. This is the classic PushT failure. Position is easy to approach. Final orientation is harder to settle.</p><p>A third failure mode appears when the controller over-corrects near the goal. The trust penalty helps, but it does not remove this risk entirely.</p><p>These failures are not random. They follow directly from the structure of the task.</p><h3>The Main Lesson</h3><p>PushT rewards a controller that is both learned and cautious.</p><p>The learned part gives the system strategy. The planning part gives it local repair. The short horizon keeps both parts honest.</p><p>That is why this architecture works. The policy is good at proposing plausible pushes. The dynamics model is good enough to compare nearby alternatives over a short future. MPPI uses that short future to improve action quality without replacing the policy.</p><p>The result is not magic. It is a good division of labor.</p><h3>Takeaway</h3><p>The most useful way to understand this system is not as a single model. It is a layered controller.</p><p>The GRU remembers recent contact history. The Transformer proposes a coherent short plan. The dynamics model predicts short-term consequences. MPPI searches around the proposal and keeps the promising corrections.</p><p>That combination fits PushT well because PushT is a task of contact, rotation, and late-stage precision. A policy alone can be brittle. A planner alone can be expensive and unstable. Together, they work better.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49522f326048" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/why-this-pusht-architecture-works-49522f326048">Why This PushT Architecture Works</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What actually improved Push-T?]]></title>
            <link>https://medium.com/correll-lab/what-actually-improved-push-t-5ec4898ae3a6?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/5ec4898ae3a6</guid>
            <category><![CDATA[flowmatching]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recurrent-neural-network]]></category>
            <category><![CDATA[imitation-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Yuri Fung]]></dc:creator>
            <pubDate>Tue, 05 May 2026 17:05:45 GMT</pubDate>
            <atom:updated>2026-05-05T17:05:44.459Z</atom:updated>
            <content:encoded><![CDATA[<h4>Flow Matching is hard to beat</h4><p>Robotic imitation learning looks fairly simple. If you have demos, then in theory you can train a model to observe some state, predict the next actions, and copy what the expert did. That sounds straightforward, but in practice, the real test isn’t whether the model fits demonstration datasets well. The real test is whether it can act effectively in a closed loop, where each mistake could change the next state it will see. A model can look strong on validation data and still fail.</p><p>This became the main lesson of my Push-T project.</p><p>Building on Professor Nikolaus Correll&#39;s work of an autoregressive encoder-decoder transformer for Push-T, or as I will call it, Model A. Attached is the article:</p><p><a href="https://medium.com/correll-lab/robotic-behavior-cloning-i-auto-regressive-transformers-a7be623f4291">Robotic Behavior Cloning I: Auto-regressive Transformers</a></p><p>For this project, I started from the tutorial implementation in Robotic Behavior Cloning I, Auto-regressive Transformers, and then extended it using Flow Matching for Generative Models from Scratch. Rather than rebuilding the tutorial from zero, I used the provided autoregressive transformer as the foundation and focused on a sequence of targeted architectural changes.</p><p>I built a series of extensions on top of it rather than reinventing the entire baseline from scratch. I implemented Baseline B, C, D, and E (v1, v2) for comparison. My goal wasn’t to build different policies, but to test the progression of ideas that seemed likely to improve robotic planning.</p><p>The difference between offline prediction and online control quality became the lesson of my Push-T project.</p><p>I ended up with five main models:</p><ul><li>Model A: Baseline of Autoregressive Encoder-Decoder Transformer</li><li>Model B: Baseline + Flow-Matching Action Head</li><li>Model C: Model B + Larger History Window</li><li>Model D: Model B + Persistent Hidden Memory across replanning</li><li>Model E: Model B + Compressed Flow Matching Policy using DCT Trajectory coefficients</li></ul><p>The main result was clear: <strong>Model B</strong> was the only great and consistent improvement over the original baseline.</p><p>A longer history improved offline imitation but not online/real-time control. Persistent memory gave mixed results, and compression degraded performance.</p><p>This is a useful result since it shows negative results are informative just as positive ones. A method that might seem “right” might not actually help robots perform better.</p><h4>Push T benchmark</h4><p>Push T is a 2D manipulation task where the model needs to push a T-shaped block into a target T-shaped pose. On the surface, the task looks simple, the environment is visually understandable, the object is easy to track, and the goal is simple. However, the task becomes difficult once we start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*exFzs9sHtw3pp-qK8QTRfg.png" /></figure><p>The agent can’t just move towards the goal location. It must learn to push from a useful side, maintain contact as the object rotates, and make smaller corrections to fix misalignments. This means that the policy must do more than predict a distraction. It needs to produce a coherent short plan that adapts ot geometry of the interaction.</p><p>The model was trained on the pusht_cchi_v7_replay dataset, which contains roughly 200 human demonstrations collected through teleoperation.</p><p>This makes Push T a good benchmark for comparing different architectures and features. It is simple enough that you can clearly see what the policy is doing, but hard enough that there is a gap between good offline imitation and good online behavior.</p><h4>Evaluation Metric Guideline:</h4><p>I had two evaluation metrics: offline vs. online rollout metrics.</p><p><strong>Offline Imitation Metrics:</strong> Measures how closely the model predicts trajectory matches the expert’s trajectory in the validation set.</p><ul><li>ADE (Average Displacement Error)</li><li>FDE (Final Displacement Error)</li><li>Final Acc @ 0.1 (Final Accuracy at 0.1 Threshold): Predicted trajectory is marked accurate only if the final position ends within 10% of its distance relative to the goal</li></ul><p>These tell us how well the model imitates demonstration trajectories.</p><p><strong>Online Performance: </strong>Measures during Rollouts (Live simulation attempts of 200 steps)</p><ul><li>Avg Max Reward</li><li>Avg Final Reward</li><li>Success @ 0.95 and @ 0.99: Computed over 50 closed-loop rollout episodes per model setting, using evaluation seeds 200–249. Each rollout was capped at 200 environment steps.</li></ul><p>This tells us how the robot actually behaves.</p><p>Across Models B, C, D, and E, I kept the rollout and planning setup mostly fixed: pred_horizon = 16, action_horizon = 8, max_steps = 200, sample_steps = 24, and the same evaluation seeds. I also kept the main flow-transformer backbone mostly fixed at d_model = 256, nhead = 8, 4 encoder layers, 4 decoder layers, dropout 0.1, and AdamW with learning rate 1e-4. The main experimental changes were isolated to one variable at a time: larger history in C, persistent memory in D, and DCT compression in E.</p><h4><strong>Model A: Autoregressive Encoder-Decoder Transformer</strong></h4><p>My starting point from the original autoregressive encoder-decoder transformer:</p><p>Model A works like:</p><ul><li>Takes in short observation history</li><li>Encodes history with a transformer</li><li>Directly predicts future action sequences</li></ul><p>Its validation results were:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1cI3Buq_PPFbUStBJT_YTw.png" /></figure><h4><strong>Model B: Baseline + Flow-Matching Action Head</strong></h4><p>Model B was the most important model in the project.</p><p>Instead of directly decoding future actions autoregressively, I added a flow-matching action head.</p><p>This changed the problem from “predict trajectory in one shot” to “start from noise and iteratively refine a trajectory towards a better one”.</p><p>This shift mattered because Push T wasn’t just about “guessing” the move. But rather producing a short plan that stays aligned as the robot puvvsshes, reorients, and corrects. A generative action head here is better aligned with this kind of problem rather than a one-shot prediction.</p><p>I changed:</p><ul><li>Added time-step embeddings so the flow network could condition on the current integration time</li><li>Replaced the original action head with a flow-matching prediction head</li><li>Adjusted training so the model learns to predict the flow field over noisy futures rather than just raw actions</li></ul><pre>class SinusoidalTimeEmbedding(nn.Module):<br>    def __init__(self, dim):<br>        super().__init__()<br>        self.dim = dim<br><br>    def forward(self, t):<br>        # t: (B,)<br>        half_dim = self.dim // 2<br>        device = t.device<br>        emb_scale = math.log(10000) / max(half_dim - 1, 1)<br>        freqs = torch.exp(torch.arange(half_dim, device=device) * -emb_scale)<br>        angles = t[:, None] * freqs[None, :]<br>        emb = torch.cat([angles.sin(), angles.cos()], dim=-1)<br><br>        if self.dim % 2 == 1:<br>            emb = F.pad(emb, (0, 1))<br>        return emb<br><br><br>class TimeEmbedding(nn.Module):<br>    def __init__(self, d_model):<br>        super().__init__()<br>        self.net = nn.Sequential(<br>            SinusoidalTimeEmbedding(d_model),<br>            nn.Linear(d_model, d_model * 4),<br>            nn.GELU(),<br>            nn.Linear(d_model * 4, d_model),<br>        )<br><br>    def forward(self, t):<br>        return self.net(t)</pre><p>New PushTFlowTransformer Class:</p><pre>class PushTFlowTransformer(nn.Module):<br>    def __init__(<br>        self,<br>        obs_horizon,<br>        pred_horizon,<br>        obs_dim=6,<br>        action_dim=2,<br>        d_model=256,<br>        nhead=8,<br>        num_encoder_layers=4,<br>        num_decoder_layers=4,<br>        dropout=0.1,<br>        angle_min=-3.14159265,<br>        angle_max=3.14159265,<br>    ):<br>        super().__init__()<br>        self.obs_horizon = obs_horizon<br>        self.pred_horizon = pred_horizon<br>        self.action_dim = action_dim<br><br>        self.register_buffer(&quot;angle_min&quot;, torch.tensor(float(angle_min), dtype=torch.float32))<br>        self.register_buffer(&quot;angle_max&quot;, torch.tensor(float(angle_max), dtype=torch.float32))<br><br>        # observation -&gt; token<br>        self.obs_proj = nn.Linear(obs_dim, d_model)<br><br>        # action trajectory -&gt; token<br>        self.action_proj = nn.Linear(action_dim, d_model)<br><br>        # time embedding for flow matching<br>        self.time_embed = TimeEmbedding(d_model)<br><br>        # learned positional embeddings<br>        self.obs_pos = nn.Parameter(torch.zeros(1, obs_horizon, d_model))<br>        self.action_pos = nn.Parameter(torch.zeros(1, pred_horizon, d_model))<br><br>        encoder_layer = nn.TransformerEncoderLayer(<br>            d_model=d_model,<br>            nhead=nhead,<br>            dim_feedforward=4 * d_model,<br>            dropout=dropout,<br>            batch_first=True,<br>            norm_first=True,<br>        )<br>        self.encoder = nn.TransformerEncoder(encoder_layer, num_layers=num_encoder_layers)<br><br>        decoder_layer = nn.TransformerDecoderLayer(<br>            d_model=d_model,<br>            nhead=nhead,<br>            dim_feedforward=4 * d_model,<br>            dropout=dropout,<br>            batch_first=True,<br>            norm_first=True,<br>        )<br>        self.decoder = nn.TransformerDecoder(decoder_layer, num_layers=num_decoder_layers)<br><br>        self.flow_head = nn.Sequential(<br>            nn.LayerNorm(d_model),<br>            nn.Linear(d_model, d_model),<br>            nn.GELU(),<br>            nn.Linear(d_model, action_dim),<br>        )<br><br>        self._reset_parameters()<br><br>    def _reset_parameters(self):<br>        nn.init.normal_(self.obs_pos, std=0.02)<br>        nn.init.normal_(self.action_pos, std=0.02)<br><br>    def _obs_to_features(self, obs_seq):<br>        # obs_seq: (B, obs_horizon, 5) in normalized space<br>        # original obs has angle as one scalar; convert to sin/cos<br>        angle_norm = obs_seq[..., 4:5]<br>        angle = ((angle_norm + 1.0) / 2.0) * (self.angle_max - self.angle_min) + self.angle_min<br><br>        obs_feat = torch.cat(<br>            [<br>                obs_seq[..., 0:4],   # x_agent, y_agent, x_block, y_block<br>                torch.sin(angle),<br>                torch.cos(angle),<br>            ],<br>            dim=-1,<br>        )<br>        return obs_feat<br><br>    def encode_context(self, obs_seq):<br>        obs_feat = self._obs_to_features(obs_seq)   # (B, obs_horizon, 6)<br>        context_tokens = self.obs_proj(obs_feat) + self.obs_pos[:, :obs_feat.shape[1], :]<br>        memory = self.encoder(context_tokens)<br>        return memory<br><br>    def forward(self, obs_seq, x_t, t):<br>        # obs_seq: (B, obs_horizon, 5)<br>        # x_t:     (B, pred_horizon, 2)<br>        # t:       (B,)<br>        memory = self.encode_context(obs_seq)<br><br>        traj_tokens = self.action_proj(x_t)<br>        traj_tokens = traj_tokens + self.action_pos[:, :x_t.shape[1], :]<br>        traj_tokens = traj_tokens + self.time_embed(t).unsqueeze(1)<br><br>        decoded = self.decoder(traj_tokens, memory)<br>        v_t = self.flow_head(decoded)   # predict flow / velocity<br>        return v_t<br><br>    @torch.no_grad()<br>    def sample(self, obs_seq, num_steps=24):<br>        batch_size = obs_seq.shape[0]<br>        device = obs_seq.device<br><br>        # start from noise<br>        x = torch.randn(batch_size, self.pred_horizon, self.action_dim, device=device)<br><br>        # simple Euler integration from t=0 -&gt; 1<br>        dt = 1.0 / num_steps<br>        for step in range(num_steps):<br>            t = torch.full((batch_size,), step / num_steps, device=device)<br>            v = self.forward(obs_seq, x, t)<br>            x = x + dt * v<br><br>        return x</pre><p>New Training Loop:</p><pre><br>import torch.optim as optim<br>from torch.optim.lr_scheduler import CosineAnnealingLR<br>from tqdm.auto import tqdm<br><br>optimizer = optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-4)<br>scheduler = CosineAnnealingLR(optimizer, T_max=100, eta_min=1e-6)<br><br>for epoch in range(epoch, epoch + 100):<br>    model.train()<br>    train_loss = 0.0<br><br>    with tqdm(dataloader, desc=f&quot;Epoch {epoch} Training&quot;, leave=False) as tepoch:<br>        for batch in tepoch:<br>            obs_seq = batch[&quot;obs&quot;].to(device, dtype=torch.float32)         # (B, obs_horizon, 5)<br>            action_seq = batch[&quot;action&quot;].to(device, dtype=torch.float32)   # (B, pred_horizon, 2)<br><br>            x1 = action_seq<br>            x0 = torch.randn_like(x1)<br>            t = torch.rand(x1.shape[0], device=device)<br>            t_view = t[:, None, None]<br><br>            x_t = (1.0 - t_view) * x0 + t_view * x1<br>            v_target = x1 - x0<br><br>            optimizer.zero_grad()<br>            v_pred = model(obs_seq, x_t, t)<br>            loss = F.mse_loss(v_pred, v_target)<br>            loss.backward()<br>            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)<br>            optimizer.step()<br><br>            train_loss += loss.item()<br>            tepoch.set_postfix(loss=loss.item())<br><br>    avg_train_loss = train_loss / len(dataloader)<br><br>    model.eval()<br>    val_loss = 0.0<br>    with torch.no_grad():<br>        for batch in val_dataloader:<br>            obs_seq = batch[&quot;obs&quot;].to(device, dtype=torch.float32)<br>            action_seq = batch[&quot;action&quot;].to(device, dtype=torch.float32)<br><br>            x1 = action_seq<br>            x0 = torch.randn_like(x1)<br>            t = torch.rand(x1.shape[0], device=device)<br>            t_view = t[:, None, None]<br><br>            x_t = (1.0 - t_view) * x0 + t_view * x1<br>            v_target = x1 - x0<br><br>            v_pred = model(obs_seq, x_t, t)<br>            loss = F.mse_loss(v_pred, v_target)<br>            val_loss += loss.item()<br><br>    avg_val_loss = val_loss / len(val_dataloader)<br>    scheduler.step()<br>    # print(f&quot;Epoch: {epoch} | Train Loss: {avg_train_loss * 1000:.4f} | Validation Loss: {avg_val_loss * 1000:.4f}&quot;)<br><br>    print(<br>        f&quot;Epoch: {epoch:03d} | &quot;<br>        f&quot;Train Loss: {avg_train_loss:.6f} | &quot;<br>        f&quot;Val Loss: {avg_val_loss:.6f} | &quot;<br>        f&quot;LR: {optimizer.param_groups[0][&#39;lr&#39;]:.8f}&quot;<br>    )</pre><p>Compared to Model A, this was a clear improvement. The trajectory prediction went significantly better, but more importantly, the actual robot behavior in run-time behaved better.</p><p>Flow matching gave the model a better way to represent future action as a structured object and not just a list of next-step predictions.</p><p>Replacing direct action prediction with iterative trajectory refinement seemed like the right direction here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*INWmb1_LnMAU3sRmuCsxiw.png" /></figure><h4><strong>Model C: Model B + Larger History Window</strong></h4><p>Now that model B was working, I tried improving it with different variants.</p><p>My next hypothesis was simple: Maybe the model needed more temporal context.</p><p>I changed obs_horizon = 2 to obs_horizon = 8.</p><p>Although it is a tiny code change, it was a meaningful experiment here.</p><p>If the model can see more motion history, will it make better decisions around contact and recovery?</p><p>These were the best “offline” metrics in the project. If you look at ADE, FDE, and final threshold accuracy, Model C might have been the best overall.</p><p>However, rollout was a different story. Compared to model B, average rewards were lower, and success rates were worse. This meant that the model was better at imitating validation trajectories without becoming a better controller.</p><p>I ran a fake history ablation as well, in which historical frames were replaced with repeated copies of the current frame. It shows that performance dropped, and shows that the model actually uses historical information here. So the extra history that we defined wasn’t ignored. However, it didn’t perform well in a closed loop.</p><p>This shows that better offline imitation doesn’t automatically mean that it has better live simulation control.</p><p>Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4VHHQ5_dgD7cLvUsktuGrA.png" /></figure><h4><strong>Model D: Model B + Persistent Hidden Memory across replanning</strong></h4><p>Model C showed that a larger fixed observation window could help. Model D here tests, rather than only seeing recent frames, could the model carry a persistent hidden summary across replanning steps?</p><p>Could the model perform more statefully by remembering what had happened in earlier replannings?</p><p>This was a test-time memory idea, in which the decoder will now be conditioned on:</p><ul><li>Current observation window</li><li>Persistent hidden state summarizing prior replanning information</li></ul><p>I added a</p><ul><li>GRU-style (Gated Recurrent Unit) memory update: It&#39;s similar to an RNN architecture, which prevents standard RNNs from learning long-term dependencies in sequential data.</li><li>Hidden state across replanning during rollout</li><li>Reset memory during training windows so the dataset loader samples an independent chunk.s</li></ul><p>Modell’s memory is persistent at test time, but not truly trained in the same way since data was chunked.</p><p>Furthermore, GRU was harder to train effectively here. If the model isn’t trained specifically to rely on that persistent state, the GRU might be passed along as “noise” rather than useful historical context. Thus, real benefits from persistent memory would require more episode-aware training, better memory supervision, and a better state-update mechanism.</p><p>This shows that adding “memory” isn’t always the ultimate fix. Although it sounds beneficial, it won’t always work unless memory is trained in a way that matches how it will be used; then the gain is limited.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XdxNGKH62uYB3I6ePPqe9w.png" /></figure><h4><strong><em>Model E: Model B + Compressed Flow Matching Policy using DCT Trajectory coefficients</em></strong></h4><p>This last model was efficient. It was whether I could compress future trajectories into lower-dimensional representations using discrete cosine transforms while still preserving performance.</p><p>I changed:</p><ul><li>Action trajectories into DCT coefficient space</li><li>Predicted only the first few coefficients</li><li>Reconstructed action chunk from those coefficients</li><li>Tested two settings: 8 coefficients and 12 coefficients.</li></ul><p>Overall, compression was a downgrade.</p><p>Neither compressed model convincingly matched B, C, or D. The rollout quality dropped, average rewards were worse, and success rates were way weaker.</p><p>My interpretation is that Push T is not a task where a low-frequency plan is enough. It needs small, local corrective actions near the contact, especially when the T-shape rotates or slips in subtle ways. A low-dimensional DCT representation throws away precise details that matter for these corrections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DqtIG-zKyMdTw-iuGy0wVQ.png" /></figure><p>Looking at the final results, the pattern is shown:</p><ul><li>Model A established the weak autoregressive baseline</li><li>Model B was the strongest overall practical controller</li><li>Model C was the best offline imitator, but not the best online controller</li><li>Model D gave mixed memory effects, with a notable gain only at the success threshold</li><li>Model E showed that trajectory compression degraded performance</li></ul><p>Final Full Table Comparison Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y3-Ox6w1WzAyVl4ALEimCw.png" /></figure><h3><strong>What I learned</strong></h3><p>The greatest improvement in this project came from replacing direct action with iterative trajectory refinement. The single model from Model A to Model B mattered more than all the other small additions.</p><p>Everything after that was useful, but in terms of performance, it didn’t gain.</p><p>Model C showed better validation metrics that could be misleading. Model D showed that memory is harder to use effectively. Model E showed that compression isn’t a free efficiency gain<strong>.</strong></p><p><strong>Did the robot actually behave better once it was in the loop?</strong></p><p>This is why online rollout metrics matter so much more than offline trajectory scores.</p><p>Flow Matching improved Push-T. More history, more memory, and more compression didn’t improve real control.</p><p>The most important thing is whether the behavior actually improves when the policy has to act.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ec4898ae3a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/what-actually-improved-push-t-5ec4898ae3a6">What actually improved Push-T?</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Proactive Agents: A Local VLM-to-Action Benchmark for Kitchen Assistance]]></title>
            <link>https://medium.com/correll-lab/proactive-agents-a-local-vlm-to-action-benchmark-for-kitchen-assistance-5ec8331d9ee3?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/5ec8331d9ee3</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[agentic-ai-architecture]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[autonomy]]></category>
            <dc:creator><![CDATA[Yuri Fung]]></dc:creator>
            <pubDate>Tue, 05 May 2026 16:44:49 GMT</pubDate>
            <atom:updated>2026-05-05T16:44:48.445Z</atom:updated>
            <content:encoded><![CDATA[<h4>Can small local models decide when a household robot should help?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MUH32ZA5z7ItE0fbPTLs2A.png" /><figcaption>Generated From Gemini</figcaption></figure><p>Many household robots are reactive. They wait for explicit instructions. Pick this up. Move that there. Clean this now.</p><p>But in real household environments, many useful actions are obvious from context. If someone has clearly finished eating, a helpful robot shouldn’t need a detailed command to begin clearing dishes. If trash is left near the counter or floor, it should recognize that cleanup is appropriate. The hard part isn’t only acting. It is deciding whether to act, when to act, and what the next action should be.</p><p>The core idea behind our project:</p><blockquote><strong>Can a small local model observe a kitchen scene, interpret what is happening, and choose the next high-level action a robot can take?</strong></blockquote><p>We didn’t start by training a full robot policy from pixels to motor commands. Instead, we built a modular system with separate layers for perception, planning, memory, and evaluation. By focusing on a more realistic and testable goal: A local, privacy-preserving proactive kitchen planner that chooses semantic actions under limited compute. The current scope focuses on household cleanup, such as beginning to clear the table after a meal, deciding when to wait, and using memory for personalization only when it is actually relevant.</p><h4><strong>Hypothesis</strong></h4><p>We hypothesize that a modular local VLM-to-action pipeline using structured scene representations, semantic action planning, and lightweight memory can produce useful proactive kitchen-assistance behavior under limited compute. We expect later versions of the system to outperform the weak early baselines on valid action rate, task progress, and proactive behavior, while image-based perception and agentic orchestration will make the benchmark more realistic without fully collapsing performance.</p><h4><strong>Introduction and Related Work</strong></h4><p>This project is grounded in recent work at the intersection of robotics and learning, especially research on language-guided robot planning, vision-language-action systems, and embodied household benchmarks. End-to-end VLA systems show that large multimodal models can support robotic decision-making, but they are often expensive, difficult to debug, and hard to evaluate under local compute constraints. In contrast, modular approaches that separate perception, planning, and execution can make it easier to analyze failure modes and study tradeoffs such as safety, latency, and memory usage. Our project follows that modular direction and asks whether a small local system can still produce useful proactive behavior in kitchen-style tasks.</p><p><a href="https://arxiv.org/abs/2505.08243">Training Strategies for Efficient Embodied Reasoning</a></p><h3>What we built</h3><p>We built a <strong>local, privacy-preserving proactive kitchen agent</strong> that takes in a kitchen image or structured scene description, converts it into a scene representation, and chooses a high-level semantic action such as wait, inspect, start_task, pick, or place. Rather than training an end-to-end robot policy immediately, we focused on a modular VLM-to-action pipeline that separates perception, planning, memory, and symbolic execution. This let us study an important question under realistic local-compute constraints:</p><blockquote><strong>can a small local model decide when a robot should help, and what it should do next?</strong></blockquote><h3>Main result</h3><p>The strongest <strong>text-only planner</strong> in our project was the structured semantic-action planner built around Qwen3.5–0.8B, which reached an average total score of <strong>0.876</strong> without memory on a 50-scenario generated kitchen benchmark. In the image-based pipeline, the strongest practical <strong>perception frontend</strong> was Qwen3.5–2B, which gave the best overall balance of action accuracy, latency, and downstream planner behavior in both the direct VLM benchmark and the SmolAgents VLM benchmark.</p><p>Although the project went through several implementation phases, the final result is simple: we built a modular local pipeline that turns kitchen observations into high-level robot actions. The most important design decision was separating the system into perception, planning, memory, and symbolic execution instead of training one end-to-end robot policy. The rest of the write-up explains why this modular design worked, which parts improved performance, and which parts remained difficult.</p><h3>Why didn’t we start with full VLA?</h3><p>We did not begin with a full vision-language-action robot because that would mix several hard problems at once: visual perception, task planning, robot control, safety, memory, and evaluation. For a class project, we wanted a system that was realistic enough to study proactive household behavior but modular enough to debug.</p><p>Our project is therefore closer to a VLM-assisted planning benchmark than a full robot controller. The system follows this structure:</p><p><strong>Image → structured scene description → local planner → high-level semantic action</strong></p><p>rather than:</p><p><strong>Image + robot state → low-level motor policy</strong></p><p>This distinction is important. We are not claiming that the system can already control a household robot end-to-end. Instead, we evaluate whether small local models can make useful proactive decisions before connecting those decisions to full robot control.</p><h3><strong>Final System Overview</strong></h3><h4>The high-level architecture we planned looks like this:</h4><p><strong>Input observation: </strong>LIBERO-style scene image, real kitchen photo, or simulated kitchen snapshot<br> ↓<br><strong> VLM perception module: </strong>Qwen 2.5-VL-3B or Qwen 3.5 (2B or 4B) extracts structured scene state.<br> ↓<br><strong> Agent/tool wrapper: </strong>SmolAgents-style orchestration around perception, memory, and planning<br> ↓<br><strong> Local planner: </strong>Qwen 3.5–0.8B reads scene state + memory and chooses a high-level semantic action.<br> ↓<br><strong> Output: </strong>wait / inspect / start_task / pick / place</p><h4>Example:</h4><pre><br>[scenario] dish_cleanup_0001<br><br>{<br>  &quot;scenario_id&quot;: &quot;dish_cleanup_0001&quot;,<br>  &quot;scene_state&quot;: {<br>    &quot;template_name&quot;: &quot;dish_cleanup&quot;,<br>    &quot;user_state&quot;: &quot;still_eating&quot;,<br>    &quot;objects&quot;: [&quot;dirty bowl&quot;, &quot;dirty plate&quot;, &quot;dirty plate&quot;],<br>    &quot;surfaces&quot;: [&quot;left_counter&quot;, &quot;right_counter&quot;, &quot;sink&quot;, &quot;drying_rack&quot;],<br>    &quot;held_object&quot;: &quot;bowl_1&quot;<br>  },<br>  &quot;planner_action&quot;: {<br>    &quot;action_type&quot;: &quot;wait&quot;,<br>    &quot;reason&quot;: &quot;The user is still eating, so clearing dishes now would be premature.&quot;,<br>    &quot;memory_used&quot;: [<br>      &quot;User prefers clean dishes on the left counter&quot;,<br>      &quot;User prefers starting from the right side first&quot;<br>    ]<br>  },<br>  &quot;evaluation&quot;: {<br>    &quot;valid_action_type&quot;: true,<br>    &quot;valid_target&quot;: true,<br>    &quot;proactive_score&quot;: 1.0,<br>    &quot;restraint_score&quot;: 1.0,<br>    &quot;progress_score&quot;: 1.0,<br>    &quot;total_score&quot;: 0.95<br>  }<br>}</pre><p>This example is useful because the correct proactive behavior is not always to act. Since the user is still eating, the best action is to wait. This shows why we included both proactive score and restraint score: a helpful robot needs initiative, but it also needs to avoid acting too early.</p><h4>Real Kitchen Images:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vACrQHJtIq09CJ6fAzEUVw.png" /><figcaption>Example kitchen image: a used cup on a table. The desired behavior is to recognize this as a possible cleanup object, but only act if the broader scene suggests the user is finished.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*djX5WMnoN-HdXzq1DRbXdg.png" /><figcaption>Example kitchen image: a used dish after eating. This represents the dish-cleanup task family, where the system should usually begin cleanup if the user appears finished.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XswoaN1aHeM12q-FtvqGrQ.png" /><figcaption>Example kitchen image: possible trash on the floor. Trash scenes were the hardest because the system had to decide whether the object was actually trash and whether picking it up was safe.</figcaption></figure><h4>Evaluation metrics:</h4><p>Before looking at the results, we should go over what scores we were measuring.</p><p>We didn’t want to evaluate the planner using only simple right-or-wrong labels. We tracked 5 core metrics:</p><ol><li>Valid action rate: Did the planner choose an action that was valid for the current state? For example, did it choose a legal action type, and did it target an object or surface that actually existed?</li><li>Proactive Score: Did the robot act when it should have acted, instead of waiting too long?</li><li>Restraint Score: Did the robot avoid acting when it should have waited?</li><li>Progress Score: Did the chosen action actually move the task in the right direction?</li><li>Memory alignment score: When memory mattered, did the chosen action reflect stored preferences or relevant prior experience?</li></ol><p>These metrics mattered since our real question wasn’t just whether the model picked an action.</p><p>It was:</p><ul><li>Can it act usefully?</li><li>Can it act safely?</li><li>Can it act proactively without being intrusive?</li><li>Can it use memory without becoming brittle?</li></ul><p>The rest of the project can be read as evidence for this final system design. We first built a text-only local planner to test whether a small model could choose useful kitchen actions from structured scene states. Once that planner became reliable, we tested whether memory helped or hurt. We then tried lightweight LoRA fine-tuning, moved into symbolic kitchen simulation, connected the planner to image-based VLM perception, and finally wrapped the perception-planning-execution loop in SmolAgents.</p><p>In other words, the phases below are not separate, unrelated versions. They are an incremental argument for the final system: structured semantic actions made local planning reliable, naive memory was not enough, fine-tuning did not beat the strongest prompt-based planner, and the best practical image-based pipeline used Qwen3.5–2B perception with a Qwen3.5–0.8B local planner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*Qc5ugBRzzJfCFYrGvE7jOA.png" /><figcaption>Table for Metrics Evaluation Description</figcaption></figure><h3>Materials and Methods</h3><p>We implemented the project as a modular local pipeline rather than a single end-to-end robot policy. The pipeline has four main components: perception, planning, memory, and evaluation.</p><p><strong>Perception.</strong> The perception stage converts a kitchen image into a structured scene description. In the image-based experiments, we compared Qwen2.5-VL-3B-Instruct, Qwen3.5–2B, and Qwen3.5–4B as candidate vision-language frontends. These models were not used to directly control robot motion. Their role was to identify the task type, relevant objects, surfaces, and whether an object appeared actionable.</p><p><strong>Planning.</strong> The planner uses Qwen3.5–0.8B running locally through LM Studio. Given a structured scene state, it chooses one high-level semantic action: wait, inspect, start_task, pick, or place. We intentionally used semantic actions instead of low-level motor commands because the goal was to evaluate proactive decision-making, not robot control.</p><p><strong>Memory.</strong> The memory module stores simple preferences and experiences, such as where a user prefers clean dishes to be placed. We tested whether adding memory improved decisions or made the planner more brittle. This became one of the central abstractions in the project.</p><p><strong>Agentic orchestration and execution.</strong> SmolAgents was used to wrap the perception, planning, and symbolic execution steps into a tool-based loop. KitchenSim provided a simple symbolic environment where actions could be executed and scored. This let us test whether the system could inspect a scene, plan an action, execute it symbolically, and receive an evaluation result.</p><p><strong>Hardware and setup.</strong> Experiments were run locally. The planner used LM Studio, and LoRA fine-tuning experiments were run on an RTX 4070. Separate visual simulator smoke tests used WSL/Linux, LeRobot, LIBERO, and MuJoCo to verify that a Franka Panda simulation backend could run, but that simulator was future integration work rather than the core evaluated system.</p><h3>Design Roadmap:</h3><p>Once the baseline planner was working, the roadmap became much clearer:</p><ul><li><strong>Phase A (Baseline local planner): </strong>built the first local Qwen-based planner that maps structured scene states to semantic actions.</li><li><strong>V1 (First scenario benchmark):</strong> ran the first generated kitchen/task benchmark; useful, but weak overall.</li><li><strong>V2 (Prompt/schema rewrite):</strong> tightened the prompt and output format, but performance stayed weak.</li><li><strong>V3 (Strong semantic action planner):</strong> introduced the stronger action_type-based schema and became the main planner baseline.</li><li><strong>Phase B (Compressed long history):</strong> explored using a more compact historical context instead of dumping raw memory.</li><li><strong>Phase C (Selective memory retrieval):</strong> tested retrieving only memory relevant to the current scene.</li><li><strong>V4 (Selective memory result):</strong> improved validity, but overall behavior became too conservative compared with V3.</li><li><strong>Phase D (LoRA fine-tuning):</strong> tested lightweight fine-tuning on synthetic planning/history/action data, but it did not beat the best prompt-based baseline.</li><li><strong>Phase E (Agentic kitchen benchmark direction):</strong> moved toward SmolAgents-style orchestration, kitchen simulation, and eventual LIBERO alignment.</li><li><strong>LIBERO setup (Visual Franka smoke test):</strong> proved that the visual backend could run with Franka Panda, even though Qwen was not yet controlling it.</li><li><strong>V5 (VLM perception bridge):</strong> connected real kitchen images to structured scene_state JSON and then into the Qwen planner.</li><li><strong>V6 (Direct VLM benchmark):</strong> compared multiple VLM perception models on real kitchen images.</li><li><strong>V7 (SmolAgents VLM benchmark):</strong> routed the perception-to-action loop through a real SmolAgents tool pipeline, giving us a more fully agentic benchmark.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T9Rgq0wrXj1IjAX_BdWrww.png" /></figure><h4>Results and Ablations:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xOEs4SfcEuptiy3OqRjVtg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aHL7k3GTIbAA-2mbjS3hLg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f5X2dXX9LQOVLmijfdl5QA.png" /></figure><p><a href="https://docs.google.com/spreadsheets/d/e/2PACX-1vQ3e6OTi0o7-Y4YLYw2Zb5eO71b3SUA3werBV2LfQGWv5krVvKE83vIbDnuO5DdJRmTFVF32WsRRolz/pubhtml">https://docs.google.com/spreadsheets/d/e/2PACX-1vQ3e6OTi0o7-Y4YLYw2Zb5eO71b3SUA3werBV2LfQGWv5krVvKE83vIbDnuO5DdJRmTFVF32WsRRolz/pubhtml</a></p><h3>V1: Planner First Phase</h3><p>The first planner-only versions tested whether Qwen3.5–0.8B could choose semantic actions from structured kitchen scenes. The planner was constrained to use the scene state first, avoid invented objects, return valid JSON, and choose from a small action space such as wait, start_task, pick, clean_surface, inspect, and place.</p><p>These early versions were weak but useful. On the first 10-scenario benchmark, V1 and V2 both reached an average total score of 0.515 and a valid action rate of 0.4. This showed that the model sometimes had useful proactive instincts, but it was not yet reliable enough for the full benchmark. The main value of V1/V2 was proving that the pipeline, parser, memory module, scenario generator, and evaluator could work together.</p><h3><strong>Results</strong></h3><h4>First benchmarks V1 and V2:</h4><p>Our earlier versions were weak, but useful:</p><h4>V1: The planner achieved:</h4><pre>=== Evaluation Summary ===<br>{<br> &quot;num_scenarios&quot;: 10, <br> &quot;avg_total_score&quot;: 0.515, <br> &quot;valid_action_rate&quot;: 0.4,<br> &quot;avg_proactive_score&quot;: 0.75, <br> &quot;avg_restraint_score&quot;: 0.5,<br> &quot;avg_progress_score&quot;: 0.4, <br> &quot;avg_memory_alignment_score&quot;: 0.6<br>}</pre><p>This was over 10 scenarios. This result showed that the planner sometimes made proactive instincts, but it was still unreliable about choosing valid actions and making consistent progress.</p><h4>V2: prompt and schema tightening</h4><pre>=== Evaluation Summary ===<br>{<br>  &quot;num_scenarios&quot;: 10,<br>  &quot;avg_total_score&quot;: 0.515,<br>  &quot;valid_action_rate&quot;: 0.4,<br>  &quot;avg_proactive_score&quot;: 0.75,<br>  &quot;avg_restraint_score&quot;: 0.5,<br>  &quot;avg_progress_score&quot;: 0.4,<br>  &quot;avg_memory_alignment_score&quot;: 0.65<br>}</pre><p>This performed similarly, which showed that this round was more about stabilizing schema and planner structure than improving behavior. The model interface got clearer, but the underlying decision quality didn’t change.</p><p>We rewrote parts of prompt.py and planner.py and used stricter instructions. Making the prompt stricter wasn’t aligned well enough with the evaluator and parsers.</p><h4>V3: The breakthrough</h4><p>The first genuinely strong result came in V3.</p><p>The key change was moving from a simple action field to a richer semantic schema built around action_type, plus fields such as:</p><ul><li>target_object</li><li>target_surface</li><li>target_zone</li><li>parameters</li><li>reason</li><li>memory_used</li><li>confidence</li></ul><p>We also explicitly restricted object and surface names to exact scene names, blocked invented objects, and told them to choose or inspect when appropriate.</p><p>This change worked:</p><p>With memory enabled: V3 achieved:</p><pre><br>=== Evaluation Summary ===<br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.857,<br>  &quot;valid_action_rate&quot;: 0.98,<br>  &quot;avg_proactive_score&quot;: 0.8,<br>  &quot;avg_restraint_score&quot;: 0.9,<br>  &quot;avg_progress_score&quot;: 0.8,<br>  &quot;avg_memory_alignment_score&quot;: 0.63<br>}</pre><p>and</p><p>Without memory:</p><pre>=== Evaluation Summary ===<br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.876,<br>  &quot;valid_action_rate&quot;: 0.98,<br>  &quot;avg_proactive_score&quot;: 0.9,<br>  &quot;avg_restraint_score&quot;: 0.95,<br>  &quot;avg_progress_score&quot;: 0.88,<br>  &quot;avg_memory_alignment_score&quot;: 0.36<br>}</pre><p>This became our strongest planner baseline. It gave us one of the most important findings in the project. Memory improved memory alignment, but hurt overall task performance. In other words, the problem wasn’t that memory was useless. The problem was that the memory needed to be retrieved and integrated much more carefully. Naive memory could make the planner worse.</p><h3>Phase B and Phase C: Better Memory</h3><p>After V3 became the strongest planner baseline, we tested whether better memory retrieval could improve performance. Phase B compressed long-term history into summaries, and Phase C retrieved only task-relevant memory, such as placement preferences or prior failures.</p><p>This produced V4, which achieved perfect validity but lower overall task performance:</p><h4>V4: Selective Memory Retrieval</h4><pre>=== Evaluation Summary ===<br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.77,<br>  &quot;valid_action_rate&quot;: 1.0,<br>  &quot;avg_proactive_score&quot;: 0.62,<br>  &quot;avg_restraint_score&quot;: 0.81,<br>  &quot;avg_progress_score&quot;: 0.62,<br>  &quot;avg_memory_alignment_score&quot;: 0.6<br>}<br><br></pre><p>The result showed that memory was useful but brittle. It improved structural correctness and memory alignment, but it also made the planner less decisive. For that reason, V3 remained the main text-only planner baseline.</p><h3>Phase D: LoRA</h3><p>We also tested whether lightweight LoRA fine-tuning could improve the planner. This phase used a synthetic SFT dataset and fine-tuned Qwen3.5–0.8B on local hardware with an RTX 4070.</p><pre> Phase D-Lora-Small:<br> &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.662,<br>  &quot;valid_action_rate&quot;: 1.0,<br>  &quot;avg_proactive_score&quot;: 0.4,<br>  &quot;avg_restraint_score&quot;: 0.7,<br>  &quot;avg_progress_score&quot;: 0.4,<br>  &quot;avg_memory_alignment_score&quot;: 0.62<br>}</pre><pre><br>Phase-D-Lora-Big:<br><br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.804,<br>  &quot;valid_action_rate&quot;: 0.84,<br>  &quot;avg_proactive_score&quot;: 0.82,<br>  &quot;avg_restraint_score&quot;: 0.91,<br>  &quot;avg_progress_score&quot;: 0.72,<br>  &quot;avg_memory_alignment_score&quot;: 0.62<br>}</pre><p>The small LoRA run underperformed clearly, reaching an average total score of 0.662. The larger LoRA run was stronger, reaching 0.804, but it still did not beat the best prompt-based V3 planner, which reached 0.876 without memory. This made LoRA a useful experiment, but not the main result of the project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4pXWxS4JWGfAPUclwFSYYw.png" /></figure><h3>Phase E: Agentic Kitchen Simulation</h3><p>At this point, the project needed to align directly with expectations. Kitchen-focused proactive behavior, semantic simulated test-described scenes, and an agentic framework like SmolAgents, with Qwen as a backend, VLM-ready image dataset, and eventually a LIBERO/Franka Panda connection.</p><p>So we added</p><ul><li>trash_cleanup</li><li>counter_cleanup</li><li>KitchenSim</li><li>scene_text.py</li><li>run_kitchen_agentic_benchmark.py</li><li>smolagents_kitchen_demo.py</li></ul><p>After several iterations and fixes, the benchmark settled around <strong>three core kitchen scenarios</strong>:</p><ol><li>Dish cleanup after eating</li><li>Trash on the floor</li><li>Picking up used cups from the counter or workspace</li></ol><p>This was an important transition point in the project. Up to this stage, most of the work had focused on planner quality in synthetic or structured scene settings. Phase E moved the project closer to an actual agentic kitchen benchmark, where the planner had to operate over multiple task types and produce actions that could be symbolically executed and evaluated.</p><h4>Agentic Kitchen Benchmark:</h4><pre><br>=== Overall Agentic Kitchen Benchmark Summary ===<br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.8,<br>  &quot;valid_action_rate&quot;: 0.94,<br>  &quot;avg_proactive_score&quot;: 0.8,<br>  &quot;avg_restraint_score&quot;: 0.88,<br>  &quot;avg_progress_score&quot;: 0.72,<br>  &quot;avg_memory_alignment_score&quot;: 0.38,<br>  &quot;avg_decision_time_sec&quot;: 0.4468,<br>  &quot;median_decision_time_sec&quot;: 0.4375,<br>  &quot;execution_success_rate&quot;: 0.92<br>}<br></pre><p>The symbolic Agentic Kitchen benchmark reached an average total score of 0.800 across 50 scenarios, with a valid action rate of 0.94, proactive score of 0.80, restraint score of 0.88, progress score of 0.72, and execution success rate of 0.92.</p><p>That was a strong sign that the system could already support useful proactive kitchen behavior in a symbolic setting. The planner was valid most of the time, generally proactive when it should be, and fairly restrained when waiting was more appropriate.</p><p>However, the biggest weakness was immediately clear: <strong>memory alignment remained low</strong>. The system could often act correctly, but it was still inconsistent about using stored preferences and prior experience in a way that meaningfully shaped the action choice.</p><h4>Per-Task Performance</h4><pre>=== Per-task summaries ===<br><br>[counter_cleanup]<br>{<br>  &quot;num_scenarios&quot;: 20,<br>  &quot;avg_total_score&quot;: 0.875,<br>  &quot;valid_action_rate&quot;: 1.0,<br>  &quot;avg_proactive_score&quot;: 0.9,<br>  &quot;avg_restraint_score&quot;: 0.95,<br>  &quot;avg_progress_score&quot;: 0.9,<br>  &quot;avg_memory_alignment_score&quot;: 0.25<br>}<br><br>[dish_cleanup]<br>{<br>  &quot;num_scenarios&quot;: 17,<br>  &quot;avg_total_score&quot;: 0.8765,<br>  &quot;valid_action_rate&quot;: 1.0,<br>  &quot;avg_proactive_score&quot;: 0.8235,<br>  &quot;avg_restraint_score&quot;: 0.9118,<br>  &quot;avg_progress_score&quot;: 0.8235,<br>  &quot;avg_memory_alignment_score&quot;: 0.6471<br>}<br><br>[trash_cleanup]<br>{<br>  &quot;num_scenarios&quot;: 13,<br>  &quot;avg_total_score&quot;: 0.5846,<br>  &quot;valid_action_rate&quot;: 0.7692,<br>  &quot;avg_proactive_score&quot;: 0.6154,<br>  &quot;avg_restraint_score&quot;: 0.7308,<br>  &quot;avg_progress_score&quot;: 0.3077,<br>  &quot;avg_memory_alignment_score&quot;: 0.2308<br>}</pre><p>This breakdown was useful. It showed that the planner handled dish cleanup and counter cleanup relatively well, but struggled much more with trash-focused scenes. In particular, the planner often had difficulty maintaining progress and using memory effectively in those cases.</p><p>Rather than treating that as a failure, we treated it as a diagnostic result: trash cleanup exposed weaknesses in prompt design, memory routing, and task-specific handling that were less obvious in the easier dish-cleanup scenarios.</p><h4>SmolAgents Benchmark:</h4><p>After that, we pushed the benchmark into a more explicitly agentic form by routing the pipeline through a SmolAgents tool loop.</p><p>This gave us a benchmark where the system had to:</p><ol><li>Inspect the scene</li><li>Plan a proactive action</li><li>Execute the action symbolically</li><li>Produce a scored result</li></ol><p>That produced the following overall benchmark result:</p><pre>=== SmolAgents Kitchen Benchmark Summary ===<br>{<br>  &quot;num_scenarios&quot;: 50,<br>  &quot;avg_total_score&quot;: 0.828,<br>  &quot;valid_action_rate&quot;: 1.0,<br>  &quot;avg_proactive_score&quot;: 0.76,<br>  &quot;avg_restraint_score&quot;: 0.86,<br>  &quot;avg_progress_score&quot;: 0.74,<br>  &quot;avg_memory_alignment_score&quot;: 0.56,<br>  &quot;avg_decision_time_sec&quot;: 0.53,<br>  &quot;median_decision_time_sec&quot;: 0.5052,<br>  &quot;avg_agent_time_sec&quot;: 1.8809,<br>  &quot;execution_success_rate&quot;: 0.98,<br>  &quot;agent_error_rate&quot;: 0.0<br>}<br></pre><p>This result was especially encouraging for three reasons.</p><ol><li>The SmolAgents tool loop worked reliably.</li><li>The valid action rate reached 100%.</li><li>The agent error rate was 0%, meaning the orchestration framework itself was stable in this benchmark setting.</li></ol><pre>=== Per-task summaries ===<br>{<br>  &quot;counter_cleanup&quot;: {<br>    &quot;num_scenarios&quot;: 20,<br>    &quot;avg_total_score&quot;: 0.8825,<br>    &quot;valid_action_rate&quot;: 1.0,<br>    &quot;avg_proactive_score&quot;: 0.85,<br>    &quot;avg_restraint_score&quot;: 0.925,<br>    &quot;avg_progress_score&quot;: 0.85,<br>    &quot;avg_memory_alignment_score&quot;: 0.575<br>  },<br>  &quot;dish_cleanup&quot;: {<br>    &quot;num_scenarios&quot;: 17,<br>    &quot;avg_total_score&quot;: 0.9088,<br>    &quot;valid_action_rate&quot;: 1.0,<br>    &quot;avg_proactive_score&quot;: 0.8824,<br>    &quot;avg_restraint_score&quot;: 0.9412,<br>    &quot;avg_progress_score&quot;: 0.8824,<br>    &quot;avg_memory_alignment_score&quot;: 0.6765<br>  },<br>  &quot;trash_cleanup&quot;: {<br>    &quot;num_scenarios&quot;: 13,<br>    &quot;avg_total_score&quot;: 0.6385,<br>    &quot;valid_action_rate&quot;: 1.0,<br>    &quot;avg_proactive_score&quot;: 0.4615,<br>    &quot;avg_restraint_score&quot;: 0.6538,<br>    &quot;avg_progress_score&quot;: 0.3846,<br>    &quot;avg_memory_alignment_score&quot;: 0.3846<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mrit-OPSii0KvgbwhKg0A.png" /></figure><p>The pattern stayed consistent. Dish cleanup remained the strongest task, counter cleanup also performed well, and trash cleanup was still the hardest. But compared with the earlier symbolic benchmark, the SmolAgents version improved overall reliability, memory alignment, and execution success.</p><p>That made Phase E one of the most important steps in the project: it moved the system from being “just a planner benchmark” toward being a real agentic benchmark.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9iIreCS_w8dVLZS4Lw-9Bw.png" /></figure><h3>LIBERO and Franka Panda Smoke Test</h3><p>We also ran a LIBERO/Franka Panda smoke test to confirm that a visual robot simulation backend could run. This was future integration work rather than a core evaluated result: Pi0 controlled the Franka in this test, not Qwen, and the main contribution of this phase remained the kitchen-focused agentic benchmark.</p><h3>V5: VLM perception Bridge.</h3><p>This was the point at which the project became image-based rather than just a semantic text/planner-based.</p><p>The new pipeline became:</p><p>Real kitchen image<br> → VLM perception<br> → structured scene_state JSON<br> → Qwen 3.5–0.8B planner<br> → semantic action.</p><p>We added:</p><ul><li>vlm_client.py</li><li>vlm_perception.py</li><li>run_vlm_to_action.py</li><li>image normalization rules</li><li>a real kitchen image dataset (39 images with annotation for validation)</li></ul><p>At this point, we had a real image to action bridge.</p><p>We used Qwen 2.5 VL 3B as our basemodel.</p><h3>V6: Direct VLM model benchmark</h3><p>Once the image-to action bridge worked, the next step was obvious:</p><p>Compare VLM perception models systematically and see which one gives the best output, in which the planner could perform as close to the V3 baseline as possible.</p><p>To do this we built a direct VLM benchmark around real annotated kitchen images and compared three candidate perception frontends:</p><ul><li>Qwen 2.5-VL 3B Instruct</li><li>Qwen 3.5–2B</li><li>Qwen 3.5–4B</li></ul><p>We added vlm_annotations.csv and benchmark_vlm_models.py, first on 19 images, then on an expanded 39-image benchmark. We evaluated and tracked perception/action quality, planner-facing behavior, and latency.</p><pre>=== Summary ===<br>{<br>  &quot;qwen2.5-vl-3b-instruct&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br>      &quot;avg_score&quot;: 0.7128,<br>      &quot;template_accuracy&quot;: 0.6923,<br>      &quot;object_presence_accuracy&quot;: 0.7949,<br>      &quot;action_accuracy&quot;: 0.6667,<br>      &quot;unsafe_false_positive_rate&quot;: 0.0513,<br>      &quot;avg_vlm_latency_sec&quot;: 10.0772,<br>      &quot;avg_planner_latency_sec&quot;: 3.3028,<br>      &quot;avg_total_latency_sec&quot;: 13.3803<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 39,<br>      &quot;avg_total_score&quot;: 0.759,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;avg_proactive_score&quot;: 0.6667,<br>      &quot;avg_restraint_score&quot;: 0.7564,<br>      &quot;avg_progress_score&quot;: 0.6667,<br>      &quot;avg_memory_alignment_score&quot;: 0.641<br>    }<br>  },<br>  &quot;qwen3.5-2b&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br>      &quot;avg_score&quot;: 0.7333,<br>      &quot;template_accuracy&quot;: 0.6667,<br>      &quot;object_presence_accuracy&quot;: 0.7692,<br>      &quot;action_accuracy&quot;: 0.7949,<br>      &quot;unsafe_false_positive_rate&quot;: 0.1026,<br>      &quot;avg_vlm_latency_sec&quot;: 6.7662,<br>      &quot;avg_planner_latency_sec&quot;: 3.3177,<br>      &quot;avg_total_latency_sec&quot;: 10.0841<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 39,<br>      &quot;avg_total_score&quot;: 0.8077,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;avg_proactive_score&quot;: 0.8333,<br>      &quot;avg_restraint_score&quot;: 0.8205,<br>      &quot;avg_progress_score&quot;: 0.7949,<br>      &quot;avg_memory_alignment_score&quot;: 0.641<br>    }<br>  },<br>  &quot;qwen3.5-4b&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br>      &quot;avg_score&quot;: 0.2513,<br>      &quot;template_accuracy&quot;: 0.1795,<br>      &quot;object_presence_accuracy&quot;: 0.2821,<br>      &quot;action_accuracy&quot;: 0.2821,<br>      &quot;unsafe_false_positive_rate&quot;: 0.0,<br>      &quot;avg_vlm_latency_sec&quot;: 53.2404,<br>      &quot;avg_planner_latency_sec&quot;: 3.1842,<br>      &quot;avg_total_latency_sec&quot;: 56.4248<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 39,<br>      &quot;avg_total_score&quot;: 0.8221,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;avg_proactive_score&quot;: 0.859,<br>      &quot;avg_restraint_score&quot;: 0.7179,<br>      &quot;avg_progress_score&quot;: 0.7128,<br>      &quot;avg_memory_alignment_score&quot;: 0.641<br>    }<br>  }<br>}<br><br>Saved:<br>results\vlm_model_benchmark_20260430_110534\all_results.csv<br>results\vlm_model_benchmark_20260430_110534\summary.json<br>results\vlm_model_benchmark_20260430_110534\details</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*707DCsyjslpBoO4q17tuAw.png" /></figure><p>The pattern was clear.</p><p>Qwen 3.5 2B was faster and had a better overall perception for the current pipeline. It had the highest average score, highest action accuracy, fastest latency, and strongest overall planner-facing behavior among the usable models.</p><p>Qwen 2.5 VL 3B Instruct was still the “safer” model. It had a lower unsafe false positive rate, but it was slower and less accurate overall.</p><p>Despite sounding stronger on paper, Qwen 3.5 4B performed poorly as a perception frontend in our current setup. It had low template accuracy, object-presence accuracy, and action accuracy, while being significantly slower than the other two models. Its unsafe false positive rate was 0, but that came with weak perception quality overall. It was not a competitive choice for the current benchmark.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5buBSnuy7PFsDi4eR0n80Q.png" /></figure><p>Thus, the conclusion here was that Qwen 3.5 2B was the current perception winner because it gave the best overall balance of accuracy, speed, and downstream planner performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jQTHCZM9PK-AlJnjThORNA.png" /></figure><h3>V7: SmolAgent VLM Benchmark</h3><p>The last major version was V7, in which VLM-to-action benchmark was routed through an actual SmolAgent tool loop, so the “agentic framework” was real.</p><p>The pipeline was:</p><p>image<br> → SmolAgents calls inspect_image_scene<br> → VLM extracts scene_state<br> → SmolAgents calls plan_proactive_robot_action<br> → Qwen 3.5 0.8B plans<br> → SmolAgents calls execute_last_robot_action_symbolically<br> → benchmark scores output.</p><p>On the 39-image SmolAgents benchmark:</p><pre>=== Summary ===<br>{<br>  &quot;qwen2.5-vl-3b-instruct&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br><br>      &quot;avg_score&quot;: 0.7026,<br>      &quot;template_accuracy&quot;: 0.6923,<br>      &quot;object_presence_accuracy&quot;: 0.7949,<br>      &quot;action_accuracy&quot;: 0.641,<br>      &quot;unsafe_false_positive_rate&quot;: 0.0513,<br>      &quot;valid_action_rate&quot;: 0.9744,<br>      &quot;agent_error_rate&quot;: 0.0256,<br>      &quot;avg_vlm_latency_sec&quot;: 9.7976,<br>      &quot;avg_planner_latency_sec&quot;: 0.5263,<br>      &quot;avg_execution_latency_sec&quot;: 0.0,<br>      &quot;avg_agent_latency_sec&quot;: 15.3496,<br>      &quot;avg_total_latency_sec&quot;: 15.3887<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 38,<br>      &quot;avg_total_score&quot;: 0.7618,<br>      &quot;valid_action_rate&quot;: 0.9744,<br>      &quot;avg_proactive_score&quot;: 0.6579,<br>      &quot;avg_restraint_score&quot;: 0.75,<br>      &quot;avg_progress_score&quot;: 0.6579,<br>      &quot;avg_memory_alignment_score&quot;: 0.6447<br>    }<br>  },<br>  &quot;qwen3.5-2b&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br>      &quot;avg_score&quot;: 0.7333,<br>      &quot;template_accuracy&quot;: 0.6667,<br>      &quot;object_presence_accuracy&quot;: 0.7692,<br>      &quot;action_accuracy&quot;: 0.7949,<br>      &quot;unsafe_false_positive_rate&quot;: 0.1026,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;agent_error_rate&quot;: 0.0,<br>      &quot;avg_vlm_latency_sec&quot;: 6.3599,<br>      &quot;avg_planner_latency_sec&quot;: 0.5257,<br>      &quot;avg_execution_latency_sec&quot;: 0.0,<br>      &quot;avg_agent_latency_sec&quot;: 11.7101,<br>      &quot;avg_total_latency_sec&quot;: 11.7101<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 39,<br>      &quot;avg_total_score&quot;: 0.7923,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;avg_proactive_score&quot;: 0.8333,<br>      &quot;avg_restraint_score&quot;: 0.8205,<br>      &quot;avg_progress_score&quot;: 0.7949,<br>      &quot;avg_memory_alignment_score&quot;: 0.641<br>    }<br>  },<br>  &quot;qwen3.5-4b&quot;: {<br>    &quot;perception_action_summary&quot;: {<br>      &quot;num_cases&quot;: 39,<br>      &quot;avg_score&quot;: 0.2513,<br>      &quot;template_accuracy&quot;: 0.1795,<br>      &quot;object_presence_accuracy&quot;: 0.2821,<br>      &quot;action_accuracy&quot;: 0.2821,<br>      &quot;unsafe_false_positive_rate&quot;: 0.0,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;agent_error_rate&quot;: 0.0,<br>      &quot;avg_vlm_latency_sec&quot;: 53.5753,<br>      &quot;avg_planner_latency_sec&quot;: 0.4998,<br>      &quot;avg_execution_latency_sec&quot;: 0.0,<br>      &quot;avg_agent_latency_sec&quot;: 112.593,<br>      &quot;avg_total_latency_sec&quot;: 112.5932<br>    },<br>    &quot;planner_evaluator_summary&quot;: {<br>      &quot;num_scenarios&quot;: 39,<br>      &quot;avg_total_score&quot;: 0.8297,<br>      &quot;valid_action_rate&quot;: 1.0,<br>      &quot;avg_proactive_score&quot;: 0.8718,<br>      &quot;avg_restraint_score&quot;: 0.7436,<br>      &quot;avg_progress_score&quot;: 0.7128,<br>      &quot;avg_memory_alignment_score&quot;: 0.641<br>    }<br>  }<br>}<br><br>Saved:<br>results\smolagents_vlm_benchmark_20260429_233539\all_results.csv<br>results\smolagents_vlm_benchmark_20260429_233539\summary.json<br>results\smolagents_vlm_benchmark_20260429_233539\details<br><br><br></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mVqhkWcNOOZ_bQH468ve3g.png" /></figure><p>That gave us a very usable result:</p><ul><li>The SmolAgents loop worked reliably</li><li>The error rate was 0%</li><li>Valid action rate was 100%</li><li>Qwen3.5–2B remained the best overall perception frontend once the full agentic loop was included.</li><li>However, Qwen3.5–2B also had the highest unsafe false-positive rate among the usable perception models. This means it was the best overall performer, but not automatically the safest choice. In a real robot setting, this tradeoff would need to be handled with stronger safety filters or a more conservative action threshold.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mVqhkWcNOOZ_bQH468ve3g.png" /></figure><h3>Discussion and Conclusion</h3><p>The main result is that structured semantic planning made small local models much more reliable for proactive kitchen-assistance decisions. The V3 planner showed that Qwen3.5–0.8B could achieve high validity, proactivity, and restraint when the action schema was explicit and the planner was constrained to use valid scene objects and surfaces.</p><p>Memory was complicated. It improved alignment with stored preferences, but it did not consistently improve overall task performance. The selective-memory version became more structurally correct but less decisive, which suggests that memory retrieval needs stronger filtering and better integration before it can reliably improve proactive behavior.</p><p>The image-based experiments showed that perception quality and latency strongly affect the full system. Qwen3.5–2B was the strongest practical perception frontend because it gave the best tradeoff between action accuracy, speed, and downstream planner behavior. However, it also had the highest unsafe false-positive rate among the usable perception models, so safety filtering remains necessary before this kind of system could be used in a real robot.</p><p>Overall, this project is not a completely autonomous household robot. It is a modular VLM-to-action kitchen benchmark with image-based perception, a local Qwen planner, SmolAgents-style orchestration, symbolic one-step execution, and explicit evaluation metrics. The broader conclusion is that small local models can support useful proactive kitchen-action planning under realistic compute constraints, but memory integration, perception reliability, safety tradeoffs, and movement beyond symbolic execution remain open challenges.</p><h3>Future Work</h3><p>The next step is to expand the benchmark, add safety filters, and move from one-step symbolic execution toward longer simulated rollouts. Future work should also test whether memory can be integrated selectively, whether perception and planning should remain separate, and how this modular pipeline compares with a lightweight end-to-end VLA baseline.</p><p>These next steps follow directly from the main finding of the project: modular local planning is viable, but safe proactive household behavior requires better perception, better memory use, and stronger execution grounding.</p><p>Github:</p><p><a href="https://github.com/SUPERINTIALD/FinalProject_CSCI7000">GitHub - SUPERINTIALD/FinalProject_CSCI7000: CSCI 7k Final Proj, Generative AI &amp; Encoder-Decoder for Robotics</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ec8331d9ee3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/proactive-agents-a-local-vlm-to-action-benchmark-for-kitchen-assistance-5ec8331d9ee3">Proactive Agents: A Local VLM-to-Action Benchmark for Kitchen Assistance</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Train Once, Adapt Online: A Hands-On Tour of Rapid Motor Adaptation (RMA)]]></title>
            <link>https://medium.com/correll-lab/train-once-adapt-online-a-hands-on-tour-of-rapid-motor-adaptation-rma-77359aaa5502?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/77359aaa5502</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[rapid-motor-adaptation]]></category>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Niraj Pudasaini]]></dc:creator>
            <pubDate>Mon, 04 May 2026 18:20:23 GMT</pubDate>
            <atom:updated>2026-05-04T18:20:21.668Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A hands-on walk-through of Domain Randomization vs Rapid Motor Adaptation on `MountainCarContinuous-v0`</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HM3dq5kDpTMNPPO6q0oDfA.png" /></figure><h3><strong>Why this matters</strong></h3><p>You train a policy in a perfect simulator. You deploy it on a real robot. It fails. The simulator’s motors are exactly 0.0025 strong; your robot’s motors are 0.0019 today (battery’s a bit low) and 0.0027 tomorrow (just charged). Same trained policy, different outcomes.</p><p>This is the <strong>sim-to-real gap</strong>, and a huge fraction of modern robot-learning research is about closing it. The two dominant strategies bracket the design space:</p><ul><li><strong>Domain Randomization (DR)</strong>— train your policy across a d<em>istribution</em> of physics parameters. The policy learns to be robust by seeing all of them.</li><li><strong>Rapid Motor Adaptation (RMA)</strong> — train the policy to <em>infer</em> the physics from a short history of observations and actions, then condition its behavior on what it inferred.</li></ul><p>Read more about RMA here:</p><p><a href="https://medium.com/correll-lab/rma-rapid-motor-adaptation-for-walking-and-manipulation-154d3315cb50">RMA: Rapid Motor Adaptation for Walking and Manipulation</a></p><p>This post is a head-to-head comparison of continuous MountainCar with <strong>three randomized physics parameters</strong> — motor power, gravity coefficient, and starting position. Across 100 random samples from the joint distribution, <strong>RMA-deploy beats DR on every metric</strong> — mean return +96.0 vs +94.1, solve rate 1.00 vs 0.99, and steps-to-goal mean 61.2 vs 66.5 (~10% faster).</p><h3>The problem</h3><p>We use Gymnasium’s `MountainCarContinuous-v0`. The classic setup: a car in a valley needs to reach a flag on top of the right hill, but its motor isn’t strong enough to climb directly — it has to oscillate to build momentum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*IyBSqbWTmXX3ubGw.gif" /><figcaption>Mountain Car Continuous Environment</figcaption></figure><p>We chose “Mountain Car” because it cannot be solved with a one-size-fits-all controller. Instead, the optimal control policy depends on the parameters of the system: if the motors are strong enough to overcome gravity, simply driving up-hill will solve the problem. If they are too weak, the cart needs to build up momentum by going back and forth. Here, we will show that this “bifurcation” in the systems dynamics is exactly why domain randomization (DR), which averages over different parameters, fails, and how rapid motor adaptation (RMA) can solve this problem.</p><h4><strong>State and action spaces</strong></h4><p>The state is two scalars:</p><ul><li><strong>x</strong> — position, in [−1.2, 0.6]</li><li><strong>v</strong> — velocity, in [−0.07, 0.07]</li></ul><p>The action is one scalar:</p><ul><li><strong>a</strong> — throttle, in [−1, +1]. Positive pushes right, negative pushes left</li></ul><p>That’s it. Two-dimensional state, one-dimensional action, fully continuous.</p><h4><strong>Dynamics</strong></h4><p>Each timestep updates velocity, then position:</p><pre>v[t+1] = clip(v[t] + a[t]·P − G·cos(3·x[t]), −0.07, +0.07)<br>x[t+1] = clip(x[t] + v[t+1], −1.2, +0.6)</pre><p>Two interesting parameters here:</p><ul><li><strong>P</strong> — motor power. Scales the throttle. Higher P = stronger motor.</li><li><strong>G</strong> — gravity coefficient. Default 0.0025. Drives the cos(3x) term that makes the slope.</li></ul><p>The cos(3x) term is what makes MountainCar interesting: it&#39;s a sinusoidal hillscape with a valley around x ≈ -0.5 and a peak around x = 0. To reach the goal at x = 0.45, you have to climb out of the valley <em>and</em> over the central hump.</p><h3>Reward</h3><pre>r[t] = -0.1·a²[t] + (100 if reached_goal else 0)</pre><p>Two terms: a small per-step throttle penalty (don’t waste motor) and a big terminal bonus (reach the flag). The second term dominates — a typical solving episode pays maybe 3–5 throttle units total against a +100 goal bonus.</p><h3>What gets randomized</h3><p>Per episode, we sample three parameters from uniform distributions:</p><pre>P  ~ Uniform[0.0010, 0.0040]   # motor power<br>G  ~ Uniform[0.0015, 0.0035]   # gravity coefficient<br>x0 ~ Uniform[-0.85, -0.20]     # starting position</pre><p>P is the actuator (internal to the robot). G and x0 are environmental (slope, where you start). Together they form a 3-dimensional parameter distribution. The optimal policy depends on all three — and crucially, the optimal <em>strategy</em> changes across the distribution.</p><h3>The bifurcation: what makes this benchmark interesting</h3><p>Here’s the part that makes MountainCar a <em>good</em> benchmark for adaptation methods, not just a toy. Look at the gravity drag along the climb:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KMr5o8FXzyOFhys6uqy9tA.png" /></figure><p>The car starts somewhere in the gray band (x ∈ [-0.85, -0.20] — wide on purpose, to make the problem harder) and needs to reach the green flag at x = 0.45. To get there, it has to climb past x = 0 where gravity drag is at its peak. With max throttle (a = 1), the per-step velocity gain is:</p><pre>Δv = P − G·cos(3x)</pre><p>At the steepest point, x = 0 this is where Δv = P − G. So whenever P &gt; G, i.e. motor power exceeds gravity, the car never decelerates and trivially climbs out. Whenever P &lt; G, there&#39;s a region where Δv &lt; 0 and the car decelerates while climbing. That region might be survivable if the car carries enough kinetic energy, but at some critical P*, drive-direct stops working entirely.</p><p>When G is fixed at 0.0025 and x₀ ≈ -0.5, a bisection search against the simulator pinned this critical value at <strong>P* ≈ 0.00206</strong>. In practice, both G and x₀ are also randomized, so the bifurcation isn&#39;t a single number anymore — it&#39;s a <em>surface</em> in (P, G, x₀) space. Starting close to x₀ = -0.85 is actually easier because gravity on the left slope helps push the car rightward (free initial velocity); starting closer to x₀ = -0.20 is harder because you start halfway up the central hump with no gravity assist. The qualitative point survives: somewhere inside the joint distribution, drive-direct becomes physically impossible and oscillation is mandatory.</p><h4>Why this is a real bifurcation, not just “needing more power”</h4><p>This is the part that often confuses people. A bifurcation isn’t “you need enough power to succeed.” It’s that <strong>the optimal policy class qualitatively changes</strong> at the critical value.</p><p>To see this concretely, here are four rollouts from the same initial state (x₀, v₀) = (−0.5, 0) with two different powers and two different policies</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZRZ51vMIDe_AkcoMZn606w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*OV9i8qep6QhxjYCJOKBZaw.png" /></figure><p>Notice what happens at the same starting state (x = −0.5, v = 0):</p><ul><li>At <strong>high motor power </strong><strong>P</strong>, the optimal first action is +1 (push right and keep pushing).</li><li>At <strong>low </strong><strong>P</strong>, the optimal first action is −1 (back up first to build energy through oscillation, only then push right).</li></ul><p><strong>The optimal action sign flips depending on a parameter that the policy can’t observe.</strong> No single feed-forward map π(x, v) → a can be optimal in both rows — it would have to output +1 and −1 for the same input, which is impossible.</p><blockquote><strong>Hypothesis</strong>: This is the gap RMA exists to fill: it gives the policy <em>extra information</em> (the encoded latent z) so that π(x, v, z) can output different actions depending on which regime the car is in. Domain Randomization has no such escape; it has to commit to one action per state and live with the compromise.</blockquote><h3>Three methods to compare</h3><p>In simulation, we have ground truth — we know (P, G, x₀) for every episode. On a real robot, we don&#39;t. So we compare three settings:</p><ol><li><strong>DR</strong>: trains one policy π(x, v) → a across the parameter distribution. No knowledge of (P, G, x₀) ever, training, or deployment.</li><li><strong>RMA-oracle</strong>: trains a policy π(x, v, z) → a conditioned on a learned latent z = μ(P, G, x₀). At evaluation, we give it the <em>true</em> parameters. This is the <strong>upper bound</strong> on what any adaptive method could do.</li><li><strong>RMA-deploy</strong>: same trained policy as oracle, but at evaluation we replace the encoder with a learned adapter that infers z from a 30-step history of (state, action) pairs. <strong>Realistic deployment number.</strong></li></ol><p>The oracle is the diagnostic that tells us where future debugging efforts should go. If RMA-deploy is close to the oracle, the adapter is doing its job; if there’s a gap, the adapter (not the policy) is the bottleneck.</p><h3>Domain Randomization</h3><p>The objective is to maximize expected return <em>over the parameter distribution</em>:</p><pre>π_DR* = argmax E_(P,G,x0) [ J(π; environment with these params) ]</pre><p>In code, this is one line: at every env.reset(), sample fresh (P, G, x₀) from the joint distribution and run the episode with those values. Train SAC on top, normally. Done.</p><p>The policy never sees (P, G, x₀) — only (x, v). So whatever policy SAC converges to, it has to be a single map (x, v) → a that performs well <em>on average</em> across the entire 3-D parameter cube. This is structural: <strong>a feed-forward map can&#39;t change its output based on information it doesn&#39;t have.</strong></p><p>Training on the wider 3-parameter distribution is harder than DR usually has it. The training curve shows real instability:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*ImCooWazAYjyzL0joZJ3Xg.png" /></figure><p>What’s happening: SAC converges fast to a decent policy in the first 10k steps, then <strong>collapses</strong> as the optimization gets pulled in conflicting directions by samples from different regions of the parameter space. The policy oscillates between strategies that work for one corner of the distribution but not another. It eventually finds a single feed-forward map that works “well enough” everywhere — but the recovery takes another 50k steps.</p><p>This is exactly DR’s known failure mode. When the optimal action at a given (x, v) differs across the parameter distribution — say, &quot;push right hard&quot; at high P versus &quot;back up to build energy&quot; at low P — DR has to pick one that&#39;s &quot;okay&quot; in both regimes. The result is a conservative oscillation policy: works everywhere, optimal nowhere. After 150k steps, it does converge to a stable solution: 99% solve rate, mean return +94.1, mean steps-to-goal 66.5.</p><h3>Rapid Motor Adaptation</h3><p>RMA’s idea: don’t ask the policy to be robust. Ask it to <em>figure out</em> what physics it’s facing, and then act accordingly. This is split into two training phases.</p><h3>Phase 1 — privileged training</h3><p>Take the same SAC pipeline, but augment the observation with the true parameters:</p><pre>augmented_obs = [x, v, P_normalized, G_normalized, x0_normalized]   # ∈ ℝ⁵</pre><p>Then feed this through a learned <strong>encoder</strong>:</p><pre>μ : ℝ³ → ℝ⁴      # takes the 3 normalized params, outputs a 4-dim latent z</pre><p>The actor and critics see [x, v, μ(P, G, x₀)] ∈ ℝ⁶ — the original state plus the encoded latent. SAC then trains everything jointly: actor, twin critics, <em>and the encoder</em> all receive gradients from the same RL update.</p><p>This is the elegant part: <strong>no auxiliary loss for the encoder.</strong> The encoder learns a low-dimensional representation of (P, G, x₀) that makes the downstream conditional policy easy to learn. If it&#39;s useful for the actor to know &quot;you&#39;re in a high-power regime,&quot; that information will end up in z.</p><p>Training Phase 1 alongside DR-multi:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*xnVXPbOdADpne3BePBEjTQ.png" /></figure><p>This plot is striking. <strong>DR (blue)</strong> suffers from the long instability dip we just saw. <strong>RMA Phase 1 (orange)</strong> dips too — but recovers much faster, crosses the solved threshold by ~35k steps, and stabilizes by ~50k. The encoder-conditioned policy doesn’t have to compromise across regimes; it can specialize per-z. <strong>The training-time evidence already favors RMA on v2 even before we get to evaluation.</strong></p><h3>Phase 2 — supervised adaptation</h3><p>The encoder is frozen. We need to replace it with something that doesn’t require privileged access. Train an <strong>adapter</strong>:</p><pre>φ : ℝ⁹⁰ → ℝ⁴     # takes a 30-step (state, action) history, outputs ẑ</pre><p>The training data: roll out the Phase-1 policy on 1000 fresh episodes, recording the (s, a) history at every step plus the <em>true</em> latent z = μ(true_params) for that episode. Train φ by mean-squared regression to match z from the history alone:</p><pre>L = E_(h, z) [ ‖φ(h) − z‖² ]</pre><p>Pure supervised learning. Fast. Converges cleanly:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*1whgRPgcAcTlX3DrlZoiWg.png" /></figure><p>In v2 (3-parameter), MSE drops from ~0.41 at epoch 0 to ~0.088 by epoch 49 — a ~5× reduction on log-scale. Recovering a 3-D vector from history is structurally harder than recovering a 1-D scalar, but still tractable.</p><p>The intuition for <em>why</em> this works: the dynamics make the parameters identifiable from a few state transitions — Δv = a·P − G·cos(3x) is linear in P once you observe Δv, a, x. The adapter is a learned finite-history estimator for this regression. We&#39;re identifying 3 parameters jointly from the same 30-step window.</p><h4>Deployment</h4><p>At inference time:</p><ol><li>Run the policy normally, but maintain a rolling 30-step buffer of recent (state, action) pairs.</li><li>Each timestep, compute ẑ = φ(buffer).</li><li>Feed [state, ẑ] to the actor (bypassing the SB3 features extractor, since the deployment obs is now 2-d, not 5-d).</li><li>Append the chosen action to the buffer for next step.</li></ol><p>The first ~30 timesteps the buffer is mostly zeros, so we deliberately include zero-padded windows in Phase-2 training data so the adapter handles the cold start gracefully.</p><h3>Implementation Details</h3><p><strong>SAC + gSDE for exploration.</strong> MountainCar’s reward is sparse — random actions almost never reach the goal. Standard SAC with iid Gaussian action noise time-averages to zero and never builds momentum. We use <strong>generalized state-dependent exploration</strong> (gSDE), which replaces per-step noise with a smooth state-dependent perturbation that gets resampled every K steps. This is the difference between “SAC solves MountainCar in 30k steps” and “SAC never learns MountainCar.”</p><p><strong>The env wrapper does manual dynamics.</strong> Gym’s MountainCar hardcodes gravity = 0.0025 inside its step() function, so to randomize G we re-implement step() in our wrapper. Same equations, just with self.current_params[1] substituted for the constant.</p><p><strong>Parallel envs for throughput.</strong> SB3’s SubprocVecEnv lets you run 8 parallel environments per training, with one shared GPU model. With small networks (64–64 MLPs), the bottleneck is env stepping, not gradient compute — VecEnv shaves training from ~15 minutes to ~3 minutes per method.</p><p><strong>Architecture sizes.</strong></p><ul><li>Actor: 64–64 MLP, ReLU</li><li>Twin critics: 64–64 MLP, ReLU</li><li>Encoder μ: 32–32 MLP, Tanh, 3-d input → 4-d output</li><li>Adapter φ: 64–64 MLP, Tanh, 90-d input → 4-d output</li></ul><p>Tiny by deep-RL standards. Total parameters across the whole RMA pipeline are under 50,000.</p><p><strong>Training budget.</strong> 150,000 SAC steps for both DR and RMA Phase 1. RMA Phase 2 is 50 epochs of MLP regression over ~55k history-window samples.</p><h3>Results</h3><p>We evaluate by Monte-Carlo sampling 100 random (P, G, x₀) triples from the joint distribution and running one deterministic episode at each. For each method we report <strong>mean return</strong>, <strong>solve rate</strong>, and <strong>mean steps-to-goal</strong> (over solved episodes only).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H8IvWriify545f50-felqA.png" /><figcaption>The headline figure</figcaption></figure><p>Three panels, three angles on the same story:</p><ul><li><strong>Left (return distribution)</strong>: DR has a small tail of failed/poor episodes (the bar near -100 in the histogram); RMA-oracle and RMA-deploy concentrate almost entirely near the +95 to +100 cap. The tail is what drags DR&#39;s mean down.</li><li><strong>Middle (mean return + solve rate)</strong>: RMA-oracle and RMA-deploy bars sit slightly above DR on both mean return <em>and</em> solve rate. The gap is small in absolute terms but consistent.</li><li><strong>Right (steps-to-goal)</strong>: RMA’s box sits visibly below DR’s, with the IQRs barely overlapping. This is the cleanest visual evidence that RMA’s policy is structurally faster, not just luckier.</li></ul><p>Numerically,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*52ung4oxXpTNRm7jWL5ZqA.png" /></figure><p><strong>RMA-deploy beats DR on every metric.</strong> Mean return is +96.0 vs +94.1 (a ~2-point gap that&#39;s <em>not</em> explainable by adapter noise — the deployed adapter is within 0.1 points of the privileged oracle). Solve rate is 1.00 vs 0.99. Steps-to-goal mean is 61.2 vs 66.5 (~8% faster); steps-to-goal median is 57 vs 63 (~10% faster).</p><h3>Per-corner rollouts</h3><p>To see what’s happening in individual episodes, here are head-to-head rollouts at three hand-picked corners of the parameter cube:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*sDQmJaoJK1LLWT_0nc_1Aw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*pV8T3u4TU7khRM6s32fM-w.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*9Q-2nKCNHQ70m4t8jTznHw.gif" /><figcaption><em>[Left] RMA-multi-deploy at the same easy corner: drives more directly because the adapter recognized this is a high-power regime. 55 steps. [Right]DR-multi at the easy corner : oscillates anyway because that&#39;s its single amortized policy. 61 steps.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*N4_zP_L8Jrq0Q7Kg3fRbvg.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*G68VD0clm88l3vC3_sdZzw.gif" /><figcaption><em>DR-multi at the hard corner: oscillates, escapes in 50 steps. RMA-multi-deploy at the same hard corner: also oscillates, escapes in 49 steps. Both have to do the same thing here, and they do.</em></figcaption></figure><h4>A geometry quirk worth flagging</h4><p>The “hard” case isn’t actually the slowest — at x₀ = −0.80 the car starts on the <em>left</em> slope of the valley where cos(3·−0.80) ≈ −0.74, and the gravity term −G·cos(3x) becomes <em>positive</em>. Gravity is helping you accelerate rightward. Both DR and RMA exploit this and finish in ~50 steps despite the low motor power.</p><p>The truly slow corner is low P combined with x₀ ≈ −0.30 — uphill side of the valley, no gravity assist, weak motor. That&#39;s where both methods take 100+ steps.</p><h3>Discussion</h3><h4>Why the headline gap is small in absolute terms</h4><p>The +1.9 mean-return gap (+96.0 vs +94.1) sounds modest, and that&#39;s because the reward function caps how dramatic any return gap can be. Mean return is dominated by the +100 goal bonus — as long as a method reaches the goal, it scores around +95 to +98 regardless of how it got there. The per-step throttle penalty (-0.1·a²) is small and varies the result by only a few points either way.</p><p>So the +1.9 gap breaks down roughly into two pieces:</p><ol><li><strong>DR fails ~1% of episodes</strong> (solve rate 0.99 vs RMA&#39;s 1.00). Each failure returns roughly −100, so a 1% failure rate alone drags DR&#39;s mean down by ~1 point.</li><li><strong>DR is also slightly less efficient when it does solve</strong> — ~6 extra steps on average translate to ~1 more point of accumulated throttle penalty.</li></ol><p>The steps-to-goal advantage is much bigger in <em>relative</em> terms (~10% faster), but mean return doesn&#39;t reward that directly: the goal bonus dominates, and the squared-throttle penalty makes &quot;many gentle steps&quot; cost about the same as &quot;few aggressive steps.&quot; RMA wins both metrics — return <em>and</em> steps — but the headline number only shows a small slice of the underlying strategy difference.</p><h4>What would amplify the gap</h4><p>Sharpen the reward. Replace the +100 goal bonus with a −1-per-step penalty (the discrete-MountainCar reward style). Then each saved step adds a point of return directly, and the throttle absorption goes away.</p><h4>The diagnostics tell us the implementation is sound</h4><p>Three checks rule out implementation bugs as the explanation for the small absolute gap:</p><ol><li><strong>Phase-2 adapter MSE converges from 0.41 to 0.088</strong> over 50 epochs — a ~5× reduction.</li><li><strong>Deployed adapter </strong><strong>ẑ lands within </strong><strong>0.1 points of the privileged oracle on mean return</strong> (+96.0 vs +96.1). The cold-start cost of the adapter is essentially zero — online identification works as designed.</li><li><strong>The training curves themselves favor RMA</strong> — RMA Phase 1 reaches the solved threshold ~30k steps earlier than DR and avoids DR’s long instability dip.</li></ol><p>Every component of the RMA pipeline works. The remaining gap to a “perfect” RMA isn’t in the encoder, the adapter, or the deployment loop — it’s the reward function’s ceiling.</p><h3>What would widen the gap further?</h3><p>Two options, in order of impact:</p><ol><li><strong>Sharpen the reward.</strong> Replace the +100 goal bonus with a −1-per-step penalty (the discrete-MountainCar reward style). Each saved step then adds a point of return. RMA&#39;s ~5-step average lead would become +5 more points of return on top of the existing +1.9 gap, and a clean visible separation. Easy one-line code change in the env wrapper.</li><li><strong>Push parameter dimensionality further.</strong> the 3-parameter randomization already broke DR with 3 random parameters; pushing to 5+ (e.g. add motor delay, link mass perturbation, sensor noise) would presumably widen the gap further.</li></ol><h3>Closing</h3><p>The takeaway, in two sentences:</p><ul><li>On the wider 3-parameter randomization, <strong>RMA beats DR on every metric we measured</strong> — return +96.0 vs +94.1, solve rate 1.00 vs 0.99, steps-to-goal ~61 vs ~67. The wider joint distribution broke DR&#39;s ability to amortize, and RMA&#39;s online specialization paid off.</li><li>The training-time evidence (RMA Phase 1’s faster convergence, DR’s long instability dip) and deployment-time evidence (steps-to-goal, return distribution) both point the same way: <strong>DR is fighting itself across the joint distribution; RMA is not.</strong></li></ul><p>The MountainCar bifurcation is the smallest interesting sim-to-real benchmark we know of: the entire RMA pipeline (encoder, adapter, deployment) fits in a ~250-line notebook with three randomized physics parameters and a single throttle action, end-to-end in ~15 minutes on one GPU. In practice, RMA is applied to systems that are orders of magnitude more complex:</p><ul><li><strong>Legged quadrupeds</strong> are RMA’s canonical home. The original Kumar et al. (2021) paper trains an A1 robot (12 actuated joints) with a privileged vector spanning ~30+ parameters — motor strengths, link masses, joint frictions, terrain heightfields, payload mass and center-of-mass offset, external force perturbations, sensor latencies. Follow-up work has scaled the same scaffolding to ANYmal, Spot, Aliengo, and other quadruped platforms walking on grass, ice, debris, and stairs.</li><li><strong>Bipedal humanoids</strong> (Cassie, Atlas, Unitree G1) push to 20–30 actuated joints with even denser reward shaping for balance, gait smoothness, and footstep placement, where any wasted joint motion costs battery and risks a fall.</li><li><strong>Dexterous manipulation</strong> uses closely related teacher-student adaptation schemes — OpenAI’s Shadow Hand cube manipulation randomizes ~100 physics and visual parameters across a 24-DOF hand.</li><li><strong>Drone flight controllers</strong> in turbulent or unmodeled aerodynamic regimes use adapter-style modules to infer wind, payload, and motor wear online.</li></ul><p>In all of those settings the conditions for RMA’s payoff are met simultaneously: the parameter distribution is high-dimensional enough that DR genuinely cannot find a single feed-forward map that works everywhere, and the reward function is dense enough — penalizing every joule of wasted torque, every degree of body roll, every unsafe foot contact — that the strategy difference between DR and RMA shows up immediately in the score. Published gaps over DR on these platforms are typically much bigger than what we measured here: 20–40% improvement on rough terrain, on out-of-distribution payloads, or after motor wear has shifted the actuator response — versus the modest ~10% steps win and +2-point return gap we get on MountainCar.</p><h4>Open Sourced</h4><p>Clean Google Colab Implementation : <a href="https://colab.research.google.com/drive/1dJuPYzVJsCLDEvMDTMZbkDiSOrNvAccH?usp=sharing">https://colab.research.google.com/drive/1dJuPYzVJsCLDEvMDTMZbkDiSOrNvAccH?usp=sharing</a></p><p>Messy Github Code : <a href="https://github.com/NirajPudasaini/rma_tutorial">https://github.com/NirajPudasaini/rma_tutorial</a></p><p>Questions and corrections welcome.</p><p><em>Author: Niraj Pudasaini, Department of Computer Science, University of Colorado Boulder.</em></p><p><em>Thanks to the RMA authors (Kumar et al. 2021) and the SB3-zoo maintainers for the SAC + gSDE hyperparameters that make MountainCar solvable in the first place.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77359aaa5502" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/train-once-adapt-online-a-hands-on-tour-of-rapid-motor-adaptation-rma-77359aaa5502">Train Once, Adapt Online: A Hands-On Tour of Rapid Motor Adaptation (RMA)</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenVLA Paper Review]]></title>
            <link>https://medium.com/correll-lab/openvla-paper-review-1da121891f88?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/1da121891f88</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Antony Zhao]]></dc:creator>
            <pubDate>Sun, 03 May 2026 21:49:15 GMT</pubDate>
            <atom:updated>2026-05-03T21:49:13.433Z</atom:updated>
            <content:encoded><![CDATA[<h4>From Large-Scale Robot Data to Open Generalist Policies</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*qt-kRiqRaLmAfLbbwzPo-g.png" /></figure><h3>Open-X Embodiment (OXE)</h3><p>Large datasets have been a contributing factor to the success of vision models and language models. There wasn’t a comprehensive robotics dataset at the time, with many various scattered datasets. This aimed to consolidate these many datasets into a single dataset, creating ~1 million robot trajectories, across various embodiments, allowing for similar large scale training that enabled the success in other fields.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oe_knx1jGIVGf2YFT7PFhA.png" /></figure><h3>RT-2-X</h3><p>RT-2-X was a Vision-Language-Action Model (VLA) released alongside and trained on the OXE dataset. It is a 55 billion transformer model, built on a pretrained PaLI-X VLM, that is finetuned on internet data and OXE data simultaneously. This broad training allows it to generalize to new tasks and scenarios, and works across all embodiments present in training. It showed that the OXE dataset does enable large scale training similar to those present in modern vision and language models.</p><h3>OpenVLA</h3><p>While RT-2-X showed that these results were possible, it was not open source or open weights, and they did not exactly describe the training “recipe” to produce such a model. OpenVLA aims to do this, keeping the training details public, providing the pretrained models to the public, and also doing it for cheaper.</p><h3>Prismatic VLM</h3><p>The Prismatic VLM is a 7B parameter model used as the backbone for the VLA. It consists of a dual vision encoder combining SigLIP and DinoV2, where SigLIP handles semantic visual understanding and DinoV2 contributes spatial reasoning. A LLaMA 2 LLM handles the language side. At 7B parameters compared to RT-2-X’s 55B, it enables significantly cheaper training and potentially faster inference.</p><p>They also tested other VLMs for the backbone, and found the Prismatic performed the best, approximately 10% better than the next best, LLaVA, which they suggest is likely due to the fused image encoder, providing more information to the model. They also tested different image sizes such as 384x384, but found that it didn’t improve performance, while slowing down the training process by around 3x.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9fpQ8lM2X7Z5M9RIXLSNig.jpeg" /><figcaption>Model Architecture</figcaption></figure><h3>Actions</h3><p>Now that they had the vision language part of the VLA handled, they needed to handle the actions. They simplified this by discretizing actions, using quantiles to divide each action dimension into 256 bins, preventing outliers from skewing bin spacing. These bins are then mapped to tokens inside the Llama tokenizer. Specifically, they take 256 of the least used tokens, and remap them to the action outputs (rather than language).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*hO8G2nZA_dsrgJQP2DRe9Q.png" /><figcaption>LLM Tokens, with the least commonly used being replaced in the training process.</figcaption></figure><h3>Training</h3><p>Now that they had the VLA, they trained the model on the OXE dataset (no internet data). For this, they actually continued to fine tune the vision encoders on the new data, rather than freezing it, noting that this resulted in better VLA performance for them. Additionally, compared to the 1–2 epochs of training for LLMs or VLMs, they had to take 27 epochs to get the action token accuracy to around 95%, where they stopped training.</p><p>For compute, they used 64 A100s for 14 days, or 21,500 A100-hours. For inference though, it only uses around 15 GB (small enough to fit on consumer GPUs) at bfloat16 precision, and is able to run at 6Hz on a RTX 4090, which is enough to give it some amount of control over the robot (where RT-2-X runs at around 1–3 Hz).</p><h3>Performance</h3><p>The authors show that this resulting model is able to match or even beat RT-2-X in terms of performance on in-distribution tasks, a notable result given it has less than one-seventh the parameters. It is also capable of generalizing to new tasks to a similar level as RT-2-X, though gaps emerged in cases requiring real-world knowledge not present in the OXE dataset. The main example the authors present is OpenVLA failing the task of “move the can to the picture of Taylor Swift”, whereas RT-2-X, likely due to its co-fine-tuning on internet data, succeeds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNw4G9UJ76ROztcH0abc9Q.jpeg" /><figcaption>Performance of various models on tasks</figcaption></figure><h3>Fine-Tuning</h3><p>It is also possible to fine-tune the model to new tasks, with full fine tuning for a task requiring around 8 A100s and 5–15 hours. This is much smaller than the pretraining phase, with around 40–120 A100-hours compared to the 21,500 A100-hours for full scale pretraining. The authors also show that it is possible to do parameter-efficient fine-tuning, showing that LoRA gives the best performance. It also only requires a single A100 for around 10–15 hours. This makes it practical for other labs to take the pretrained model and adapt it to their specific tasks without significant compute resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IUP89ivRQzaMRuPrRwOnAg.png" /></figure><h3>Limitations</h3><p>Despite the many positives of the OpenVLA model, there are some limitations. One notable example is that the model only takes a single image as the observation, it doesn’t support multiple images or any of the various robot observations like proprioception.</p><p>Second, while 6Hz allows for the model to have good control over the robot, some tasks and robots need more, such as ALOHA which uses 50Hz.</p><p>Finally, while it is decent at most tasks, it isn’t reliable enough for most use cases, with success rates of &lt;90% for many of them. This makes it suitable for research but not yet reliable enough for production deployment.</p><h3>Modern VLAs</h3><p>OpenVLA helped to enable further research into robotic VLA models. Since then, there have been many developments improving various parts of the OpenVLA model. Some of the most notable are:</p><ol><li>Discrete actions are replaced with diffusion or flow matching heads for continuous actions.</li><li>Action chunking allows the model to output multiple actions simultaneously, reducing the effect of the slow autoregressive process of OpenVLA, enabling for faster inference speeds.</li></ol><p>These improvements are often combined in newer models, with more recent work incorporating several simultaneously. For anyone interested in reading further, some of these models or papers to look at include π0 and π0.5. Another model to look at would be OpenVLA-OFT, which directly builds on OpenVLA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TurfcexpNffyeIGiEcwedg.png" /></figure><h3>Conclusion</h3><p>OpenVLA represents a meaningful step towards accessible, generalist robot learning, demonstrating that large-scale VLA training doesn’t require massive models or even datacenter-scale training. While various limitations remain, the rapid pace of follow-up work suggests these are solvable problems. The combination of open weights, a reproducible training recipe, and a strong baseline provides a solid foundation for the field to build on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1da121891f88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/openvla-paper-review-1da121891f88">OpenVLA Paper Review</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[One-step Generative Modeling with Flow matching, Mean Flows and Solution Flow Models for Robotic…]]></title>
            <link>https://medium.com/correll-lab/one-step-generative-modeling-with-flow-matching-mean-flows-and-solution-flow-models-for-robotic-701d2ae3a48b?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/701d2ae3a48b</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[flowmatching]]></category>
            <category><![CDATA[application]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[YiChun Liao]]></dc:creator>
            <pubDate>Sat, 02 May 2026 17:40:21 GMT</pubDate>
            <atom:updated>2026-05-04T22:02:58.454Z</atom:updated>
            <content:encoded><![CDATA[<h3>One-step Generative Modeling with Flow matching, Mean Flows and Solution Flow Models for Robotic Manipulation</h3><h4>A Comparison on the LIBERO dataset</h4><p>Is one step really enough? In this article, we dive into the math behind these trending models, share the results we achieved, and explore what this means for the future of robotic trajectory generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j90xQgOSkK1Zizx0fxnW_g.png" /></figure><h3>What is one-step generative modeling? And how will it change the game?</h3><p>We consider a robotic manipulation problem. We will use 2 current trending one-step generative modeling methods, <a href="https://doi.org/10.48550/arXiv.2505.13447">Mean Flows</a> and <a href="https://doi.org/10.48550/arXiv.2512.15657">Solution Flow</a>, for our models. I will talk about these 2 methods briefly and make sure you know what happened in the models. We also trained a traditional Flow matching model as our comparison baseline.</p><h4>Flow Matching</h4><p><a href="https://medium.com/correll-lab/flow-matching-for-generative-models-from-scratch-8264bad4e0ba">This article</a> is great work explaining Flow Matching:</p><p><a href="https://medium.com/correll-lab/flow-matching-for-generative-models-from-scratch-8264bad4e0ba">Flow Matching For Generative Models From Scratch</a></p><p>Let’s talk about Flow matching method first. The core idea behind Flow Matching (FM) is straightforward: we want to learn a smooth transformation that maps a simple, easy-to-sample distribution (like pure Gaussian noise) into a complex target data distribution (in our case, robotic manipulation trajectories from the LIBERO dataset).</p><p>If you are familiar with Diffusion Models, Flow Matching is a close relative, but it offers a more straightforward and generalized mathematical framework based on Continuous Normalizing Flows (CNFs).</p><p><strong>The Intuition: </strong>Imagine a bunch of dust particles floating randomly in the air (our initial noise). We want to move these particles into the exact shape of a successful robotic action. Flow Matching is the process of learning the exact velocity — or <strong>vector fields</strong> — needed to push the noise into the right shape over a period of time.</p><p><strong>The Math: </strong>In Flow Matching, we define a continuous probability path <em>pₜ(x)</em> that smoothly morphs from our simple prior distribution at time <em>t=0, p₀(x)</em>, to our target data distribution at time <em>t=1, p₁(x)</em>.</p><p>This movement is governed by an Ordinary Differential Equation (ODE):</p><p><em>dx/dt = vₜ(x)</em></p><p>Here, <em>vₜ(x) </em>is the time-dependent vector field. Our goal is to train a neural network, <em>v_θ(x, t)</em>, to predict this vector field. During training, we know the starting point (noise) and the end point (the real robotic action). Flow Matching defines a target vector field, <em>uₜ(x|x₁)</em>, which represents the straightest, most efficient path from the noise to the data.</p><p>The model is trained using a remarkably simple, simulation-free loss function — we just want our network to mimic this target path:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*vAwP16S5mDGL17Y2zSSXpA.png" /></figure><h4>Do we need something better? Yes!!!</h4><p>Training a Flow Matching model is fast and stable. However, the bottleneck happens during <strong>inference</strong> (generation). To generate a robotic action in the real world, we sample random noise <em>x₀</em> and then use an ODE solver to step forward in time from <em>t=0</em> to <em>t=1</em>.</p><p>Because the path is continuous, the ODE solver cannot just jump to the end. It has to take <strong>multiple small steps</strong> (often 10, 20, or even 50 steps) to accurately follow the curved vector field and arrive at a high-quality action. In fast-paced robotic manipulation, calculating these multiple steps takes precious computing time, introducing latency that can make real-time control difficult.</p><p>This multi-step limitation is exactly why researchers are pushing the boundaries to find a faster way. What if we could jump straight from noise to the final action in just a single leap, without losing the quality of traditional Flow Matching?</p><h4>One straight forward method</h4><p>To solve the multi-step latency of traditional Flow Matching, researchers initially turned to distillation techniques. An example is InstaFlow, which relies on a teacher-student model architecture. In this paradigm, you must first spend massive computational resources training a heavy, highly accurate multi-step “teacher” model. Once that is done, you train a faster “student” model to mimic the teacher’s final output in just one or two steps.</p><p>While distillation works, it is a massive headache. It requires training multiple models sequentially, effectively doubling the time, effort, and compute required. If we can find a method to train a model that generates perfectly in one step from scratch, we can bypass the teacher entirely, saving resources while keeping inference lightning-fast.</p><p>This brings us to the game-changers: <strong>Mean Flows</strong> and <strong>Solution Flow Models</strong>.</p><h3><strong>Mean Flows: The Power of Average Velocity</strong></h3><p><strong>The Intuition:</strong> In traditional Flow Matching, the model learns the <em>instantaneous velocity</em> — the exact direction the “wind” is blowing at one specific micro-second. MeanFlow takes a different approach: it models the <em>average velocity</em>. Average velocity is simply the total displacement (from pure noise to the final robotic action) divided by the time it takes. Instead of following a curving path step-by-step, MeanFlow learns the straightest possible trajectory to reach the end goal in one calculation.</p><p><strong>The Math: </strong>Mathematically, the true average velocity <em>u</em> is defined as an integral of the instantaneous velocity <em>v</em> over a time interval from <em>r</em> to <em>t</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*l7VSXYZF9VLuvV9yunmxJg.png" /></figure><p>However, calculating an integral during training is far too slow. To bypass this, the authors derived this MeanFlow Identity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*Pi7vWfA8W52swX4LkWmwdg.png" /></figure><p>This identity allows the neural network to learn the average velocity using only the instantaneous velocity as its target, avoiding the integral entirely.</p><p><strong>The Catch</strong> There is a structural bottleneck here. To compute the derivative term, the model must rely on Jacobian-vector product (JVP) calculations. In deep learning frameworks like PyTorch, JVP operations are less optimized than standard forward propagation. This introduces a new computational bottleneck that eats up GPU memory and slows down training.</p><h3><strong>Solution Flow Models (SoFlow): Learning the Destination Directly</strong></h3><p><strong>The Intuition:</strong> SoFlow completely discards the idea of calculating iterative velocities or dealing with the JVP. Instead of learning the velocity, SoFlow directly learns the <strong>solution function</strong>. Think of it as a teleporter: you give the model a starting state and a target time, and it instantly predicts the exact physical state of the robot at that future time.</p><p><strong>The Math: </strong>SoFlow introduces a parameterized neural network,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/153/1*aOi9lObvKk8FvI5BADrVPw.png" /></figure><p>which explicitly maps a state <em>xₜ</em> at time <em>t</em> directly to its evolved state at a later time <em>s</em>.</p><p>To train this “teleporter” efficiently without JVP, SoFlow combines a standard Flow Matching loss with a unique <strong>Solution Consistency Loss</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*rTVRL5XSA-qHufpiHXJ4sA.png" /></figure><p>The model is trained to ensure that taking one massive leap from time <em>t</em> to time <em>s</em> produces the exact same robotic action as taking a tiny step to an intermediate time <em>l</em>, and <em>then</em> teleporting to <em>s</em>.</p><p>The approximation of the parameterized neural network looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*KqxCbB6smrvb6NR_Q69afg.png" /></figure><p>From the paper, this method gives a better result compare to the MFM and a faster training speed. We wonder how well will SoFlow model do in the Robotic Manipulation case.</p><h3>Result: Which method is better? What can we learn from these results?</h3><p>Before we dive into the numbers in the table above, I need to show how this specific data snapshot was collected.</p><h4>Examples</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*2eQh9zHFaV7-6cHhWlRXRg.gif" /><figcaption>Success case (Left) v.s. Fail case (Right) example in libero_spatial dataset</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*ObgZjpCascubaoB7K_bRDQ.gif" /><figcaption>Success case (Left) v.s. Fail case (Right) example in libero_object dataset</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*ZS2P5XgIRW0TJRnojyeGGw.gif" /><figcaption>Success case (Left) v.s. Fail case (Right) example in libero_goal dataset</figcaption></figure><p>For dataset libero_10, the tasks are more special. These are some examples</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/128/1*oCTIuIOO2oPV-qIpmZUcjg.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/128/1*tpc0161Illbm8-TDc2ijlA.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/128/1*hIcVO2ZhQgAFkQfq5P2dOQ.gif" /></figure><h4>Statistic Result</h4><p><strong>All results were evaluated across 100 seeds per task</strong>, providing a highly rigorous and stable average.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lO2enoF2wGxfidkHB1bgXg.png" /></figure><h4>Conclusion</h4><p>Looking at the overall success across all 40 LIBERO tasks, the one-step models are holding their own. <strong>FM (N=1)</strong> currently leads the pack with a 53.7% overall success rate.</p><p>However, SoFlow claims the crown in efficiency. It is the fastest, with a <strong>~2.0 ms generation time</strong>.</p><p>Also, the FM (N=10) get a lower score compare to FM (N=1). We think this is because the iterative ODE solver compound errors during inference. However, for MFM the N = 10 gives a better results! Which makes our guess of ODE solver compounding errors during inference meaningless. This is something we are looking forward to explain.</p><h4>Future work</h4><p>This is just a brief start of the comparison. We can engineer all models and see what will the result be. Such as adding <strong>Noise Injection</strong> to improve robustness. Adding <strong>Exponential Moving Average (EMA) </strong>to keep generative models stable performance. These might improve the comparison accuracy!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=701d2ae3a48b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/one-step-generative-modeling-with-flow-matching-mean-flows-and-solution-flow-models-for-robotic-701d2ae3a48b">One-step Generative Modeling with Flow matching, Mean Flows and Solution Flow Models for Robotic…</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[World Model-Augmented Imitation Learning for Bimanual Garment Folding]]></title>
            <link>https://medium.com/correll-lab/world-model-augmented-imitation-learning-for-bimanual-garment-folding-a072edb9fd51?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/a072edb9fd51</guid>
            <category><![CDATA[world-models]]></category>
            <category><![CDATA[imitation-learning]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Antony Zhao]]></dc:creator>
            <pubDate>Sat, 02 May 2026 06:30:40 GMT</pubDate>
            <atom:updated>2026-05-02T06:30:39.388Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using DreamerV3 representations to enhance ACT policies on the LeHome Challenge</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*jRoXVStlLF0CFXkTQ4hn_Q.png" /></figure><h3>1. Introduction</h3><p>Manipulating deformable objects such as garments remains one of the most challenging open problems in robotic manipulation. Unlike rigid objects, cloth exhibits a high-dimensional configuration space, stochastic dynamics under contact, and severe self-occlusion — properties that make both perception and control fundamentally difficult [1, 2]. Bimanual garment folding, in which two robot arms must coordinate to transform a crumpled garment into a neatly folded state, compounds these challenges by requiring precise inter-arm coordination over extended time horizons.</p><p>Recent imitation learning (IL) methods, particularly Action Chunking with Transformers (ACT) [3] and Diffusion Policy [4], have shown remarkable success on contact-rich manipulation tasks, learning policies directly from human demonstrations. However, these methods operate primarily on single-frame observations. At each timestep, the policy observes the current camera images and joint states but has no explicit representation of how the scene has evolved or where it is headed. For rigid objects, this is often sufficient. A block on a table looks essentially the same regardless of how it got there. For cloth, it is a critical limitation. The same visual appearance of a partially folded garment can arise from many distinct folding trajectories, and the correct next action depends heavily on the history of manipulation.</p><p>World models, such as from the DreamerV3 algorithm [5], offer a complementary capability: they learn compressed latent representations of environment dynamics from sequential experience. Through a Recurrent State-Space Model (RSSM), world models maintain a hidden state that summarizes the history of observations and actions, capturing temporal dynamics that single-frame encoders miss. While world models have primarily been used to enable reinforcement learning through imagined rollouts, the learned representations themselves, particularly the recurrent hidden state, encode exactly the kind of temporal context that frame-based imitation learning policies lack.</p><p>In this work, we investigate whether augmenting an imitation learning policy with temporal latent states from a world model can improve performance on bimanual garment folding. Rather than using DreamerV3 for its intended purpose of actor-critic learning in imagination, we train it purely as a representation learner on demonstration data and inject its hidden states as additional input tokens to an ACT policy. We evaluate this approach on the LeHome Challenge [6], a benchmark for bimanual garment folding in simulation using the Isaac Sim physics engine.</p><p>Our hypothesis is that the world model’s hidden state provides the ACT policy with temporal context about cloth dynamics — information about the fold&#39;s trajectory, the fabric&#39;s momentum, and the cumulative effect of past actions — that is not available from a single observation frame. We find that this augmentation yields modest but consistent improvements over the baseline ACT policy, particularly on garment categories that require longer manipulation sequences.</p><h3>2. Related Work</h3><h4>2.1 Imitation Learning for Robotic Manipulation</h4><p>Behavioral cloning — learning a policy as a supervised mapping from observations to actions — remains one of the most practical approaches to robotic manipulation, owing to its simplicity and data efficiency relative to reinforcement learning [7]. However, naive behavioral cloning suffers from compounding errors: small mistakes early in a trajectory push the policy into states it has not seen during training, leading to cascading failures [8].</p><p>Several recent methods address this limitation through architectural innovations. Action Chunking with Transformers (ACT) [3] predicts a chunk of future actions simultaneously rather than one action per timestep, using a conditional variational autoencoder (CVAE) with a transformer backbone. By committing to a sequence of actions and temporally ensembling overlapping chunks during inference, ACT reduces the effective number of decision points and produces smoother, more consistent behavior. The method was originally developed for bimanual manipulation with the ALOHA teleoperation system and demonstrated 80–90% success rates on fine-grained tasks from as few as 50 demonstrations.</p><p>Diffusion Policy [4] takes a different approach, framing action prediction as a conditional denoising diffusion process. Starting from Gaussian noise, the policy iteratively denoises action sequences conditioned on current observations. The key advantage of the diffusion formulation is its natural ability to represent multimodal action distributions — when multiple valid strategies exist for a given observation, the diffusion process can capture all modes without the mode collapse issues that affect CVAE-based methods. Both methods predict action chunks rather than single actions, and both operate on per-timestep visual and proprioceptive observations without explicit temporal modeling beyond the chunk horizon.</p><p>Vision-Language-Action (VLA) models such as SmolVLA [9] and OpenVLA [10] extend this paradigm by conditioning on natural language task descriptions, enabling multi-task generalization. However, VLA models typically require significantly more training data and computation, and their language conditioning adds complexity that may not benefit single-task settings.</p><p>A common limitation across all these approaches is the lack of explicit modeling of temporal dynamics. Each forward pass processes the current observation independently (or with a short history window), relying on the action chunk to provide implicit temporal consistency. For tasks where the current visual observation is ambiguous — as is frequently the case with deformable objects — this can lead to inconsistent or suboptimal behavior.</p><h4>2.2 World Models and Learned Dynamics</h4><p>World models learn a predictive model of environment dynamics in a compact latent space, enabling an agent to simulate future outcomes without interacting with the real environment [11]. The Dreamer family of algorithms [12, 13, 5] has established a particularly successful paradigm built around the Recurrent State-Space Model (RSSM). The RSSM maintains two complementary state components: a deterministic recurrent hidden state that summarizes history, and a stochastic categorical latent that captures the current observation. Together, these components provide a compressed representation of the environment state that supports both reconstruction of observations and prediction of future dynamics.</p><p>DreamerV3 [5] introduced several robustness improvements, including symlog-transformed predictions, two-hot-encoded reward bins, and KL balancing between the encoder and the dynamics predictor. These techniques enabled a single algorithm with fixed hyperparameters to achieve strong performance across over 150 diverse tasks, including the landmark achievement of collecting diamonds in Minecraft from scratch. The algorithm was recently published in Nature [14], establishing world models as a general-purpose approach to sequential decision-making.</p><h4>2.3 World Models as Representation Learners</h4><p>World models have primarily been developed for reinforcement learning, using imagined rollouts to train actor-critic policies without environment interaction. Their learned representations have independent value. The RSSM hidden state encodes a compressed summary of observation history and dynamics that is predictive of future states and rewards.</p><p>Recent work has begun to exploit this property explicitly. The World-Action Model (WAM) [19] demonstrated that world-model latent states serve as effective inputs for diffusion policy training on the CALVIN manipulation benchmark, achieving 92.8% success after PPO fine-tuning. PreLAR [20] showed that pre-training world models on unlabeled video data produces representations that transfer effectively to downstream manipulation tasks. The FRAPPE framework [21] enhanced diffusion-based manipulation policies by incorporating auxiliary future-prediction objectives within a world-model, improving robustness to visual perturbations.</p><p>These results suggest that world model representations capture aspects of scene dynamics that are complementary to the static visual features learned by standard encoders. However, prior work has largely focused on either joint training of the world model and policy or on using world-model latents as the sole input to the policy. Our approach is distinct in that we train the world model independently on demonstration data, freeze it, and augment an existing imitation learning policy with the world model’s hidden states as additional input. This modular design preserves the strengths of the original policy while adding temporal context at minimal architectural cost.</p><h3>3. Materials and Methods</h3><h3>3.1 Environment and Task</h3><p>We evaluate on the LeHome Challenge [6], a benchmark for bimanual garment folding in simulation. The environment runs on NVIDIA Isaac Sim with PhysX-based particle cloth simulation, providing realistic deformable dynamics at the cost of significant computational overhead (~10–30 FPS per environment instance).</p><p><strong>Robot setup.</strong> Two SO101 robot arms are mounted on opposite sides of a table, each with 6 degrees of freedom. The policy outputs 12-dimensional joint position targets at each timestep.</p><p><strong>Observations.</strong> The environment provides three RGB camera views (top, left wrist, right wrist) at 480×640 resolution, along with a 12-dimensional proprioceptive state vector containing the joint positions of both arms.</p><p><strong>Task and reward.</strong> The task is to fold a garment from a randomized initial configuration into a target folded state. Success is determined by a set of keypoint-distance conditions on the cloth mesh (specific pairs of keypoints must be brought within threshold distances, while others must remain separated). The thresholds are garment-specific, scaled by garment size.</p><p><strong>Evaluation protocol.</strong> The Release test set contains 48 garments across four categories: long-sleeved tops (12), short-sleeved tops (12), long pants (12), and shorts (12). Within each category, 10 garments are “Seen” (with corresponding demonstrations in the training data), and 2 are “Unseen” (with no demonstrations). The official competition evaluation additionally includes held-out garments not present in the Release set.</p><h3>3.2 Demonstration Dataset</h3><p>Training data is provided as the four_types_merged dataset containing teleoperated demonstrations across all four garment categories. The dataset comprises approximately 270,000 frames stored in LeRobot format [27] — a HuggingFace-compatible format that uses parquet files for structured data and MP4 videos for camera observations. Each frame contains synchronized camera images, joint states, and actions recorded during human teleoperation via leader-follower arms.</p><h3>3.3 Baseline: Action Chunking with Transformers (ACT)</h3><p>Our baseline policy uses ACT [3] as implemented in the LeRobot library (v0.4.3). ACT frames imitation learning as training a conditional variational autoencoder (CVAE) to predict future action chunks given current observations.</p><p><strong>Architecture.</strong> The visual encoder consists of three separate ResNet-18 backbones (one per camera), pretrained on ImageNet, each followed by a spatial softmax layer that extracts keypoint coordinates from the final convolutional feature maps. These visual features are concatenated with a linearly projected proprioceptive state vector (12-dim joint positions) to form the input token sequence for a transformer encoder.</p><p>The transformer encoder (4 layers, 512-dim, 8 heads) processes these tokens with self-attention, producing a context representation. A transformer decoder (1 layer) then autoregressively generates action chunks — sequences of 100 future joint position targets. At inference time, overlapping action chunks are temporally ensembled by exponentially weighted averaging, producing smooth continuous behavior.</p><p><strong>Training.</strong> The CVAE training objective combines an action reconstruction loss with a KL divergence term that regularizes the latent style variable. We train with a batch size of 128 for 200,000 gradient steps using the AdamW optimizer. Input images are processed at 128×128resolution.</p><h3>3.4 DreamerV3 World Model</h3><p>We train a DreamerV3 world model [5] to learn compressed temporal representations of garment-manipulation dynamics. Critically, we use the world model solely as a representation learner — we do not train the actor-critic components or perform imagination-based reinforcement learning. The world model’s purpose is to provide the ACT policy with a recurrent hidden state that summarizes the history of observations and actions, encoding temporal context that single-frame observations cannot capture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAbS7_ifmnoO9QNVTWeAAA.png" /></figure><h3>3.4.1 Architecture</h3><p><strong>Multi-modal encoder.</strong> The encoder processes both visual and proprioceptive inputs. Camera images from all three views are stacked channel-wise and resized to 128×128, producing a 9-channel input tensor (3 cameras × 3 RGB channels). A convolutional encoder with 5 layers (base filters: 64, doubling per layer, kernel size 4) processes this into a flattened feature vector. The 12-dimensional joint state is passed through a linear layer. Both are concatenated and projected through an MLP to produce logits over the categorical latent space.</p><p><strong>Categorical latent.</strong> Following DreamerV3, the latent representation uses 32 categorical variables, each with 32 classes, resulting in a 32×32 (or 1024 flattened) latent vector. A uniform mixture (unimix) of 1% prevents categorical collapse.</p><p><strong>RSSM.</strong> The Recurrent State-Space Model maintains a 1024-dimensional deterministic hidden state​ via a GRU with block-linear structure. At each step, the RSSM takes as input the concatenation of the previous latent state, hidden state, and action, processes them through an input projection, normalization, an MLP, and a GRU cell to produce the updated hidden state.</p><p><strong>Dynamics predictor.</strong> An MLP predicts the categorical latent distribution from the hidden state alone (without observing the current frame), producing the prior. This is trained to match the encoder’s posterior via KL divergence.</p><p><strong>Decoder.</strong> A transposed convolutional decoder reconstructs the 9-channel image from the world model state, while a separate MLP decoder reconstructs the 12-dimensional joint state. For the multi-modal (“multi”) observation type, both decoders operate in parallel from the same state vector.</p><p><strong>Continue predictor.</strong> An MLP predicts the probability of episode continuation from the world model state, trained with binary cross-entropy against the episode termination signal.</p><h3>3.4.2 Training</h3><p>The world model is trained offline on the demonstration dataset, loaded into a replay buffer, and sampled in sequences of 32 timesteps with a batch size of 16. Training proceeds for 90,000 gradient steps.</p><p><strong>No reward predictor.</strong> Because the demonstration dataset contains no reward signals (only observations and actions from teleoperation), we do not train the reward prediction head. This head would be necessary for imagination-based actor-critic training, but is irrelevant for our representation-learning use case.</p><h4>3.5 Integration Pipeline</h4><p>The integration of the world model with ACT follows a modular, four-stage pipeline that requires no modification to the core ACT architecture beyond accepting an additional input feature.</p><h4>Stage 1: World Model Training</h4><p>The DreamerV3 world model is trained offline on the demonstration dataset as described in Section 3.4.2. After training, the world model is frozen — all parameters are fixed, and no further gradient updates occur.</p><h4>Stage 2: Latent State Extraction</h4><p>The frozen world model is run sequentially over every episode in the demonstration dataset to produce temporal hidden states. For each episode:</p><ol><li>The RSSM hidden state is initialized to a learned initial state.</li><li>At each timestep t, the observation is encoded alongside the current hidden state to produce the posterior latent.</li><li>The hidden state​ (1024-dim) is recorded as the temporal context for frame t.</li><li>The RSSM advances one recurrent step.</li><li>At episode boundaries, the hidden state is reset again.</li></ol><p>This produces a 1024-dimensional hidden state vector for each frame in the dataset, which is stored alongside the original observations and actions.</p><h3>Stage 3: Dataset Augmentation and ACT Training</h3><p>The extracted hidden states are added to the LeRobot dataset as an additional STATE feature (observation.dreamer_state with shape [1024]). The ACT training configuration is updated to include this feature as an input:</p><p>yaml</p><pre>input_features:<br>  observation.state:<br>    type: STATE<br>    shape: [12]<br>  observation.dreamer_state:<br>    type: STATE<br>    shape: [1024]<br>  observation.images.top_rgb:<br>    type: VISUAL<br>    shape: [3, 480, 640]<br>  observation.images.left_rgb:<br>    type: VISUAL<br>    shape: [3, 480, 640]<br>  observation.images.right_rgb:<br>    type: VISUAL<br>    shape: [3, 480, 640]</pre><p>LeRobot’s ACT implementation concatenates all STATE features before projecting to the transformer’s model dimension. The 12-dim joint state and 1024-dim dreamer state are thus concatenated into a 1036-dim vector, linearly projected to 512-dim, and added as a token in the transformer encoder’s input sequence alongside the visual tokens.</p><p>The augmented ACT is trained from scratch on this dataset using the same hyperparameters as the baseline (batch size 128, AdamW optimizer), but for 300,000 steps rather than 200,000 to allow the policy sufficient time to learn from the higher-dimensional input.</p><h3>Stage 4: Inference with Online World Model</h3><p>At evaluation time, the frozen world model runs alongside the ACT policy to provide live temporal context:</p><ol><li><strong>Episode start</strong>: The RSSM hidden state is initialized</li><li><strong>Each timestep</strong>: a. The current observation is passed through the world model encoder, alongside the current hidden state, to produce the latent state. b. The hidden state is passed to the ACT policy as the observation.dreamer_state feature. c. ACT produces an action (with temporal ensembling of overlapping chunks). d. The RSSM advances.</li><li><strong>Episode end</strong>: Hidden state is reset.</li></ol><p>This adds minimal computational overhead — one forward pass through the world model encoder (a small CNN + MLP) and one GRU step per timestep, compared to ACT’s three ResNet-18 forward passes and transformer inference.</p><h3>3.6 Implementation Details</h3><p><strong>Software stack.</strong> The simulation environment uses NVIDIA Isaac Sim 5.1.0 with IsaacLab. Policy training uses the LeRobot library (v0.4.3) with PyTorch 2.7.0. The DreamerV3 world model is a custom implementation in PyTorch.</p><p><strong>Compute budget.</strong> World model training (90k steps): ~30 hours. Latent state extraction: ~20 minutes per dataset. ACT training (300k steps): ~100 hours. Full evaluation (48 garments × 5 episodes): ~5 hours.</p><h3>4. Results</h3><h4>4.1 Baseline ACT Performance</h4><p>We first established a baseline by training ACT on the full demonstration dataset (four_types_merged, approximately 270k frames) for 200k gradient steps with a batch size of 128. This model achieved a self-reported success rate of 60.00% across 240 evaluation episodes (48 garments × 5 episodes each). Performance varied substantially across garment categories: Pant Short was strongest at 77%, followed by Top Short at 62%, Top Long at 54%, and Pant Long at 44%. Unseen garments were considerably harder, averaging 38.8% compared to 63.3% on seen garments — indicating that the policy learned category-specific strategies but struggled to generalize to novel garment geometries.</p><h4>4.2 World Model Training</h4><p>We trained the DreamerV3 world model offline on the same demonstration dataset, using the multimodal encoder (9-channel images from 3 cameras + 12-dim joint state) with reconstruction, KL divergence, and continuation-prediction losses. The reconstruction loss decreased from approximately 1100 early in training to around 700 after 90k steps, indicating that the encoder and decoder learned meaningful representations of the garment and robot state.</p><p>However, the KL divergence between the encoder posterior and the dynamics prior remained persistently high at 20–30 nats throughout training, despite tuning the loss coefficients (dynamics coefficient of 2.0, representation coefficient of 0.5). We attribute this to the stochastic nature of the particle-based cloth simulation in Isaac Sim: the cloth exhibits continuous micro-oscillations even when no external forces are applied, meaning that consecutive frames of an untouched garment produce visibly different observations. This inherent simulation noise creates an irreducible floor on the KL divergence, as the dynamics predictor cannot anticipate random particle jitter.</p><p>This behavior contrasts with prior applications of DreamerV3 to rigid-body environments (e.g., Atari, DMControl) where KL values typically converge to near the free-nats threshold. The high KL suggests that, for deformable-object tasks, the RSSM hidden state captures temporal trends (fold progression, arm trajectory) rather than precise frame-to-frame predictions, which is arguably more useful for downstream policy conditioning.</p><h4>4.3 Dreamer-Augmented ACT</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SdH5GN1PdIgw2tqwtUgiwA.png" /></figure><p>We extracted the RSSM hidden states (1024-dim) for every frame in the demonstration dataset by running each episode sequentially through the frozen world model encoder and recurrence. These hidden states were added as an additional STATE feature to the LeRobot dataset, and ACT was retrained with this augmented input.</p><p>At 200k steps (a controlled comparison with the baseline), the Dreamer-augmented ACT achieved 59.58% — essentially equivalent to the 60.00% baseline. The world model features did not provide an immediate benefit at matched training steps. However, with continued training through 300k steps, the augmented model reached 65.42%, with improvements across all four garment categories. Performance peaked at 300k steps and declined to 60.00% at 350k, suggesting the onset of overfitting.</p><p>We note an important caveat: because we did not train a baseline ACT model beyond 200k steps, we cannot definitively attribute the improvement at 300k to the world-model features rather than simply to more training. The controlled comparison at 200k shows no benefit from the augmentation; the gains emerge only with additional training. One interpretation is that the world model features expand the effective input space, making learning slower but ultimately enabling better performance. Another is that baseline ACT would have shown similar improvement with more training. Disentangling these factors would require training both models to 300k under identical conditions.</p><h4>4.4 Seen vs. Unseen Generalization</h4><p>Across all models, the gap between seen and unseen-garment performance was substantial — typically 20–30 percentage points. At the 300k checkpoint, seen garments averaged 67.0% while unseen averaged 50.0%. Compared to the baseline (63.3% seen, 38.8% unseen), the Dreamer-augmented model showed larger relative improvement on unseen garments (+11.2%) than on seen garments (+3.7%). This is consistent with the hypothesis that temporal context from the world model aids generalization: the hidden state captures dynamic patterns (how cloth responds to manipulation) that transfer across garment geometries, rather than memorizing category-specific visual features.</p><p>Two specific garments — Top_Short_Unseen_0 and Pant_Short_Unseen_0 — remained at or near 0% success across every model and checkpoint we evaluated. These likely represent garment geometries that fall sufficiently far outside the training distribution that no amount of temporal context can bridge the gap without explicit demonstrations.</p><h4>4.5 Category-Specific Fine-Tuning</h4><p>We attempted to improve performance on weak categories through targeted fine-tuning. Training the 228k checkpoint for 25k additional steps on only Pant Short data caused significant catastrophic forgetting: overall performance dropped from 63.33% to 57.71%, with degradation across all categories except the targeted one. A brief 4k-step recovery phase on the full dataset restored performance to 63.33%, but the fine-tuning itself provided no lasting benefit.</p><p>This result highlights a well-known limitation of sequential fine-tuning without replay mechanisms. More sophisticated approaches — such as elastic weight consolidation (EWC), experience replay mixing, or multi-task training with category-weighted sampling — would be needed to effectively address category imbalances.</p><h4>4.6 Official Leaderboard Results</h4><p>Our first submission (the 228k checkpoint) received an official score of 47.38%, placing 22nd out of 39 teams. This represents a significant drop from the self-reported 63.33%, with the largest discrepancy on Short-Sleeved Tops (75% self-reported vs. 38% official). The gap indicates that the official holdout garments are substantially harder than those in the Release test set, particularly for tops, where garment geometry varies more (neckline shape, sleeve attachment, body proportions).</p><p>The top-performing team achieved 68.88% on the official evaluation, suggesting that even the best approaches face significant challenges on holdout garments. Notably, the top teams exhibited more uniform performance across categories, whereas our model showed strong category imbalance — competitive on Shorts (63%) but weak on Short-Sleeved Tops (38%) and Long Pants (38%).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3LNGxgHxBu-eyZt_ZsE_qA.png" /></figure><h3>5. Discussion</h3><h4>Effectiveness of World Model Augmentation</h4><p>Our results present a nuanced picture of world model augmentation for imitation learning. At matched training steps (200k), the augmentation provides no measurable benefit, with the baseline ACT and Dreamer-augmented ACT achieving nearly identical performance. The gains emerge only with continued training, suggesting that the 1024-dimensional hidden-state input represents a significant expansion of the policy’s input space, which requires more gradient steps to utilize effectively.</p><p>The most encouraging signal is the disproportionate improvement on unseen garments, which suggests the world model representations encode transferable dynamics information rather than category-specific features. If confirmed by a properly controlled experiment (with both models trained to 300k), this would support the hypothesis that temporal context from a learned dynamics model aids generalization in deformable object manipulation.</p><h4>Simulation Fidelity as a Bottleneck</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*98O5GXvGAr6Z2RycUX9wrg.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*-S0j1ljtXm55ZgqcyZ4hAQ.gif" /></figure><p>A recurring theme in our experiments was the impact of simulation fidelity on the world model’s learned representations. Isaac Sim’s particle-based cloth physics, while enabling differentiable simulation of complex garment dynamics, introduces persistent micro-oscillations that never fully settle. This stochastic jitter introduces noise into the training signal, fundamentally limiting the world model’s predictive accuracy.</p><p>The high KL divergence throughout training (20–30 nats vs. the typical 1–5 nats in rigid-body environments) is a direct consequence: the encoder’s posterior distribution shifts unpredictably between frames due to particle noise, and the dynamics predictor cannot anticipate these random perturbations.</p><p>A world model trained on real-world video data, where cloth genuinely settles to rest between manipulations, might produce substantially more useful latent states. The signal-to-noise ratio for meaningful dynamics (folds, drapes, slides) relative to spurious motion would be much higher, potentially enabling a lower KL and more informative hidden states.</p><h4>The Self-Reported to Official Gap</h4><p>The 16-point gap between self-reported (63.33%) and official (47.38%) results reveals that generalization to held-out garment geometries remains the central challenge. Our model’s performance was particularly brittle on Short-Sleeved Tops, where the official score dropped to 38% compared to a self-reported 75%. This category exhibits high geometric variation (tank tops vs. t-shirts vs. polo shirts), and the training demonstrations may not have covered sufficient diversity.</p><p>This gap also suggests that the Release test set’s “Unseen” garments (2 per category) provide a limited signal about true generalization capability. Our model achieved reasonable unseen performance on the Release set (50% at 300k) but clearly struggled more on the larger and more diverse official holdout set. Future work should investigate data augmentation strategies — both visual (color jitter, crops, as supported by LeRobot’s transform pipeline) and geometric (simulating garments with varied proportions) — to improve robustness.</p><h4>Catastrophic Forgetting in Sequential Fine-Tuning</h4><p>Our attempt at category-specific fine-tuning demonstrated the fragility of sequential training on subset data. Even a modest 25k steps on Pant Short data caused performance degradation across all other categories, and recovery required explicit retraining on the full dataset. This is a practical limitation for competition settings where rapid iteration on weak categories is desirable.</p><p>More principled approaches exist: training with weighted sampling (oversampling underperforming categories without removing others), elastic weight consolidation to preserve learned representations, or maintaining a replay buffer from all categories during fine-tuning. We attempted oversampling through dataset merging but encountered technical difficulties with the data pipeline that prevented evaluation within the competition timeline.</p><h3>Limitations and Future Work</h3><p>Several limitations constrain the conclusions we can draw from this work:</p><p><strong>Experimental controls.</strong> The primary comparison lacks a baseline ACT model trained beyond 200k steps. The observed improvement at 300k could reflect the benefits of additional training rather than specifically from the world model augmentation. A rigorous evaluation would train both models to identical step counts under identical conditions.</p><p><strong>World model architecture.</strong> We used a standard DreamerV3 RSSM with a GRU recurrence. Alternative sequence models, particularly Mamba or Transformer-based architectures, might capture cloth dynamics more effectively. The GRU’s fixed-size hidden state may be a bottleneck for representing the high-dimensional state space of deformable objects.</p><p><strong>Offline-only world model training.</strong> Our world model was trained exclusively on demonstration data, which contains only successful folding trajectories. It has never observed failed attempts or the recovery behaviors that follow mistakes. Training on a more diverse set of trajectories (including failures collected from the simulator) could produce representations that better capture the full dynamics landscape.</p><p><strong>Integration architecture.</strong> We injected the world model hidden state as a simple additional token in ACT’s transformer encoder. More sophisticated integration — such as cross-attention between ACT’s visual features and the world model&#39;s latent state, or using the world model to predict future observations for planning — might yield greater benefits.</p><p>Despite these limitations, the modular nature of our approach — train world model independently, extract representations, augment existing policy — offers a practical and low-risk framework for incorporating temporal dynamics into imitation learning systems. The world model adds no overhead at training time (representations are pre-computed) and minimal overhead at inference time (one forward pass through the encoder and RSSM per step). This makes it straightforward to apply to other manipulation domains where temporal context might benefit frame-based policies.</p><h3>References</h3><p>[1] J. Zhu, A. Cherubini, C. Dune, D. Navarro-Alarcon, F. Alambeigi, D. Berenson, F. Ficuciello, K. Harada, J. Kober, X. Li, J. Lu, and U. Bhatt. “A survey on robotic manipulation of deformable objects: Recent advances, open challenges and new frontiers.” arXiv:2312.10419, 2023.</p><p>[2] J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, and Y. Mezouar. “Modeling, learning, perception, and control methods for deformable object manipulation.” Science Robotics, 2021.</p><p>[3] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. “Learning fine-grained bimanual manipulation with low-cost hardware.” In Proceedings of Robotics: Science and Systems (RSS), 2023.</p><p>[4] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. “Diffusion policy: Visuomotor policy learning via action diffusion.” In Proceedings of Robotics: Science and Systems (RSS), 2023.</p><p>[5] D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. “Mastering diverse domains through world models.” arXiv:2301.04104, 2023.</p><p>[6] LeHome Challenge. <a href="https://lehome-challenge.com/">https://lehome-challenge.com</a>, 2026.</p><p>[7] D. A. Pomerleau. “ALVINN: An autonomous land vehicle in a neural network.” In Advances in Neural Information Processing Systems (NeurIPS), 1989.</p><p>[8] S. Ross, G. J. Gordon, and J. A. Bagnell. “A reduction of imitation learning and structured prediction to no-regret online learning.” In International Conference on Artificial Intelligence and Statistics (AISTATS), 2011.</p><p>[9] HuggingFace. “SmolVLA: A small vision-language-action model for robotic manipulation.” 2025.</p><p>[10] M. J. Kim, K. Pertsch, S. Karamcheti, et al. “OpenVLA: An open-source vision-language-action model.” arXiv:2406.09246, 2024.</p><p>[11] D. Ha and J. Schmidhuber. “World models.” arXiv:1803.10122, 2018.</p><p>[12] D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi. “Dream to control: Learning behaviors by latent imagination.” In International Conference on Learning Representations (ICLR), 2020.</p><p>[13] D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba. “Mastering Atari with discrete world models.” In International Conference on Learning Representations (ICLR), 2021.</p><p>[14] D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. “Mastering diverse control tasks through world models.” Nature, 2025.</p><p>[15] V. Micheli, E. Alonso, and F. Fleuret. “Transformers are sample-efficient world learners.” In International Conference on Learning Representations (ICLR), 2023.</p><p>[16] E. Alonso, A. Jelley, V. Micheli, A. Kanervisto, A. Storkey, T. Pearce, and F. Fleuret. “Diffusion for world modeling: Visual details matter in Atari.” In Advances in Neural Information Processing Systems (NeurIPS), 2024.</p><p>[17] A. Gu and T. Dao. “Mamba: Linear-time sequence modeling with selective state spaces.” arXiv:2312.00752, 2023.</p><p>[18] N. Hansen, H. Su, and X. Wang. “TD-MPC2: Scalable, robust world models for continuous control.” In International Conference on Learning Representations (ICLR), 2024.</p><p>[19] World-Action Model (WAM). “Enhancing policy learning with world-action model.” arXiv:2603.28955, 2026.</p><p>[20] PreLAR. “World model pre-training with learnable action representation.” In European Conference on Computer Vision (ECCV), 2024.</p><p>[21] FRAPPE. “Infusing world modeling into generalist policies via multiple future representation alignment.” arXiv:2602.17259, 2026.</p><p>[22] “Deep learning-based robotic cloth manipulation applications: Systematic review, challenges and opportunities for physical AI.” Frontiers in Robotics and AI, 2026.</p><p>[23] “Unfolding the literature: A review of robotic cloth manipulation.” Annual Review of Control, Robotics, and Autonomous Systems, 2025.</p><p>[24] R. Wu, H. Lu, Y. Wang, Y. Wang, and H. Dong. “Learning foresightful dense visual affordance for deformable object manipulation.” In IEEE/CVF International Conference on Computer Vision (ICCV), 2023.</p><p>[25] A. Lee, I. Chuang, L.-Y. Chen, and I. Soltani. “InterACT: Inter-dependency aware action chunking with hierarchical attention transformers for bimanual manipulation.” arXiv:2409.07914, 2024.</p><p>[26] T. Weng, S. M. Bajracharya, Y. Wang, K. Agrawal, and D. Held. “FabricFlowNet: Bimanual cloth manipulation with a flow-based policy.” In Conference on Robot Learning (CoRL), 2021.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a072edb9fd51" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/world-model-augmented-imitation-learning-for-bimanual-garment-folding-a072edb9fd51">World Model-Augmented Imitation Learning for Bimanual Garment Folding</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic Orchestration of Vision-Language-Action Models for Fine-Motor Industrial Manipulation]]></title>
            <link>https://medium.com/correll-lab/agentic-orchestration-of-vision-language-action-models-for-fine-motor-industrial-manipulation-1c7a79874b6c?source=rss----fc0bf7de342f---4</link>
            <guid isPermaLink="false">https://medium.com/p/1c7a79874b6c</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[robotics-automation]]></category>
            <category><![CDATA[dataset]]></category>
            <category><![CDATA[vision-language-action]]></category>
            <dc:creator><![CDATA[Rayaan Lodhi]]></dc:creator>
            <pubDate>Fri, 01 May 2026 22:46:09 GMT</pubDate>
            <atom:updated>2026-05-03T22:34:16.109Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>EV Battery Disassembly via Imitation Learning and Agent-Driven Reactive Planning</strong></h4><p><strong><em>Team: </em></strong><a href="https://medium.com/@ralo7738"><strong><em>Rayaan Lodhi</em></strong></a><strong><em>, </em></strong><a href="https://medium.com/@ised7881"><strong><em>Isaac Edelman</em></strong></a><strong><em>, </em></strong><a href="https://medium.com/@semo3882"><strong><em>Sepideh Mohammadi</em></strong></a></p><p><em>The code for the agentic framework can be found here: </em><a href="https://github.com/rlodhicode/xlerobot-agent"><em>https://github.com/rlodhicode/xlerobot-agent</em></a></p><p><em>Integration code with the XLeRobot-Pro platform is located here: </em><a href="https://github.com/Minko82/xlerobot-pro/tree/sandbox/xlerobot-agentic-ai"><em>https://github.com/Minko82/xlerobot-pro/tree/sandbox/xlerobot-agentic-ai</em></a></p><h3>Abstract</h3><p>Can a robot learn a fine-motor manipulation task purely from human demonstrations — no hand-crafted controllers, no task-specific reward shaping, just watching and imitating? And when it can, is it better to train one model for the whole task, or break the task into pieces and let an AI agent coordinate them?</p><p>We explore these questions using bolt removal from an EV battery chassis as a demanding test case: a task that requires millimeter-level precision, sequential contact-rich manipulation, and reliable recovery from small errors. Using imitation learning on a modest set of teleoperated demonstrations, we fine-tuned SmolVLA, a lightweight Vision-Language-Action model, and evaluated two systems — a single end-to-end policy versus an LLM-driven agentic framework orchestrating two shorter-horizon sub-policies. The agentic system achieved 43% end-to-end success; the monolithic policy achieved 0%. We open-source our full pipeline — datasets, model checkpoints, and orchestration code — as a reproducible guide for applying this approach to other precision manipulation tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*xGwSS07ZpI63lMTEDc0Unw.jpeg" /><figcaption>The xlerobot-pro platform seated by an Hyundai Ioniq5 EV battery</figcaption></figure><p><strong>Question:</strong> Are current generative models capable of performing fine-motor, complex manipulation tasks — given appropriate demonstrations?</p><p><strong>Problem:</strong> An EV battery pack contains seated bolts on its chassis that must be removed as part of the disassembly process, currently requiring direct human involvement.</p><p><strong>Our Approach:</strong> We explored whether generative AI models could learn to remove a small bolt from an EV battery pack and place it into a bin, entirely without human intervention. Using imitation learning and fine-tuning a range of Vision-Language-Action (VLA) models, we trained an SO-101 arm to perform this task autonomously, with the broader goal of keeping human workers out of harm’s way.</p><blockquote>Note from the authors: We want this project to demonstrate how simple VLA models can be used for complex tasks while also providing a guide to others on how to replicate similar results. Please use this as your guide to explore the world of generative models for robotic manipulation.</blockquote><h3><strong>Introduction</strong></h3><p>Learning from demonstration is one of the most natural ways to teach a robot. You show it what to do, it learns to imitate, and ideally it generalizes. But how far does that actually get you on tasks that genuinely require precision — where small errors cascade, where contact matters, and where a single misstep undoes everything before it?</p><p>That is the question this work is built around. We are not primarily interested in bolt removal. We are interested in whether imitation learning, combined with modern Vision-Language-Action models, is a viable path to automating the class of tasks that have historically resisted it: fine-motor, contact-rich manipulation that tolerates almost no positional error. Bolt removal from an EV battery chassis is a particularly honest stress test for that question. The robot has to locate a bolt head at millimeter precision, close its gripper on the lip correctly, apply enough rotational force to break it free, extract it cleanly without catching on the threading, and transport it to a bin — all from demonstrations alone, with no engineered controllers or reward signal.</p><p>The secondary question we investigate is architectural: given a capable base model and a set of demonstrations, is it better to train a single policy for the full task, or to decompose the task into sub-policies and let an LLM-driven agent coordinate them? This turns out to matter enormously in practice, and the answer has implications well beyond this specific task.</p><p>We offer this work as both a research contribution and a practical tutorial. The full pipeline — teleoperation recording, dataset formatting, SmolVLA fine-tuning, and agentic orchestration with live camera verification — is open-source and designed to be reproduced on accessible hardware. If you want to apply learning-from-demonstration to a precision manipulation task of your own, this is meant to be your starting point.</p><p>We make the following contributions:</p><ul><li>A dataset of teleoperated demonstrations for bolt picking and placing on an EV battery chassis, collected using a custom leader-follower recording pipeline — comprising 100 picking episodes, 49 placing episodes and 50 pick-&amp;-place episodes.</li><li>Two fine-tuned SmolVLA sub-policies, screw_picking and screw_placing, were trained on these datasets.</li><li>A LangGraph ReAct agent that orchestrates these policies using live camera observations, VLM-based grasp verification, and YOLO-World object detection.</li><li>An evaluation of this pipeline across individual sub-tasks and the full end-to-end sequence, with analysis of failure modes.</li><li>Open-source tooling including ZMQ camera servers, a dataset recorder, and the full agentic orchestration stack.</li></ul><h3>Related Work</h3><h4><strong>Imitation Learning for Manipulation</strong></h4><p>Imitation learning from human demonstrations has become the dominant paradigm for teaching robots complex manipulation skills without explicit programming. Action Chunking with Transformers (ACT) <em>[Zhao et al., 2023]</em> demonstrated that a relatively small number of demonstrations (50–100) could train effective pick-and-place policies. Diffusion Policy <em>[Chi et al., 2023]</em> extended this with probabilistic action generation, showing improved multi-modal performance. SmolVLA <em>[Hugging Face, 2024]</em> represents a further step: a compact, open-weight VLA model pretrained on a large cross-embodiment dataset and designed for efficient fine-tuning on task-specific data. Unlike its predecessors, SmolVLA conditions on both visual input and language task descriptions, enabling more flexible generalization, though still within the distribution of its fine-tuning data.</p><h4><strong>Long-Horizon Manipulation and Task Decomposition</strong></h4><p>End-to-end VLA models struggle with long-horizon tasks because small early errors compound across later steps. Task decomposition, breaking a complex skill into a sequence of modular sub-policies, is a well-established mitigation <em>[Gupta et al., 2019; Shridhar et al., 2022]</em>. Our work decomposes bolt removal into two sub-policies (pick, place) coordinated by an external agent, following the modular design principle of hierarchical robot learning.</p><h4>LLM-Based Robot Agents</h4><p>Large language models have emerged as effective high-level planners for robot tasks <em>[Ahn et al., 2022; Liang et al., 2023]</em>. SayCan <em>[Ahn et al., 2022]</em> showed that an LLM can select from a library of learned skills to accomplish open-ended goals, grounded by affordance functions. More recent work has integrated LLMs with vision via VLMs to close the perception loop <em>[Driess et al., 2023]</em>. Our orchestrator extends this line of work</p><p>by implementing a full ReAct <em>[Yao et al., 2023]</em> loop — Reason, Act, Observe — over a live robot, with the LLM deciding when to start and stop VLA policies based on real-time camera evidence.</p><h4>Industrial Disassembly</h4><p>Automated disassembly for remanufacturing and recycling has received growing attention <em>[Allen et al., 2026]</em>, with most work focusing on structured environments with rigid task specifications <em>[Bdiwi et al., 2017; Wegener et al., 2020]</em>. VLA-based approaches to unstructured industrial disassembly remain largely unexplored, making our task a meaningful extension of the current frontier.</p><h3><strong>Materials and Methods</strong></h3><h4>The Hardware Platform</h4><p>We run all our experiments on the <strong>XLeRobot-Pro </strong><em>[Shaw et al., 2026]</em>, an open-source, 3D-printed, low-cost robot arm built by our lab, and powered by an NVIDIA Jetson Orin Nano. Full specs and bill of materials are on the<a href="https://github.com/Minko82/xlerobot-pro"> project’s GitHub</a>.</p><p>For our bolt-removal task, the arm was fitted with a <strong>2-finger gripper</strong> featuring fingertip ridge modifications to improve grip on bolts. The target was a Hyundai Ioniq5 EV battery pack with seated bolts on its chassis, and a collection bin placed to the left of the arm for bolt disposal.</p><p>Perception was provided by two cameras: a <strong>wrist-mounted RGB camera</strong> for close-up alignment near the bolt, and an <strong>overhead Intel RealSense</strong> for a wide view of the workspace, both feeding live visual input to the AI model at inference time.</p><p>For our bolt-removal task, the key components are:</p><p><strong>The Arms</strong> — There is a leader and follower arm. The SO-101 arms are a 6-joint manipulator driven by STS3215 servo motors.</p><p><strong>Two cameras</strong> — a base-mounted Intel RealSense for a wide view of the workspace, and a wrist-mounted USB camera on the end-effector for close-up alignment near the bolt.</p><p><strong>The computer</strong> — a Jetson Orin Nano onboard for real-time motor control, communicating over WiFi with a laptop with a GTX 1650 Max-Q 4GB card.</p><p><strong>Power</strong> — fully untethered, running off an Anker portable power station for 10+ hours of continuous operation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*J56pHnyi1-xbUdBigHehIQ.png" /><figcaption>Annotated image of the xlerobot-pro platform as it is used in the battery disassembly task</figcaption></figure><h4>The Brain: SmolVLA</h4><p>The AI model we used is called <strong>SmolVLA</strong>, developed by HuggingFace: <a href="https://huggingface.co/docs/lerobot/il_robots#train-a-policy">Imitation Learning on Real-World Robots</a>, and for proper learning <a href="https://huggingface.co/docs/lerobot/notebooks#training-act">example notebooks for using LeRobot</a></p><p>It comes pre-trained on thousands of robot demonstrations from labs around the world; we just gave it extra practice on our specific task.</p><p>While SmolVLA is capable of generating extended action sequences, our implementation focuses on high-frequency, single-step execution to ensure closed-loop reactivity. Consequently, we structured our training data such that complex behaviors — such as picking and placing a bolt — are decomposed into discrete, manageable sub-tasks. This granularity allows the model to maintain higher spatial precision and adapt to environmental changes between movements.</p><h3>Dataset Collection</h3><p>Training data was collected via leader-follower teleoperation. The leader arm (SO-101 Leader, port /dev/ttyACM1) is kinematically identical to the follower and is operated by hand. Joint positions are read from the leader at ~16 Hz and replayed on the follower in real time, while both cameras record at 30 fps. We implemented a custom <em>dataset_recorder.py</em> script to manage the full episode lifecycle: positioning, recording, resetting, and background encoding to HuggingFace LeRobot dataset format.</p><p>The recorder supports headless operation (keyboard commands via stdin), background episode encoding using a thread-pool executor to avoid blocking the control loop, and video encoding via the LeRobot VideoEncodingManager. Episodes are encoded with h264 and pushed directly to the HuggingFace Hub.</p><p>We collected demonstrations across three conditions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XMwjf8-7cIfBK-laA7Q6VQ.png" /><figcaption>Table 1: Dataset collection and model information</figcaption></figure><p>You can visualize the datasets using the following links:</p><ul><li>Screw Picking: <a href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_picking%2Fepisode_0">https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_picking%2Fepisode_0</a></li><li>Screw Placing: <a href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_placing_deprecated%2Fepisode_0">https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_placing_deprecated%2Fepisode_0</a></li><li>Screw Pick &amp; Place: <a href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_pick_place%2Fepisode_0">https://huggingface.co/spaces/lerobot/visualize_dataset?path=%2Frlodhi%2Fscrew_pick_place%2Fepisode_0</a></li></ul><p>Each demonstration captures full joint-state trajectories and stereo video (base + wrist) at 30 fps. Demonstration duration averaged approximately <em>17</em> <em>seconds</em> for picking and <em>5.66</em> <em>seconds</em> for placing.</p><h4><strong>Teaching the Robot: Imitation Learning</strong></h4><p>We used <strong>kinesthetic teaching</strong>: a human physically guided (by a leader to teach the follower leader) the robot arm through the full task while the system recorded everything, joint positions, camera frames, timestamps, at 30 frames per second.</p><blockquote>One insight from our sessions: Slowing the leader’s arm down significantly during the second round of demonstrations produced noticeably better model performance. The model learned cleaner, more deliberate trajectories when the training data moved more slowly. Our dataset also has issues with the cameras having frame drop and image collection issues throughout the episodes, which cause jumps in visual continuity in the dataset.</blockquote><p>Our overall target action had 5 phases:</p><ol><li><strong>Approach</strong> — move the end-effector toward the bolt</li><li><strong>Open the gripper</strong> — before closing to the screw/bolts</li><li><strong>Align</strong> — use the wrist camera to center the tool precisely on the bolt head</li><li><strong>Grip </strong>— grip the bolt by the lip of the head</li><li><strong>Return Home</strong> — Once the bolt is gripped, the arm removes the bolt and returns to a home position.</li><li><strong>Place</strong> — transport the bolt to the bin and release</li></ol><h4>Model Training</h4><p>To fine-tune a model from our collected datasets, we used the following Google Colab Notebook:</p><p><a href="https://colab.research.google.com/drive/1LBO75JmylF8aARQZ1fIWtkpVcoov7dCi?usp=sharing">Google Colab</a></p><p>This notebook fine-tunes SmolVLA, a vision-language-action model for robot control, on a custom dataset for a specific manipulation task (picking up screws).</p><p>Here’s how it works end-to-end: the notebook first clones a custom fork of the xlerobot-pro repo library and installs its SmolVLA-specific dependencies. It then authenticates with Hugging Face to pull down a robot demonstration dataset (stored in LeRobot’s V3 dataset, standard format with episodes, frames, and camera feeds) into a local cache. From there, it kicks off training via the lerobot-train CLI, which fine-tunes the pretrained smolvla_base checkpoint — a base model that already understands visual and language inputs — on that dataset for 25,000 gradient steps. Key training parameters like batch size are tuned to the available GPU (T4/L4/A100), and checkpoints are saved every 5,000 steps. The final trained policy is pushed back to a Hugging Face Hub repo so it can be deployed on a real robot.</p><p>We fine-tuned SmolVLA for the complete task and separately for each sub-task, producing three policies: screw_pick_place (rlodhi/screw_pick_place_50k_smolvla), screw_picking (rlodhi/screw_picking_25k_smolvla), screw_placing (rlodhi/screw_placing_5k_smolvla), all hosted on Hugging Face. The goal of having the three separate policies was to explore long-horizon task completion with and without the aid of an AI orchestrator. No task-specific reward shaping or auxiliary supervision was applied; only the standard imitation learning objective was used.</p><p>The model receives two camera inputs per forward pass (base and wrist frame), encoded jointly, along with the task string as a language signal. Action outputs are joint position deltas, chunked into sequences of 25 steps. At inference time, policies run via an asynchronous client that maintains a rolling buffer of predicted action chunks, keeping motor control continuous without gaps caused by model inference latency.</p><h3>Agentic Orchestration Framework</h3><p>The core contribution of our system architecture is the agentic orchestrator that coordinates VLA policies, camera observations, and task verification. We implemented a ReAct (Reason + Act + Observe) agent using LangGraph, backed by OpenAI’s GPT5.4-nano as the reasoning LLM.</p><blockquote>The tool is designed to allow users to point to any common provider for the orchestrating LLM, available currently are Anthropic, OpenAI, Gemini, and local Ollama models.</blockquote><h4><strong>Agent Architecture</strong></h4><p>The agent operates as a LangGraph StateGraph with two primary nodes: a reason node (LLM inference) and a tools node (capability execution). The state machine implements the following loop:</p><ul><li><strong>Reason</strong>: the LLM receives the current conversation history, system prompt, and all prior observations, and decides the next action.</li><li><strong>Act</strong>: the chosen capability is dispatched via the tool registry (list_capabilities_tool, read_capability_tool, run_capability_tool).</li><li><strong>Observe</strong>: the tool result (VLM description, YOLO detections, policy status) is appended to the message history.</li><li>Repeat until the LLM concludes the task is complete, or the maximum iteration limit (20) is reached.</li></ul><p>Conversation history is persisted across turns via LangGraph’s AsyncPostgresSaver checkpoint system, enabling multi-turn interactions within a thread. Long contexts are compacted by summarizing older messages when the estimated token count exceeds 25,000 tokens.</p><h4>Capability Registry</h4><p>The agent has access to eight capabilities exposed as LangChain tools:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*RZqEuRBWh7_a79RdlwEe8w.png" /><figcaption>Table 2: Available capabilities to the orchestration agent</figcaption></figure><p>The object detector (YOLO-World) is configured with an EV battery-specific vocabulary, Bolt, BusBar, InteriorScrew, Nut, OrangeCover, Screw, Screw Hole adapted from the class taxonomy introduced by <em>[Allen et al. 2026]</em>. Detection results tell the agent whether objects are present; they are not consumed directly by the VLA policy, which is purely vision-driven.</p><h4>Policy Execution Protocol</h4><p>When the agent decides to execute a VLA policy, it follows a structured protocol enforced by the system prompt:</p><ul><li>Call start_vla_policy with the policy_id. This call blocks until the model is loaded on the inference server and the robot is already executing (signaled by the ‘Action receiving thread starting’ log line). No additional wait is needed for model loading.</li><li>Wait for the policy’s typical_wait_time (75 seconds for screw_picking, 25 seconds for screw_placing), as returned in the start response.</li><li>Call observe_with_wrist_camera to verify the outcome (grasp success or bin placement).</li><li>If the outcome is ambiguous or negative, wait for a while longer (currently the agent prefers to wait in increments halving the original wait time i.e. 75 -&gt; 37.5 etc.) and re-observe. If confirmed failure, the agent may stop the policy and report failure.</li><li>Call stop_vla_policy to terminate the inference client before proceeding to the next sub-task.</li></ul><blockquote>A key design decision was implementing start_vla_policy as a blocking call that writes subprocess output to a file rather than a pipe. Pipe-based subprocess communication has a finite kernel buffer (~64 KB on Linux); if the orchestrator’s log handler is slow, the buffer fills and the subprocess blocks — freezing the robot’s control loop. File-based logging eliminates this bottleneck entirely.</blockquote><h4>VLM Integration</h4><p>Visual observations are handled by Gemini 2.5 Flash via the Vertex AI SDK. Frames are downsampled to 400px max dimension and JPEG-encoded at 85% quality before submission to minimize token cost. Camera-specific preambles condition the model on the workspace context: the base camera preamble describes the overview perspective and EV battery components; the wrist camera preamble explicitly describes the gripper’s appearance (black ribbed cylindrical fingers) to prevent the model from misidentifying the gripper as a grasped object.</p><p>The orchestrator can capture images from the robot’s cameras and send them to the VLM alongside a question. This allows the robot to effectively ask generic questions like, “what is in the workspace”, or “what grey objects are in the scene” and get back an answer in natural language.</p><blockquote>Base-64 encoded frames are stripped from the agent’s context window after each observation step (before re-entering the LLM) to prevent runaway token accumulation. The VLM’s natural language description is retained in context for subsequent reasoning steps.</blockquote><h3>Results</h3><p>To determine if our architecture has an impact on the full VLA action success rate, we compared the comprehensive VLA model without the agentic AI agent orchestrator to the agentic AI orchestrator with two shorter-horizon VLA tasks that completed the same overall action goal of removing the bolt and placing it in the bin.</p><p>We evaluated the two systems with varying level of semantic task descriptions as seen in Table 3:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*WU0KnEl6c4-TLdpVZ9nuuQ.png" /><figcaption>Table 3: Mapping of task keys to the task label that are used in the experiments</figcaption></figure><p>The results show a clear performance gap between the two systems. The pure VLA model failed across all 14 trials (0% success), struggling primarily with navigation to the grasp pose and precise gripper closure — failures that reflect the difficulty of chaining a long-horizon task within a single policy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*lLsZ_Mb6OZd7bisWKnhL6Q.gif" /><figcaption>Grasp Failure</figcaption></figure><p>Even in the one trial where the screw was successfully picked (Trial 6), the policy dropped it before reaching the placement target, highlighting how compounding errors across a long action sequence make end-to-end success rare with the model at this stage of training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*3vixqPV8y624rFyEIZaxMg.png" /><figcaption>Table 4: VLA only trials</figcaption></figure><p>The Agent + VLA system achieved a 43% success rate (6 of 14 trials), demonstrating that decomposing the task into two shorter-horizon policies — pick and place — meaningfully improves reliability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*ODgBtEQ05nZ9BdntDjTdZw.gif" /><figcaption>Successful Pick and Place via Agent Orchestration (sped up) — Encoding from video to GIF slightly occludes the placing process, but the screw was dropped successfully in the bin</figcaption></figure><p>The agent’s ability to reason about task state between policies also proved valuable: in several trials, it correctly identified a failed grasp and extended VLA execution time or requeued the policy entirely, recovering that a standalone VLA has no mechanism to perform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*PnOuTO6pSWhIo-qRnEMBZw.png" /><figcaption>Table 5: Agent Orchestrated VLA Trials</figcaption></figure><p>The primary failure mode for the agentic system was hallucinated grasp confirmation — the vision-language model incorrectly assessed the screw as secured and triggered the place policy prematurely. This accounted for the majority of failures across prompt types A and B. Notably, the system handled all three prompt types with reasonable competence, including the highly verbose Prompt C, correctly parsing natural language descriptions that the pure VLA would have had no mechanism to interpret at all. This suggests that the agent layer adds meaningful robustness not just in task execution but in language generalization.</p><h4>Sub-Task Analysis</h4><p><strong>Picking:</strong> The screw_picking policy succeeded in roughly 43% of agent trials (the 6 passing trials all achieved a successful pick). The most common failure mode was an initial alignment miss: the gripper approached the bolt head but contacted the chassis surface rather than seating on the bolt itself. This failure was typically detectable via the wrist camera (the bolt head remained visible between open fingers). In 4 of 8 failed agent trials, the VLM correctly identified this failure; trials 8, 11, and 12 explicitly show the agent recognizing a failed grasp and either extending VLA time or requeuing the policy, while trials 5 and 7 show the opposite, where the agent failed to detect the miss and prematurely handed off to the place policy.</p><p><strong>Placing:</strong> The screw_placing policy, run with a bolt pre-loaded in the gripper, achieved 100% success across all trials where the pick was confirmed secured; every trial that passed the pick stage went on to place successfully. The dominant failure mode was bin overshoot; the gripper released the bolt with lateral velocity sufficient to cause it to bounce out of the bin. Trial 14 is the clearest example of a place-stage failure, where the screw was extracted successfully but was dropped before the place policy could execute due to async inference lag. This was less detectable via wrist camera (the bolt exits the frame on release) and required base camera observation for verification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*qmzuzcDhNxhhZ97qBREoKg.gif" /><figcaption>Screw Placing</figcaption></figure><h4>Agent Coordination Overhead</h4><p>The agentic orchestration added an estimated 16 LLM calls per full trial (initial task interpretation, post-pick verification, and place trigger, with additional calls in retry scenarios), with an average reasoning time of 1.15 seconds using GPT5.4-nano. The total wall-clock overhead of the orchestrator (excluding policy loading execution time) was approximately 25–35 seconds per trial, derived by subtracting average pick time (~86s), place time (~35s), and policy loading time from the average total thread time (~305s across successful runs). Note: We use Langsmith observability to track exact metrics on time taken and tokens spent. This overhead is negligible relative to policy execution time (~86s pick + ~35s place, based on observed averages). The agent operated within the 20-iteration limit in all trials, though trials 11 and 12 came closest, each requiring two full VLA pick attempts before terminating. In all 14 trials the accumulated cost was $0.30. Average token use per trial was about 55k.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ueeH8vPkGGfkdkWLudR8Cw.png" /><figcaption>Example Trace Snippit From Langsmith Demonstrating Agent Coordination — Visualized with Gemini</figcaption></figure><h3>Discussion</h3><h4>What Worked</h4><p>The modular pick-then-place decomposition proved effective. Separating the two sub-tasks allowed independent fine-tuning on datasets of appropriate size, 100 demonstrations for the harder picking task, 50 for placing, and enabled per-stage verification. This design would be difficult to replicate with a single end-to-end policy trained on the full sequence. However, if more training is used and longer tuning steps, perhaps a better model can be achieved. In addition, separating the tasks in our case does create two VLA models, which doubles the size; if the same performance can be achieved with one, the storage requirements for local devices would benefit. This would be the first natural thing to look into, as VLAs on the LeRobot platform are configurable to bake in multiple datasets into one.</p><p>The agentic orchestrator’s most concrete contribution was verification: the wrist camera observation after picking provided a reliable signal about the grasp state, allowing the agent to abort failed trials rather than proceeding into a placing attempt that would definitely fail. This is a meaningful improvement over blind policy chaining. It does need improvement, but it shows great promise for VLA error checking. With the VLA error checking we get VLA policy termination OOTB; the agent decides when a policy should carry on and when it should stop, which is actually a non trivial problem <em>[Zhang et al., 2026]</em>.</p><p>The custom hardware contributions were significant. The static base camera mount eliminated the frame drift that corrupted early trials. The modified gripper fingertips improved contact consistency with the bolt head, particularly important for the initial grasp alignment.</p><h4>Failure Analysis and Limitations</h4><p>The dominant failure mode across both sub-policies was alignment sensitivity. M8 bolt heads have a hexagonal cross-section approximately 13mm across; the gripper must approach within approximately 2–3mm of the center to achieve a stable grip. The VLA policy learns this alignment implicitly from demonstrations, but does not always generalize correctly; small variations in bolt position or lighting cause the learned approach trajectory to miss. This is fundamentally a distribution problem: our training set was relatively narrow, and the policy has not developed the adaptive re-alignment behavior that an expert human teleoperator uses naturally. A compounding issue is the policy’s inability to recover from partial failures mid-execution. If the gripper contacts the bolt off-center, the policy does not back off and retry; it continues executing its learned trajectory, which may produce a marginal grip that fails during extraction. External intervention from the agent is the only current recovery mechanism.</p><p>The more operationally significant failure mode, however, was the interaction between async inference and the action buffer. SmolVLA’s async inference client maintains a rolling action buffer to ensure continuous motor control, which means the robot is always executing actions computed several steps in the past. When a pick attempt fails midway through execution, the robot continues consuming buffered actions that were planned under the assumption of a successful grasp. There is no mechanism to flush the buffer and replan in response to a detected failure. In practice, this means the gripper can be observed closing on air while the arm continues its extraction trajectory, producing failures that are difficult to interrupt cleanly. This latency between physical state and planned action is the underlying cause of the “premature handoff” failures seen in trials 5 and 7, where the agent’s grasp check occurred while stale buffered actions were still executing. As a matter of fact, in the SmolVLA paper, the authors used synchronous inference when in the real environment to deal with this issue. That is something that we would like to look into but weren’t able to do so given the circumstances. The Orin, containing programs and services for other collaborators was not able to load the policies into its unified 8GB memory. Because of this we offload inference to the laptop and lose synchronous capability. In order to achieve that adjustments would have to be made to the LeRobot codebase to allow for offloaded synchronous inference.</p><p>A closely related limitation is the agent’s susceptibility to visual hallucination during grasp verification. The orchestrator relies on Gemini 2.5 Flash to assess wrist camera frames and determine whether the screw is secured before triggering the place policy. In 4 of 8 failed agent trials, Gemini incorrectly confirmed a successful grasp despite the bolt not being secured, in some cases because the gripper’s partial closure around the bolt head produced an image that superficially resembled a successful grip. This is a meaningful reliability ceiling for the current architecture: the agent’s recovery logic is only as good as its visual verification, and hallucinated confirmations bypass all retry logic entirely. Improving this likely requires either higher-confidence verification signals (force/torque sensing, multiple-frame confirmation) or a more conservative confirmation threshold that accepts false negatives over false positives.</p><p>Finally, the YOLO-World detector contributes limited value during active policy execution. Its normalized image-space coordinates are not consumed by the VLA control loop, and bolt head detection degrades precisely when it would be most useful, during gripper approach, when the bolt is partially occluded by the fingers themselves. Its role is effectively limited to <strong>initial scene assessment </strong>(which is still incredibly useful!), and future iterations should explore tighter integration between detection confidence and VLA action gating.</p><h4>The Role of Agentic Supervision</h4><p>A natural question is whether the agentic orchestrator adds meaningful value over simply running the policies in sequence with fixed waits. Our results suggest it does, but primarily through verification rather than active replanning. The agent does not modify the VLA policy’s behavior during execution; it observes outcomes and decides whether to proceed, wait, or abort. This is a reactive supervisory role, not a control role.</p><p>This distinction points toward a key open problem: closed-loop agentic control, where the agent can interrupt and redirect a running policy based on real-time observation, rather than only acting between policy completions. Achieving this would require either a faster feedback loop (sub-second observation and decision) or a policy architecture that can accept mid-execution conditioning from an external supervisor.</p><h4><strong>Broader Implications</strong></h4><p>Our results are limited in scale, 14 trials on a single hardware configuration with a fixed bolt position, and should not be over-generalized. Nevertheless, they suggest that the combination of imitation-learned sub-policies and lightweight agentic supervision is a viable architecture for fine-motor industrial tasks. The pure VLA approach achieved 0% end-to-end success across the same 14 trials, while the agentic architecture achieved 43%, entirely without changes to the underlying policy weights; the improvement comes purely from decomposition and orchestration. This delta is meaningful: it suggests that for tasks where a single long-horizon policy struggles, a coordination layer adds real value even at the current state of VLA capability.</p><p>The components are relatively accessible. SmolVLA fine-tuning requires no proprietary infrastructure and runs on a single A100 in approximately 1 hour for 25,000 steps, and the orchestration stack runs on commodity hardware with a lightweight LLM API call overhead. The primary cost is demonstration collection; we collected demonstrations across picking and placing sub-tasks to achieve 43% end-to-end success, and the gap between that figure and human-level performance (a trained operator completes this task reliably in under 30 seconds) points to where the work remains.</p><p>The most significant remaining barrier is demonstrating efficiency and distribution coverage. Our training distribution was narrow, and the policy’s alignment sensitivity to small variations in bolt position or lighting reflects that directly. A human expert learns this task in minutes and generalizes immediately to new bolt positions, lighting conditions, and partial occlusions, capabilities that currently require explicit retraining or significant data augmentation to replicate. Improving data efficiency, whether through better base models, sim-to-real transfer, or active learning, is the most important open direction for this line of work. A secondary implication of our failure analysis is that agentic verification quality is now a bottleneck: with 4 of 8 agent failures attributable to LLM hallucination during grasp confirmation, improvements to the orchestration layer, better visual verification, multi-modal confirmation signals, or more conservative decision thresholds may yield as much as further policy training at this stage.</p><h3>Conclusion</h3><p>We presented an end-to-end system for robotic bolt removal from Hyundai Ioniq5 EV battery packs, combining SmolVLA imitation learning with a LangGraph-based agentic orchestrator. Our system achieves 43% end-to-end success on M8 bolt removal from a EV battery chassis, compared to 0% for a standalone VLA policy on the same task, representing a meaningful step toward practical automation of a dangerous and labor-intensive disassembly process.</p><p>The central finding is that task decomposition combined with camera-based verification substantially improves the reliability of VLA policies in long-horizon, contact-rich manipulation. By splitting a single difficult policy into two shorter-horizon sub-policies and interposing an agentic verification step, the system gains the ability to detect and respond to partial failures that a monolithic policy has no mechanism to handle. The agent’s verification loop, using a wrist-mounted VLM to assess grasp state, provides an actionable signal that enables informed decisions about whether to proceed, extend execution time, or abort and retry, in a way that fixed-timing policy chains cannot. That said, our failure analysis makes clear that the reliability of this verification step is itself a meaningful bottleneck: hallucinated grasp confirmations accounted for the majority of remaining failures, and closing this gap is as important as further policy training.</p><p>The system also demonstrated unexpectedly strong natural language generalization. Across three prompt types ranging from precise task instructions to highly colloquial descriptions, the agent successfully interpreted the task and invoked the correct sub-policies in every trial, a capability that a standalone VLA, which operates on a fixed task string, cannot replicate.</p><p>Our codebase, datasets, and model checkpoints are open-source to enable other groups to build on this work for EV disassembly and related industrial tasks. We hope this serves as a practical baseline for combining lightweight VLA fine-tuning with agentic orchestration in real-world complex settings, where demonstration efficiency, failure recovery, and natural language flexibility are all first-class requirements.</p><p><em>The code for the agentic framework can be found here: </em><a href="https://github.com/rlodhicode/xlerobot-agent"><em>https://github.com/rlodhicode/xlerobot-agent</em></a></p><p><em>Integration code with the XLeRobot-Pro platform is located here: </em><a href="https://github.com/Minko82/xlerobot-pro/tree/sandbox/xlerobot-agentic-ai"><em>https://github.com/Minko82/xlerobot-pro/tree/sandbox/xlerobot-agentic-ai</em></a></p><h3>References</h3><p>[Ahn et al., 2022] Ahn, M., et al. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. arXiv:2204.01691.</p><p>[Allen et al., 2026] Allen, Z., et al. Robotic Agentic Platform for Intelligent Electric Vehicle Disassembly (RAPID). arXiv:2603.18520. University of Colorado Boulder, Correll Lab. March 2026.</p><p>[Bdiwi et al., 2017] Bdiwi et al. AR-based interaction for human-robot collaborative manufacturing. <a href="https://doi.org/10.1016/j.ifacol.2020.12.2744">https://doi.org/10.1016/j.ifacol.2020.12.2744</a>.</p><p>[Chi et al., 2023] Chi, C., et al. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. RSS 2023.</p><p>[Driess et al., 2023] Driess, D., et al. PaLM-E: An Embodied Multimodal Language Model. ICML 2023.</p><p>[Gupta et al., 2019] Gupta, A., et al. Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning. CoRL 2019.</p><p>[Hugging Face, 2024] HuggingFace Robotics Team. SmolVLA: A Vision-Language-Action Model for Efficient Robot Control. HuggingFace Blog.</p><p>[Liang et al., 2023] Liang, J., et al. Code as Policies: Language Model Programs for Embodied Control. ICRA 2023.</p><p>[Shaw et al., 2026] Shaw, A., et al. Cutting the Cord: System Architecture for Low-Cost, GPU-Accelerated Bimanual Mobile Manipulation. arXiv:2603.09051. University of Colorado Boulder.</p><p>[Shridhar et al., 2022] Shridhar, M., et al. Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation. CoRL 2022.</p><p>[Wegener et al., 2020] Wegener et al. Human-robot collaborative disassembly task planning for retired power battery based on Stackelberg game and multi-agent deep reinforcement learning. <a href="https://doi.org/10.1016/j.jmsy.2025.07.024">https://doi.org/10.1016/j.jmsy.2025.07.024</a>.</p><p>[Yao et al., 2023] Yao, S., et al. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023.</p><p>[Zhang et al., 2026] Zhang, W., et al. From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models. ICASSP 2026. arXiv:2602.01811.</p><p>[Zhao et al., 2023] Zhao, T., et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. RSS 2023.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c7a79874b6c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/agentic-orchestration-of-vision-language-action-models-for-fine-motor-industrial-manipulation-1c7a79874b6c">Agentic Orchestration of Vision-Language-Action Models for Fine-Motor Industrial Manipulation</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>