<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Muhammad Angga Muttaqien on Medium]]></title>
        <description><![CDATA[Stories by Muhammad Angga Muttaqien on Medium]]></description>
        <link>https://medium.com/@muhamuttaqien?source=rss-34645ca0105b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FxViIp4oPxClGY2FaoWVRw.png</url>
            <title>Stories by Muhammad Angga Muttaqien on Medium</title>
            <link>https://medium.com/@muhamuttaqien?source=rss-34645ca0105b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 13:14:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@muhamuttaqien/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Explicit vs. Implicit vs. Diffusion Policy: Three Ways Robots Learn to Act]]></title>
            <link>https://medium.com/@muhamuttaqien/explicit-vs-implicit-vs-diffusion-policy-three-ways-robots-learn-to-act-ad66fa714236?source=rss-34645ca0105b------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad66fa714236</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Muhammad Angga Muttaqien]]></dc:creator>
            <pubDate>Sat, 19 Sep 2026 08:30:24 GMT</pubDate>
            <atom:updated>2026-09-19T17:21:57.469Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4XzESAU5on0_hIQqs_-qjQ.png" /><figcaption>Mug-flipping task from Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.</figcaption></figure><p>A robot sees a mug on the table. Its camera captures the scene. Its sensors tell it where the arm currently is. The instruction might be simple: “Pick up the mug.” or “Flip the mug”.</p><p>But the robot still needs to answer one important question:</p><p><strong>What should I do next?</strong></p><p>For a robot, that answer is usually an action. It could be the next joint positions. It could be the movement of the end-effector. It could also be something like:</p><pre>[x, y, z, roll, pitch, yaw, gripper]</pre><p>In other words, the robot needs to transform what it observes into a physical movement.</p><p>The policy is the decision-making model in the middle. It receives information about the current situation and produces the action the robot should execute.</p><p>This sounds straightforward.</p><p>See the mug.</p><p>Predict the motion.</p><p>Move the robot.</p><p>But there is a complication.</p><p>In many robot tasks, there is not only one correct action.</p><p>Imagine a robot trying to reach an object while avoiding an obstacle. It might go around the obstacle from the left. Or it might go around from the right. Both trajectories can be perfectly correct. So the real question is not simply:</p><p><strong>“Can the model predict an action?”</strong></p><p>A more interesting question is:</p><p><strong>“How the model represent all the possible actions that could be correct?”</strong></p><p>This is where different policy formulations begin to matter. In the <a href="https://arxiv.org/pdf/2303.04137">Diffusion Policy</a> paper, the authors discuss three useful ways to think about this problem: <strong>Explicit Policy</strong>, <strong>Implicit Policy</strong> and <strong>Diffusion Policy</strong>.</p><p>They all try to solve the same basic problem. But they represent the answer in very different ways. Let’s break it down!</p><h3>Explicit Policy: Predict the Action Directly</h3><p>The most straightforward approach is an <strong>explicit policy</strong>.</p><p>The idea is simple:</p><p>Give the model an observation.</p><p>Ask it to directly predict the action.</p><p>Suppose the robot sees a cup in front of it.</p><p>The policy receives the camera image and robot state.</p><p>Then it directly produces something like:</p><pre>[0.03,,-0.01,,0.05,,0.00,,0.02,,-0.01,,1]</pre><p>These numbers might represent a small end-effector movement and a gripper command.</p><p>The important point is that the model directly tells the robot:</p><p><strong>“Do this.”</strong></p><p>There is no search.</p><p>There is no iterative refinement.</p><p>The observation goes into the neural network, and the action comes out.</p><p>This is the basic idea behind many behavioral cloning systems.</p><p>During training, we collect demonstrations from an expert.</p><p>For example:</p><pre>[(o_1,a_1), (o_2,a_2), (o_3,a_3), ...]</pre><p>At each observation, the expert provides the correct action.</p><p>The neural network then learns to imitate those examples.</p><p>If the expert moves the gripper slightly to the left, the model learns to predict that movement. If the expert closes the gripper near the object, the model learns to predict that too.</p><p>A simple regression policy can be trained by minimizing the difference between the predicted action and the expert action.</p><p>Conceptually:</p><pre>loss = predicted_action - expert_action</pre><p>Or more commonly, with mean squared error (MSE):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/145/1*aD7Q95RT4Z_ur_cxClfejA.png" /></figure><p>This works surprisingly well when the relationship between observation and action is quite simple.</p><p>If the robot sees this situation:</p><p><strong>there is one obvious thing to do.</strong></p><p>Then direct prediction is a very natural solution.</p><p>You can think of it like asking someone:</p><p><strong>“Where should I go?”</strong></p><p>And expecting them to immediately point in one direction.</p><p>No map. No discussion. Just one answer.</p><p>The problem begins when several answers are equally valid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3CYXK3fRoibVzOyMlEDacA.png" /><figcaption>Two successful robot trajectories pass around the obstacle from opposite sides.</figcaption></figure><p>Imagine a robot arm needs to reach a target behind an obstacle. During data collection, the human operator demonstrates two strategies. In some demonstrations:</p><p><strong>Go around the left side.</strong></p><p>In others:</p><p><strong>Go around the right side.</strong></p><p>Both strategies succeed. From the robot learning’s perspective, this creates a difficult situation. The same or very similar observation can be associated with two very different actions.</p><p>One demonstration says:</p><p><strong>Move left.</strong></p><p>Another says:</p><p><strong>Move right.</strong></p><p>A simple regression policy is often trained to minimize average prediction error. So what happens? It may try to compromise. Left and right get averaged. And the model predicts something in the middle. Unfortunately, the middle is exactly where the obstacle is.</p><p>This phenomenon is often called <strong>mode averaging</strong>.</p><p>The demonstrations contain multiple valid modes of behavior. But the model tries to summarize them with one prediction.</p><p>Let’s consider a very simplified example. Suppose the demonstrations contain only two possible horizontal movements:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/89/1*FuokQ88IXjAEEEbkoFHFzA.png" /></figure><p>and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/89/1*voYF2evs39uF890xs-m5ew.png" /></figure><p>If the model tries to minimize squared error, a reasonable average prediction becomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/59/1*sSC_kBzHL11pjdDh2riKwQ.png" /></figure><p>Mathematically, that can reduce the average error.</p><p>But behaviorally, it can be completely wrong.</p><p>Neither expert ever chose zero.</p><p>The model has produced an action that exists between the demonstrations but may not correspond to any successful behavior.</p><p>Another related problem is <strong>mode switching</strong>.</p><p>Even if a policy avoids averaging, it may switch between different valid strategies during execution.</p><p>For example, starting to go left around an obstacle and then suddenly changing toward the right.</p><p>This is especially important in robotics because many tasks are naturally multimodal. A robot can grasp an object from different angles.</p><p>It can move around an obstacle in different directions.</p><p>It can place an object at different valid locations.</p><p>It can perform the same task using different motion strategies.</p><p>So the problem is deeper than simply predicting accurately. The model needs to represent the fact that: <strong>there may be several correct answers at the same time.</strong></p><p>An explicit policy that predicts only one direct answer can struggle with this. So researchers began asking a different question. Instead of forcing the model to immediately answer:</p><p><strong>“What action should I take?”</strong></p><p>What if we ask:</p><p><strong>“How good is each possible action?”</strong></p><p>That leads us to the next idea: <strong>Implicit Policy.</strong></p><h3><strong>Implicit Policy: Score Possible Actions Instead</strong></h3><p>An explicit policy tries to answer one question directly:</p><p><strong>“What action should I take?”</strong></p><p>An implicit policy asks a different question:</p><p><strong>“How good is this action?”</strong></p><p>Instead of directly predicting an action from the observation, the model receives both: the observation, and a candidate action, then produces a score. This is often written as an energy function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/84/1*svNq7vJUmhf_2E-sjsKuHw.png" /></figure><p>where:</p><ul><li>(o) is the observation</li><li>(a) is a candidate action</li></ul><p>A good action should have low energy.</p><p>A bad action should have high energy.</p><p>Sounds simple, right? So instead of learning:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/64/1*_wAPoHw-gJxu_lVih5l2yg.png" /></figure><p>the model learns something closer to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/175/1*xEjjgem1Tat_4eq9yXvGog.png" /></figure><p>A useful way to imagine this is as a landscape.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fKr8YDjbrfE6si8SNbkEtA.png" /><figcaption>Two low-energy valleys represent different valid action modes, separated by a high-energy region</figcaption></figure><p>Suppose every possible robot action is a location on a map. Some regions are mountains. Others are valleys. The valleys represent actions that fit the demonstrations well. The mountains represent actions that are unlikely or incorrect.</p><p>Now return to our earlier example.</p><p>The robot can move around an obstacle either from the left or from the right. An explicit regression policy may try to average those two behaviors. But an implicit policy does not have to collapse them into one answer. It can represent:</p><p><strong>one low-energy region on the left</strong></p><p>and</p><p><strong>another low-energy region on the right</strong></p><p>Both can exist at the same time. That is the main attraction of an implicit policy. It can naturally represent <strong>multimodal behavior</strong>.</p><p>Instead of saying:</p><p><strong>“There is only one correct action.”</strong></p><p>it can represent:</p><p><strong>“There are several actions that are all plausible.”</strong></p><p>This is especially useful in robotics. A robot might grasp the same object from different directions. It might move around an obstacle using different trajectories. It might complete the same task with several equally valid motion strategies.</p><p>An implicit policy gives the model a way to represent all of these possibilities without averaging them into one compromise.</p><p>But unfortunately this flexibility creates a new problem. The model can tell us how good an action is. It still does not directly tell us which action to execute.</p><p>Suppose the implicit policy gives us an energy landscape. Somewhere in that landscape is a good robot action. Maybe there are several. Now we need to find one.</p><p>The usual goal is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/226/1*tRz3MIstke3ocZJZfweZWQ.png" /></figure><p>In simple words:</p><p><strong>Find the action with the lowest energy.</strong></p><p>That sounds easy.</p><p>But in robotics, the action space can become very large.</p><p>Imagine a robot with a 7-dimensional action as mentioned before:</p><pre>[x, y, z, roll, pitch, yaw, gripper]</pre><p>If the model only needs to choose one action, the search space is already continuous. Now imagine predicting a sequence of 16 actions (predicted actions in chunk).</p><p>The model is no longer searching in a 7-dimensional space. <strong>It is searching in: 16 * 7 = 112 dimensions</strong>. And that is only one short trajectory!</p><p>This makes inference much harder. The model knows what good actions look like. But actually finding one may require exhaustive optimization, sampling, or repeated search.</p><p>It is a little like having a perfect terrain map but no direct route to the valley. You know where the low areas are. But you still need to figure out how to get there.</p><p>This is where the practical limitation of implicit policies appears. They are expressive. They can represent several valid solutions. But inference can be difficult, especially in high-dimensional action spaces.</p><p>So the next question becomes:</p><p><strong>Can we keep the ability to represent multiple possible actions without solving a difficult global optimization problem every time?</strong></p><p>Now, here we go. This is where Diffusion Policy comes in!</p><h3>Diffusion Policy: Start With Noise and Refine It</h3><p>Diffusion Policy takes a very different approach.</p><p>Instead of directly predicting the final action…</p><p>and instead of globally searching for the lowest-energy action…</p><p>it starts with noise.</p><p>Yes, noise.</p><p>At the beginning of inference, the candidate action trajectory can look almost completely random.</p><p>Then the model repeatedly improves it.</p><p>Step by step.</p><p>A noisy trajectory becomes slightly more structured.</p><p>Then more structured again.</p><p>Eventually, it becomes a plausible robot action sequence.</p><p>Conceptually:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*DUn699AP9SG8vRSWXdevJA.png" /></figure><p>This may sound strange at first. Why start from something random when we want a precise robot motion?</p><p>The key idea is that the model learns how to move noisy actions toward the distribution of expert actions.</p><p>You can also think of it like navigating through fog. Imagine that you cannot see the destination directly. But at every step, someone gives you an arrow:</p><p><strong>“Move slightly this way.”</strong></p><p>Then another arrow.</p><p>Then another.</p><p>Eventually, those small corrections guide you toward a valid destination.</p><p>Diffusion Policy works in a similar way.</p><p>It does not need to know the final action immediately.</p><p>It only needs to know:</p><p><strong>“How should this noisy action be changed so that it becomes more like an expert action?”</strong></p><p>This is especially useful when there are multiple valid solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XKRY6mk9hSfAYO6ZdB0v5g.png" /><figcaption>Diffusion Policy gradually transforms a noisy action sequence into a smooth, executable robot trajectory.</figcaption></figure><p>Suppose the robot can move around an obstacle either from the left or the right. A diffusion model does not need to average those two possibilities.</p><p>The action distribution can contain both.</p><p>Depending on the initial noise and the learned denoising process, the trajectory can evolve toward one valid mode or the other.</p><p>This is one of the main reasons diffusion models are attractive for robot control.</p><p>They can represent complex, multimodal action distributions without forcing everything into one direct prediction.</p><p>And unlike a conventional implicit policy, they do not need to solve a global optimization problem from scratch.</p><p>Instead, they iteratively refine a sample using a learned direction.</p><p>In simple terms:</p><p><strong>Explicit Policy </strong>predicts the answer directly.</p><p><strong>Implicit Policy </strong>scores possible answers.</p><p><strong>Diffusion Policy </strong>starts from noise and gradually move toward a good answer.</p><h3>What Does Diffusion Policy Actually Learn?</h3><p>Now we know what happens during inference.</p><p>Start from noise.</p><p>Then gradually turn that noise into a plausible robot action.</p><p>But how does the model learn to do that?</p><p>The basic idea is surprisingly simple.</p><p>During training, we begin with a real action trajectory from an expert demonstration.</p><p>For example, the robot might have executed a sequence like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/177/1*J4_3VWV-hzwUr0ydMCQGPw.png" /></figure><p>Then we deliberately add Gaussian noise to that trajectory.</p><p>The clean action becomes corrupted.</p><p>The model is then asked to predict the noise that was added.</p><p>Suppose the true noise is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/10/1*83WneDN-wPhJX-btZbJz5A.png" /></figure><p>and the model predicts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/20/1*sLQUX8rtoJF6Suaob13JqA.png" /></figure><p>A common training objective is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*-0cDgiUGKfjiOEsmscq2Mg.png" /></figure><p>In other words:</p><p><strong>How close is the predicted noise to the actual noise we added (artifically)?</strong></p><p>If the model predicts the noise accurately, we can subtract that noise from the noisy action.</p><p>The action becomes cleaner.</p><p>Then we repeat the process.</p><p>Over training, the model becomes better and better at answering:</p><p><strong>“Given this noisy action and the current observation, what part of this action looks like noise?”</strong></p><p>This is the key.</p><p>The model is not directly memorizing one final trajectory.</p><p>It is learning how noisy action samples should move toward the distribution of expert demonstrations.</p><p>You can think of training as repeatedly showing the model:</p><p><strong>Here is a good robot action.</strong></p><p><strong>Here is the same action after I disturbed it.</strong></p><p><strong>Can you tell me how to undo that disturbance?</strong></p><p>After seeing enough examples, the model learns a useful denoising direction. At inference time, that knowledge can be applied repeatedly to an action that starts as pure noise.</p><p>Noise becomes structure.</p><p>Structure becomes motion.</p><p>Motion becomes an executable robot trajectory.</p><p>This is also where the connection to the <strong>score function</strong> appears.</p><p>In diffusion models, the learned denoising direction is closely related to the gradient of the log-probability of the action distribution.</p><p>Intuitively, it tells us:</p><p><strong>“Which direction makes this action more likely to look like an expert action?”</strong></p><p>This is why Diffusion Policy can be connected conceptually to an energy landscape.</p><p>But instead of only knowing which areas have high or low energy, the model learns a direction that helps move the sample toward a good region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E1uHSHqJ5Xg_av--JhxTMA.png" /><figcaption>Diffusion Policy learns directions in the action landscape that guide a sample from high-energy regions toward valid low-energy actions.</figcaption></figure><h3>Why Diffusion Policy Fits Robot Manipulation</h3><p>Robot actions are not always simple.</p><p>A classification model, for example, might choose between:</p><p><strong>cat</strong></p><p>or</p><p><strong>dog</strong></p><p>Very simple right. But a robot policy often needs to generate continuous movement.</p><p>And it may need to generate many of these actions over time.</p><p>That means robot control naturally involves <strong>high-dimensional continuous trajectories</strong>. There is another complication.</p><p>The same task may have many correct solutions.</p><p>A robot can grasp a cup from slightly different angles.</p><p>It can move around an obstacle in different ways.</p><p>It can approach an object from different directions.</p><p>It can sometimes even use different strategies to complete the same task.</p><p>So a good robot policy needs to deal with two things at once:</p><p><strong>High-dimensional actions</strong></p><p>and</p><p><strong>multiple valid behaviors</strong></p><p>This is exactly where Diffusion Policy becomes attractive.It does not require all demonstrations to collapse into one average action.</p><p>Instead, it can model a richer distribution over possible trajectories.</p><p>There is also another important design choice.</p><p>Diffusion Policy predicts an <strong>action sequence</strong>, not just one isolated action.</p><p>Instead of predicting only:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/22/1*RCdjVBSJuq9OR6a6-C9XGg.png" /></figure><p>the model can predict something like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/238/1*RiVqqy4HEeQE1F8RjMiXcg.png" /></figure><p>This means the model considers a short horizon of future motion together. That is useful because robot movement is inherently temporal.</p><p>Reaching for a cup is not a collection of completely independent movements. The next motion depends on what the robot is trying to accomplish over the next few moments.</p><p>But the robot does not necessarily execute the entire predicted sequence.</p><p>Instead, it can use <strong>receding-horizon control</strong>.</p><p>The idea is:</p><p>Predict a short action sequence.</p><p>Execute the first few actions.</p><p>Observe the world again.</p><p>Predict a new sequence.</p><p>This gives the robot a balance between planning ahead and reacting to changes.</p><p>The result is a policy that can generate coherent short-term motion while still repeatedly updating its decision from new observations.</p><p>This helps explain why Diffusion Policy works well for manipulation tasks.</p><p>It combines several useful properties:</p><p><strong>It can represent multimodal behavior.</strong></p><p><strong>It can generate high-dimensional continuous actions.</strong></p><p><strong>It can predict temporally consistent action sequences.</strong></p><p><strong>And it can continuously replan as the robot interacts with the environment!</strong></p><p>The big picture is therefore not just that diffusion adds noise and removes it. The deeper idea is about how we represent robot decisions.</p><p>Instead of demanding the final motion in one shot, it gradually transforms uncertainty into a good possible action!</p><h3>References</h3><p>For more technical details and further exploration, please see:</p><ul><li><a href="https://arxiv.org/abs/2303.04137"><em>Diffusion Policy: Visuomotor Policy Learning via Action Diffusion</em> by Chi et al. (2023)</a></li><li><a href="https://arxiv.org/abs/2405.12213"><em>Octo: An Open-Source Generalist Robot Policy</em> by Octo Model Team et al. (2024)</a></li><li><a href="https://arxiv.org/abs/2410.24164"><em>π₀: A Vision-Language-Action Flow Model for General Robot Control</em> by Black et al. (2024)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad66fa714236" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>