<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Umair Amjid on Medium]]></title>
        <description><![CDATA[Stories by Umair Amjid on Medium]]></description>
        <link>https://medium.com/@umairamjid18?source=rss-20828675cebc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*-YZ-mKUOLAVmV160H1WhqQ.jpeg</url>
            <title>Stories by Umair Amjid on Medium</title>
            <link>https://medium.com/@umairamjid18?source=rss-20828675cebc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 03:27:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@umairamjid18/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Representation Collapse Over Self-Supervised Learning Architectures.]]></title>
            <link>https://medium.com/@umairamjid18/representation-collapse-over-self-supervised-learning-architectures-1edfc88e42f9?source=rss-20828675cebc------2</link>
            <guid isPermaLink="false">https://medium.com/p/1edfc88e42f9</guid>
            <dc:creator><![CDATA[Umair Amjid]]></dc:creator>
            <pubDate>Sun, 16 Aug 2026 09:36:01 GMT</pubDate>
            <atom:updated>2026-08-16T09:36:01.318Z</atom:updated>
            <content:encoded><![CDATA[<p>This problem occurs mostly in self-supervised learning architectures. Several architectures and methods have been discussed throughout this blog such as non contrastive, such as BYOL, SimSiam, VICREG and the contrastive methods such as SimCLR. The non contrastive methods were introduced to avoid representation collapse without relying on the negative pairs.</p><p>Multiple architecture before JEPA were introduced in self-supervised format.</p><h4>BYOL <strong>Architectures</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hw4J68XKWOt796nPrsPx7Q.png" /></figure><p>First x image is taken and different augmentations are implemented x~t and x~t’ in the view which outputs a representation which is then passed through some projection of q which results in qθ(zθ) from online network and in parallel the target network produces its own results by same process. Now the sg represents stop gradients, so the online network weights are updated by gradient descent while the target is updated by exponential moving average to avoid collapse. The equation for the loss function is written as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*r-vyVteINkPKzwov3Xhr0g.png" /></figure><p>Also from this equation can see if q(z) and ze map to same vector the result will be one and so 2– 2(1) = 0 representing representation collapse. Also note that we are normalizing by the norm of its own vector which will result in the second equation.</p><p>And the update will be written as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*moOvyulfuFfq7SGxJ7z8iA.png" /></figure><p>The online and target architectures are structurally identical but the update rule makes the architecture asymmetric. The online architecture is updated by gradient descent while the target architecture is updated by exponential moving average (EMA). This asymmetric update rule was carried into I-JEPA and V-JEPA to avoid representation collapse.”</p><h4>SimSiam architecture</h4><p>This architecture is fundamentally different from BYOL. In this we have a Resnet can be represented by f and an mlp head for prediction.</p><p>p1 = h(f(x1)), z2 = f(x2).</p><p>The idea is the same that we have two augmented view of same image. We minimize the cosine similarity</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/194/1*a7zu596M_v7jaTCvJiv5Yg.png" /></figure><p>The total loss will be written as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*Y1qSw5fiKTabmBmeWhoFrA.png" /></figure><p>Since stop gradients will be used to prevent anti collapse and no need for exponential moving average because there is no target architecture at all. So, when updating p1 stop gradients is applied on z2 and vice versa of z1. So the final equation becomes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*Li68qVku0T90kEfqniskxg.png" /></figure><p>We need to take into account that this is for same image view so we want the view to align as closely as possible — landing on the same vector is the goal not the collapse. Collapse would mean different image being mapped to the same vector.</p><h4>SimCLR</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2pqvKID7KsaC4xPm8_2LEw.png" /></figure><p>Since we discussed few non contrastive methods. Now we will discuss a contrastive method. The difference is that for contrastive method we also have to deal similarities and dissimilarities as well. The architecture is similar overall as shown in figure 1 consider an image x with different augmentation. Then both xi and xj are passed through f(.) encoder architecture resulting in hi and hj. Then (MLP) projection head g(.) is applied to produce zi and zj. Which is what the contrastive loss is computed on.</p><p>In the final step we want to maximize the agreement. So, the loss is represented as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/274/1*z9uBE8Aq_vHt4phminXD1w.png" /></figure><p>So, the inner terms compute the similarity between same augmented image which has to be maximum but division takes place by computing the similarity wrt to all the other inputs as well. Then the -log is taken to convert the loss into the positive value. if the similarity between zi, zj = 1 then the whole result will be 1 and log(1) = 0. So, the loss will be 0. And this is the goal maximize the similarity among the same pairs while minimize the similarity among negative pairs.</p><h4>VICREG:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LGUkWnzpXNYeOemBnpcUrQ.png" /></figure><p>Its a regularization technique which contains three components.</p><p>Invariance: The mean squared distance between the embedding vectors</p><p>Variance: a hinge loss to maintain the standard deviation of each variable, across the batch above a certain threshold. This term forces the embedding vectors of samples within a batch to be different.</p><p>Covariance: Normally covariance tell us how one variable changes by the change in another variable. But here we don’t want two different embeddings learning the same representation so the covariances over the batch of embedding variables has to be zero. The architecture used will be Siamese architecture for self supervised learning.</p><p>In the case of trying to maximize the variance The hinge loss will be written over the whole batch as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*szIxhie7jD3ex-m8MiccuQ.png" /></figure><p>Where the standard deviation can be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/173/1*jtapqk453hlJodW0Imy4fQ.png" /></figure><p>Here gamma is criterion for all the embeddings in particular dimension in the batch which is set equal to 1. So standard deviation is computed along that dimension and averaged over. Which is added in loss as regularization. Furthermore, a small noise is added to avoid numerical instabilities.</p><p>Consider the gradient of variance wrt to x: as xi​ moves closer to xˉ the gradient goes to zero which results in no way of avoiding the collapse. But in case of standard deviation, the gradients involves vi = (x — xˉ) divided by Std(x) — near the point of collapse, vi / Std(x) ≈ ±1,which means the gradient signal remains active and it will be hard to fall into collapse.</p><p>The covariance matrix along dimensions is represented by C(z) to be close to 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/415/1*uqaMbavl-XDDSSE95cNBtg.png" /></figure><p>Because we don’t want two different embedding dimensions carrying the same information. So, decorrelation effect at the embedding level will have decorrelation at the representation level. Finally the first component is represented as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*bRoushbVfolDuMkvKZyqAA.png" /></figure><p>The full loss will be written by combination of distances, variances and covariances</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*gOe6Le3nl0sQzb4TeD0ThQ.png" /></figure><p>where λ, µ and ν are hyper-parameters which control the importance of each term.</p><h4>JEPA (Le World models):</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*rnw5XmYGWlvBEM2m6rNL-g.png" /></figure><p>Now from the figure suppose x is encoded to some vector (zx) and the target y is encoded to zy. And we compute MSE loss in the encoded space. Now what can happen is in order to minimize the loss the optimizer will change the weights of both the inputs and outputs in such a way such that they are mapped to same vector. Now this will definitely decrease the loss but ultimately the encoders will not learn the correct representation of both input and output. So, this is called representation collapse.</p><p>Since I_JEPA and V_JEPA use non contrastive method specifically using gradient descent for input encoder and Exponential Moving Average for target encoder. But a new technique was introduced which is called</p><h4>SIGREG:</h4><p>This method is more effective than I_JEPA or V_JEPA which just uses different gradient update techniques to update the weights. While this method explicitly introduces a regularization terms along with loss function that the optimizer has to minimize.</p><p>SIGREG’s approach is very simple. Consider that we have N embeddings in our space. If we can prove all embeddings are Gaussian we will have proof that there is no collapse because the embeddings will be far from each other. <br>So for that first we have to project the embedding Z into unit vector u(m). Which is in same dimension as in Z embedding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/102/1*ujbssBfx_AftYHvdb9xzGw.png" /></figure><p>Then we project all embeddings into this u(m) and result is a scalar.</p><p>So if we do that for N embedding projected to single u(m). Then we get a vector where each value is scalar for a particular embedding as the inner dimensions will be same.</p><p>After this projection we have a vector. Then for each value we want to compute Empirical characteristic distribution for different values of t.</p><p>The empirical characteristic function for normal gaussian can be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/101/1*HTcGpqZOezhdjPiWAeU0dQ.png" /></figure><p>For each projection, we compute the empirical characteristic function, defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*2XJlwMR0ITHhTmjjZCOQrQ.png" /></figure><p>We will use Epps–Pulley test which to check the Gaussianity of the embeddings across multiple values of t for certain projection of u(m). The formal equation is represented as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/1*hewJUv-l5qP-dCNUNLdTAQ.png" /></figure><p>But we cannot evaluate for infinite value of t. But we will evaluate over certain values of t. Compute φ_N(t) for full embedding sequence at certain t than subtract with original Gaussian ECF. And we combine all the results. W(t) is introduced which is equal to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/98/1*dNwbnnuQJyqSrAGaKicyqw.png" /></figure><p>Which is evaluated at different values of t. and dt is the difference moving from t1 to t2 and so on for further values of t. Also sigma σ controls the bandwidth of the kernel. Smaller sigma makes it sensitive to the overall spread of the embeddings while higher values of sigma makes it sensitive to finer structure in the distribution.</p><p>Afterwards the results for all t are<strong> </strong>combined which will be result along single direction u(m1). After summing all contributions of t. And the final result is multiplied by no. of embeddings N. We repeat this process for m projections and average over all the results of m. if the total result is zero, then the embedding is perfect Gaussian. The formulation of average is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/184/1*lvAwOR0xx3Tm-B7XBlRK-Q.png" /></figure><p>This SIGREG is added to the loss function as regularizer that the optimizer tries to minimize. if the values of SIGREG is very large then it means the embedding dimensions are not spread out which will increase the loss, so the optimizer will try to update the weights so the embeddings are scattered enough. But if SIGREG is small then the embeddings are well scattered close to a Gaussian. The loss can be written as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/249/1*Y536VT3cdSE5T9dzeGQUDw.png" /></figure><p>Also, the bigger values of lambda means that optimizer would force the embeddings more to be scattered and vice versa.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1edfc88e42f9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the JEPA Family: From Images to Robots]]></title>
            <link>https://medium.com/@umairamjid18/inside-the-jepa-family-from-images-to-robots-9bc406594d6b?source=rss-20828675cebc------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bc406594d6b</guid>
            <dc:creator><![CDATA[Umair Amjid]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 12:16:01 GMT</pubDate>
            <atom:updated>2026-08-14T12:16:01.424Z</atom:updated>
            <content:encoded><![CDATA[<p>In this post we will discuss about the difference architectures that have been proposed we will not discuss all of them. We will discuss just some recent ones. Which are namely I_JEPA, V_JEPA, JEPA_WM and Le World models.</p><p>The general idea of JEPA is that we have input x and we want to produce output y. We encode both inputs and outputs mapping x_t → z_x_t and</p><p>y_t → z_y_t. In the encoded space or latent space. In the latent space we have predictor which predicts the encoded target z_y_t.</p><h4><a href="https://arxiv.org/pdf/2301.08243">I_JEPA</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YSTKoZkINEnp3_r7t2zaNQ.png" /><figcaption>Image taken from paper <a href="https://arxiv.org/pdf/2301.08243">I-JEPA</a></figcaption></figure><p>In I_JEPA consider an image. Some parts of the image are hidden and some are kept as context and model has to predict the rest of the image in encoded image. The context encoder is a vision transformer ViT which only processes the visible context patches. The predictor is also a narrow ViT which conditions the prediction on positional tokens that needs to be predicted.</p><p>For context encoder we also use a sampling method to sample a context x and masking corresponding to the remaining representation of encoder is B_x. So the goal is to predict M patch level representation.</p><p>In order to produce the encoded targets we consider N non overlapping sequences and pass them through target\_encoder and convert them into patches. s_y1, ..., s_yN</p><p>where s_yk is considered as kth patch of the image. To obtain the targets for the predictor we sample M overlapping sequence after encoding targets.</p><p>The predictor takes in the context s_x corresponding to a target mask B_i we want to predict. The masked tokens are parameterized by shared learnable vector with an added positional embedding. Since we wish to make predictions M for M blocks so we apply predictor M times. And the loss is computed over the masked tokens. Since sampling for context block is independent from the target blocks generated. So, there could be overlapping samples which are removed. The equation over the loss:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/179/1*_8d4dqCzqrbaf8ykyjI9NA.png" /></figure><p>The encoder and predictor are updated by gradient descent while the target encoder is updated by exponential moving average. Both of them use different methods because of the representation collapse. Which will be discussed later.</p><h4><a href="https://arxiv.org/pdf/2404.08471">V_JEPA</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*29Oo_m50P2G9-3V5_To0bg.png" /><figcaption>Image taken from paper <a href="https://arxiv.org/pdf/2404.08471">V_JEPA</a></figcaption></figure><p>While the I_JEPA was restricted to images. V_JEPA was introduced to learn from videos. The goal was to learn the feature frames in an unsupervised learning format with no labeling involved about any particular task.</p><p>V_JEPA works by taking a sequence of T frames and combines all of them in such a way that we have N patches for T frames. Each patch can thought of as token.</p><p>Before encoding the frames masking is applied and the unmasked frames are given to encoder. There are two types of masking. Short range masking which will sample 8 blocks from a single temporal block and mask them with some aspect ration masking about 15% of the area. After masked patches are selected. Then the union of all blocks are taken. While long range masking sample 2 blocks only covering 70% for each block. Then union of all blocks are taken. The aspect ratio is chosen between the range of (0.75,1.5).</p><p>The whole observation sequences is taken by target encoder which converts into patches and the loss computed only on the patches which are masked.</p><p>The masked sequence is combined later with encoder outputs. The masked sequence is replaced by learnable vector at those particular positions. The predictors job is predict the masked positions. The predictor is narrow transformer implemented with 12 blocks with embedding dimensions of 384.</p><p>This is the process of pre-training. In the evaluation process. The predictor is not involved. The weights of encoder are fixed. And separate learnable parameters are involved which will conditioned according to the task at hand. The encoder and predictor are updated by gradient descent while the target encoder is updated by exponential moving average to avoid representation collapse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*Gg4hXJ5us_ueY4zsdAe2WA.png" /></figure><h4><a href="https://arxiv.org/pdf/2512.24497">Jepa_wm in robotics</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1BFIXL5zcZxxtXoLg3lVeQ.png" /><figcaption>Image taken from <a href="https://arxiv.org/pdf/2512.24497">JEPA_wm</a></figcaption></figure><p>The model specifically designed to handle tasks for robotics. It is split into three sections as discussed in the blog # 1 Encoder, Predictor, Planner.</p><p>The frozen pretrained encoder one of choices being Dino_WM. Because at the backend it use Dinov2 as pretrained encoder which is pretrained and its able to capture both spatial and object centric representation.</p><p>The general idea is we have a video of a robot doing some task. The video frame at x_t is taken and encoded through to get z_t. Also the states are also passed through an encoder as well which can be position across x, y, and z axis along with velocity etc. The state is explicitly required because the predictor needs to know explicitly the states as it might not be able to know it from the images explicitly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/133/1*TXtwf7z0f04qX0vRGQjv_w.png" /></figure><p>The predictor takes in as encoded visual inputs, actions and encoded states and predicts next frame in the sequence along with the states. Also, we introduce a frame skip of n. Which is how many intermediate frames we want to skip. Because predicting every next frame will be redundant and it will only increase the computational cost.</p><p>Furthermore the predictor can be conditioned in two ways namely feature conditioning and token conditioning. Feature conditioning embeds the actions and states along the embedding dimensions. While token conditioning embeds states and actions along token dimension of transformer so attention mechanism is applied for each encoded state with encoded actions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*pH6HzNsqaVi4vmey1KUqCA.png" /></figure><p>Once the predictor is trained. Then the goal is to get the best action sequence for particular task at hand. The Encoder and predictor weights are fixed. The strategies to train the planner have been discussed in the blog #2 in details. But CEM(cross entropy method) was used for planner. Which samples random action sequences and try each action sequence by passing it through the predictor. Since planning is happening in the latent space the start position of task and goal position have to be encoded as well. So, using the sampled action sequences a rollout is performed for each one of them. And the loss is calculated. <br>Then the top k sequence which gives the lowest loss are used to update the distribution and whole processed is repeated again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*nxl5HGWNdAKat83wym_vDg.png" /></figure><p>This process continues until the horizon H is reached.<br>The loss is then computed at the final horizon step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*CO-CPf9QWedk9YzTFDX0Uw.png" /></figure><h4><a href="https://arxiv.org/pdf/2603.19312">Le World Models</a></h4><p>This model is a further extension of JEPA. In terms of architecture, the encoder and predictor are both Vision Transformers. The goal is to predict the next s_t+1.</p><p>Also, the encoder is not pretrained, which means it isn’t restricted to some fixed representation and doesn’t require pretraining on large datasets — a process that would otherwise require a lot of computation time. Furthermore, this is the smallest model related to JEPA in terms of parameters and can be trained on a single GPU.</p><p>Representation collapse is explicitly handled using a technique called SIGREG — a regularization technique used to promote feature diversity in the embedding space.</p><p>The whole point of this technique is to match the embeddings to a Gaussian distribution, which isn’t naturally possible in high-dimensional space. So, two methods are combined: the Cramér method, which projects the embedding into a lower-dimensional space, and the Pulley test, which checks for Gaussianity using an empirical characteristic function.</p><p>This method works by adding a loss function as a regularizer, which forces the encoder to project the embeddings far enough that they are close to Gaussian — which also suggests that the encoder and predictor are learning useful representations, avoiding representation collapse. The final equation can be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/253/1*qkhDNHejlhUn9oGQsgYVvQ.png" /></figure><p>The planner also works in the same way as discussed in JEPA_WM, using the Cross-Entropy Method combined with MPC. This concept is called receding MPC. The idea is the same: we sample an action sequence, compute the rollout, and at the encoded horizon compute the loss. We select the top-k sequences with minimum cost and update the sampling distribution. Once we’ve selected the correct sequence of actions, we execute just the action a₁ in the real-world environment to see how far we are from the goal, and repeat the process.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bc406594d6b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Planning in Latent Space: CEM, Gradients, and MPC]]></title>
            <link>https://medium.com/@umairamjid18/planning-in-latent-space-cem-gradients-and-mpc-ca1ecf4b3639?source=rss-20828675cebc------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca1ecf4b3639</guid>
            <dc:creator><![CDATA[Umair Amjid]]></dc:creator>
            <pubDate>Tue, 11 Aug 2026 11:21:01 GMT</pubDate>
            <atom:updated>2026-08-11T11:21:01.299Z</atom:updated>
            <content:encoded><![CDATA[<p>In the case of world models different architectures have been designed for the encoder and predictor which will be detailed in another blog.<br>For this post I will be detailing the different design choices proposed for the planner. The goal is still the same: find the best action sequence that can get us to the encoded goal state.<br>There are mainly three different methods that have been proposed for the planner<br> 1- CEM (Sampling based methods)<br> 2- Gradient Based methods<br> 3- MPC with CEM (model predictive control with CEM)</p><h4>Cross Entropy Method</h4><p>This method is a sampling based method in which we sample n action sequences of a fixed Horizon length from a random distribution, aiming to reach a goal state. For each sample we pass it to the predictor to perform a rollout. For each rollout we compute the loss for the predicted goal state <strong>z_hat_g</strong> and original encoded goal state z_g.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/65/1*hw5rEwFH_rJER-YbTIc1uQ.png" /></figure><p>Afterwards we take the top k action sequences which gave us minimum loss and use the mean and standard deviation of those k action sequences to update our original random distribution. And this process is repeated upto J iterations. The process is slow as it requires more iterations and larger action sequences in order to reach the goal state.</p><h4>Gradient Descent</h4><p>In this method we sample a random action sequence. We compute the rollout and calculate the loss from the goal state. We update the gradients of the action sequences, then we repeat the same process again up to a certain iteration.</p><p>But there is a problem in this method. During inference time the gradients of action sequence are updated to minimize the cost, which pushes the rollout in the regions of state space the predictor was never trained on. Because this is a distribution shift, the predictor is not reliable in these unseen regions. It might report a lower loss in encoded state but the action sequence will not be able to reach the goal state.</p><p>To handle this problem <a href="https://arxiv.org/pdf/2512.09929">Adversarial WM</a> method was introduced which trained the predictor during training time by adding noise which would increase the loss function and predictor would minimize it. As a result it would learn the local representation around that encoded state. So, when ever there is a shift in distribution the predictor has already seen nearby regions at the inference time so the predictor will output accurate and reliable predictions.</p><h4>Model Predictive Control</h4><p>For this section consider two components: 1) the outer wrapper being the Model Predictive Control, 2) the CEM method or Gradient based method.<br>The gradient based method or CEM is considered as the inner section of MPC. So, the inner section works the same way as described above. The goal is to get the best sequence of actions. Once it has selected the best sequence of actions that has the minimum loss, the outer section MPC executes from action sequence a₁, …, a_h, picks a₁, and executes it in the ground-truth environment to see where the object ended up. Afterwards the inner section (CEM) is reinitialized and the same process is executed again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mqiPv7mj0Bzxj3uFHPEEng.png" /><figcaption>Code taken the paper <a href="https://arxiv.org/pdf/2512.24497">WHAT DRIVES SUCCESS IN PHYSICAL PLANNING WITH JOINT-EMBEDDING PREDICTIVE WORLD MOD ELS?</a></figcaption></figure><p>The pseudocode shows the CEM-within-MPC loop, including the replanning step described above. N candidate trajectories are sampled per round so each can be evaluated individually, and the resulting costs are then used together to update the search for the next round.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca1ecf4b3639" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Trial-and-Error to Imagination: An Introduction to World Models]]></title>
            <link>https://medium.com/@umairamjid18/from-trial-and-error-to-imagination-an-introduction-to-world-models-e12320f94597?source=rss-20828675cebc------2</link>
            <guid isPermaLink="false">https://medium.com/p/e12320f94597</guid>
            <dc:creator><![CDATA[Umair Amjid]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 07:57:53 GMT</pubDate>
            <atom:updated>2026-08-10T07:57:53.771Z</atom:updated>
            <content:encoded><![CDATA[<p>Consider a robot arm which wants to move an object from start to goal state. In the case of Reinforcement Learning, it will try to move the object to goal state iteratively by trying to update action policy which will require thousands of iterations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/107/1*KnPwvxOJnYXA_28JbkE8NA.png" /></figure><p>But with world model instead of doing the task in the real environment which can be expensive the model has the ability to imagine possible futures and choose the best action in that scenario. So, it builds an internal simulation first which allows it to search over the best set of actions to reach the goal state.</p><p>In Reinforcement learning considering that we are in state sₜ we find the possible best action aₜ while being at sₜ current state. The goal is to learn a policy π for which at each state we take the correct set of actions to reach the goal state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/106/1*C9uD1JA-fE1VTOUuiiAD2Q.png" /></figure><p>In order to predict the imagined trajectory in real space requires predicting millions of pixels for an image at every step every time step. Which is very expensive in terms of time and computations. In order to solve this problem, we want to learn a more compact representation. The model predicts the compact latent representation that preserves the information needed for planning. So, first, an encoder maps the observation sₜ into a latent representation zₜ, and a predictor then uses zₜ and an action aₜ to predict the next latent state zₜ₊₁. This makes the long horizon prediction more efficient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/106/1*sp21pKmvzOiflnJH0VTOug.png" /></figure><p>The action sequence is not learned inside the predictor. At planning the encoder and predictor parameters are frozen and planner searches over action sequences, rolling out each frozen encoder and predictor and picks the one that reaches the goal state. So, instead of running the robot through thousands of trial and error iterations, the rollouts are computed in imagined low latent space, which makes the rollout cheap to compute. The model is also frozen so weights don’t update. Additionally, real planners reach a good solution faster than pure random search — e.g. CEM refines the distribution for guessing over the few rounds instead of guessing blindly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e12320f94597" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>