<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adwita on Medium]]></title>
        <description><![CDATA[Stories by Adwita on Medium]]></description>
        <link>https://medium.com/@Nayonae?source=rss-f6c5c8196601------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*BGTP1jDRbNNZSsPNlPZ-3w.jpeg</url>
            <title>Stories by Adwita on Medium</title>
            <link>https://medium.com/@Nayonae?source=rss-f6c5c8196601------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:29:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@Nayonae/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Azure App Service vs Container Apps: Which one does your service need and Why?]]></title>
            <link>https://medium.com/@Nayonae/azure-app-service-vs-container-apps-which-one-does-your-service-need-and-why-c46eadab79fa?source=rss-f6c5c8196601------2</link>
            <guid isPermaLink="false">https://medium.com/p/c46eadab79fa</guid>
            <category><![CDATA[azure-app-service]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[microsoft-azure]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[azure-container-apps]]></category>
            <dc:creator><![CDATA[Adwita]]></dc:creator>
            <pubDate>Wed, 01 Oct 2025 21:15:15 GMT</pubDate>
            <atom:updated>2025-10-01T21:15:15.895Z</atom:updated>
            <content:encoded><![CDATA[<p>I was working on deploying my backend application on Azure when I came across the two. While Azure provides multiple options for deploying your web apps, we’ll be discussing the details of the above two today, particularly, how they differ and which one you should choose for your use-case.</p><h3>Azure Container App (ACA)</h3><p>As per the documentation:</p><blockquote>Azure Container Apps is a serverless platform that allows you to maintain less infrastructure and save costs while running containerized applications.</blockquote><p>Before delving into the specifics, we need to understand what exactly is a Container.</p><p><strong>WHAT IS A CONTAINER?</strong></p><p>A container is essentially an isolated package of your application code and its dependencies. Unlike a Virtual Machine which emulates a whole operating system, containers are hosted above a layer on the hosted VM and share resources while remaining isolated in terms of processes, filesystem and network. This makes them <strong>fast, efficient, and portable</strong> across environments as well as ideal for faster implementation loops.</p><p>A good way to represent the layers is shown below. As we can see, the Docker engine [it is what runs your built container] is hosted upon a shared host kernel</p><pre>                [ Container 1 ] ───&gt; [ Application A ] ← Isolated Process IDs<br>                [ Container 2 ] ───&gt; [ Application B ]   IP addresses, ports,<br>                [ Container 3 ] ───&gt; [ Application C ]   etc<br>                                 |<br>                                 ↓<br>                         [ Docker Engine ]<br>                                 |<br>                                 ↓<br>                          [ Host OS Kernel ] ← Single shared kernel for all <br>                                 |             containers<br>                                 |             Shared CPU cycles, network <br>                                 ↓             stack<br>                            [ Hardware ]<br><br>Fig 1: Diagramatic representation of How and Where Containers Share Resources</pre><p>A point of confusion might be how network elements like IP addresses are said to be isolated for containers when the network stack is shared at the kernel level. To understand this, we must know the difference between a network namespace and a network stack:</p><ol><li>Network Stack (shared): it refers to the actual kernel code and hardware that handles networking. There is only one kernel, and therefore this information needs to be shared among all containers. This includes:<br>q. The TCP/IP protocol implementation in the kernel<br>w. Physical Network Interface Cards, etc.</li><li>Network Namespace (isolated): it is a virtualised view of the network stack. Each container thinks it’s the only thing on the network with its own IP, ports, etc. This implies that each container gets its own:<br>q. Virtual network interface<br>w. IP address<br>e. Routing table<br>r. Port space (important because this means that Container 1 and 2 can both listen to port 8080 at the same time).</li></ol><p>So while networking appears completely isolated to each container, but under the hood they’re all using the same kernel code to actually send/receive packets.</p><p>Containers can again be of different types. These can be based on runtimes, use cases, etc. A standard example is a Docker (platform/tool that helps you <strong>create, package, and run containers</strong>) container, which is a running instance of a docker image.</p><p>If that sounds like too many terms, think of the docker image as a snapshot of your application with its dependencies. It is the container the runs this image (think of it as a projector? imagine capturing a snapshot but now you need something to run it for it to be usable). An important differentiation is that a Docker container is not an OS or a VM; it is just a process on the host system.</p><p>And while this is not a Docker tutorial, another advantage worth mentioning is that a docker image is portable (same image runs on all OS) and immutable (any changes made to the application would require a new docker image to be built).</p><p><strong>BACK TO ACA</strong></p><p>ACA is built on Kubernetes which provides container orchestration services. If Docker was built to create and manage a single container, Kubernetes essentially allows you to manage multiple containers. This is exactly what ACA aims to achieve, while abstracting away the complexities of this management.</p><p>ACA can be used to spin up multiple micro-services for your application by running multiple container images. An example of this can be deploying separate images for different API endpoints that serve your frontend application.</p><p>Every container app is tied to a container environment, which allows multiple apps within the same environment to communicate with each other (they scale independently though).</p><p>Now, replicas are multiple instances of same version (horizontal scaling) and revisions are multiple versions running simultaneously (deployment strategy)</p><p>ACA also allows to version these container apps with the help of revisions, which are a snapshot of each version of your app. Revisions are immutable (of course, you cannot make changes to a container’s image once built) act as a record of the container app’s versions, capturing its state at various stages.</p><p>ACA supports revision management, allowing you to run multiple versions of your app concurrently for scenarios like traffic splitting between old and new versions. For fault tolerance and traffic management, ACA can spin up multiple replicas of the same container image with automatic load balancing and auto-scaling capabilities.</p><pre>Container Environment<br>│<br>├── Container App 1 (e.g., API service)<br>│   ├── Revision v1 (10% traffic)<br>│   │   ├── Replica 1<br>│   │   └── Replica 2<br>│   │<br>│   └── Revision v2 (90% traffic)<br>│       ├── Replica 1<br>│       ├── Replica 2<br>│       └── Replica 3<br>│<br>└── Container App 2 (e.g., Background worker)<br>    └── Revision v1 (100% traffic)<br>        ├── Replica 1<br>        └── Replica 2<br><br>Fig 2: Hierarchy of Replicas and Revisions for your ACA</pre><p>ACA also provides stack specific features for Java and .NET as can be seen below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhLLemCa_gHhEYuryijf0Q.png" /><figcaption>Fig 3: Container App creation on Azure</figcaption></figure><p>Being serverless means that management of servers is relieved off you. However, ACA does not support Windows containers as of now and it does not provide management access to the underlying kubernetes cluster. It mainly abstracts the complexities of Kubernetes and container orchestration, allowing developers to deploy containers using their preferred runtimes, programming languages and development stacks.</p><h3>Azure App Service</h3><p>According to the Azure documentation:</p><blockquote>Azure App Service is a platform that lets you run web applications, mobile back ends, and RESTful APIs without worrying about managing the underlying infrastructure.</blockquote><p>So it is literally a fully-managed Platform-as-a-Service (PaaS) for hosting your web apps. This automatically means that Azure is providing you with platform that includes hardware, software, infrastructure, operating systems, databases, and tools, all encapsulated and abstracted from your end.</p><p>While the underlying details are hidden from the user (the developer in this case), the platform lets you deploy and manage your web app while it runs. It lets you:</p><ol><li>Configure and setup a logging stream for debugging</li><li>Securely connect to databases (either creates one during the creation of the app service or lets you continue with pre-existing databases)</li><li>Provide program files either by direct upload or uses GitHub Actions for deployment via CICD for GitHub repositories</li><li>Configure auto-scaling in case the scope of your application grows large</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pgaJwpSVvyaGsH5JUl1tog.png" /><figcaption>Fig 4.1: Creation of an Azure App Service</figcaption></figure><p>It also provides a lot of other features such as sidecars with Linux Apps (you can check out the details in the documentation linked below). A managed identity associated with the App Service is created every time you deploy an application when choosing Deploy with GitHub on the portal, that lets you connect to the application from external sources, for eg, GitHub Actions.</p><p>But how does App service accomplish all this? The app service plan provisions Virtual Machines or VMs running Linux/Windows environments (depending on whichever you select), which could be isolated or shared depending on your plan. When creating a web app with app service, it gives you the choice between “Code” or “Container” and two operating systems are provided, namely Windows and Linux</p><p>(We will see this later but Linux uses Containers for both Code and Container cases under the hood)</p><p>The uploaded code firstly gets loaded onto Azure Storage, and the App service runtime then pulls it, i.e, App Service control plane detects the deployment and orchestrates getting your code onto the VM.</p><p>For Linux app Service architecture, every time you select a runtime stack (such as Node, .NET, Java, etc), a pre-built Docker container is selected for you. Azure uses Blessed images for this, and your code is injected into this running container. Blessed images are pre-configured, Microsoft-maintained container images optimized for App Service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xYdDkaotlEX2LPGctOzz_A.png" /><figcaption>Fig 4.2: Creation of an Azure App Service</figcaption></figure><p>However, this container is abstracted away, and you can neither configure nor manage it. You have root access to the container but no access to the host operating system. The file system is rooted at /home. These files are persistent, meaning that you can rely on them staying there until you do something to change them. Also, they are shared between all instances of your site (when you scale it up to multiple instances). Internally, the way this works is that they are stored in Azure Storage instead of living on the local file system.</p><p>So essentially, your app service runs on a VM, and your code is deployed with built in blessed images of docker which again reside and run on the VM in the case of Linux based environment. If we consider a Node js application (or any other stack for that matter, but let us consider a backend application with multiple API endpoints running through server.js or a similar file), the the following flow diagram demonstrates the deployment process for it:</p><pre>                          Azure DevOps Pipeline<br>                                    ↓<br>                         Builds your Node.js app<br>                                    ↓<br>            Azure App Service Deploy task uploads to Azure Storage<br>                                    ↓<br>            API Controller (in scale unit) orchestrates deployment<br>                                    ↓<br>            File Server mounts Azure Storage blob as network drive<br>                                    ↓<br>                Worker maps network drive to /home/site/wwwroot<br>                                    ↓<br>                      Oryx generates startup script<br>                                    ↓<br>                     Container starts with your code<br><br>      <br>      Fig 5: Azure App Service Deployment via an Azure DevOps Pipeline</pre><p>The Azure DevOps Pipeline simply automates the build and deployment process provides a workload and is use-case dependent here. Your App Service deployment can work without it.</p><p>On the other hand, for a Windows App service architecture, Internet Information Services (IIS) is used as the web server and no containers are used for code deployment. File system changes to D:\ drive.</p><p>There is also a frontend layer involved before the end-user request hits our worker VM. This Frontend layer acts as a Load Balancer that terminates SSL (this means that TLS/SSL requests never get to your app and therefore the app need not provide any support for these), routes to correct application, and forwards the end-user request to the Worker VM which is hosting our application. This implies that it is always going to be an HTTP request that hits your app while it is on.</p><pre>Internet User Request (HTTPS)<br>↓<br>Azure DNS ([yourapp.azurewebsites.net](http://yourapp.azurewebsites.net/) → VIP address)<br>↓<br>Scale Unit Public VIP (e.g., 168.62.20.37)<br>↓<br>Front End (Layer 7 Load Balancer)<br>├─ Terminates SSL<br>├─ Routes to correct application (round robin load balancing)<br>└─ Forwards HTTP request to Worker<br>↓<br>Worker VM (Linux)<br><br>├─ Docker Container (blessed image)<br>│   ├─ Oryx-generated startup script [runs npm install and npm server.js or your startup cmd]<br>│   ├─ Node.js runtime<br>│   ├─ Your application code (/home/site/wwwroot)<br>│   │   ├─ server.js (or app.js)<br>│   │   ├─ node_modules/<br>│   │   └─ package.json<br>│   └─ Process: node server.js (listening on PORT env variable; default 8080 for Node)<br><br>└─ Mounted network drive from File Server<br>↓<br>File Server<br>↓<br>Azure Storage Blob (your actual files stored here)<br>↓<br>SQL Azure (metadata about your app)</pre><h3>Conclusion</h3><p>It goes without saying that whichever you choose depends on your specific environment and requirements. While ACA is definitely suitable for a microservice architecture and for quicker implementation and testing cycles, App Service provides you with the flexibility to select between Windows and Linux environments and for monolithic architectures.</p><h3>Resources</h3><ul><li><a href="https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans">Azure App Service Plans - Azure App Service</a></li><li><a href="https://learn.microsoft.com/en-us/azure/app-service/overview-sidecar">Sidecars overview - Azure App Service</a></li><li><a href="https://edi.wang/post/2023/9/1/how-azure-app-service-for-linux-runs-your-net-application">How Azure App Service for Linux Runs Your .NET Application</a></li><li><a href="https://github.com/projectkudu/kudu/wiki/understanding-the-azure-app-service-file-system">Understanding the Azure App Service file system</a></li><li><a href="https://learn.microsoft.com/en-us/answers/questions/758546/azure-app-serivce-load-balancer">Azure App Serivce load balancer - Microsoft Q&amp;A</a></li><li><a href="https://learn.microsoft.com/en-us/azure/container-apps/overview">Azure Container Apps overview</a></li><li><a href="https://azureossd.github.io/2023/03/24/Default-exposed-ports-of-Azure-App-Service-Linux-Blessed-Images/">Default exposed ports of Azure App Service Linux Blessed Images</a></li><li><a href="https://learn.microsoft.com/en-us/azure/container-apps/start-serverless-containers">Using serverless containers on Azure</a></li><li><a href="https://learn.microsoft.com/en-us/azure/container-apps/revisions">Update and deploy changes in Azure Container Apps</a></li><li><a href="https://stackoverflow.com/questions/54332024/difference-between-azure-app-services-and-web-app-for-containers">Difference between Azure App Services and Web App for Containers</a></li><li><a href="https://stackoverflow.com/questions/54551192/what-is-the-difference-between-azure-container-instances-and-web-app-for-contain">What is the difference between Azure Container Instances and Web App for Containers?</a></li><li><a href="https://learn.microsoft.com/en-us/answers/questions/33865/difference-between-container-instances-and-contain">Difference between Container Instances and Container for Webapps - Microsoft Q&amp;A</a></li><li><a href="https://developersvoice.com/blog/cloud/azure-app-service-aks-container-apps-benchmark/">Cost vs. Performance on Azure: App Service, AKS, and Container Apps Benchmark</a></li><li><a href="https://learn.microsoft.com/en-us/answers/questions/1339752/help-me-understand-azure-container-apps-scaling-an">Help me understand Azure Container Apps scaling and replicas - Microsoft Q&amp;A</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c46eadab79fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding ADAM Optimizer]]></title>
            <link>https://medium.com/@Nayonae/understanding-adam-optimizer-f525dfdd63e0?source=rss-f6c5c8196601------2</link>
            <guid isPermaLink="false">https://medium.com/p/f525dfdd63e0</guid>
            <category><![CDATA[optimizer]]></category>
            <category><![CDATA[adam]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Adwita]]></dc:creator>
            <pubDate>Wed, 02 Jul 2025 19:03:36 GMT</pubDate>
            <atom:updated>2025-07-02T19:03:36.713Z</atom:updated>
            <content:encoded><![CDATA[<p>The article builds on the <a href="https://arxiv.org/abs/1412.6980">original paper</a> from 2014, based on convex functions theoretically. I would recommend reading <a href="https://medium.com/@Nayonae/how-to-write-the-backward-pass-of-any-neural-network-7bf04e0b4ee2">How To Write The Backward Pass of Any Neural Network</a> before starting this article, since they use the same Network template and well, you must know how to write the backward pass. If you’re already familiar with the process of backpropagation, feel free to continue.</p><p>Okay, so we all know of the <a href="https://www.youtube.com/watch?v=Nutpusq_AFw">Unofficial PyTorch Optimization Loop Song</a> by Daniel Burke, don’t we?</p><p>And I’m pretty sure we all have some basic idea about how loss is calculated in a standard neural network (and why it is needed) (spoiler: to check how far our predictions are from true values).</p><p>But once the loss has been computed, and gradients have been calculated via the backward pass, how do we actually adjusts our parameters (or in our case, weights and biases) so as to make sure that the next round of forward pass in our model would produce a lower loss value, and therefore predictions that would be closer to the true values in our data?</p><p>Too much to unpack maybe? Let’s understand with an example</p><h3>Why Do We Need An Optimizer?</h3><p>The thing with a loss function is that it only tells us how bad our model is, and not exactly how to improve it (kinda like that one Math teacher we all had).</p><p>We begin with random weights when starting with the training of our model. These become the parameters for our model, and are represented by θ.</p><p>During the backward pass, we compute the gradient so we can adjust the values of these parameters for our model. These gradients are <strong>stored</strong> in each parameter’s .grad attribute.</p><p>Now, the <strong>optimizer’s job</strong> is to then <strong>use those gradients to update the parameters</strong>, according to its specific algorithm.</p><p>We have Optimizers like Stochastic Gradient Descent (SGD), which use fixed updates for updating θ. Equation for SGD updates can be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*BkA_hDdf3-ikuCIQWn77aA.png" /></figure><p>where:</p><ul><li>η: learning rate</li><li>∇θL(θ): gradient of the loss function L with respect to θ</li></ul><p>Pretty standard? It is almost like the equation of a straight line if you think about it. The problem here, is that besides having these fixed updates, SGD is also sensitive to noisy gradients, and is poor at navigating local minimas.</p><p>This is exactly where the Adam Optimizer comes in. How?</p><h3>Algorithm</h3><p>Let us use the same RandomNet we used for <a href="https://medium.com/@Nayonae/how-to-write-the-backward-pass-of-any-neural-network-7bf04e0b4ee2">‘How To Write The Backward Pass of Any Neural Network’</a></p><pre>class RandomeNet:<br>    def __init__(self):<br>        self.w1 = torch.randn(4, 6)   # input, hidden<br>        self.w2 = torch.randn(6, 5)   # hidden, hidden<br>        self.w3 = torch.randn(5, 2)   # hidden → output<br>        self.params = [self.w1, self.w2, self.w3]<br><br>    def forward(self, x):<br>        z1 = x @ self.w1             <br>        a1 = torch.relu(z1)          <br><br>        z2 = a1 @ self.w2             <br>        a2 = torch.tanh(z2)          <br><br>        z3 = a2 @ self.w3            <br>        return z1, a1, z2, a2, z3    <br><br>    def loss(z3, target):<br>        return (z3-target)**2 <br>    <br>    def backward2(self, x, z1, a1, z2, a2, z3, target):<br>        dz3 = 2 * (z3 - target)           # (batch_size, 2)<br>        dw3 = a2.T @ dz3                  # (5, 2)<br><br>        da2 = dz3 @ self.w3.T            # (batch_size, 5)<br>        dtanh = 1 - torch.tanh(z2) ** 2  # (batch_size, 5)<br>        dz2 = da2 * dtanh                # (batch_size, 5)<br>        dw2 = a1.T @ dz2                 # (6, 5)<br><br>        da1 = dz2 @ self.w2.T            # (batch_size, 6)<br>        drelu = (z1 &gt; 0).float()         # (batch_size, 6)<br>        dz1 = da1 * drelu                # (batch_size, 6)<br>        dw1 = x.T @ dz1                  # (4, 6)<br><br>        return dw1, dw2, dw3</pre><p>Before jumping to the algorithm, we need to understand a terms here.</p><p>Moment of a gradient:</p><ul><li>It refers to the running average of gradients (or their powers).</li><li>It is used to smooth out updates during training a network.</li><li>It also helps avoid oscillations and helps converge faster.</li><li>Oscillations here mean cases where the gradient direction <strong>keeps changing sharply</strong>, especially in areas where the loss surface is steep in one dimension but flat in another (e.g, ravines)</li><li>This causes the parameter updates to <strong>zig-zag</strong>, bouncing side-to-side across the loss surface, <strong>slowing convergence</strong>.</li><li>As we can see below: SGD zig-zags its way around. This happens especially in narrow, curved valleys where gradients flip direction each step</li><li>The <strong>red line</strong> shows how <strong>Adam</strong> moves more <strong>directly</strong> toward the minimum because it uses moments (1st and 2nd). It remembers the direction <strong>(because of the 1st moment)</strong> it’s been going and avoids abrupt swings.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6wJdRSAMmPyJAKI87bHKgQ.jpeg" /></figure><p>However, raw moments can be noisy, which is a problem since noise is like the exact opposite of what we want. If a gradient is noisy, it means we have high variance. There must be some mechanism to ensure that the optimizer can be cautious in these cases and smaller steps-sizes are taken (IK it sounds insane rn but it will make sense 10/10 trust) since if we take a huge step during noisy gradients, we might end up missing our direction of convergence entirely.</p><p>So to avoid these noisy gradients, Adam uses two types of moment of gradients:</p><ul><li>1st moment (m_t): this one is supposed to help with directional stability. it is called the Exponential Moving Average (EMA) and is supposed to give us the average direction in which the optimizer is supposed to go (kinda resembles the mean if you ask me)</li><li>2nd moment (v_t): this one helps with the adaptive learning rates. It is EMA of squared gradients and tracks magnitude. for eg, if the gradient is huge, the updates must be small.</li></ul><p><strong>Defining Stochastic Function</strong></p><p>Besides moments, we need to consider 𝑓 ( 𝜃 ) , which is our noisy objective function with parameters θ (as in it is differentiable with respect to θ, i.e our weights biases).</p><p>Mathematically, function 𝑓 ( 𝜃 ) must be minimised since it represents the loss function. But it gives <strong>different outputs</strong> every time we run it — because of random data, noise, or sampling.</p><p>Why is the data noisy?While this would have worked if our use-case was deterministic (Every evaluation of 𝑓 ( 𝜃 ) gives the same value) and the algorithm would have converged easily, it is not so.</p><ul><li>We’re supposed to be minimizing the expected values over a random process.</li><li>Moreover, the network will take data in batches. So minimizing one output might just make you good on one batch — not overall.</li></ul><p>However, instead of minimizing just <strong>one</strong> value of 𝑓 ( 𝜃 ) , we want to minimize the <strong>average value</strong> over all possible random situations. This average is written as:</p><p>𝐸 [ 𝑓 ( 𝜃 ) ](“expected value of 𝑓 ( 𝜃 )”)</p><p>Say we’re training a model on random batches of data.</p><ul><li>One batch might give a loss of 0.45</li><li>Another batch might give a loss of 0.39</li><li>Another might give 0.51</li></ul><p>Each one is a different output of 𝑓 ( 𝜃 ) . We can’t trust just <strong>one</strong>. This is why we need the <strong>average loss</strong> across many batches.</p><p>That average is 𝐸 [ 𝑓 ( 𝜃 ) ]. That’s what we are trying to minimize. Also, stochasticity helps escape local minima or saddle points.</p><p>It’s a little confusing, but not too complicated. It means the average value of the function 𝑓 ( 𝜃 ) when there is randomness involved.</p><p>In practice, we approximate this expectation using a batch:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*0JMb_ueTAQHrIGQ4-aaXmw.png" /></figure><p>Honestly, this might sound like it overlaps with the concept of fixing random seeds for reproducibility, but those ensure that the order and composition of our batches during training are kept the same every time we run it.</p><p>Loss functions are theoretically noisy. This is because:</p><ul><li>We’re not evaluating the loss on the entire dataset each time. We’re using mini-batches.</li><li>So even with a fixed seed, each individual update step in our optimizer sees a subset of the data → hence the loss and its gradient still fluctuate.</li><li>The function we’re minimizing — the expected loss over the full data distribution — remains inherently stochastic unless weevaluate over the entire dataset every time (which is rare and expensive).</li></ul><p>Think of it this way: Wehave <strong>1 million samples</strong> total, split into <strong>80% training</strong>, <strong>20% validation</strong> → So <strong>800,000 training samples and a batch size = 32</strong>. One <strong>epoch</strong> = 1 full pass over all 800,000 training samples.</p><p>Setting a seed, say 42, would ensure that if In <strong>epoch 1</strong>, sample #237 might be in batch #3, then sample #237 will always be in batch #3 <strong>for that specific seed.</strong></p><p>At <strong>each step</strong>, our model only sees a batch of 32 → and computes <strong>a loss value for that batch</strong>. That batch-loss is <strong>noisy</strong> — it’s not based on all 800,000 samples. So we’re only seeing an <strong>estimate of the true expected loss</strong> (which would require computing over all training data).</p><p>Code wise, 𝐸 [ 𝑓 ( 𝜃 ) ] can represent the loss produced during a single epoch when the optimizer is called (remember Daniel’s song?).</p><p>Tl;dr: Suppose you have infinite data from some real-world process (e.g., speech, medical scans, etc). You <strong>can’t</strong> compute loss on all of it. So, you take samples (mini-batches), compute their losses 𝑓 ( 𝜃 ) , and use them to <strong>estimate</strong> the true expected loss 𝐸 [ 𝑓 ( 𝜃 ) ].</p><p>Phew! Now the actual algorithm. There’s some other stuff like convex and non-convex loss functions too, but we will cover that later.</p><p><strong>Coding the Algorithm</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ADWRSkmWlFsmtrR-k65gpQ.png" /><figcaption>Adam’s Algorithm as described in ‘Adam: A Method for Stochastic Optimization’</figcaption></figure><p>Consider this pseudocode that was provided in the paper. We will code the function we these equations. Firstly, the constants provided are:</p><pre>max_steps = 1000<br>alpha = 0.001<br>beta_1 = 0.9<br>beta_2 = 0.999<br>epsilon = 1e-8</pre><p>beta_1 and beta_2 are decay rates for our 1st and 2nd moments here, while alpha is the step size by which the parameters are updated. Furthermore:</p><p>∇ θ f(θ): the gradient of the loss function with respect to parameters 𝜃. In other words, remember the derivatives of the weights we calculated previously in our backward pass (dw1, dw2 and dw3)? Those are going to form these gradients.</p><p>θ represents our original parameters. Since our RandomNet did not use any biases, params would simply include all the weights stacked as a list.</p><pre>params = [self.w1, self.w2, self.w3]. # defined in __init__<br>grads = [dw1, dw2, dw3]</pre><p>These values would be passed, accessible to our function. Another important thing to note here is that generalising the condition “while θt has not converged do” is a little difficult since we don’t actually know when the algorithm will converge. So we fix the maximum number of steps that the loop can take for attempting this convergence as max_steps =1000</p><p>The first and second moment are initialised as a tensor of zeros. We’ll first attempt to code this as close as to it’s mathematical interpretation (or rather the way I interpreted it the first time I read it, and then see why this is a bad idea in terms of efficiency)</p><pre>def literal_math_of_adam_optimizer(self, grad, max_steps = 1000, alpha = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 10**(-8)):<br>      self.params = params # list of tensorsjkjjkk<br>      mt=[[torch.zeros_like(p) for p in params]]<br>      vt=[[torch.zeros_like(p) for p in params]]</pre><p>So we create both moments as a list of lists of the form m[t][params], where t represents the current step as t, and params represents the list of parameters we created earlier. Essentially, our list is going to be a list of stacked parameters across each step in the loop. Why this way? Let us see below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OsJBAI_FRk_Z-jQs721pMw.jpeg" /></figure><p>The list of list was primarily created so we could acc the 1st and 2nd moment from the previous step (t-1). We code our function further as:</p><pre>def literal_math_of_adam_optimizer(self, grad, max_steps = 1000, alpha = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 10**(-8)):<br>      self.params = params # list of tensorsjkjjkk<br>      mt=[[torch.zeros_like(p) for p in params]]<br>      vt=[[torch.zeros_like(p) for p in params]]<br><br>      for t in range(max_steps):<br>        for i in range(len(params)):<br>          g = grad[i]<br><br>          mt = beta_1 * mt[-1] + (1-beta_1) * g<br>          vt = beta_2 * vt[-1] + (1-beta_2) * g**2</pre><p>We create two more temporary lists, m_curr and v_curr to store the current the updated moments for each parameter in the loop.</p><p>However, both these moments are bias corrected and represented as m_hat and v_hat, and it is these bias corrected terms which are used for updation. We’ll explore why this is need more in the Initialisation Bias Correction section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MxX4aTz5GMHuDTtrg0uW4g.jpeg" /></figure><p>These updates are then used to update the final parameters depending on the final step-size alpha x (some ratio).</p><pre>def literal_math_of_adam_optimizer(self, grad, max_steps = 1000, alpha = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 10**(-8)):<br>      self.params = params # list of tensorsjkjjkk<br>      mt=[[torch.zeros_like(p) for p in params]]<br>      vt=[[torch.zeros_like(p) for p in params]]<br><br>      m_curr = []<br>      v_curr = []<br><br><br>      for t in range(max_steps):<br>        for i in range(len(params)):<br>          g = grad[i]<br><br>          mt = beta_1 * mt[-1] + (1-beta_1) * g<br>          vt = beta_2 * vt[-1] + (1-beta_2) * g**2<br><br>          m_hat = m_t / (1-beta_1**t)<br>          v_hat = v_t/ (1-beta_2**t)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NPQ2MSAM1p4uijOn3iOIgg.jpeg" /></figure><p>Clubbing these all together, our literal mathematical translation of the algorithm becomes:</p><pre>def literal_math_of_adam_optimizer(self, grad, max_steps = 1000, alpha = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 10**(-8)):<br>      self.params = params # list of tensorsjkjjkk<br>      mt=[[torch.zeros_like(p) for p in params]]<br>      vt=[[torch.zeros_like(p) for p in params]]<br><br>      m_curr = []<br>      v_curr = []<br><br><br>      for t in range(max_steps):<br>        for i in range(len(params)):<br>          g = grad[i]<br><br>          mt = beta_1 * mt[-1] + (1-beta_1) * g<br>          vt = beta_2 * vt[-1] + (1-beta_2) * g**2<br><br>          m_hat = m_t / (1-beta_1**t)<br>          v_hat = v_t/ (1-beta_2**t)<br><br>          params[i] -= alpha * m_hat / (v_hat**0.5 + epsilon)<br><br>          m_curr.append(mt)<br>          v_curr.append(vt)<br><br>        mt.append(m_curr)<br>        vt.append(v_curr)<br><br>      <br>      return params</pre><p><strong>Improving Our Implementation</strong></p><p>I mean this implementation is an engineer’s nightmare (not entirely, since this is how the algorithm is inferred).</p><p>We don’t have to store the moments vectors of every parameter across every timestep. We just need the moments from the previous step- this is borderline abusing all that memory space (ever struggled with fitting our kv cache into RAM even with higher GPUs? exactly) In our usecase, we’re storing 3 params across each of the 1000 steps — that is literally 3000 tensors</p><p>Like if each tensor is 100 KB, and we store 3000 tensors, that is 300 MB of space we are using. However, if we store only 3 tensors from previous batch, it reduces to 3KB. That’s huge bishes.</p><p>(It also makes our code slower because processing a list of list that’s like a whole matrix of size (no_of_params, no_of_time_steps))</p><p>So what do we do? We simply use a single list of params and update it with the loop on runtime instead of forcefully accessing the previous value. The equations remain the same, however our optimizer looks neater and closer to it’s algorithmic representation:</p><pre>def adam_optimizer(self, grad, max_steps = 1000, alpha = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-8):<br>      # m0 = torch.zeros(params.shape)<br>      # v0 = torch.zeros(params.shape)<br>      m0 = [torch.zeros_like(p) for p in self.params]<br>      v0 = [torch.zeros_like(p) for p in self.params]<br>      # mt = [m0]<br>      # vt = [v0]<br><br>      for t in range(1,max_steps+1):<br>        for i in range(len(self.params)):<br>          gt = grad[i];<br>          # mt = beta_1 * mt[-1] + (1-beta_1) * gt<br>          mt[i] = beta_1 * mt[i] + (1-beta_1) * gt<br>          # vt = beta_2 * vt[-1] + (1-beta_2) * gt**2<br>          vt[i] = beta_2 * vt[i] + (1-beta_2) * gt**2<br>          # m_1.append(mt)<br>          # v_1.append(vt)<br><br>          m_hat = mt[i] / (1-beta_1**t)<br>          v_hat = vt[i]/ (1-beta_2**t)<br><br>          self.params[i] -= alpha * m_hat / (v_hat**0.5 + epsilon)<br><br>      <br>      return params</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*-0op7iokUjizIu7zdcG5WA.png" /><figcaption>Output Produced by our RandomNet being optimised via Adam</figcaption></figure><p>Coolspace! Also, as you can see, we actually didn’t have to consider the RandomNet specifically, since the optimizer simply requires the previously computed gradients and parameters.</p><p>Below is the mathematical explanation from the paper. The basic implementation via code is done.</p><h3>Update Rule Math</h3><p>Bound 1: Now, in our update rule, if we take epsilon = 0 and use ∆t to represent 𝜃(t-1)-𝜃(t), we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DAC0T118nskN0kHqKLSyxg.jpeg" /></figure><p>Now, to bind (should be bound but grammar?) this, we make two assumptions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*laFzH3sS3c798OGVsjChtA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kApyvB5_VYyjEsj1_MxjTw.jpeg" /></figure><p>However, this is a super special case: when a gradient has been 0 at all time-steps except for the current time-step.</p><p>As per the paper:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3GzwqUfZ3vfDCosFANqYEQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KYbKTc9pvFe419tP54uvvg.jpeg" /></figure><p>So yes, this bound is larger than α, but it still depends linearly on α. This only happens when gradients have been 0 for all previous steps. If gradient was 0 at, say (t-1)th step, then:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zcxl3o6RfmoNV00TW6Qo6g.jpeg" /></figure><p>This is where the <strong>bound 1</strong> comes from. It’s rare but possible in sparse gradient settings. If a gradient was 0 at previous step, this means that Adam has no prior information — it’s doing a step based only on the current gradient. Therefore, this step is <strong>larger</strong> than normal (could be &gt; α), which is why they compute that as an upper bound.</p><p><strong>Bound 2</strong>: In most common cases, we will have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-IfDaZPMxEhLfxvUvkdFzw.jpeg" /></figure><p>In both bounds (1) and (2), the effective step size is of the form:</p><p>∣Δt∣=α x (some ratio)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pHdgcKsqTY4wAoSEc1ZPog.jpeg" /></figure><p>Cauchy-Schwarz inequality ensures step ratio is ≤ 1 unless weird edge case</p><p>So the bounds are really saying: No matter what α is, Adam will multiply it by a ratio that’s ≤ 1 or slightly above 1 in extreme cases.</p><p>Remember how we said that:</p><p>m_t → moving averages of mean</p><p>v_t → moving averages of mean and variance</p><p>If we assume m_t to be the signal, and v_t to be noise/variance in gradient, then m_t/√v_t ****would be analogous to Signal to Noise ratio (SNR), where m_t is the true direction of the gradient, and v_t is how uncertain we are about this direction.</p><p>IF noise is high→ v_t is large → we take a small step</p><p>This means that Δt would be extremely small (inverse proportionality, although that’s the wrong terminology for this subject i suppose but whatever helps) and quite closer to 0, in fact.</p><p>Now this is the caution mechanism we mentioned earlier. Noisy gradient means high variance and small SNR (since denominator would be huge). Adam naturally becomes cautious in this case, and takes smaller steps.</p><p><strong>P.S:</strong> SNR becomes 0 towards an optimum value. This means that since gradients start hovering around 0, we get m_t →0 and v →+ve since it is a squared value. Therefore m_t/√v_t →0 and Δt → 0, i.e., our step size is already shrinked.</p><p>This is what we call authentic annealing: the learning rate decreases slowly as the optimization proceeds. And this is something Adam does automatically because of how it is designed.</p><p>Another trivial application here is that step-size is invariant to gradient scale.</p><h3>Initialisation Bias Correction</h3><p>The initialisation bias arises since we start m_t and v_t as a tensor of 0s. We will here derive the term for the second moment estimate (same as the paper). It’s similar for the first one tbh will add that too I suppose.</p><p>Let g be the gradient of the stochastic objective f, and we wish to estimate its second raw moment. We rewrite the equation of v_t as a function of gradients across all previous time-steps as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eQueQpPFOVebzgcAMpP55A.jpeg" /></figure><p>Assuming gradients come from a stationary distribution E[gi²].</p><p>I said stochastic earlier not stationary there is a difference. stationary implies that gradients do not change overtime, but they are also not constant. To be more specific, it is because the gradients are sampled from a distribution which is not changing overtime either. This assumption only holds true for shorter windows, eg: batch training, since neural networks themselves are non-stationary as the parameters change at every step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5HZP3XettBtmKvk5BQS8ew.jpeg" /></figure><p>This is why we divide v_t by (1−β2) to cancel out the bias and obtain v_hat. The same applies to m_hat too.</p><h3>Convergence Analysis</h3><p>Alright so for understanding the convergence Analysis, we firstly need to know that there are two types of loss functions: Convex and Non-Convex.</p><p>We have something called the minima of the function, which is the lowest point on the graph of that function, found by setting its first derivative to 0. These are called critical points (points where derivative of a function is 0), but it is easy to confuse a local minima with a global minima for a lot of these optimizers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*CZzfobOJZaRZoYPcsFLQTQ.jpeg" /></figure><p>In the graph above, ‘x’ represents a parameter in the parameter space (or the weight space) where our Model exists. Each dimension corresponds to a parameter, and since each ML model is dependent on multiple parameters, we abstract this a little and represent this by considering a model with a single parameter (hence the 2D diagram).</p><p>f(x) represents the value of the loss function for that parameter. The graph forms a 2 dimensional optimization landscape</p><ol><li>Global minimum: smallest value of lost function across the entire parameter space. These are the optimal set of parameters for a model</li><li>Local minimum: this refers to a value where the loss function is minimum for a set of parameter values in the parameter space.</li></ol><p>We have saddle points too, which graphically lie between a local minima and global minima. The gradient of the loss function is 0 at these points, and the value of the loss function is greater on one side and minimum on the other.</p><p>In high dimensions, l<strong>ocal minima are less of an issue</strong> due to the geometry of high-dimensional spaces (more saddle points than true local minima).</p><p>We define our Convex and Non-convex loss functions as:</p><p>Convex Function: It has only 1 global minima and no local minima. This means that optimizing a convex loss function would ensure that our algorithm always stumbles upon the global minimum, making optimization easier. Examples of these are L2 loss, Log loss, etc.</p><p>A way to visualise convex loss functions is that any two points (x1 and x2) on the graph of a loss function would always lie above its curve as shown below with the help of L2 norm:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pxd9ZHG3F75rzr0KUvxllQ.jpeg" /></figure><p>This represents a bowl shaped curve where function’s value would increase as we move away from the bottom of the curve, where a represented by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bS4XJEOCelFv0haClkNTmg.png" /></figure><p>Non-Convex Functions: These have both local and global minima and require advanced optimization techniques to find the global minimum. Graphically, it has multiple peaks and valleys and two points (x1 and x2) may lie below the graph at instances. However, a non-convex function is different from a concave function, and we can’t simply flip the sign on the convex equality to define them.</p><p>Non-convex functions are convex in certain regions of the graph and concave across others, but across the entire graph, they are neither convex nor concave. For a function to be non-convex, it simply and literally means that the function is not convex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YgsSewcjk1ocjyBe2nfY4g.jpeg" /></figure><p>Which means that the equation for representing a non-convex function becomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p_zgkPo22I6XAFFSmuRERA.png" /></figure><p>Regret bounds are typically defined as the maximum possible regret that an online learning algorithm can incur over a sequence of rounds, i.e, the difference between the cumulative reward of the optimal strategy and the cumulative reward of the algorithm.</p><p>R(T) = ∑ (optimal — algorithm)</p><p>An important thing to note before we begin is that neural network based problems (like RNNs) are non-convex optimisation problems, due to non-linear nesting of weight values from previous layers.</p><p>Non-linearity from previous layers introduces non-convexity. While in convex/concave problems we’re looking for a global maxima or minima, for non-convex problems we actually need to understand what kind of points need to be looked for exactly. This makes non-convex problems much more difficult. However, the current paper theoretically considers a convex scenario, while practically solving experiments of non-convex nature. Since this article considers the original proof of 2014 paper, we’re going to stick to its implementation (however Adam’s proof for non-convex problems was later introduced in a paper titled ‘Adaptive Methods for Non-convex Optimisations’ and y’all can check it out later). Now back to our proof.</p><p>We represent our predictions as 𝑓t ( 𝜃 ) and the best fixed point parameters as 𝑓t ( 𝜃* ). Therefore, for a given unknown sequence of functions 𝑓1 ( 𝜃 ), 𝑓2 ( 𝜃 ), …, 𝑓T ( 𝜃 ), our aim is to predict 𝜃t and evaluate it on a previously unknown cost function 𝑓t ( 𝜃 ). Since we don’t know the nature of this sequence, we use regret bounds, which can be defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7iVH7otPsQ7nDAtg8KyTUg.jpeg" /></figure><p>This implies that the function at the middle input is not more than any function values at endpoints. This means our curve is going to be bowl shaped, one where the function never bulges over the line between two points (refer to convex figure diagram above).</p><p>According to the diagram, we can say that a tangent drawn at any point on the curve of a convex function will always lie below the curve. Hence, we can use this tangent to lowerbound the function. However, we’re using regret to bound our function here, which is where Lemma 2 comes at:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kZbyB8a6L4xVWFj7p_dSYQ.jpeg" /></figure><p>The next two lemmas are basically deriving results for proving the convergence algorithm at Theorem 10.5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LaJMin2OE1s8AJCE0LT2MA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sZIMSfimOPNMNHFbTELfEQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iOUXzDyQJkrDLgRubfVQkw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZX-E2BYwNXHUzj8GAjRZlA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fjYus1WiCGG86kX655XwPA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D3kv7sQ5260SNxbODCaKfQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WEs0zwGLJvJnih7zbpaChw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X9xqMLl6QgKjJCvqtHG4kQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eJ3sPju1-iKG899udntGwA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xcY2f_vubW1YBy8yYDSxAg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NuxHX6fK08e_ctUHUecB3A.jpeg" /></figure><p>We finally prove the Theorem 10.4, which proves Adam’s convergence via regrets bounds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3S90VfVeJmnmkVgvRlx1VA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELstjVsCDSejTa6ndsffqA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gh9pDzkfleTi_AtxY6Q4zg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ob9GNqxEzLcAey_E2Wl9OQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wcTWdgJWu2GXYS1p4lImHg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E1Eillm_ahYH8ixHniDVLA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K33W3JP1UP-0uuDMm4O7oQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3m5tRBHQnxBO8Tuzz-SFZg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hIFCE3lIO6x9gfG0mom9ew.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K7p6tnD-XWPNRt9lJnpEWA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Oa4-h7EAgiOPKLNuoTcpxA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-gUtEmSFaNJyiHePgsZv9w.jpeg" /></figure><p>Convergence in online optimization usually means that average regret goes to 0:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*ao2eZLy78zrGCZ4jeJOcuQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*moVSxFIn08sFNQBJ8BTQ6A.png" /></figure><p>All the code is available at the GitHub repository for <a href="https://github.com/AdwitaSingh1711/from-scratch-lab">Scratch Lab.</a> If you spot any mistakes or inconsistencies, feel free to point them out. It would actually help a lot since I’m in the process of learning and figuring things out too.</p><p>Also going to link some of the resources i referred to while working on this article.</p><p>Thanks for stopping by.</p><h3>Resources</h3><p><a href="https://arxiv.org/abs/1412.6980">Adam: A Method for Stochastic Optimization</a></p><p><a href="https://stackoverflow.com/questions/65243290/how-to-get-the-second-moment-of-the-gradient">https://stackoverflow.com/questions/65243290/how-to-get-the-second-moment-of-the-gradient</a></p><p><a href="https://satishkumarmoparthi.medium.com/why-l1-norm-creates-sparsity-compared-with-l2-norm-3c6fa9c607f4">Why L1 norm creates Sparsity compared with L2 norm</a></p><p>How To Prove It by Daniel J.Velleman for Proof by Induction</p><p><a href="https://www.learnpytorch.io/08_pytorch_paper_replicating/">08. PyTorch Paper Replicating - Zero to Mastery Learn PyTorch for Deep Learning</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fn65pElMp6x8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dn65pElMp6x8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fn65pElMp6x8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2655a8c9f8e018032f3bb93ec7e409ee/href">https://medium.com/media/2655a8c9f8e018032f3bb93ec7e409ee/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f525dfdd63e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Byte Pair Encoding]]></title>
            <link>https://medium.com/@Nayonae/implementing-byte-pair-encoding-3d7311978f88?source=rss-f6c5c8196601------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d7311978f88</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[byte-pair-encoding]]></category>
            <dc:creator><![CDATA[Adwita]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 16:28:13 GMT</pubDate>
            <atom:updated>2025-06-27T16:30:43.208Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have a terrible attention span like me, buckle up folks cuz this is actually pretty straightforward lol.</p><p>This article actually came about as I was reading a section from <em>An Introduction to Information Retrieval by Christopher D. Manning, Hinrich Schütze, and Prabhakar Raghavan.</em></p><p>In the 2nd chapter titled <em>The term vocabulary and posting lists</em>, section 2.2.2 on <em>Dropping common terms: stop words</em>:</p><blockquote><em>“The general trend in IR systems over time has been from standard use of quite large stop lists (200–300 terms) to very small stop lists (7–12 terms) to no stop list whatsoever. Web search engines generally do not use stop lists”</em></blockquote><p>This made little sense to me because every tutorial I had read (prior to 2021) on this topic for RNN type models went into great detail about how stop-words are the bane of our existence while building language models, and must be excluded from being passed to it.</p><p>While the book essentially explains their mitigation via compression techniques and further goes on to explain why common words have little impact on document rankings due to standard term-weighing techniques (the book is a banger read 10/10 recommended; my French classes felt funnier when I thought about tokenizing “<em>l’ordonnance”</em> and watching my pipeline struggle).</p><p>All this might sound unrelated to tokenizer algorithms, but the question is still worthwhile: how are large systems tokenizing words such that perfect sentence reconstruction is able to happen? Because when you think about it, if you train an LLM with no stop-words, the LLM will inherently not know that stop-words exist in the human vocabulary.</p><p>How else are the inputs then prepared for LLM training and how are they passed to the model at inference time?</p><p>You all have probably used some kind of tokenizer.encode() call while working with LLMs to vectorise the input prompt before passing to the LLM. Similarly, the output produced by the LLM is converted to a readable format using tokenizer.decode().</p><p>Most LLMs have their own Custom tokenizers. LLMs like GPT use Sentence Piece, which is a popular tokenizing algorithm. The Sentence Piece model is based on Byte Pair Encoding (BPE), which was first introduced in this <a href="https://arxiv.org/abs/1508.07909">paper</a>.</p><p>While the exact implementation for commercial models might differ to support their systems, we will implement a basic structure to understand why it works effectively.</p><h3>ALGORITHM</h3><p>BPE is a subword tokenisation algorithm — which means that instead of tokenizing words on the basis of trivial characters like ‘ ’ or ‘-’ (<em>An Introduction to Information Retrieval</em> also explains why these are terrible methods), we make our model learn the representations in the form of subwords (literally means part of a word).</p><p>We start with initial data, which is going to be a small sentence in our example (honestly I just picked up the first paragraph from Wikipedia’s featured article on May 2nd, 2025), which is going to act as the source of all words our model is supposed to know. We then construct:</p><ul><li>A list of tokens: includes all words that appear in the vocabulary</li><li>A vocabulary: which includes all unique words in our sentence. We store this as a set</li><li>Hyperparameter K: it is the number of times our algorithm will go hunting for a new subword from the list of tokens, or the number of merges operations that increase our vocabulary size.</li></ul><pre>sentence=&quot;Nosy Komba (Malagasy pronunciation: [nusʲ ˈkuᵐba]; lit. &#39;island of lemurs&#39;), also known as Nosy Ambariovato, is a small island in Madagascar, situated between the island of Nosy Be and the northwestern coast of the main island of Madagascar. Roughly circular, it rises sharply towards a plateau and the summit of Antaninaomby at the center of the island. Administered as an arrondissement of the unitary commune and district of Nosy-Be within Diana Region, the island is divided into five fokontany (villages), with Ampangorina as the main village and administrative center. The population is mainly restricted to the northern half of the island. The economy is reliant on tourism and handicrafts, supplemented by a wide range of agricultural products. Hotels and guest houses support tourists to the island, mainly on excursions from Nosy Be.&quot;<br><br>tokens = []<br><br>for i in sentence:<br>  if i not in tokens:<br>    tokens.append(i)<br><br>vocab = set(tokens)<br>print(f&quot;Initial vocab: {vocab}&quot;)<br>k=3</pre><p>Once initialised, we start our our initial loop. K essentially decides how many new subwords will be added to our vocabulary this once.</p><p>Next, for our current model, we will consider bigram level tokens. To track them, we also create a dictionary called freq_tokens={}, which will track the frequency of tokens for the current pass through our tokens.</p><p>freq_tokens follows your average structure of storing pairs (current_char, next_char). The most frequently appearing tokens are simply added to our vocab.</p><pre>for _ in range(k):<br>  <br>  freq_tokens = {}<br>  for char in range(len(tokens)-1):<br>      pair = (tokens[char], tokens[char+1])<br>      freq_tokens[pair] = freq_tokens.get(pair, 0)+1<br>  <br>  highest_freq_pair = max(freq_tokens, key = freq_tokens.get)<br>  new_token = &#39;&#39;.join(highest_freq_pair)<br>  vocab.add(new_token)</pre><p>Now, once the most frequent bigram has been picked from the current pass, it is added to vocabulary. But your token list must also be updated so that you don’t end up picking the same token at every pass.</p><pre>i=0<br>  updated_tokens = []<br><br>  while i&lt;len(tokens)-1:<br>    if (tokens[i],tokens[i+1])==highest_freq_pair:<br>      updated_tokens.append(new_token)<br>      i +=2<br>    else:<br>      updated_tokens.append(tokens[i])<br>      i +=1<br>  <br>  tokens = updated_tokens</pre><p>Combining all of the above, we can see the final code and corresponding output obtained below.</p><h3>FINAL CODE AND OUTPUT</h3><pre>sentence=&quot;Nosy Komba (Malagasy pronunciation: [nusʲ ˈkuᵐba]; lit. &#39;island of lemurs&#39;), also known as Nosy Ambariovato, is a small island in Madagascar, situated between the island of Nosy Be and the northwestern coast of the main island of Madagascar. Roughly circular, it rises sharply towards a plateau and the summit of Antaninaomby at the center of the island. Administered as an arrondissement of the unitary commune and district of Nosy-Be within Diana Region, the island is divided into five fokontany (villages), with Ampangorina as the main village and administrative center. The population is mainly restricted to the northern half of the island. The economy is reliant on tourism and handicrafts, supplemented by a wide range of agricultural products. Hotels and guest houses support tourists to the island, mainly on excursions from Nosy Be.&quot;<br><br>tokens = []<br>for i in sentence:<br>  if i not in tokens:<br>    tokens.append(i)<br>vocab = set(tokens)<br>print(f&quot;Initial vocab: {vocab}&quot;)<br>k=3<br>for _ in range(k):<br>  <br>  freq_tokens = {}<br>  for char in range(len(tokens)-1):<br>    pair = (tokens[char], tokens[char+1])<br>    freq_tokens[pair] = freq_tokens.get(pair, 0)+1<br>  highest_freq_pair = max(freq_tokens, key = freq_tokens.get)<br>  new_token = &#39;&#39;.join(highest_freq_pair)<br>  vocab.add(new_token)<br>  i=0<br>  updated_tokens = []<br>  while i&lt;len(tokens)-1:<br>    if (tokens[i],tokens[i+1])==highest_freq_pair:<br>      updated_tokens.append(new_token)<br>      i +=2<br>    else:<br>      updated_tokens.append(tokens[i])<br>      i +=1<br>  <br>  tokens = updated_tokens<br>    <br>print(f&quot;New Vocab: {vocab}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ctX7aONlUSwtURtqX5zzQ.png" /><figcaption>Output of our BPE Tokenizer</figcaption></figure><p>That it. It’s not too complicated, unless you’re a huge AI lab working with production level code (thoughts and prayers for all of us to get there). You can check out the code at <a href="https://github.com/AdwitaSingh1711/from-scratch-lab/tree/master/BPE">Scratch Lab</a>.</p><p>Thanks for stopping by.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d7311978f88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Write The Backward Pass of Any Neural Network]]></title>
            <link>https://medium.com/@Nayonae/how-to-write-the-backward-pass-of-any-neural-network-7bf04e0b4ee2?source=rss-f6c5c8196601------2</link>
            <guid isPermaLink="false">https://medium.com/p/7bf04e0b4ee2</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[backpropagation]]></category>
            <dc:creator><![CDATA[Adwita]]></dc:creator>
            <pubDate>Tue, 24 Jun 2025 19:09:47 GMT</pubDate>
            <atom:updated>2025-06-24T19:09:47.122Z</atom:updated>
            <content:encoded><![CDATA[<p>This blog focusses more on the coding of the backward pass than the full mathematical theory. There are loads of good articles available online for the derivations if you wish to check them out. For this though, simple chain rule from high school with partial derivatives is enough. If you know that, you’re good to go.</p><p>Also, I’ve used PyTorch for implementing this, since most projects are built with it and it helps in familiarising oneself with Tensors.</p><h3>Why Partial Derivatives though?</h3><p>We use partial derivative because remember that the goal of the backward pass is to compute gradients. Gradients tell us how much the loss would change if we make a small change in a variable (could be weights here), and accordingly the updates needed in our weights and biases to ensure that the network produces an output closest to the desired result (or true value).</p><p>For this, we must compute the change in the loss computed after each pass through the network with respect to the change in the value of the variable. But most neural networks use loss functions which are functions of more than one variable. This is why Partial derivative comes in: we need to compute how much each of these variables (weights, biases, etc) are influencing the loss while the other variables remain fixed.</p><h3>Picking Any Network</h3><p>Let us suppose we have a random net that looks as follows (don’t ponder too hard upon why I chucked random activations into it):</p><pre>class RandomNet:<br>    def __init__(self):<br>        self.w1 = torch.randn(4, 6)   # input, hidden<br>        self.w2 = torch.randn(6, 5)   # hidden, hidden<br>        self.w3 = torch.randn(5, 2)   # hidden, output<br>	<br>    def forward(self, x):<br>	 z1 = x @ self.w1<br>         a1 = torch.relu(z1)<br>	<br>	 z2 = a1 @ self.w2   <br>	 a2 = torch.tanh(z2)<br>	<br>	 z3 = a2 @ self.w3<br>	 return z1, a1, z2, a2, z3</pre><p>For ease of visualization, we can represent the above RandomNet as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nGcD7VtGTbR50o4oKI6y4g.jpeg" /><figcaption>RandomNet Structure</figcaption></figure><p>Our first goal always, in my opinion, if to check the dimensions of each transformation, activation and weight matrix here. In that way, while writing down the backward pass, we can easily check for the order of matrix multiplication.</p><pre>x -&gt; input(batch_size, 4)<br>z1 -&gt; matrix transformation (batch_size, 6)<br>a1 -&gt; Relu activation (batch,6)<br>w1 -&gt; weights to z1 (4,6)<br>z2 -&gt; matrix multiplication (batch_size, 5)<br>a2-&gt; tanh activation (batch_size, 5)<br>w2 -&gt; weight to z2 (6,5)<br>z3 -&gt; matrix transformation (batch_size, 2)<br>w3 -&gt; weights to z3 (5,2)</pre><p>Once we have checked through the dimensions, we use chain rule to compute the derivative of the loss with respect to the variable whose derivative is to be computed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sm1ejeTbq3yV0dEq612JIg.jpeg" /><figcaption>Representing the tensors and functions</figcaption></figure><p>We compute the loss after the output given by activation z3 with the help of Mean Squared Loss (although any formula could have been used. Cross-entropy is one that is commonly used):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*KH3hq10G6QxW6uTi2zzVdg.jpeg" /></figure><p>Once we have our Loss Function Sorted, we get the difference between our predicted values and the true values. This gives us how far our neural net is from it’s ideal form. Once we have the loss, we can compute it’s derivative with respect to the weights (w1, w2, and w3) of the layers of the network.</p><h3>Now We Go Back</h3><p>We represent these derivatives as dw1, dw2 and dw3. To calculate the change in weights, we must first compute the ∂Loss/∂z3, since z3 is the final activation. It must be noted that the backward pass takes place in the opposite direction of the forward pass (…which is obvious I suppose).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3vLh1ikoflXiRbJRSV_HoQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zGCDWnAys0UzSBdjxzvwig.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*duEt-NDPL1pqrAhYlWT3gA.jpeg" /></figure><p>We have our equations, so we can start writing our function for the backward pass as follows:</p><pre>def backward2(self, x, z1, a1, z2, a2, z3. target):<br>  dz3 = 2*(z3 - target)<br>  dw3 = a2.T @ dz3</pre><p>Similarly, we go back the path. ALthough, it must be noted that the order of computing dw3 is decided by checking dimensions of a2 and dz3 here.</p><p>dz3 will have the same dimension as z3, which would be (batch_size, 2) where as a2 would be (batch_size, 5). We simply wouldn’t be able to get dw3 if we had not taken the transpose of a2 which we got via computing it’s derivative (as shown above).</p><p>Furthermore, doing dw3 = a2.T @ dz3 means [5, batch_size]X[batch_size, 2], which gives us a tensor of [5,2], which is literally the dimension of dw3 too (Please I’m hoping this is making sense).</p><p>Similarly, we compute the derivatives of all variables down the equation as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*22Ne-t8dco-AY4ki6gWmQA.jpeg" /></figure><p>This gives us da2 = [dz3][w3.T] is the same as dz3@w3.T. It’s matrix multiplication but I seem to have used both notations throughout so yeah you’ll figure it out. Our function now becomes:</p><pre>def backward2(self, x, z1, a1, z2, a2, z3. target):<br>  dz3 = 2*(z3 - target)<br>  dw3 = a2.T @ dz3<br>  da2 = dz3 @ self.w3.T</pre><p>The next thing we encounter is tanh(x) activation. It is a little tricky to handle activations for me, which is why I break them down into computing the actual derivative of tanh(x) first (represented by dtanh here), then compute the derivative of the activation dz2, just to minimise any changes of errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o2WSU1RQ68j_73aBduOB4Q.jpeg" /></figure><pre>def backward2(self, x, z1, a1, z2, a2, z3. target):<br>  dz3 = 2*(z3 - target)<br>  dw3 = a2.T @ dz3<br>  da2 = dz3 @ self.w3.T<br>  dtanh = (1-torch.tanh(z2)**2)<br>  dz2 = da2*dtanh</pre><p>We continue up the variables in a steady manner. In hindsight, you can see that we’re using the computing the same variables and utilising them as we move back in the forward pass, which is exactly what back propagation is supposed to be doing.</p><p>As you can see below, we simply substitute the value of ∂Loss/ ∂z2 as dz2 and multiply by ∂z2/ ∂w2 in order to obtain the derivative of w2 with respect to the loss.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*asg4zHhq61-6kfm4OgikjA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LcaMpcLL30TWgaLeMPaKSA.jpeg" /></figure><p>You can try computing the derivatives of all variables in the same fashion. The tricky part i suppose might be the RELU activation, which is max(x,0), i.e.,</p><p>RELU→x→ if (x&gt;0)</p><p>RELU→0→ if (x≤0)</p><p>This is represented in PyTorch as dz1 = da1 * (z1 &gt; 0).float(). And this dz1 will be used to calculate ∂Loss/∂w1.</p><p>The final backward pass would be represented as:</p><pre>def backward2(self, x, z1, a1, z2, a2, z3, target):<br>    dz3 = 2 * (z3 - target)          # (batch_size, 2)<br>    dw3 = a2.T @ dz3                 # (5, 2)<br>    da2 = dz3 @ self.w3.T            # (batch_size, 5)<br>    dtanh = 1 - torch.tanh(z2) ** 2  # (batch_size, 5)<br>    dz2 = da2 * dtanh                # (batch_size, 5)<br>    dw2 = a1.T @ dz2                 # (6, 5)<br> <br>    da1 = dz2 @ self.w2.T            # (batch_size, 6)<br>    drelu = (z1 &gt; 0).float()         # (batch_size, 6)<br>    dz1 = da1 * drelu                # (batch_size, 6)<br>    dw1 = x.T @ dz1                  # (4, 6)<br> <br>    return dw1, dw2, dw3</pre><p>Therefor the updated network along with the loss can be represented as:</p><pre>import torch<br>import torch.nn as nn<br><br>class RandomeNet:<br>    def __init__(self):<br>        self.w1 = torch.randn(4, 6)   # input, hidden<br>        self.w2 = torch.randn(6, 5)   # hidden, hidden<br>        self.w3 = torch.randn(5, 2)   # hidden → output<br>    def forward(self, x):<br>        z1 = x @ self.w1             <br>        a1 = torch.relu(z1)          <br>        z2 = a1 @ self.w2             <br>        a2 = torch.tanh(z2)          <br>        z3 = a2 @ self.w3            <br>        return z1, a1, z2, a2, z3    <br>    def loss(z3, target):<br>        return (z3-target)**2 <br>    <br>    def backward2(self, x, z1, a1, z2, a2, z3, target):<br>        dz3 = 2 * (z3 - target)           # (batch_size, 2)<br>        dw3 = a2.T @ dz3                  # (5, 2)<br>        da2 = dz3 @ self.w3.T            # (batch_size, 5)<br>        dtanh = 1 - torch.tanh(z2) ** 2  # (batch_size, 5)<br>        dz2 = da2 * dtanh                # (batch_size, 5)<br>        dw2 = a1.T @ dz2                 # (6, 5)<br>        da1 = dz2 @ self.w2.T            # (batch_size, 6)<br>        drelu = (z1 &gt; 0).float()         # (batch_size, 6)<br>        dz1 = da1 * drelu                # (batch_size, 6)<br>        dw1 = x.T @ dz1                  # (4, 6)<br>        return dw1, dw2, dw3</pre><h3>End Notes</h3><p>We’re done! The explanations might seem redundant, but that’s a good sign because it means that you’ve truly grasped it.</p><p>If you would like to check out the working, I have implemented the Adam optimiser for this RandomNet at <a href="https://github.com/AdwitaSingh1711/from-scratch-lab/blob/master/adam/adam.py">Scratch Lab</a> (including the full backward pass of course, since that is important for demonstrating how Adam works).</p><p>Thanks for stopping by.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7bf04e0b4ee2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>