<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Elanthirayan on Medium]]></title>
        <description><![CDATA[Stories by Elanthirayan on Medium]]></description>
        <link>https://medium.com/@elanthirayan?source=rss-d28ca620d8a7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*IbxbvnhCYNZnU01ZVSX8aQ.png</url>
            <title>Stories by Elanthirayan on Medium</title>
            <link>https://medium.com/@elanthirayan?source=rss-d28ca620d8a7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:42:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@elanthirayan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[HPC Isn't About More Computers. It's About the Workload.]]></title>
            <link>https://elanthirayan.medium.com/hpc-isnt-about-more-computers-it-s-about-the-workload-3da65e342702?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/3da65e342702</guid>
            <category><![CDATA[gpu-computing]]></category>
            <category><![CDATA[hpc]]></category>
            <category><![CDATA[beginners-guide]]></category>
            <category><![CDATA[high-performace-computing]]></category>
            <category><![CDATA[supercomputer]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 09:22:51 GMT</pubDate>
            <atom:updated>2026-11-02T09:22:51.883Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Part 1 of HPC Foundations: what a high-performance computing system is made of, and the seven questions to answer before you use one.</em></p><p>High-performance computing, or HPC, uses powerful computing resources to solve complex problems within a practical time.</p><p>An HPC system combines:</p><ul><li>CPUs and GPUs</li><li>Memory</li><li>Storage</li><li>Networks</li><li>Software</li><li>Computing systems</li></ul><p>HPC is not only about using more computers. It is about using these resources together in an efficient way.</p><figure><img alt="HPC Foundations_ Login and Compute Nodes.png" src="https://cdn-images-1.medium.com/max/1024/52826a858022fe60375942f513b88b5c654828cf914e4d50083de0c9c69bda88" /><figcaption>AI-generated illustration for conceptual understanding</figcaption></figure><h3>Why do we need HPC?</h3><p>Many scientific, engineering, and artificial intelligence workloads require more computing power than a normal computer can provide.</p><p>Examples include:</p><ul><li>Weather and climate modelling</li><li>Computational fluid dynamics</li><li>Energy research</li><li>Molecular simulation</li><li>Large-scale AI training</li><li>Data analysis</li><li>Digital simulations</li></ul><p>These workloads may require large amounts of processing power, memory, storage, or time. HPC helps reduce the time required to complete such workloads.</p><h3>How does HPC improve performance?</h3><p>HPC systems can use several computers, processors, or GPUs to process a workload. Some workloads can be divided into independent tasks. These tasks can run at the same time. Other workloads contain connected tasks. These tasks must exchange information while they run. The correct approach depends on the problem and the application.</p><p>For example, one application may run many independent simulations. Another application may divide one large simulation across several computers.</p><h3>HPC is a complete system</h3><p>HPC performance depends on more than the processor.</p><p>The following components can affect the result:</p><ul><li>Compute power</li><li>Memory capacity</li><li>Memory speed</li><li>Storage performance</li><li>Network speed</li><li>Software design</li><li>Application behaviour</li></ul><p>A powerful GPU may not improve performance if the application cannot use it efficiently. Similarly, adding more computers may not help if the workload requires frequent communication. The hardware and software must match the workload.</p><h3>Start with the workload: 7 questions to ask.</h3><p>Before using an HPC system, we should understand the workload.</p><p>Important questions include:</p><ul><li>How large is the problem?</li><li>How much compute power is required?</li><li>How much memory is required?</li><li>How much data must be read or written?</li><li>Can the work be divided?</li><li>Do the tasks need to exchange information?</li><li>Is the goal to complete one task faster or to complete more tasks?</li></ul><p>The answers help determine the required HPC resources.</p><h3>What&#39;s next in HPC Foundations.</h3><p>HPC combines computing power, memory, storage, networking, and software to solve demanding problems efficiently. It supports many types of workloads. Some workloads use independent tasks. Others require several computing resources to work together. Understanding the workload is the first step in using HPC effectively.</p><p>The next articles will explain parallel computing, distributed computing, HPC cluster architecture, Slurm, job submission, memory, and cache in more detail.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3da65e342702" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Universal Guidance for Diffusion Models: One Model, Many Ways to Control It]]></title>
            <link>https://elanthirayan.medium.com/universal-guidance-for-diffusion-models-one-model-many-ways-to-control-it-344469a05ecd?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/344469a05ecd</guid>
            <category><![CDATA[diffusion-models]]></category>
            <category><![CDATA[universal-guidance]]></category>
            <category><![CDATA[research-paper]]></category>
            <category><![CDATA[image-generation]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Fri, 23 Oct 2026 07:33:38 GMT</pubDate>
            <atom:updated>2026-10-23T07:39:17.284Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine asking an artist to paint a dog sitting in a garden. The painting looks beautiful but the dog is in the center, and you wanted it on the left. So you give the artist a sketch showing where everything should go. Now imagine the artist only understands text instructions, not sketches.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nQ0m0ht3e9qflzX-gfumGg.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><p>What if you wanted to use a photograph to preserve someone’s face? Or a segmentation map to control the layout? Would you need to retrain the artist every time?</p><p>This is the problem explored in the paper <strong>Universal Guidance for Diffusion Models</strong>.</p><p>The idea is simple: instead of training a diffusion model for every type of control, use existing AI models to guide it.</p><h4>The Problem: Beautiful Images, Limited Control</h4><p>Diffusion models are excellent at generating realistic images, but controlling exactly what appears and where isn’t always easy.</p><p>Different tasks require different kinds of conditioning:</p><ul><li><strong>Text prompts:</strong> What should appear?</li><li><strong>Semantic Segmentation:</strong> Which regions belong to which objects?</li><li><strong>Bounding Boxes:</strong> Where should objects appear?</li><li><strong>Face Recognition:</strong> Does the generated person match a reference?</li><li><strong>Style Guidance:</strong> Does the image follow a particular visual style?</li></ul><p>Traditionally, introducing a new conditioning mechanism often requires additional training or model modifications but what if we could use models that already understand these requirements?</p><h4>The Big Idea: An Artist with Different Guides</h4><p>Think of the diffusion model as an artist and the artist knows how to paint. It doesn’t necessarily need to understand every instruction directly. Instead, we introduce specialists who evaluate its work.</p><p>For example:</p><ul><li><strong>CLIP (Contrastive Language–Image Pre-training)</strong> checks whether an image matches a text description.</li><li><strong>Semantic Segmentation Network</strong> checks whether the image follows the requested layout.</li><li><strong>Face Recognition Network</strong> checks whether the generated face resembles a reference person.</li><li><strong>Object Detection Network</strong> checks whether objects appear in the correct locations.</li></ul><p>These networks don’t generate images, they provide feedback. The diffusion model uses that feedback to adjust its generation process.</p><h4>The Challenge: How Do You Guide an Image That Doesn’t Exist Yet?</h4><p>Diffusion models don’t create complete images instantly. They start with random noise and gradually remove it through a process called <strong>Denoising</strong>.</p><p>Now imagine passing that noisy intermediate image to a Face Recognition Network. How would it recognize a face that isn’t clearly visible? The same issue affects CLIP, segmentation networks, and object detectors.</p><p>These models are generally trained on clean, recognizable images, not heavily noisy diffusion states. This mismatch is called <strong>Domain Shift </strong>and it’s one of the main problems Universal Guidance addresses.</p><h4>The Clever Trick: Predict the Finished Image</h4><p>Instead of giving the noisy image directly to the guidance network, Universal Guidance first estimates what the clean image might look like and this is called the <strong>Predicted Clean Image (x₀)</strong>.</p><p>Think of it as showing the art director a preview of the finished painting rather than the messy work in progress.</p><p>The process works like this:</p><ol><li>The diffusion model predicts a clean image from the current noisy state.</li><li>An external guidance network evaluates that prediction.</li><li>It calculates a <strong>Guidance Gradient</strong>, indicating how the result should change.</li><li>The diffusion process uses that feedback to steer generation.</li></ol><p>The important part is that the guidance model can evaluate something closer to the clean images it was trained on. This makes it possible to use different pretrained networks without retraining the diffusion model itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xe0kXBJsiBM3O0bTX1q6fQ.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><h3>Three Techniques That Make It Work</h3><p>The paper introduces three important mechanisms.</p><h4>Forward Guidance: Give Directions</h4><p>Imagine the artist is drawing a dog in the center. You tell them, “<strong>Move it slightly to the left</strong>.” That’s Forward Guidance.</p><p>It uses the guidance gradient to steer the denoising process toward the desired condition.</p><h4>Backward Guidance: Correct the Draft</h4><p>Sometimes giving directions isn’t enough. Instead, you take the draft, make a correction, and ask the artist to continue from there. Backward Guidance follows a similar idea. It optimizes the predicted clean image and maps that correction back into the diffusion process. This provides a more direct way to enforce the desired condition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NDW6YTIW8u2R3fG6j6zFZA.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><h4>Self-Recurrence: Try That Step Again</h4><p>Strong guidance can sometimes produce unnatural images. Self-Recurrence addresses this by revisiting a denoising step. The model denoises, adds some noise back, and denoises again. Think of an artist revisiting a sketch to improve the proportions before continuing. This can help balance realistic image generation with following the requested instructions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VBP7lOGVaRRzbcPD2BqShw.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><h3>What Can Universal Guidance Actually Do?</h3><p>The interesting part is how many different types of control can work with the same pretrained diffusion model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*auV6yrnqma3X_D5QvcgtsA.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><p>These don’t necessarily have to work independently. The framework can combine multiple guidance objectives.</p><p>For example, you could use <strong>Inpainting</strong> to modify a selected region, <strong>Classification Guidance</strong> to encourage a particular object category, and <strong>Segmentation Guidance</strong> to influence the layout.</p><p>It’s like having multiple specialists reviewing the same painting. Each checks something different, but all provide feedback to the same artist.</p><h3>Is There a Catch?</h3><p>Yes. Flexibility comes with additional computation. Every guidance network adds processing overhead. Backward Guidance and Self-Recurrence can make generation slower.</p><p>Guidance strength also matters. Too little guidance may fail to satisfy the condition, while too much can produce visual artifacts and the results depend on how reliable the external guidance networks are.</p><p>So Universal Guidance doesn’t guarantee perfect control. It offers a flexible alternative to training separate conditioning systems.</p><h3>Why This Paper Matters</h3><p>What I find interesting isn’t just the ability to control diffusion models. It’s the idea of <strong>separating generation from evaluation</strong>.</p><ol><li>CLIP already understands relationships between images and text.</li><li>Segmentation networks already understand image regions.</li><li>Face recognition models already understand identity similarity.</li></ol><p>Instead of retraining the diffusion model to learn all these capabilities, Universal Guidance uses them as external evaluators. The diffusion model focuses on generating images. The guidance networks help determine whether those images satisfy the requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A2QLjE-1mgQbA1O02zZfyg.png" /><figcaption>AI-generated illustration for conceptual understanding. Not an experimental result from the paper.</figcaption></figure><p><strong>The takeaway:</strong> Sometimes, we don’t need to teach an AI model a new capability. We just need to connect it to another model that already understands what we’re looking for.</p><h3>Reference</h3><p>Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein (2023). <strong>Universal Guidance for Diffusion Models.</strong></p><p>Original paper: <a href="https://arxiv.org/abs/2302.07121">https://arxiv.org/abs/2302.07121</a></p><p><em>This article explains the research through intuition and practical examples. For the complete methodology, mathematical formulations, and experimental results, refer to the original paper.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=344469a05ecd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding MultiDiffusion: How AI Creates Seamless Panoramas Without Retraining]]></title>
            <link>https://elanthirayan.medium.com/understanding-multidiffusion-how-ai-creates-seamless-panoramas-without-retraining-6b2ec9b826d6?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b2ec9b826d6</guid>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[research-paper]]></category>
            <category><![CDATA[ai-image-generator]]></category>
            <category><![CDATA[aipapersummary]]></category>
            <category><![CDATA[multidiffusion]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 09:02:55 GMT</pubDate>
            <atom:updated>2026-10-14T09:02:55.562Z</atom:updated>
            <content:encoded><![CDATA[<p>AI image generators can create impressive images from simple text prompts. But they are not always easy to control. Ask for a “beautiful landscape,” and you may get a good result. But ask for something more specific such as a wide panorama with a mountain on the left, a lake on the right, and a blue sky above and things become less predictable. This is the problem that MultiDiffusion tries to solve.</p><p>This article explains the key ideas from the original <a href="https://arxiv.org/abs/2302.08113">MultiDiffusion paper</a> in simple terms.</p><p>The central idea is simple: <em>Generate several overlapping parts of an image, make them agree with one another, and gradually combine them into one coherent image.</em></p><p>Let’s understand how this works without starting with equations.</p><h4>Why ordinary diffusion models struggle with large images</h4><p>Diffusion models such as Stable Diffusion are usually trained to generate images at particular sizes. They are very good at creating images within those boundaries, but generating a much wider image can be difficult.</p><p>Imagine asking the model to create a panorama showing:</p><ul><li>A castle on the left</li><li>A forest in the middle</li><li>A lake on the right</li><li>Mountains in the background</li></ul><p>One approach would be to generate each section separately and stitch the results together but this can produce something that looks more like a collection of unrelated images than one continuous scene. The lighting may change between sections. Objects may not line up. The horizon may suddenly move. Even the artistic style can vary from one crop to another.</p><p>The problem is not generating the individual parts. The problem is making them feel like they belong to the same image.</p><h4>The mural analogy</h4><p>Think of a large mural being painted by several artists. Each artist is responsible for one part of the wall. If they work independently, the transitions may look strange. One artist might paint the sky bright blue while the next uses an orange sunset. A road might stop suddenly at the edge of another artist’s section.</p><p>Now imagine that the artists work on overlapping parts of the mural and regularly compare their work. They can adjust their paintings so that the boundaries blend naturally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4dBxwQ75F9ucDE58viMMw.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p>MultiDiffusion works in a similar way. Instead of asking the model to generate the entire large image in one step, it looks at several overlapping crops. Each crop makes its own prediction, and those predictions are combined into a single global image.</p><h4>How MultiDiffusion works</h4><ol><li><strong>Generation begins with noise</strong></li></ol><p>Like other diffusion models, MultiDiffusion starts with random noise.At this stage, there is no clear mountain, lake, or castle. The image is gradually shaped over a series of denoising steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GwzW0Ys_SiAMZ0MXa7VHFA.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p><strong>2. The image is divided into overlapping crops</strong></p><p>The large canvas is viewed through several smaller windows.</p><p>For example:</p><pre>+----------+----------+----------+----------+<br>|  Crop 1  |  Crop 2  |  Crop 3  |  Crop 4  |<br>+----------+----------+----------+----------+</pre><p>The crops overlap with their neighbours. This overlap is important because it gives neighbouring regions a shared area where they can influence one another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3y68SOCR7NJJLVgA_WCqsQ.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p><strong>3. Each crop makes a prediction</strong></p><p>The pretrained diffusion model processes each crop and predicts how it should be denoised.</p><p>For example:</p><pre>Crop 1 → Mountain<br>Crop 2 → Mountain and trees<br>Crop 3 → Trees and river<br>Crop 4 → River and waterfall</pre><p>Each crop contributes useful information, but the predictions may not always agree perfectly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CL9NDn32qPWrzRwuPYDyWg.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p><strong>4. Conflicting predictions are combined</strong></p><p>Suppose one crop predicts a blue sky while another predicts an orange sky in their shared region.</p><p>MultiDiffusion does not simply choose one crop and ignore the other. Instead, it combines the predictions and finds a compromise that fits the overall image.</p><p>This fusion happens repeatedly throughout the denoising process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Z-D0BaBa4fVVb_M3j5FnA.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p><strong>5. The global image is updated</strong></p><p>The combined information from all the crops is used to update the full image.</p><pre>Crop predictions<br>        ↓<br>     Fusion<br>        ↓<br>Global image update</pre><p>The process continues until the noise gradually becomes a coherent image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h4_OEsjW2k4fPFLtTF2L-g.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><h4><strong>Why the overlap matters?</strong></h4><p>Without overlap, each crop has no direct connection to the next one.</p><p>The result might look like this:</p><pre>Sunny sky | Rainy sky | Sunset | Night</pre><p>With overlap, neighbouring crops share information:</p><pre>Sunny sky → softer transition → sunset</pre><p>The shared areas help the model maintain continuity in colour, lighting, shapes, and objects. This is what makes the final image feel like one scene rather than several images placed side by side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BtT1yUdcaWPEqBFQbZm9ZQ.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><h4><strong>Region-based text-to-image generation</strong></h4><p>MultiDiffusion can also be used to control different regions of an image with different prompts.</p><p>For example, we might define a simple layout:</p><pre>+----------------+<br>|      Sky       |<br>+----------------+<br>|    Mountain    |<br>+----------------+<br>|      Lake      |<br>+----------------+</pre><p>Then we can provide a prompt for each region:</p><pre>Sky      → Blue sky with soft clouds<br>Mountain → Snow-covered mountain<br>Lake     → Calm lake reflecting the mountains</pre><p>The model generates predictions for the different regions and fuses them into one image.</p><p>This gives the user more control over where objects and concepts appear, while still allowing the entire image to remain visually connected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7R4avR2Ovbhu1joSlP2jA.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><h4><strong>Bounding boxes and masks</strong></h4><p>MultiDiffusion can work with different kinds of spatial guidance.</p><p><strong>Bounding boxes</strong></p><p>A bounding box tells the model that an object should appear somewhere inside a rectangular area. This gives the model a general location, but it does not define the exact shape of the object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*drx3EKRUs4FSA4xX7Ol0rQ.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p><strong>Masks</strong></p><p>A mask provides a more precise shape or region. A mask tells the model to place the content within a specific shape. This can provide stronger spatial control than a simple rectangle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X1jRfDyIb2L3mSYrmz70RA.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><h4><strong>What is bootstrapping?</strong></h4><p>The paper also describes a technique called bootstrapping. Without bootstrapping, an object may appear somewhere inside the selected region but fail to fit the region particularly well. It may be too small, too large, or poorly aligned.</p><p>Bootstrapping provides an early spatial hint that helps the object settle into the intended area. For example, if we ask for a small object inside a narrow region, bootstrapping can help the model position and scale it more accurately before the rest of the image develops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hJipcGNAg1gKGK-yHHH3ig.png" /><figcaption>AI Generated Image for understanding purpose</figcaption></figure><p>In simple terms, it gives the generation process an early push in the right direction.</p><h4><strong>The main insight</strong></h4><p>Many improvements to image-generation models require additional training, fine-tuning, or a new network but MultiDiffusion takes a different approach. It keeps the original diffusion model and changes how the generation process is organised:</p><pre>Same pretrained model<br>        +<br>Overlapping predictions<br>        +<br>Repeated fusion<br>        ↓<br>More controlled image generation</pre><p>There is no need to retrain the original model. The control comes from coordinating multiple diffusion paths during generation.</p><h4><strong>Strengths</strong></h4><p>MultiDiffusion provides several useful advantages:</p><ul><li>It works with pretrained diffusion models.</li><li>It can generate images larger or wider than the model’s usual format.</li><li>It supports panorama generation.</li><li>It allows different prompts in different regions.</li><li>It can use bounding boxes and masks for spatial control.</li><li>It improves transitions between neighbouring image regions.</li><li>It avoids the need for model retraining.</li></ul><h4>Limitations</h4><p>MultiDiffusion does not completely replace the underlying diffusion model.</p><p>If the original model struggles to generate a particular object, style, or composition, MultiDiffusion may still struggle with it. Combining multiple predictions cannot automatically fix weaknesses in the base model.</p><p>There is also a practical trade-off: processing several crops and repeatedly fusing their predictions can increase the computational work required during generation.</p><h4>Final takeaway</h4><p>The most important idea behind MultiDiffusion is easy to remember:</p><blockquote><em>Instead of generating one large image in isolation, generate overlapping parts, encourage them to agree, and fuse them into one coherent result.</em></blockquote><p>This simple change gives a pretrained diffusion model more control over image layout, panoramas, and region-specific prompts without retraining the original model.</p><p>MultiDiffusion is a good example of how improving an AI system does not always require a bigger model. Sometimes, the most useful improvement comes from changing how an existing model is used.</p><h4>Paper Reference</h4><p>Bar-Tal, O., Yariv, L., Lipman, Y., &amp; Dekel, T. (2023). <a href="https://arxiv.org/abs/2302.08113">MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation.</a> arXiv:2302.08113.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b2ec9b826d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 5: MCP From Scratch — MCP vs Agent: What’s the Difference?]]></title>
            <link>https://elanthirayan.medium.com/part-5-mcp-from-scratch-mcp-vs-agent-whats-the-difference-c6f23372534c?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6f23372534c</guid>
            <category><![CDATA[mcp-client]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Sun, 16 Aug 2026 16:41:20 GMT</pubDate>
            <atom:updated>2026-08-16T16:41:20.793Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 4, something interesting happened.</p><p><a href="https://elanthirayan.medium.com/part-4-mcp-from-scratch-how-does-it-know-which-tool-to-use-154e969441b7?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-4-mcp-from-scratch-how-does-it-know-which-tool-to-use-154e969441b7</a></p><p>We asked: <em>Which department does John work in?</em></p><p>Our application looked at the available tools and chose:</p><pre>search_employee</pre><p>Then we asked: <em>Add Sarah to the IT department.</em></p><p>This time, it chose:</p><pre>create_employee</pre><p>So now you might be wondering: <strong><em>Isn’t this what an Agent does?</em></strong></p><p>Yes, an Agent can use tools too but <strong>MCP and an Agent are not the same thing. </strong>Let’s understand the difference.</p><h4>Remember Our USB Example?</h4><p>Throughout this series, we have compared MCP to USB.</p><p>A computer can connect to:</p><pre>Keyboard<br>Mouse<br>Webcam<br>Pen Drive</pre><p>using USB but USB does not decide: <em>I should use the keyboard now.</em></p><p>USB only provides a common way for the computer and device to communicate. MCP works in a similar way. It provides a common way to connect and use tools. It does not decide what your goal is or what steps should be taken. That is an important difference.</p><h4>Back to Our HR Example</h4><p>Our MCP server currently provides two tools:</p><pre>HR MCP Server<br>      │<br>      ├── search_employee<br>      │<br>      └── create_employee</pre><p>The MCP server is basically saying: <em>Here are the things I can do.</em></p><p>It does not decide when those things should happen.</p><h4>Then What Does an Agent Do?</h4><p>Think of an Agent as something that receives a goal and works out what needs to happen.</p><p>For example: <em>Find John and create a new employee in the same department as John.</em></p><p>This is no longer one simple action. There are multiple steps.</p><p>First:</p><pre>Find John</pre><p>Then:</p><pre>Find John&#39;s department</pre><p>Then:</p><pre>Create the new employee</pre><p>An Agent can work through these steps. Our MCP server simply provides the tools needed to do them.</p><h4>Let’s See the Difference</h4><p>We already have:</p><pre>search_employee</pre><p>and:</p><pre>create_employee</pre><p>Now imagine the request is: <em>Find John’s department and add David to the same department.</em></p><p>The Agent can work through it like this:</p><pre>Goal<br>↓<br>Find John&#39;s department<br>↓<br>Use search_employee<br>↓<br>John works in Finance<br>↓<br>Now add David to Finance<br>↓<br>Use create_employee<br>↓<br>David added to Finance</pre><p>Notice what happened. The MCP server did not create this plan.</p><p>It only provided:</p><pre>search_employee<br>create_employee</pre><p>The Agent decided:</p><pre>First use search_employee.<br>Take its result.<br>Then use create_employee.</pre><p>That is the main difference.</p><h4>MCP Gives the Tools</h4><p>Our MCP server says:</p><pre>I can search employees.<br>I can create employees.</pre><p>That’s its job. It provides the capabilities.</p><h4>Agent Uses the Tools to Reach a Goal</h4><p>The Agent receives:</p><pre>Find John&#39;s department<br>and add David to the same department.</pre><p>It needs to work out:</p><pre>What should I do first?<br>Which tool should I use?<br>What did that tool return?<br>What should I do next?<br>Am I finished?</pre><p>That is more than simply calling one tool.</p><h4>A Simple Way to See It</h4><p>Think about a toolbox. Inside the toolbox you have:</p><pre>Hammer<br>Screwdriver<br>Drill</pre><p>The toolbox provides the tools but the toolbox does not look at a broken chair and decide: <em>First I need the screwdriver, then I need the drill.</em></p><p>Someone using the toolbox makes those decisions. That is similar to MCP and Agents.</p><pre>MCP<br>↓<br>Provides tools<br>↓<br>Agent<br>↓<br>Decides how to use them</pre><h4>One Tool Call Does Not Automatically Mean Agent</h4><p>This is important.</p><p>In Part 4, we did this:</p><pre>Question<br>↓<br>Choose search_employee<br>↓<br>Call tool<br>↓<br>Get result</pre><p>Just because a model chose a tool does not mean we suddenly built a full Agent. An Agent usually has a bigger job.</p><p>It may need to:</p><pre>Understand a goal<br>↓<br>Choose an action<br>↓<br>Use a tool<br>↓<br>Check the result<br>↓<br>Choose another action<br>↓<br>Continue until the goal is complete</pre><p>That repeated decision-making is the important idea.</p><h4>MCP and Agent Can Work Together</h4><p>We don’t have to choose between MCP and Agents, they can work together.</p><p>Imagine our company has different MCP servers:</p><pre>HR MCP<br>├── search_employee<br>└── create_employee<br><br>IT MCP<br>├── create_ticket<br>└── check_ticket<br><br>Document MCP<br>├── search_document<br>└── read_document</pre><p>An Agent can use the tools provided by these MCP servers.</p><p>For example: <em>Find John’s department, check the laptop policy for his department, and create an IT ticket if he needs a new laptop.</em></p><p>Now several things may need to happen:</p><pre>Agent<br>  │<br>  ├── HR MCP<br>  │      ↓<br>  │   Find John<br>  │<br>  ├── Document MCP<br>  │      ↓<br>  │   Find laptop policy<br>  │<br>  └── IT MCP<br>         ↓<br>      Create ticket</pre><p>The Agent handles the goal and MCP servers provide access to the tools.</p><h4>Back to USB One Last Time</h4><p>Let’s return to where this series started. USB gives your computer a common way to communicate with many devices.</p><pre>Computer<br>   │<br>  USB<br>   │<br>   ├── Keyboard<br>   ├── Mouse<br>   └── Webcam</pre><p>MCP gives an application a common way to communicate with many tools.</p><pre>Application<br>     │<br>    MCP<br>     │<br>     ├── HR Tools<br>     ├── IT Tools<br>     └── Document Tools</pre><p>An Agent can sit above this and decide how those tools should be used to complete a goal.</p><pre>              User Goal<br>                 ↓<br>               Agent<br>                 ↓<br>                MCP<br>          ┌──────┼──────┐<br>          ↓      ↓      ↓<br>         HR      IT   Documents</pre><h4>The Easy Way to Remember</h4><p>If you remember only two lines from this article, remember these: <strong><em>MCP provides the tools. Agent decides how to use the tools to complete a goal.</em></strong></p><p>MCP is the connection. Agent is the decision maker.</p><p>They solve different problems, but they can work together.</p><p>The systems may change. The tools may change. but the idea behind MCP stays simple: <strong><em>Give applications one common way to discover and use what other systems can do.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6f23372534c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 4: MCP From Scratch — How Does It Know Which Tool to Use?]]></title>
            <link>https://elanthirayan.medium.com/part-4-mcp-from-scratch-how-does-it-know-which-tool-to-use-154e969441b7?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/154e969441b7</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[mcp-protocol]]></category>
            <category><![CDATA[mcp-client]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 04:50:18 GMT</pubDate>
            <atom:updated>2026-08-16T16:42:05.098Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 3, we connected our client to the MCP server and called:</p><p><a href="https://elanthirayan.medium.com/part-3-mcp-from-scratch-lets-connect-to-our-mcp-server-6280143bba00?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-3-mcp-from-scratch-lets-connect-to-our-mcp-server-6280143bba00</a></p><pre>session.call_tool(<br>    &quot;search_employee&quot;,<br>    {&quot;name&quot;: &quot;John&quot;}<br>)</pre><p>It worked but there was something important hiding in that code. <strong>We chose the tool ourselves.</strong></p><p>We wrote:</p><pre>search_employee</pre><p>directly in our code but what if someone simply asks: <em>Which department does John work in?</em></p><p>How does the application know that it should call search_employee?</p><p>Let’s find out.</p><h4>Let’s Add One More Tool</h4><p>Right now, our HR server has:</p><pre>search_employee</pre><p>Let’s add another tool:</p><pre>create_employee</pre><p>Our server now has two things it can do:</p><pre>HR MCP Server<br>      │<br>      ├── search_employee<br>      │<br>      └── create_employee</pre><p>Adding create_employee to hr_server.py:</p><pre>@mcp.tool()<br>def create_employee(name: str, department: str) -&gt; str:<br>    &quot;&quot;&quot;Add a new employee to the company.&quot;&quot;&quot;<br>  EMPLOYEES.append({<br>        &quot;name&quot;: name,<br>        &quot;department&quot;: department<br>    })<br>    return f&quot;{name} added to {department}&quot;</pre><p>Now we have two tools.</p><h3>How Does Someone Know What These Tools Do?</h3><p>Look closely at this:</p><pre>@mcp.tool()<br>def search_employee(name: str) -&gt; list:<br>    &quot;&quot;&quot;Find an employee by name.&quot;&quot;&quot;</pre><p>There are three useful pieces of information here:</p><pre>Name<br>↓<br>search_employee<br>Description<br>↓<br>Find an employee by name.<br>Input<br>↓<br>name</pre><p>The same applies to our second tool:</p><pre>Name<br>↓<br>create_employee<br>Description<br>↓<br>Add a new employee to the company.<br>Inputs<br>↓<br>name<br>department</pre><p>Our MCP server can share this information.</p><h4>Let’s Ask the Server What It Can Do</h4><p>Instead of directly calling a tool, let’s first ask: <strong><em>What tools do you have?</em></strong></p><p>In our client, listing the available tools:</p><pre>await session.initialize()<br>tools = await session.list_tools()<br>for tool in tools.tools:<br>    print(tool.name)<br>    print(tool.description)</pre><p>Run:</p><pre>python hr_client.py</pre><p>You should see:</p><pre>search_employee<br>Find an employee by name.<br>create_employee<br>Add a new employee to the company.</pre><p>Something important just happened. The client did not keep its own list of tools. It asked the MCP server: <em>What can you do?</em></p><p>The server answered with its available tools. This is called <strong>tool discovery</strong>.</p><h4>Back to Our USB Example</h4><p>Remember our USB example? When you connect a keyboard, the computer can understand what kind of device is connected and what it can do. Our MCP example follows a similar idea. The client connects and asks: <em>What can you do?</em></p><p>The MCP server answers:</p><pre>search_employee<br>→ Find an employee by name<br>create_employee<br>→ Add a new employee</pre><p>Now the client knows what is available but we still have one problem.</p><p><strong>Who chooses between them?</strong></p><h4>Let’s Make the Tool Selection Work</h4><p>Until now, we manually wrote:</p><pre>session.call_tool(<br>    &quot;search_employee&quot;,<br>    {&quot;name&quot;: &quot;John&quot;}<br>)</pre><p>Let’s change that.</p><p>We want to type:</p><pre>Which department does John work in?</pre><p>and let a language model choose the correct tool. For this example, we will use the OpenAI Python library.</p><p>Install it:</p><pre>pip install openai</pre><p>You will also need your API key set as an environment variable.</p><p>For macOS or Linux:</p><pre>export OPENAI_API_KEY=&quot;your-api-key&quot;</pre><p>For Windows PowerShell:</p><pre>$env:OPENAI_API_KEY=&quot;your-api-key&quot;</pre><p>Now let’s update our client.</p><h3>Our New Client</h3><p>Replace hr_client.py with:</p><pre>import asyncio<br>import json<br><br>from openai import OpenAI<br>from mcp import ClientSession, StdioServerParameters<br>from mcp.client.stdio import stdio_client<br><br>client = OpenAI()<br><br>async def main():<br>    server = StdioServerParameters(<br>        command=&quot;python&quot;,<br>        args=[&quot;hr_server.py&quot;]<br>    )<br>    async with stdio_client(server) as (read, write):<br>        async with ClientSession(read, write) as session:<br>            await session.initialize()<br>            # Ask MCP server what tools are available<br>            mcp_tools = await session.list_tools()<br>            # Convert MCP tools into a format the model can understand<br>            tools = []<br>            for tool in mcp_tools.tools:<br>                tools.append({<br>                    &quot;type&quot;: &quot;function&quot;,<br>                    &quot;name&quot;: tool.name,<br>                    &quot;description&quot;: tool.description or &quot;&quot;,<br>                    &quot;parameters&quot;: tool.inputSchema<br>                })<br>            question = &quot;Add sarah to IT department and then list all the employees and their departments&quot;<br>            # Give the question and available tools to the model<br>            response = client.responses.create(<br>                model=&quot;gpt-5-mini&quot;,<br>                input=question,<br>                tools=tools<br>            )<br>            # Find the tool the model selected<br>            for item in response.output:<br>                if item.type == &quot;function_call&quot;:<br>                    tool_name = item.name<br>                    tool_args = json.loads(item.arguments)<br>                    print(&quot;Selected tool:&quot;, tool_name)<br>                    print(&quot;Input:&quot;, tool_args)<br>                    # Call the selected MCP tool<br>                    result = await session.call_tool(<br>                        tool_name,<br>                        tool_args<br>                    )<br>                    print(&quot;Result:&quot;, result.content[0].text)<br><br>asyncio.run(main())</pre><p>Don’t worry about every line. Let’s look only at what is happening.</p><h4>Step 1: Get the MCP Tools</h4><p>First:</p><pre>mcp_tools = await session.list_tools()</pre><p>Our client asks the MCP server: <em>What can you do?</em></p><p>The server returns:</p><pre>search_employee<br>create_employee</pre><p>along with their descriptions and inputs.</p><h4>Step 2: Give Those Tools to the Model</h4><p>Next, we give the question and available tools to the model:</p><pre>response = client.responses.create(<br>    model=&quot;gpt-5-mini&quot;,<br>    input=question,<br>    tools=tools<br>)</pre><p>Our question is:</p><pre>Which department does John work in?</pre><p>The model can also see:</p><pre>search_employee<br>→ Find an employee by name<br>create_employee<br>→ Add a new employee to the company</pre><p>It now has enough information to choose.</p><h4>Step 3: See What It Chose</h4><p>We read the tool call:</p><pre>tool_name = item.name<br>tool_args = json.loads(item.arguments)</pre><p>For our question, we expect something similar to:</p><pre>Selected tool: search_employee<br>Input: {&#39;name&#39;: &#39;John&#39;}</pre><p>Notice what changed.</p><p>We never wrote:</p><pre>session.call_tool(<br>    &quot;search_employee&quot;,<br>    {&quot;name&quot;: &quot;John&quot;}<br>)</pre><p>The tool name and input now come from the model:</p><pre>session.call_tool(<br>    tool_name,<br>    tool_args<br>)</pre><p>That is the important part.</p><h4>Let’s Run It</h4><p>Make sure you have:</p><pre>mcp-hr<br>│<br>├── hr_server.py<br>└── hr_client.py</pre><p>Now run:</p><pre>python hr_client.py</pre><p>You should see something similar to:</p><pre>Selected tool: search_employee<br>Input: {&#39;name&#39;: &#39;John&#39;}<br><br>Result: [{&quot;name&quot;:&quot;John&quot;,&quot;department&quot;:&quot;Finance&quot;}]</pre><p>Now try changing:</p><pre>question = &quot;Which department does John work in?&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AyO21L3zOQ3byv3Fh4wc7g.png" /></figure><p>to:</p><pre>question = &quot;Add Sarah to the IT department&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NNApc2tdrb5uLNMJ1NT5bw.png" /></figure><p>Run it again.</p><p>This time, you should see something similar to:</p><pre>Selected tool: create_employee<br>Input: {&#39;name&#39;: &#39;Sarah&#39;, &#39;department&#39;: &#39;IT&#39;}<br><br>Result: Sarah added to IT</pre><p>Same client, same MCP server but a different question resulted in a different tool.</p><h4>Now We Can See the Full Picture</h4><p>Our program is finally doing this:</p><pre>&quot;Which department does John work in?&quot;<br>                ↓<br>        Available MCP Tools<br>                ↓<br>         Language Model<br>                ↓<br>        search_employee<br>                ↓<br>          MCP Client<br>                ↓<br>          MCP Server<br>                ↓<br>     search_employee(&quot;John&quot;)<br>                ↓<br>        John → Finance</pre><p>Now try:</p><pre>&quot;Add Sarah to IT&quot;</pre><p>The path changes:</p><pre>&quot;Add Sarah to IT&quot;<br>        ↓<br>Available MCP Tools<br>        ↓<br> Language Model<br>        ↓<br> create_employee<br>        ↓<br>   MCP Client<br>        ↓<br>   MCP Server<br>        ↓<br>create_employee(<br>    &quot;Sarah&quot;,<br>    &quot;IT&quot;<br>)</pre><p>This is the part that was missing from our previous example.</p><h4>Why Are Tool Descriptions Important?</h4><p>Imagine our server had:</p><pre>tool1<br>tool2</pre><p>with no useful descriptions, it would be much harder to know what they are for.</p><p>Compare that with:</p><pre>search_employee<br>→ Find an existing employee and their department by name.<br>create_employee<br>→ Add a new employee to the company.</pre><p>Now the purpose is much clearer. That is why good tool names and descriptions matter.</p><h3>The Easy Way to Remember</h3><p>The MCP server says: <strong><em>Here is what I can do.</em></strong></p><p>The language model says: <strong><em>This tool matches the request.</em></strong></p><p>The MCP client says: <strong><em>I’ll call that tool.</em></strong></p><p>The MCP server runs it and returns the result.</p><pre>User Request<br>     ↓<br>Available Tools<br>     ↓<br>Language Model<br>     ↓<br>Selected Tool<br>     ↓<br>MCP Client<br>     ↓<br>MCP Server<br>     ↓<br>Result</pre><h4>What’s Next?</h4><p>Now our program can understand a request, choose a tool, and perform an action. That brings up an interesting question: <strong><em>Isn’t this an Agent?</em></strong></p><p>And if it is, where does MCP fit?</p><p>In <strong>MCP From Scratch — Part 5</strong>, we’ll understand:</p><p><strong>MCP vs Agent — What’s the Difference?</strong></p><p><a href="https://elanthirayan.medium.com/part-5-mcp-from-scratch-mcp-vs-agent-whats-the-difference-c6f23372534c?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-5-mcp-from-scratch-mcp-vs-agent-whats-the-difference-c6f23372534c</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=154e969441b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 3: MCP From Scratch — Let’s Connect to Our MCP Server]]></title>
            <link>https://elanthirayan.medium.com/part-3-mcp-from-scratch-lets-connect-to-our-mcp-server-6280143bba00?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/6280143bba00</guid>
            <category><![CDATA[mcp-client]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[llmops]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Thu, 13 Aug 2026 05:14:14 GMT</pubDate>
            <atom:updated>2026-08-14T04:51:04.306Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 2, we created our first MCP server.</p><p><a href="https://elanthirayan.medium.com/part-2-mcp-from-scratch-lets-build-our-first-mcp-server-af14f2df41bf?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-2-mcp-from-scratch-lets-build-our-first-mcp-server-af14f2df41bf</a></p><p>Our server had one simple job:</p><pre>HR MCP Server<br>      ↓<br>search_employee</pre><p>We also started it using:</p><pre>python hr_server.py</pre><p>But we stopped with one question: <strong><em>Who is going to talk to our MCP server?</em></strong></p><p>That is called a <strong>client</strong>.</p><pre>Client<br>   ↓<br>MCP<br>   ↓<br>HR MCP Server</pre><p>Don’t worry about the word “client.” For now, simply remember: <strong><em>Server provides something. Client connects to it.</em></strong></p><h4>Let’s Create Our Client</h4><p>In the same folder as hr_server.py, create another file:</p><pre>hr_client.py</pre><p>Our folder should now look like:</p><pre>mcp-hr<br>│<br>├── hr_server.py<br>│<br>└── hr_client.py</pre><p>We already built the server. Now let’s work only on the client.</p><h3>Connect to Our Server</h3><p>Add this code to hr_client.py:</p><pre>import asyncio<br><br>from mcp import ClientSession, StdioServerParameters<br>from mcp.client.stdio import stdio_client<br><br>async def main():<br>    server = StdioServerParameters(<br>        command=&quot;python&quot;,<br>        args=[&quot;hr_server.py&quot;]<br>    )<br>    async with stdio_client(server) as (read, write):<br>        async with ClientSession(read, write) as session:<br>            await session.initialize()<br><br>asyncio.run(main())<br></pre><p>This may look like a lot of code but don’t focus on every line yet.</p><p>The important part is this:</p><pre>server = StdioServerParameters(<br>    command=&quot;python&quot;,<br>    args=[&quot;hr_server.py&quot;]<br>)</pre><p>We are telling the client: <em>Connect to </em><em>hr_server.py</em></p><p>The client starts our server and opens a connection to it.</p><h4>Now Let’s Call Our Tool</h4><p>Remember the tool we created in Part 2?</p><pre>search_employee</pre><p>Let’s search for John.</p><pre>await session.initialize()<br>result = await session.call_tool(<br>    &quot;search_employee&quot;,<br>    {&quot;name&quot;: &quot;John&quot;}<br>)<br><br>print(result.content[0].text)</pre><p>Now our client is saying: <em>Call </em><em>search_employee and search for John.</em></p><p>Our complete client becomes:</p><pre>import asyncio<br><br>from mcp import ClientSession, StdioServerParameters<br>from mcp.client.stdio import stdio_client<br><br>async def main():<br>    server = StdioServerParameters(<br>        command=&quot;python3&quot;,<br>        args=[&quot;hr_server.py&quot;]<br>    )<br>    async with stdio_client(server) as (read, write):<br>        async with ClientSession(read, write) as session:<br>            await session.initialize()<br>            result = await session.call_tool(<br>                &quot;search_employee&quot;,<br>                {&quot;name&quot;: &quot;John&quot;}<br>            )<br>            print(result.content[0].text)<br><br>asyncio.run(main())</pre><p>That’s our client.</p><h4>Let’s Run It</h4><p>Open your terminal.</p><p>Now run:</p><pre>python hr_client.py</pre><p>Notice something important. You don’t need to run hr_server.py separately this time.</p><p>Our client starts it using:</p><pre>command=&quot;python&quot;,<br>args=[&quot;hr_server.py&quot;]</pre><p>Then it connects to the server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R4-LkanrCNcQB-QkDW7N1w.png" /></figure><h4>What Happens?</h4><p>Our client sends:</p><pre>Tool:<br>  search_employee<br>Name:<br>  John</pre><p>The server receives it and server runs:</p><pre>search_employee(&quot;John&quot;)</pre><p>It finds:</p><pre>John → Finance</pre><p>Then it sends the result back to the client.</p><p>So the complete journey is:</p><pre>hr_client.py<br>     │<br>     │  search_employee(&quot;John&quot;)<br>     ↓<br>hr_server.py<br>     │<br>     ↓<br>Search employee list<br>     │<br>     ↓<br>John → Finance<br>     │<br>     ↓<br>hr_client.py</pre><p>We now have both sides working.</p><h3>Back to Our USB Example</h3><p>Think about connecting a keyboard to a computer.</p><pre>Keyboard<br>   ↓<br>USB<br>   ↓<br>Computer</pre><p>Both sides agree on how to communicate.</p><p>The client knows how to communicate using MCP. The server knows how to communicate using MCP. That common connection is what makes them work together.</p><h4>But Did It Choose the Tool?</h4><p>Here is something very important.</p><p>Look at our code again:</p><pre>session.call_tool(<br>    &quot;search_employee&quot;,<br>    {&quot;name&quot;: &quot;John&quot;}<br>)</pre><p>Who chose search_employee? <strong>We did. </strong>We wrote the tool name directly in our code. Our client did not look at the request: <em>Does John work in Finance? </em>and decide what to do.</p><p>We told it exactly what to call. So there is still one missing piece.</p><h4>What’s Next?</h4><p>Imagine our MCP server has many tools:</p><pre>search_employee<br>create_employee<br>get_leave_balance<br>list_departments</pre><p>Now someone asks: <em>Which department does John work in?</em></p><p>How does the application know that it should use:</p><pre>search_employee</pre><p>instead of the other tools?</p><p>That is where <strong>tool names, descriptions, and available tools</strong> become important.</p><p>In <strong>MCP From Scratch — Part 4</strong>, we will understand: <strong><em>How does an application know which MCP tool to use?</em></strong></p><p><a href="https://elanthirayan.medium.com/part-4-mcp-from-scratch-how-does-it-know-which-tool-to-use-154e969441b7?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-4-mcp-from-scratch-how-does-it-know-which-tool-to-use-154e969441b7</a></p><p>That is where our simple MCP example becomes much more interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6280143bba00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 2: MCP From Scratch — Let’s Build Our First MCP Server]]></title>
            <link>https://elanthirayan.medium.com/part-2-mcp-from-scratch-lets-build-our-first-mcp-server-af14f2df41bf?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/af14f2df41bf</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mcp-client]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Wed, 12 Aug 2026 04:46:37 GMT</pubDate>
            <atom:updated>2026-08-13T05:14:48.373Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 1, we compared MCP to <strong>USB</strong>.</p><p><a href="https://elanthirayan.medium.com/what-is-mcp-the-secret-that-helps-ai-talk-to-other-apps-f9fd49150a28?sharedUserId=elanthirayan">https://elanthirayan.medium.com/what-is-mcp-the-secret-that-helps-ai-talk-to-other-apps-f9fd49150a28</a></p><p>A keyboard, mouse, webcam, and pen drive are all different devices but they can use the same USB connection, that makes things simple. Now let’s take the next step. <strong>What does an MCP server actually look like? </strong>And can we build one ourselves? Yes.</p><p>Let’s build a very small one.</p><h4>Remember Our USB Example?</h4><p>Imagine plugging a keyboard into your computer. The keyboard can do certain things. It can type letters. It can send key presses.</p><p>A mouse can do different things. It can move the pointer. It can click.</p><p>Each device has something it can offer. An MCP server works in a similar way. It tells other applications: <strong><em>Here are the things I can do.</em></strong><em> </em>Let’s see that with a simple example.</p><h4>Our Small HR System</h4><p>Imagine we have a company with a few employees.</p><pre>John  → Finance<br>Alice → IT<br>Bob   → HR<br>Ela   → Marketing</pre><p>We want to create an MCP server for this HR system. Let’s start with just one thing: <strong><em>Search for an employee</em></strong></p><h4>Create Our MCP Server</h4><p>Create a file called:</p><pre>hr_server.py</pre><p>First, install MCP:</p><pre>pip install mcp</pre><p>Now add:</p><pre>from mcp.server.fastmcp import FastMCP</pre><pre>mcp = FastMCP(&quot;HR System&quot;)</pre><p>That’s our MCP server.</p><p>We simply created one and called it:</p><pre>HR System</pre><p>Right now, it doesn’t offer anything. So let’s give it something useful to do.</p><h4>Add Some Employees</h4><p>For now, we don’t need a database. We’ll keep a small list inside our program.</p><pre>EMPLOYEES = [<br>    {&quot;name&quot;: &quot;John&quot;, &quot;department&quot;: &quot;Finance&quot;},<br>    {&quot;name&quot;: &quot;Alice&quot;, &quot;department&quot;: &quot;IT&quot;},<br>    {&quot;name&quot;: &quot;Bob&quot;, &quot;department&quot;: &quot;HR&quot;},<br>    {&quot;name&quot;: &quot;Ela&quot;, &quot;department&quot;: &quot;Marketing&quot;}<br>]</pre><p>That’s our tiny HR system.</p><h4>Let It Search Employees</h4><p>Now let’s create a function:</p><pre>def search_employee(name: str):<br>    return [<br>        employee<br>        for employee in EMPLOYEES<br>        if name.lower() in employee[&quot;name&quot;].lower()<br>    ]</pre><p>This is just a normal Python function.</p><p>Give it a name:</p><pre>John</pre><p>and it searches our employee list but there is a problem. This is still just a normal function inside our Python file.</p><p>We want our <strong>MCP server to offer it</strong>.</p><p>For that, we add one line:</p><pre>@mcp.tool()</pre><p>So our function becomes:</p><pre>@mcp.tool()<br>def search_employee(name: str):<br>    return [<br>        employee<br>        for employee in EMPLOYEES<br>        if name.lower() in employee[&quot;name&quot;].lower()<br>    ]</pre><p>Now something important has happened.</p><p>Our MCP server can say:<strong> <em>I can search for employees.</em></strong></p><p>In MCP, something the server can offer like this is called a <strong>tool</strong>.</p><h4>Back to Our USB Example</h4><p>Think about our keyboard again.</p><pre>Keyboard<br>   ↓<br>Can send key presses<br>   ↓<br>USB<br>   ↓<br>Computer</pre><p>Now look at our HR example.</p><pre>HR System<br>   ↓<br>Can search employees<br>   ↓<br>MCP<br>   ↓<br>AI applications</pre><p>The HR system and keyboard are obviously very different things but the idea is similar.</p><p>USB gives devices a common way to communicate.</p><p>MCP gives applications a common way to offer what they can do.</p><h4>Start the Server</h4><p>Finally, add:</p><pre>if __name__ == &quot;__main__&quot;:<br>    mcp.run()</pre><p>Our first MCP server is ready.</p><p>Here is the complete code:</p><pre>from mcp.server.fastmcp import FastMCP<br><br>mcp = FastMCP(&quot;HR System&quot;)<br><br>EMPLOYEES = [<br>    {&quot;name&quot;: &quot;John&quot;, &quot;department&quot;: &quot;Finance&quot;},<br>    {&quot;name&quot;: &quot;Alice&quot;, &quot;department&quot;: &quot;IT&quot;},<br>    {&quot;name&quot;: &quot;Bob&quot;, &quot;department&quot;: &quot;HR&quot;},<br>    {&quot;name&quot;: &quot;Ela&quot;, &quot;department&quot;: &quot;Marketing&quot;}<br>]<br><br>@mcp.tool()<br>def search_employee(name: str):<br>    &quot;&quot;&quot;Find an employee by name.&quot;&quot;&quot;<br>    return [<br>        employee<br>        for employee in EMPLOYEES<br>        if name.lower() in employee[&quot;name&quot;].lower()<br>    ]<br><br>if __name__ == &quot;__main__&quot;:<br>    mcp.run()</pre><p>Now we have created our first MCP server.</p><h4>The Easy Way to Remember</h4><p>Go back to USB. A USB device tells the computer what it can do.</p><p>In our example, the MCP server also has something it can offer:</p><pre>HR MCP Server<br>      ↓<br>&quot;I can search employees&quot;<br>      ↓<br>search_employee</pre><h4>Let’s Start Our MCP Server</h4><p>Our code is ready. Now open your terminal and go to the folder where you created: <em>hr_server.py</em></p><p>Run:</p><pre>python hr_server.py</pre><p>Our MCP server is now running. It is waiting for something to connect and use what it offers. You may notice something strange. It might look like nothing is happening. That is completely fine.</p><p>Remember, we created a <strong>server</strong>.</p><p>The server’s job is to wait for someone to connect and ask it to do something.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6hVI3O4nxI_9SHSCOk9hbA.png" /></figure><p>We now have a working MCP server, But something is still missing. Our server is running and waiting.</p><p><strong>Who is going to talk to it?</strong></p><p>In <strong>MCP From Scratch — Part 3</strong>, we will create a small client and connect it to the MCP server.</p><p><a href="https://elanthirayan.medium.com/part-3-mcp-from-scratch-lets-connect-to-our-mcp-server-6280143bba00?sharedUserId=elanthirayan">https://elanthirayan.medium.com/part-3-mcp-from-scratch-lets-connect-to-our-mcp-server-6280143bba00</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af14f2df41bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker from Scratch — Part 5: Running Multiple Containers with Docker Compose]]></title>
            <link>https://elanthirayan.medium.com/docker-from-scratch-part-5-running-multiple-containers-with-docker-compose-954def6b82ca?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/954def6b82ca</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[docker-image]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Tue, 11 Aug 2026 05:42:16 GMT</pubDate>
            <atom:updated>2026-08-11T05:42:16.207Z</atom:updated>
            <content:encoded><![CDATA[<p>So far, we have worked with one Container at a time but imagine an application that needs:</p><pre>Backend Application<br>Database</pre><p>Now we have two Containers that need to run together, we could start and configure each Container manually but as the number of Containers grows, managing them this way becomes difficult.</p><p>This is the problem <strong>Docker Compose</strong> solves.</p><h4>What Is Docker Compose?</h4><p>Docker Compose lets us describe all the parts of an application in <strong>one file</strong>.</p><p>For example:</p><pre>services:<br>  app:<br>  database:</pre><p>This tells Docker: <strong><em>My application has two parts: an app and a database.</em></strong></p><p>Each part is called a <strong>Service, </strong>instead of starting each one separately, Compose manages them together.</p><h4>The Compose File</h4><p>A simple project might look like:</p><pre>my-project/<br>├── Dockerfile<br>├── compose.yaml<br>├── requirements.txt<br>└── app.py</pre><p>Inside compose.yaml:</p><pre>services:<br>  app:<br>    build: .<br>  database:<br>    image: postgres:16</pre><p>There are two interesting things here.</p><p>For our application, we use:</p><pre>build: .</pre><p>This means: <strong><em>Build the application using the Dockerfile in this project.</em></strong></p><p>For PostgreSQL, we use:</p><pre>image: postgres:16</pre><p>This means: <strong><em>Use an already available PostgreSQL Image.</em></strong></p><p>So:</p><pre>build → Build our own Image<br><br>image → Use an existing Image</pre><h4>Starting Everything</h4><p>Instead of starting each Container manually, we can run:</p><pre>docker compose up</pre><p>Compose reads compose.yaml and starts the required Containers.</p><p>To stop them:</p><pre>docker compose down</pre><p>This is one of the biggest conveniences of Docker Compose.</p><h4>Ports</h4><p>Suppose our application runs on port 8000 inside its Container. We want to open it from our computer.</p><p>We add:</p><pre>services:<br>  app:<br>    build: .<br>    ports:<br>      - &quot;8000:8000&quot;</pre><p>Think of this as:</p><pre>Your Computer<br>localhost:8000<br>      ↓<br>Container<br>port 8000</pre><p>Now we can open:</p><pre>http://localhost:8000</pre><p>and reach the application running inside Docker.</p><h4>How Do Containers Find Each Other?</h4><p>Now imagine our application needs to connect to the database.</p><p>Our Compose file contains:</p><pre>services:<br>  app:<br>  database:</pre><p>Docker Compose automatically creates a network for these Services more importantly, each Service can be reached using its <strong>name</strong>. Our application doesn’t need to know the database Container’s IP address.</p><p>It can simply connect to:</p><pre>database</pre><p>because that is the Service name.</p><p>So:</p><pre>app<br> │<br> ▼<br>database</pre><p>Docker handles the connection between them.</p><h4>Environment Variables</h4><p>Our application may need configuration such as:</p><pre>DATABASE_HOST<br>DATABASE_USER<br>DATABASE_NAME</pre><p>We don’t want these values scattered throughout our application code.</p><p>Compose can provide them:</p><pre>services:<br>  app:<br>    build: .<br>    environment:<br>      DATABASE_HOST: database<br>      DATABASE_NAME: myapp</pre><p>The application can then read them from its environment.</p><p>For example, in Python:</p><pre>import os<br><br>database_host = os.getenv(&quot;DATABASE_HOST&quot;)</pre><p>This allows the same application code to run with different settings in different environments.</p><h4>Putting It Together</h4><p>Our Compose file might now look like:</p><pre>services:<br>  app:<br>    build: .<br>    ports:<br>      - &quot;8000:8000&quot;<br>    environment:<br>      DATABASE_HOST: database<br>  database:<br>    image: postgres:16</pre><p>The relationship is:</p><pre>Browser<br>   │<br>   ▼<br>localhost:8000<br>   │<br>   ▼<br>App Container<br>   │<br>   ▼<br>Database Container</pre><p>Compose manages both Containers and the network between them.</p><h4>Running a One-Time Task</h4><p>Some Containers are meant to stay running.</p><p>For example:</p><pre>API<br>Database</pre><p>Others may only need to run once.</p><p>For example:</p><pre>Run Tests<br>Generate Report<br>Train Model</pre><p>For a one-time task, Compose provides:</p><pre>docker compose run &lt;service&gt;</pre><p>For services that should stay running, we normally use:</p><pre>docker compose up</pre><p>A simple way to remember:</p><pre>up  → Start the application/services<br><br>run → Execute a one-time task</pre><h4>Sharing Your Docker Image</h4><p>Until now, the Images we build exist on our computer but what if another developer or a server needs the same Image? Docker Images can be stored in an <strong>Image Registry</strong>. Docker Hub is one example.</p><p>The basic flow is:</p><pre>Build Image<br>    ↓<br>Push Image<br>    ↓<br>Image Registry<br>    ↓<br>Another Computer<br>    ↓<br>Pull Image<br>    ↓<br>Run Container</pre><p>This allows teams to build an Image once and use that same Image elsewhere. That is much better than rebuilding the application differently on every machine.</p><h4>The Docker Journey</h4><p>Across this series, we started with a simple problem: <strong><em>Why does an application work on one machine but fail on another?</em></strong></p><p>From there, we learned how Docker approaches the problem.</p><p>The complete picture is now:</p><pre>Dockerfile<br>    ↓<br>Image<br>    ↓<br>Container<br>    ↓<br>Persistent Data when needed<br>    ↓<br>Docker Compose<br>    ↓<br>Multiple Containers working together<br>    ↓<br>Image Registry<br>    ↓<br>Share the same Image with others</pre><p>You don’t need to memorize every Docker command. The important thing is understanding <strong>why each piece exists</strong>.</p><h4>Final Takeaways</h4><p>If you’re starting with Docker, remember these core ideas:</p><ul><li>A Dockerfile describes how an application should be packaged.</li><li>An Image is the packaged application.</li><li>A Container is a running instance of that Image.</li><li>Volumes keep important data outside replaceable Containers.</li><li>Docker Compose manages applications that need multiple Containers.</li><li>Services can communicate using their Service names.</li><li>Ports allow us to access applications from outside Docker.</li><li>Environment variables keep environment-specific settings outside the code.</li><li>Image Registries allow Docker Images to be shared.</li></ul><p>With these concepts, you have the foundation needed to understand and work with Docker in most development projects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=954def6b82ca" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker from Scratch — Part 4: Where Does Your Data Go?]]></title>
            <link>https://elanthirayan.medium.com/docker-from-scratch-part-4-where-does-your-data-go-08a125c75849?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/08a125c75849</guid>
            <category><![CDATA[docker-image]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[docker-volume]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 04:50:30 GMT</pubDate>
            <atom:updated>2026-08-11T05:43:06.868Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 3, we built our first Docker Image and ran a Container.</p><p><a href="https://elanthirayan.medium.com/docker-from-scratch-part-3-writing-your-first-dockerfile-a920c0516e81?sharedUserId=elanthirayan">https://elanthirayan.medium.com/docker-from-scratch-part-3-writing-your-first-dockerfile-a920c0516e81</a></p><p>We saw this simple flow:</p><pre>Dockerfile → Image → Container</pre><p>At that point, everything felt complete but now a new question appears naturally: <strong><em>If a Container runs an application, where does the data go?</em></strong></p><p>Because real applications don’t just run and stop.</p><p>They create things like:</p><ul><li>reports</li><li>database records</li><li>logs</li><li>experiment results</li><li>uploaded files</li></ul><p>So we need to understand what happens to this data.</p><h4>The Important Realization</h4><p>A Container is not meant to be permanent, you can think of it like a workspace. You open it, do your work, and close it when you’re done. If you open a new workspace later, it starts fresh again. This means something very important: <strong><em>If your data lives only inside a Container, it can disappear when the Container is removed.</em></strong></p><p>This is where many beginners get confused, the application is not the problem. The Container is just temporary by design.</p><h4>Separating Application and Data</h4><p>To understand Docker properly, we need to separate two things:</p><h4>1. The application</h4><p>This includes things like:</p><ul><li>Python</li><li>your code</li><li>libraries</li><li>configuration</li></ul><p>This belongs in the <strong>Image</strong>.</p><h4>2. The data</h4><p>This includes things like:</p><ul><li>database records</li><li>ML experiments</li><li>logs</li><li>uploaded files</li><li>generated reports</li></ul><p>This should not depend on a single Container. It should live somewhere else. So the real question becomes: <strong><em>How do we keep data safe even if Containers come and go?</em></strong></p><h4>The Idea of External Storage</h4><p>Docker solves this by introducing the idea of <strong>external storage</strong>. Instead of keeping data inside the Container, we connect the Container to storage that exists outside it.</p><p>So the relationship becomes:</p><pre>Container → External Storage → Data</pre><p>Now the Container can be replaced anytime, but the data remains safe.</p><h4>Docker Volumes (Simple Idea)</h4><p>A Volume is just a safe place for data outside the Container. You can think of it like a separate storage box. The Container can read from it and write to it, but the box itself does not disappear when the Container is removed.</p><p>So now the flow looks like this:</p><pre>Container ↔ Volume ↔ Data</pre><p>This is the key idea: <em>Containers are replaceable. Volumes are not.</em></p><h4>Why This Matters in Real Life</h4><p>Let’s take a simple ML example. Imagine your application is tracking experiments.</p><p>It stores:</p><ul><li>model versions</li><li>accuracy scores</li><li>training runs</li></ul><p>Now imagine you remove the Container and start a new one without external storage, everything is gone.</p><p>But with a Volume, the new Container can continue from where the old one left off. So the data survives even when the Container does not.</p><h4>Two Ways to Connect Storage</h4><p>Docker gives us two common ways to connect storage.</p><h4>1. Named Volumes</h4><p>This is when Docker manages the storage for you. You simply say: <strong><em>I want a place to store my data. </em></strong>Docker creates and manages it.</p><p>The important idea is:</p><ul><li>you don’t worry about where it is stored</li><li>Docker handles it internally</li><li>it is safe and consistent</li></ul><p>This is usually used for things like databases or ML tracking systems.</p><h4>2. Bind Mounts</h4><p>This is when you connect a folder from your own computer directly into the Container. So now both the Container and your computer are looking at the same files.</p><p>This is useful when:</p><ul><li>you are developing code</li><li>you want to see changes immediately</li><li>you want full control over files</li></ul><p>So instead of Docker managing storage, you are directly sharing a folder.</p><h3>When to Use What</h3><p>The simplest way to think about it is:</p><ul><li>If it is <strong>application data</strong>, use a Volume</li><li>If it is <strong>development files</strong>, use a Bind Mount</li></ul><p>For example:</p><ul><li>database → Volume</li><li>MLflow tracking → Volume</li><li>source code during development → Bind Mount</li></ul><h4>A Common Beginner Mistake</h4><p>Many beginners assume: <strong><em>If I rebuild the Image, I will keep my data. </em></strong>That is not true, An Image is like a blueprint. It describes the application, not the changing data. If you store data inside the Image or inside a Container without external storage, it is not safe. That is why Docker separates them clearly.</p><h4>What About .dockerignore?</h4><p>Now that we understand data separation, another problem appears. When Docker builds an Image, it copies files from your project but not everything in your project should go inside the Image.</p><p>For example:</p><ul><li>temporary files</li><li>virtual environments</li><li>Git history</li><li>old experiment data</li></ul><p>These are not part of the application.</p><h4>Why .dockerignore Exists</h4><p>.dockerignore is simply a way to tell Docker: <strong><em>Do not include these files when building the Image. </em></strong>This keeps your Image clean and focused only on the application.</p><p>So instead of copying everything blindly, Docker only includes what matters. This is like .gitignore for the git.</p><h4>A Simple Mental Model</h4><p>Now let’s connect everything together:</p><ul><li>The <strong>Image</strong> is your application blueprint</li><li>The <strong>Container</strong> is a running instance of that application</li><li>The <strong>Volume</strong> is where important data lives safely</li><li>.dockerignore ensures unnecessary files don’t enter the Image</li></ul><p>So the system becomes:</p><pre>Code → Image → Container → External Storage (Data)</pre><p>Each part has a clear responsibility.</p><h4>Key Takeaways</h4><p>By the end of this part, you should understand:</p><ul><li>Containers are temporary by design</li><li>Application and data must be treated separately</li><li>Volumes store data outside Containers</li><li>Bind mounts connect local folders to Containers</li><li>.dockerignore keeps Images clean and minimal</li><li>Docker is not just about running apps, but managing how data survives</li></ul><h3>What’s Next?</h3><p>So far, we have been working with single Containers, but real applications are rarely that simple.</p><p>A real system might include:</p><ul><li>an application</li><li>a database</li><li>a tracking system like MLflow</li></ul><p>Managing all of them manually becomes difficult very quickly.</p><p>In Part 5, we will solve this problem using <strong>Docker Compose</strong>, which lets us define and run multiple Containers together in a simple way.</p><p><a href="https://elanthirayan.medium.com/docker-from-scratch-part-5-running-multiple-containers-with-docker-compose-954def6b82ca?sharedUserId=elanthirayan">https://elanthirayan.medium.com/docker-from-scratch-part-5-running-multiple-containers-with-docker-compose-954def6b82ca</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=08a125c75849" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker from Scratch — Part 3: Writing Your First Dockerfile]]></title>
            <link>https://elanthirayan.medium.com/docker-from-scratch-part-3-writing-your-first-dockerfile-a920c0516e81?source=rss-d28ca620d8a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a920c0516e81</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <category><![CDATA[docker-image]]></category>
            <dc:creator><![CDATA[Elanthirayan]]></dc:creator>
            <pubDate>Sun, 09 Aug 2026 16:40:44 GMT</pubDate>
            <atom:updated>2026-08-10T13:04:56.701Z</atom:updated>
            <content:encoded><![CDATA[<p>In Part 1, we learned why Docker exists.</p><p><a href="https://elanthirayan.medium.com/docker-from-scratch-part-1-what-is-docker-and-why-was-it-created-4f60eea241f4?sharedUserId=elanthirayan">https://elanthirayan.medium.com/docker-from-scratch-part-1-what-is-docker-and-why-was-it-created-4f60eea241f4</a></p><p>In Part 2, we understood the basic flow:</p><p><a href="https://elanthirayan.medium.com/docker-from-scratch-part-2-understanding-dockerfile-image-and-container-c0d78d524e30?sharedUserId=elanthirayan">https://elanthirayan.medium.com/docker-from-scratch-part-2-understanding-dockerfile-image-and-container-c0d78d524e30</a></p><pre>Dockerfile → Image → Container</pre><p>Now it is time to build something. In this part, we will write a simple Dockerfile for a Python application and understand every line.</p><h4>Our Simple Python Application</h4><p>Imagine we have a small Python file called app.py.</p><pre>print(&quot;Hello from Docker!&quot;)</pre><p>Our project folder looks like this:</p><pre>my-docker-app/<br>├── app.py<br>└── Dockerfile</pre><p>The goal is simple: <em>Package this Python application so it can run inside Docker.</em></p><h4>What Should the Dockerfile Do?</h4><p>Our application needs:</p><ul><li>Python</li><li>The app.py file</li><li>A command to run the file</li></ul><p>So our Dockerfile needs to describe those three things.</p><p>Here is the complete Dockerfile:</p><pre>FROM python:3.12-slim<br><br>WORKDIR /app<br><br>COPY app.py .<br><br>CMD [&quot;python&quot;, &quot;app.py&quot;]</pre><p>Now let us understand it one line at a time.</p><p><strong>FROM</strong></p><pre>FROM python:3.12-slim</pre><p>Every Dockerfile starts from an existing base.</p><p>Here, we are saying: <strong><em>Start with an environment that already has Python 3.12 installed.</em></strong></p><p>This saves us from installing Python manually. The word slim means we are using a smaller version of the Python environment. Think of it as starting with a ready-made kitchen instead of building one from scratch.</p><p><strong>WORKDIR</strong></p><pre>WORKDIR /app</pre><p>This creates or selects a folder called /app inside the container. All the next steps will happen inside this folder. It is similar to opening a folder on your computer before working inside it. After this line, Docker treats /app as the current working folder.</p><p><strong>COPY</strong></p><pre>COPY app.py .</pre><p>This copies app.py from your computer into the container.</p><p>The first value:</p><pre>app.py</pre><p>is the file on your machine.</p><p>The second value:</p><pre>.</pre><p>means the current folder inside the container.</p><p>Because our working folder is /app, Docker copies the file here:</p><pre>/app/app.py</pre><h4>CMD</h4><pre>CMD [&quot;python&quot;, &quot;app.py&quot;]</pre><p>This tells Docker what to run when the container starts.</p><p>In simple words: <em>Start Python and run </em><em>app.py.</em></p><p>This command does not run while building the Image. It runs later, when a Container starts.</p><h4>Build Time and Run Time</h4><p>This is an important difference.</p><h4>Build Time</h4><p>Docker reads the Dockerfile and prepares the Image.</p><p>During this stage, Docker:</p><ul><li>Starts from the Python environment</li><li>Creates the working folder</li><li>Copies the application file</li></ul><h4>Run Time</h4><p>Docker starts a Container from the Image.</p><p>Then it executes:</p><pre>python app.py</pre><p>So:</p><pre>Build → Create the Image<br>Run → Start the Container</pre><h4>Building the Image</h4><p>Open a terminal inside the project folder and run:</p><pre>docker build -t my-python-app .</pre><p>Let us break this command down.</p><pre>docker build</pre><p>tells Docker to build an Image.</p><pre>-t my-python-app</pre><p>gives the Image a name.</p><pre>.</pre><p>tells Docker to use the current folder.</p><p>Docker looks for a file named Dockerfile in that folder and follows its instructions.</p><h4>Running the Container</h4><p>After the Image is built, run:</p><pre>docker run my-python-app</pre><p>You should see:</p><pre>Hello from Docker!</pre><p>What happened?</p><pre>Dockerfile<br>    ↓<br>Docker built an Image<br>    ↓<br>Docker started a Container<br>    ↓<br>The Container ran app.py</pre><h4>Why Did the Container Stop?</h4><p>After printing the message, the Python program finished. When the main program inside a Container finishes, the Container stops and this is normal. Some Containers run briefly and exit.</p><p>Examples include:</p><ul><li>Training a machine learning model</li><li>Generating a report</li><li>Running a script</li><li>Performing a database task</li></ul><p>Other Containers stay running because their main application keeps waiting for work.</p><p>Examples include:</p><ul><li>Web servers</li><li>APIs</li><li>Databases</li></ul><h4>What If the Application Needs Libraries?</h4><p>Our first example only uses Python, so we do not need to install anything else.</p><p>A real Python application may need packages such as:</p><ul><li>pandas</li><li>FastAPI</li><li>MLflow</li><li>scikit-learn</li></ul><p>These are usually listed in a file called requirements.txt.</p><p>For example:</p><pre>pandas<br>mlflow<br>scikit-learn</pre><p>The Dockerfile can then install them:</p><pre>FROM python:3.12-slim<br><br>WORKDIR /app<br><br>COPY requirements.txt .<br><br>RUN pip install --no-cache-dir -r requirements.txt<br><br>COPY . .<br><br>CMD [&quot;python&quot;, &quot;app.py&quot;]</pre><h4>What Does RUN Mean?</h4><pre>RUN pip install --no-cache-dir -r requirements.txt</pre><p>RUN executes a command while Docker is building the Image.</p><p>Here, Docker installs the required Python packages into the Image.</p><p>This is different from CMD.</p><ul><li>RUN happens while building.</li><li>CMD happens when the Container starts.</li></ul><p>That difference is very important.</p><h4>Why Copy requirements.txt First?</h4><p>You may notice that we copy requirements.txt before copying the rest of the project.</p><pre>COPY requirements.txt .<br><br>RUN pip install --no-cache-dir -r requirements.txt<br><br>COPY . .</pre><p>Docker builds Images in steps. If requirements.txt has not changed, Docker can reuse the package installation step.</p><p>Then, when only your Python code changes, Docker does not need to install all packages again. This makes rebuilding faster.</p><h4>Key Takeaways</h4><p>By the end of this part, you should understand:</p><ul><li>A Dockerfile describes how to build an Image.</li><li>FROM chooses the starting environment.</li><li>WORKDIR selects the working folder.</li><li>COPY moves files into the Image.</li><li>RUN executes commands while building.</li><li>CMD runs when the Container starts.</li><li>docker build creates an Image.</li><li>docker run creates and starts a Container.</li></ul><p>The full flow is:</p><pre>Write Dockerfile<br>      ↓<br>Build Image<br>      ↓<br>Run Container<br>      ↓<br>Application starts</pre><h4>What’s Next?</h4><p>Our Image currently includes everything in the project folder.</p><p>But some files should not be copied, such as:</p><ul><li>Virtual environments</li><li>Temporary files</li><li>Git history</li><li>Local databases</li><li>MLflow experiment history</li></ul><p>In Part 4, we will learn about .dockerignore, Docker storage, and why application data should live outside the Image.</p><p><a href="https://elanthirayan.medium.com/docker-from-scratch-part-4-where-does-your-data-go-08a125c75849?sharedUserId=elanthirayan">https://elanthirayan.medium.com/docker-from-scratch-part-4-where-does-your-data-go-08a125c75849</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a920c0516e81" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>