<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aditya Dawadikar on Medium]]></title>
        <description><![CDATA[Stories by Aditya Dawadikar on Medium]]></description>
        <link>https://medium.com/@aditya-dawadikar?source=rss-e5802f137581------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Ys3jXfhTn-GrY2BVxzo_1g.jpeg</url>
            <title>Stories by Aditya Dawadikar on Medium</title>
            <link>https://medium.com/@aditya-dawadikar?source=rss-e5802f137581------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:19:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aditya-dawadikar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to estimate your model’s Inference mode memory requirements]]></title>
            <link>https://medium.com/@aditya-dawadikar/how-to-estimate-your-models-inference-mode-memory-requirements-baedd7f1e9a2?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/baedd7f1e9a2</guid>
            <category><![CDATA[kv-cache]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[model-quantization]]></category>
            <category><![CDATA[vllm]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Wed, 08 Jul 2026 06:24:10 GMT</pubDate>
            <atom:updated>2026-07-08T06:24:10.114Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nXQ26d0gJhARHxks" /><figcaption>Photo by <a href="https://unsplash.com/@omilaev?utm_source=medium&amp;utm_medium=referral">Igor Omilaev</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Deploying an LLM model in the inference mode is easy, at least that’s what <a href="https://vllm.ai/"><strong>vLLM</strong></a> says and try to achieve. But the real challenge is not the one-click deployment, rather its the estimation of the model, the traffic and the target use case. So after running into a lot of OOM (out-of-memory) issues myself while deploying LLMs, I finally decided to take a deep dive in to the VRAM requirements, to ensure that your models fit on your hardware.</p><h3>The Holy Grail of VRAM estimation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0vgUbs-DapyB1zss94Pa2A.png" /><figcaption>Inference VRAM Requirement Estimation Formula</figcaption></figure><p>During the inference, the overall VRAM requirements depend on these 3 things: Model Weights, KV Cache and the CUDA operational overheads.</p><p>Model Weights is largely static, and represents the actual model memory requirements. During inference, these weights must already be loaded onto the VRAM, in order to perform the self-attention calculation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rtIU2Bub3bBMaScCy5MiTQ.png" /><figcaption>Model Weight Estimation</figcaption></figure><p>Note that the model weight is static, but an estimate. The actual size of the model can be found on the HuggingFace’s model listing page. Hence, even though the estimated weight for Llama-8B model is approximately 16GB, its an optimistic estimate, and actual size may be slightly higher than 16GB, hence always consider a buffer.</p><p>Next part is the KV cache. KV cache, as the name suggests is the cache the model can use for faster calculations. Since the model operates auto-regressively, the KV cache helps us store intermediate results. Except the catch is, KV cache can really bloat your memory requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WhMtItt8dMCHAwDT6TIG-g.png" /><figcaption>KV Cache Memory Estimation</figcaption></figure><p>You can clearly see how the KV Cache for the full context can be roughly the same size as the model weights itself. And this is estimation for just one in-flight request. In reality, you may have thousands of users, so you can clearly visualize the scale of the KV Cache.</p><p>Finally, plugging all the numbers into the overall memory estimation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c8gsbEq2C_9mbB7AfE-yNw.png" /><figcaption>Total VRAM Memory Estimation</figcaption></figure><p>We learn that for serving a single user with 128K context window, the system needs 34GB of VRAM. For realistic purposes, it may be slightly higher than 34GB…</p><p>But then what is the problem? 34GB is a piece of cake. right? RIGHT?</p><p>Nope, the problem is, there are not many Hardware options, especially cheap ones that allow you to have so much VRAM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a0d84J38qQWGZAWu4gIeqg.png" /><figcaption>GPU Pricing Table</figcaption></figure><p>So what can we do about the high prices? You pay by the hour, not based on the number of requests served. These two are opposite goals, hence we must optimize our serving strategy.</p><h3>The Holy Grail of Cost Optimization</h3><p>From the above calculations, it is clear that we will have to pay a lot for our GPUs when serving for inference. We cannot cut down on the CUDA operational VRAM costs, but we can cut down on the Model Weights and the KV Cache size.</p><h4>1. Optimizing the Model Weights</h4><p>This will directly impact our fixed costs. Instead of serving our models in FP32/FP16 precision, we can quantize our model down to INT4.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GM85eNDfOoypdyIIVF1ogg.png" /><figcaption>Model Weight Optimization</figcaption></figure><p>Watchout for framework and hardware specific limitations for the Quantization process. VLLM uses Marlin or AWQ kernels to unpack the 4Bit weights back to 16-bit on the fly. The model weights remain in 4Bit form, where as the Activations during the forward pass remains in 16Bit form.</p><ul><li>When the weights are 4Bit, the time required to fetch the weights from the slow serving global VRAM (HBM — High Bandwidth Memory, GDDR) to the local cache is cut down, hence speed up is observed.</li><li>When the batch size increases, the on-the-fly dequantization overhead catches up to us, at this point unpacking becomes a drag, thus making quantized models slightly slower than a full native model.</li></ul><blockquote>If the underlying hardware can support 4-bit quantization using dedicated 4-bit tensor cores eg: A100, and both the model weights and activations are 4-bit quantized, then on-the-fly dequantization is not needed. But if the operation is mixed-precision where Weights are 4-bit quantized, where as Activations are full precision FP16/FP32, then the 4-bit tensor cores will not be able to perform the operations.</blockquote><h4>2. Optimizing the KV Cache</h4><p>During text generation KV Cache grows dynamically with context length and user count. We can change the storage format from 16-bit to FP8 which is widely supported in (A100 &amp; H100 GPUs) or even NVFP4 (natively accelerated on Blackwell architecture)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RMgil3ESbQKzYxZCM3L2rg.png" /><figcaption>KV Cache Optimization</figcaption></figure><p>So lets look at some numbers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GMsVkJIZbCMEMXuri6z6Yg.png" /><figcaption>Some estimates after Model and KV Cache Quantization for 24GB VRAM</figcaption></figure><p>but we have some more tricks up our sleeves!</p><h4>3. Capping the Context</h4><p>This is the simplest trick you can pull off. The KV cache grows linearly with sequence length. Hence, if we reduce the sequence length, the KV Cache size will also drop. This is trick can be used when the use-case does not require a 128K context window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/1*NA8T7c0n8_suZ_WVXw5PvA.png" /><figcaption>Context Capping and Concurrency Trend</figcaption></figure><blockquote>Notice how context capping yeilds perfectly linear inverse scaling for concurrency. For production the best approach is to set the — max-model-len to 95th percentile of your actual request length. Never leave it at maximum default 128K unless your users are routinely uploading full text books, otherwise VLLM will reserve vast pools of virtual memory pages that sit completely empty, starving your system of throughput.</blockquote><h4>4. Automatic Prefix Caching (APC)</h4><p>If you are running multi-turn agent chats or multi-document RAG pipelines, or using heavy system prompts, different users consistently send the exact same context blocks over and over again.</p><p>Normally vLLM would recalculate the KV Cache for that identical 5000 token system prompt every time a new user makes a query. By turning on prefix caching, vLLM hashes your input blocks. If a new request shares the same starting tokens a.k.a prefix, vLLM completely skips the compute-heavy prefill phase and instantly mounts the existing KV cache from memory. Use --enable-prefix-caching in vLLM.</p><h4>5. Chunked Prefill &amp; Iteration Balancing</h4><p>By default vLLM prioritizes new incoming requests. If a user drops a massive 30,000 Tokens document into the queue, the engine stops generating text for all other active users to process the new prompt. This resource hogging can create massive variance in the latency metrics.</p><p>Chunked prefill slices the giant incoming prompt into smaller uniform chunks eg: 2048 tokens each, which can be interleved with the execution batch, thus providing fair share usage to all users and reduced variance in the user latencies.</p><p>Enable --enable-chunked-prefill --max-num-batched-tokens 2048 for this behavior.</p><h3>Conclusion</h3><p>In this article we studied how to estimate the VRAM requirements of your model in the inference mode, and a few tricks to optimize the model serving and the user experience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=baedd7f1e9a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating Planning Agents with LLM-as-a-Judge]]></title>
            <link>https://medium.com/@aditya-dawadikar/evaluating-planning-agents-with-llm-as-a-judge-095fd0d46c56?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/095fd0d46c56</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[software-testing]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Sat, 30 May 2026 18:05:04 GMT</pubDate>
            <atom:updated>2026-05-30T18:05:04.410Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zJuiO8HdcsuBPHrR" /><figcaption>Photo by <a href="https://unsplash.com/@kellysikkema?utm_source=medium&amp;utm_medium=referral">Kelly Sikkema</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Goal: We want to ensure our Planner Agents are making effective plans.</h4><h3>Using LLM-as-a-Judge</h3><p>An LLM’s output is non deterministic, which means your Agents may trace multiple branches for execution and reaching the same end goal. So can you ensure that your Agents are well tested under these conditions?</p><p>One of the methods to evaluate the LLM response is by using a larger, more capable LLM to evaluate the output of the LLM of interest. Interestingly, even if the Judge LLM is exactly the same as the one being judged, the Judge as the rubric and sometimes even the correct answer, which can be used to semantically understand the Agents’ output.</p><p>One place where an Agent may return non-deterministic output is at the Planning Stage.</p><p>For Instance, lets build a Planning agent to solve Mathematical equations.</p><p>Eg: 2*(3+4) = ?</p><p>We know this equation can be solved using BODMAS rule, where first step will be to apply BO (Bracket Open), i.e take a sum of 3 &amp; 4 i.e 7. Next step will be to compute the product between 7 &amp; 2 i.e 14.</p><p>For the Agent to solve this equation, the Agent will have access to the tools like Add, Subtract, Multiply, Divide. But we also need a planning step to ensure the correctness of the output.</p><p>So in this article, we are going to build and test the output of a planner agent, and see if it is able to provide the correct execution plan.</p><p>Some Sample Queries:</p><ul><li>A club had 48 snacks. It gave out 19, and then the rest were shared equally among 29 students. What plan should the agent follow?</li><li>A teacher buys 6 packs of pencils with 8 pencils in each pack and then buys 5 extra pencils. Create the tool plan to find the total number of pencils.</li><li>A farmer planted 6 rows of corn with 14 plants in each row and 5 rows of beans with 9 plants in each row. Plan the tool calls to find the total number of plants.</li></ul><p>With these queries, we are testing if the Agent can perform the following tasks:</p><ol><li>Extract numerical data from the input query, or infer numerical data from text representation.</li><li>Understand the query intent and plan the execution</li><li>For the Math equation solver agent, the sequence of operations matter, hence the Agent should apply the Math rules correctly</li></ol><p>Lastly, since there can be scenarios where multiple execution plan can be valid and result into the correct answer, eg: 2*(3+4) + 5*(6+7) = ?</p><p>In the above example, the order of the operation 3+4 and 6+7 do not matter as long as the output of 2*(3+4) and 5*(6+7) are summed up at the end. For such a non deterministic scenario, <a href="https://github.com/langchain-ai/openevals#llm-as-judge">LLM-as-a-Judge</a> is the right choice for tests. We will be using the Open Eval&#39;s create_llm_as_judge as our evaluator.</p><h3>Prerequisites</h3><ul><li>You must know how to deploy a local LLM using Ollama and preferably Docker</li><li>You must know how to use Pytest for testing traditional code</li><li>You must know basics of Langchain, Prompting and Agents</li></ul><h3>Tech Stack</h3><ul><li>Python 3.13.x</li><li>Langchain &gt; = 0.3.0</li><li>Langgraph &gt; = 0.2.0</li><li>Openevals &gt; = 0.2.0</li><li>Pytest &gt; = 9.0.3</li></ul><h3>Open Evals</h3><p>I have created the following test cases for evaluating the Agent. I prefer writing tests even before building the Agent, this way I know what behavior should the Agent show.</p><pre>[<br> {<br>  &quot;id&quot;: &quot;ambiguous-altogether-leftover-shared&quot;,<br>  &quot;category&quot;: &quot;ambiguous_language&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A club had 48 snacks. It gave out 19, and then the rest were shared equally among 29 students. What plan should the agent follow?&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 48,<br>     &quot;b&quot;: 19<br>    },<br>    &quot;purpose&quot;: &quot;Find how many snacks are left after some were given out.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;divide&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 29,<br>     &quot;b&quot;: 29<br>    },<br>    &quot;purpose&quot;: &quot;Share the remaining snacks equally among the students.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;direct-language-multiply-then-add&quot;,<br>  &quot;category&quot;: &quot;direct_language&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A teacher buys 6 packs of pencils with 8 pencils in each pack and then buys 5 extra pencils. Create the tool plan to find the total number of pencils.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 6,<br>     &quot;b&quot;: 8<br>    },<br>    &quot;purpose&quot;: &quot;Compute the number of pencils in the packs.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 48,<br>     &quot;b&quot;: 5<br>    },<br>    &quot;purpose&quot;: &quot;Add the extra pencils to the pack total.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;direct-language-subtract-then-divide&quot;,<br>  &quot;category&quot;: &quot;direct_language&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A tank contains 90 liters of water. After removing 18 liters, divide the remaining water equally into 6 buckets. Plan the tool calls.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 90,<br>     &quot;b&quot;: 18<br>    },<br>    &quot;purpose&quot;: &quot;Find the amount of water remaining after removal.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;divide&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 72,<br>     &quot;b&quot;: 6<br>    },<br>    &quot;purpose&quot;: &quot;Split the remaining water evenly into buckets.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;direct-equation-parentheses-times&quot;,<br>  &quot;category&quot;: &quot;direct_equations&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;Plan the tool calls needed to evaluate (18 + 7) * 4.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 18,<br>     &quot;b&quot;: 7<br>    },<br>    &quot;purpose&quot;: &quot;Evaluate the expression inside the parentheses.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 25,<br>     &quot;b&quot;: 4<br>    },<br>    &quot;purpose&quot;: &quot;Multiply the parenthesized result by 4.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;direct-equation-power-minus&quot;,<br>  &quot;category&quot;: &quot;direct_equations&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;Plan the tool calls for 5 squared minus 11.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;power&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;base&quot;: 5,<br>     &quot;exponent&quot;: 2<br>    },<br>    &quot;purpose&quot;: &quot;Compute the square of 5.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 25,<br>     &quot;b&quot;: 11<br>    },<br>    &quot;purpose&quot;: &quot;Subtract 11 from the squared value.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;mixed-word-number-sum-then-divide&quot;,<br>  &quot;category&quot;: &quot;mixed_word_and_number_form&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A fundraiser collected twenty four dollars on Monday and 16 dollars on Tuesday. The total was split equally among 4 teams. What is the tool plan?&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 24,<br>     &quot;b&quot;: 16<br>    },<br>    &quot;purpose&quot;: &quot;Add the amounts collected on both days.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;divide&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 40,<br>     &quot;b&quot;: 4<br>    },<br>    &quot;purpose&quot;: &quot;Split the total equally among the teams.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;mixed-word-number-product-then-difference&quot;,<br>  &quot;category&quot;: &quot;mixed_word_and_number_form&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;Leo bought three notebooks for 7 dollars each and then used a 5 dollar coupon. Plan the steps to find what he paid.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 3,<br>     &quot;b&quot;: 7<br>    },<br>    &quot;purpose&quot;: &quot;Find the total cost before the coupon.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 21,<br>     &quot;b&quot;: 5<br>    },<br>    &quot;purpose&quot;: &quot;Subtract the coupon value from the total cost.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;order-flexible-two-products-then-add&quot;,<br>  &quot;category&quot;: &quot;direct_language&quot;,<br>  &quot;order_sensitive&quot;: false,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A farmer planted 6 rows of corn with 14 plants in each row and 5 rows of beans with 9 plants in each row. Plan the tool calls to find the total number of plants.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 6,<br>     &quot;b&quot;: 14<br>    },<br>    &quot;purpose&quot;: &quot;Find the number of corn plants.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 5,<br>     &quot;b&quot;: 9<br>    },<br>    &quot;purpose&quot;: &quot;Find the number of bean plants.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 3,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 84,<br>     &quot;b&quot;: 45<br>    },<br>    &quot;purpose&quot;: &quot;Add the corn and bean plants.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;order-flexible-two-sums-then-multiply&quot;,<br>  &quot;category&quot;: &quot;mixed_word_and_number_form&quot;,<br>  &quot;order_sensitive&quot;: false,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;Morning classes used twelve boxes of crayons and 8 boxes of markers, while afternoon classes used 9 boxes of crayons and eleven boxes of markers. Each box has 10 items. Plan the tool calls to find the total number of items used.&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 12,<br>     &quot;b&quot;: 8<br>    },<br>    &quot;purpose&quot;: &quot;Find the total number of boxes used in the morning.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 9,<br>     &quot;b&quot;: 11<br>    },<br>    &quot;purpose&quot;: &quot;Find the total number of boxes used in the afternoon.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 3,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 20,<br>     &quot;b&quot;: 20<br>    },<br>    &quot;purpose&quot;: &quot;Combine the morning and afternoon box totals.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 4,<br>    &quot;tool_name&quot;: &quot;multiply&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 40,<br>     &quot;b&quot;: 10<br>    },<br>    &quot;purpose&quot;: &quot;Convert the total number of boxes into total items.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;order-flexible-two-differences-then-add&quot;,<br>  &quot;category&quot;: &quot;ambiguous_language&quot;,<br>  &quot;order_sensitive&quot;: false,<br>  &quot;should_decline&quot;: false,<br>  &quot;prompt&quot;: &quot;A store started with 90 notebooks and sold 25, and it also started with 70 sketchbooks and sold 18. What plan should the agent follow to find how many items remain altogether?&quot;,<br>  &quot;expected_steps&quot;: [<br>   {<br>    &quot;step_number&quot;: 1,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 90,<br>     &quot;b&quot;: 25<br>    },<br>    &quot;purpose&quot;: &quot;Find how many notebooks remain.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 2,<br>    &quot;tool_name&quot;: &quot;subtract&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 70,<br>     &quot;b&quot;: 18<br>    },<br>    &quot;purpose&quot;: &quot;Find how many sketchbooks remain.&quot;<br>   },<br>   {<br>    &quot;step_number&quot;: 3,<br>    &quot;tool_name&quot;: &quot;add&quot;,<br>    &quot;arguments&quot;: {<br>     &quot;a&quot;: 65,<br>     &quot;b&quot;: 52<br>    },<br>    &quot;purpose&quot;: &quot;Add the remaining notebooks and sketchbooks.&quot;<br>   }<br>  ],<br>  &quot;decline_reason&quot;: &quot;&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;decline-poem-request&quot;,<br>  &quot;category&quot;: &quot;non_math_decline&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: true,<br>  &quot;prompt&quot;: &quot;Write a short poem about winter rain.&quot;,<br>  &quot;expected_steps&quot;: [],<br>  &quot;decline_reason&quot;: &quot;The query does not ask for a math solution or a tool-use math plan.&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;decline-definition-request&quot;,<br>  &quot;category&quot;: &quot;non_math_decline&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: true,<br>  &quot;prompt&quot;: &quot;Explain what photosynthesis is in simple terms.&quot;,<br>  &quot;expected_steps&quot;: [],<br>  &quot;decline_reason&quot;: &quot;The query is explanatory and does not represent a math problem.&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;decline-personal-advice&quot;,<br>  &quot;category&quot;: &quot;non_math_decline&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: true,<br>  &quot;prompt&quot;: &quot;Help me decide whether I should move to a new city.&quot;,<br>  &quot;expected_steps&quot;: [],<br>  &quot;decline_reason&quot;: &quot;The query asks for personal advice, not a math word problem.&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;decline-summarization&quot;,<br>  &quot;category&quot;: &quot;non_math_decline&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: true,<br>  &quot;prompt&quot;: &quot;Summarize the main idea of this meeting note.&quot;,<br>  &quot;expected_steps&quot;: [],<br>  &quot;decline_reason&quot;: &quot;The query is a summarization task and should not trigger math planning.&quot;<br> },<br> {<br>  &quot;id&quot;: &quot;decline-vague-non-math-ambiguous&quot;,<br>  &quot;category&quot;: &quot;non_math_decline&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;should_decline&quot;: true,<br>  &quot;prompt&quot;: &quot;What should I do next with this project?&quot;,<br>  &quot;expected_steps&quot;: [],<br>  &quot;decline_reason&quot;: &quot;The query is ambiguous but does not contain a math problem to solve with tools.&quot;<br> }<br>]</pre><p>The above set of test cases contain following type of scenarios:</p><ol><li>Ambiguous Language</li><li>Direct Language</li><li>Mixed words &amp; numeric form</li><li>Non Math queries which should be declined</li></ol><p>Following is my Actual agent code.</p><pre>import os<br>import json<br>import re<br>from typing import Any, Literal<br><br>from langchain_core.messages import AIMessage, BaseMessage, SystemMessage<br>from langchain_ollama import ChatOllama<br>from pydantic import BaseModel, Field<br><br><br>DEFAULT_SYSTEM_PROMPT = &quot;&quot;&quot;You are a planner agent.<br><br>You are not an execution agent. Do not call tools. Do not solve the problem directly.<br>Your job is to return a structured tool-use plan only.<br><br>Available tool names:<br>- add<br>- subtract<br>- multiply<br>- divide<br>- power<br>- get_current_datetime<br><br>Return JSON only and follow this exact contract:<br>{<br>  &quot;should_decline&quot;: false,<br>  &quot;decline_reason&quot;: &quot;&quot;,<br>  &quot;order_sensitive&quot;: true,<br>  &quot;steps&quot;: [<br>    {<br>      &quot;step_number&quot;: 1,<br>      &quot;tool_name&quot;: &quot;add&quot;,<br>      &quot;arguments&quot;: {&quot;a&quot;: 1, &quot;b&quot;: 2},<br>      &quot;purpose&quot;: &quot;short explanation&quot;<br>    }<br>  ]<br>}<br><br>Rules:<br>- If the query is not a math problem or should not be handled as a math planning task, set should_decline=true, provide a short decline_reason, set steps=[], and set order_sensitive=true.<br>- If the query is a math planning task, set should_decline=false and decline_reason=&quot;&quot;.<br>- Decline requests such as creative writing, summarization, explanation, general advice, brainstorming, or other non-math tasks.<br>- Decline vague requests that do not contain a concrete math problem to solve with tools.<br>- When declining, do not invent any steps or tool calls.<br>- A valid decline response looks like this:<br>    {<br>        &quot;should_decline&quot;: true,<br>        &quot;decline_reason&quot;: &quot;The query is not a math problem.&quot;,<br>        &quot;order_sensitive&quot;: true,<br>        &quot;steps&quot;: []<br>    }<br>- Examples that should be declined:<br>    - &quot;Write a short poem about winter rain.&quot;<br>    - &quot;Explain what photosynthesis is in simple terms.&quot;<br>    - &quot;Help me decide whether I should move to a new city.&quot;<br>    - &quot;Summarize the main idea of this meeting note.&quot;<br>- Use only these tool names: add, subtract, multiply, divide, power, get_current_datetime.<br>- For add, subtract, multiply, and divide, arguments must use keys a and b.<br>- For power, arguments must use keys base and exponent.<br>- For get_current_datetime, arguments must be {}.<br>- Use numeric values in arguments, including computed intermediate values when a later step depends on an earlier one.<br>- Never use placeholders such as result_of_step_1. Substitute the numeric value directly.<br>- Set order_sensitive=false only when independent subcomputations can be performed in any order without changing correctness.<br>- Every step must include step_number, tool_name, arguments, and purpose.<br>&quot;&quot;&quot;<br><br><br>class PlannerStep(BaseModel):<br>    step_number: int = Field(description=&quot;1-based position of the step in the plan.&quot;)<br>    tool_name: Literal[&quot;add&quot;, &quot;subtract&quot;, &quot;multiply&quot;, &quot;divide&quot;, &quot;power&quot;, &quot;get_current_datetime&quot;]<br>    arguments: dict[str, float | str] = Field(description=&quot;Exact arguments for the chosen tool.&quot;)<br>    purpose: str = Field(description=&quot;Short explanation of why this step is needed.&quot;)<br><br><br>class PlannerResponse(BaseModel):<br>    should_decline: bool<br>    decline_reason: str<br>    order_sensitive: bool<br>    steps: list[PlannerStep]<br><br><br>RESULT_REFERENCE_PATTERN = re.compile(r&quot;^result_of_step_(\d+)$&quot;)<br><br><br>def _coerce_argument_value(value: float | str, step_results: list[float | None]) -&gt; float | str:<br>    if isinstance(value, (int, float)):<br>        return float(value)<br><br>    match = RESULT_REFERENCE_PATTERN.fullmatch(value.strip())<br>    if match:<br>        step_index = int(match.group(1)) - 1<br>        if step_index &lt; 0 or step_index &gt;= len(step_results) or step_results[step_index] is None:<br>            raise RuntimeError(f&quot;Unresolvable step result reference: {value}&quot;)<br>        return float(step_results[step_index])<br><br>    try:<br>        return float(value)<br>    except ValueError:<br>        return value<br><br><br>def _compute_step_result(tool_name: str, arguments: dict[str, float | str]) -&gt; float | None:<br>    if tool_name == &quot;add&quot;:<br>        return float(arguments[&quot;a&quot;]) + float(arguments[&quot;b&quot;])<br>    if tool_name == &quot;subtract&quot;:<br>        return float(arguments[&quot;a&quot;]) - float(arguments[&quot;b&quot;])<br>    if tool_name == &quot;multiply&quot;:<br>        return float(arguments[&quot;a&quot;]) * float(arguments[&quot;b&quot;])<br>    if tool_name == &quot;divide&quot;:<br>        return float(arguments[&quot;a&quot;]) / float(arguments[&quot;b&quot;])<br>    if tool_name == &quot;power&quot;:<br>        return float(arguments[&quot;base&quot;]) ** float(arguments[&quot;exponent&quot;])<br>    return None<br><br><br>def _normalize_plan(plan: dict[str, Any]) -&gt; PlannerResponse:<br>    normalized_steps: list[dict[str, Any]] = []<br>    step_results: list[float | None] = []<br><br>    for step in plan.get(&quot;steps&quot;, []):<br>        normalized_arguments = {<br>            key: _coerce_argument_value(value, step_results)<br>            for key, value in step.get(&quot;arguments&quot;, {}).items()<br>        }<br>        normalized_step = {<br>            &quot;step_number&quot;: step[&quot;step_number&quot;],<br>            &quot;tool_name&quot;: step[&quot;tool_name&quot;],<br>            &quot;arguments&quot;: normalized_arguments,<br>            &quot;purpose&quot;: step[&quot;purpose&quot;],<br>        }<br>        normalized_steps.append(normalized_step)<br>        step_results.append(_compute_step_result(step[&quot;tool_name&quot;], normalized_arguments))<br><br>    normalized_plan = {<br>        &quot;should_decline&quot;: bool(plan.get(&quot;should_decline&quot;, False)),<br>        &quot;decline_reason&quot;: str(plan.get(&quot;decline_reason&quot;, &quot;&quot;)),<br>        &quot;order_sensitive&quot;: bool(plan.get(&quot;order_sensitive&quot;, True)),<br>        &quot;steps&quot;: normalized_steps,<br>    }<br>    return PlannerResponse.model_validate(normalized_plan)<br><br><br>class PlannerAgent:<br>    def __init__(self, llm: ChatOllama):<br>        self._planner = llm.with_structured_output(<br>            PlannerResponse,<br>            method=&quot;json_mode&quot;,<br>            include_raw=True,<br>        )<br><br>    async def ainvoke(self, inputs: dict[str, Any], config: dict[str, Any] | None = None) -&gt; dict[str, Any]:<br>        del config<br>        messages = inputs.get(&quot;messages&quot;, [])<br>        prompt_messages: list[BaseMessage] = [SystemMessage(content=DEFAULT_SYSTEM_PROMPT), *messages]<br>        result = await self._planner.ainvoke(prompt_messages)<br><br>        parsed = result.get(&quot;parsed&quot;)<br>        if parsed is not None:<br>            normalized_plan = _normalize_plan(parsed.model_dump())<br>        else:<br>            parsing_error = result.get(&quot;parsing_error&quot;)<br>            raw_response = result.get(&quot;raw&quot;)<br>            raw_content = getattr(raw_response, &quot;content&quot;, raw_response)<br>            try:<br>                raw_plan = json.loads(raw_content)<br>                normalized_plan = _normalize_plan(raw_plan)<br>            except Exception as exc:<br>                raise RuntimeError(<br>                    &quot;Planner returned an invalid structured response. &quot;<br>                    f&quot;parsing_error={parsing_error!r}, raw_content={raw_content!r}&quot;<br>                ) from exc<br><br>        plan = normalized_plan.model_dump()<br>        return {<br>            &quot;plan&quot;: plan,<br>            &quot;messages&quot;: [AIMessage(content=json.dumps(plan, ensure_ascii=True))],<br>        }<br><br><br>def build_agent(with_memory: bool = False):<br>    del with_memory<br>    llm = ChatOllama(<br>        model=os.getenv(&quot;OLLAMA_MODEL_2&quot;, &quot;llama3.1:8b&quot;),<br>        base_url=os.getenv(&quot;OLLAMA_BASE_URL&quot;, &quot;http://localhost:11434&quot;),<br>        temperature=0,<br>    )<br>    return PlannerAgent(llm)</pre><p>Things to pay attention to:</p><ol><li>System Prompt</li><li>Agent response contract classes (Planner Step, Planner Response)</li></ol><p>Finally, the following code is the one responsible to test our agent using LLM-as-a-judge strategy</p><pre>import importlib<br>import json<br>import os<br>import sys<br>from pathlib import Path<br><br>import pytest<br>from langchain_core.messages import HumanMessage<br>from langchain_ollama import ChatOllama<br>from openevals.llm import create_llm_as_judge<br><br>PROJECT_ROOT = Path(__file__).resolve().parents[1]<br>if str(PROJECT_ROOT) not in sys.path:<br>    sys.path.insert(0, str(PROJECT_ROOT))<br><br>TEST_CASES = json.loads(<br>    (Path(__file__).parent / &quot;data&quot; / &quot;tool_planner_cases.json&quot;).read_text(encoding=&quot;utf-8&quot;)<br>)<br><br><br>DEFAULT_AGENT_MODULE = &quot;app.planner_agent_v1&quot;<br><br>CORRECTNESS_PROMPT = &quot;&quot;&quot;<br>You are a helpful and precise assistant for evaluating the correctness of an agent&#39;s tool calls plan.<br><br>This is the agent&#39;s system prompt: {system_prompt}<br>This is the agent&#39;s input query: {inputs}<br>This is the agent&#39;s generated structured plan: {outputs}<br>This is the expected structured plan: {reference_outputs}<br><br>Your goal is to generate a score of 1 if the agent&#39;s structured plan is equivalent to the expected structured plan, and a score of 0 if it is not.<br><br>Evaluation rules:<br>- Respect the should_decline field.<br>- Respect the decline_reason field semantically, not by exact wording.<br>- Compare the steps field structurally.<br>- If order_sensitive is false, allow equivalent reordering of independent steps.<br>- If order_sensitive is true, require the step order to be correct.<br>- Tool names and arguments must match the intended plan.<br>&quot;&quot;&quot;<br><br>judge_llm = ChatOllama(<br>    model=os.getenv(&quot;OLLAMA_JUDGE_MODEL&quot;, &quot;llama3.1:8b&quot;),<br>    base_url=os.getenv(&quot;OLLAMA_BASE_URL&quot;, &quot;http://localhost:11434&quot;),<br>    temperature=0,<br>)<br><br>tool_eval = create_llm_as_judge(<br>    prompt=CORRECTNESS_PROMPT,<br>    judge=judge_llm,<br>)<br><br>def resolve_agent_module_name(pytestconfig=None):<br>    if pytestconfig is not None:<br>        agent_module = pytestconfig.getoption(&quot;agent_module&quot;)<br>        if agent_module:<br>            return agent_module<br><br>    return os.getenv(&quot;AGENT_MODULE&quot;, DEFAULT_AGENT_MODULE)<br><br><br>def load_build_agent(agent_module_name):<br>    agent_module = importlib.import_module(agent_module_name)<br>    return agent_module.build_agent<br><br><br>def load_default_system_prompt(agent_module_name):<br>    agent_module = importlib.import_module(agent_module_name)<br><br>    if not hasattr(agent_module, &quot;DEFAULT_SYSTEM_PROMPT&quot;):<br>        raise AttributeError(f&quot;{agent_module_name} is missing DEFAULT_SYSTEM_PROMPT&quot;)<br><br>    return agent_module.DEFAULT_SYSTEM_PROMPT<br><br><br>@pytest.fixture(scope=&quot;session&quot;)<br>def agent_builder(pytestconfig):<br>    return load_build_agent(resolve_agent_module_name(pytestconfig))<br><br><br>@pytest.fixture(scope=&quot;session&quot;)<br>def agent_system_prompt(pytestconfig):<br>    return load_default_system_prompt(resolve_agent_module_name(pytestconfig))<br><br>@pytest.mark.asyncio<br>@pytest.mark.parametrize(&quot;case&quot;, TEST_CASES, ids=[c[&quot;id&quot;] for c in TEST_CASES])<br>async def test_agent_tool_planning(case, agent_builder, agent_system_prompt):<br>    agent = agent_builder(with_memory=False)<br><br>    result = await agent.ainvoke({&quot;messages&quot;: [HumanMessage(content=case[&quot;prompt&quot;])]})<br>    actual_plan = result[&quot;plan&quot;]<br><br>    print(&quot;##############################&quot;)<br>    print(&quot;Agent Invocation Result: &quot;, actual_plan)<br><br>    expected_plan = {<br>        &quot;should_decline&quot;: case[&quot;should_decline&quot;],<br>        &quot;order_sensitive&quot;: case[&quot;order_sensitive&quot;],<br>        &quot;steps&quot;: case[&quot;expected_steps&quot;],<br>        &quot;decline_reason&quot;: case[&quot;decline_reason&quot;],<br>    }<br><br>    eval_result = tool_eval(<br>        system_prompt=agent_system_prompt,<br>        inputs=case[&quot;prompt&quot;],<br>        outputs=actual_plan,<br>        reference_outputs=expected_plan,<br>    )<br><br>    print(&quot;##############################&quot;)<br>    print(eval_result)<br>    <br>    assert eval_result[&quot;score&quot;] is True, {<br>        &quot;case&quot;: case[&quot;id&quot;],<br>        &quot;prompt&quot;: case[&quot;prompt&quot;],<br>        &quot;expected&quot;: expected_plan,<br>        &quot;actual&quot;: actual_plan,<br>        &quot;eval_result&quot;: eval_result,<br>    }</pre><blockquote>Notice how the Planner agent and the Judge LLM are using the exact same model i.e llama-3.1–8B , but if you want, you may also use a more capable LLM as a judge for better reliability.</blockquote><h3>Experiment Setup</h3><p>All tests were executed 5 times per version of the Agent, and we collect per test case pass/fail frequency.</p><p>I noticed something interesting while running the tests, and this is the Autoregressive nature of the LLM slows down the test bench. So I used a parallel test execution feature of pytest using pytest-xdist-plugin</p><p>Sample Pytest Command: pytest tests/some_test_file.py::some_case -n 4</p><p>Here, the -n 4 allows us to engage 4 CPU cores parallely to execute the test cases. Interestingly, since the output can be returned in any order, I had to write the individual output to a file and then read it. Pytest does it for you out of the box using the --junitxmlflag</p><p>When combining --junitxml with pytest-xdist (-n auto), the main master process automatically collects all test results from the parallel workers and compiles them into a single, unified XML report at the end of the run. You do not need to manually merge files.</p><p>Following is the experiment runner code I used.</p><pre>import argparse<br>import subprocess<br>import sys<br>import tempfile<br>import time<br>import xml.etree.ElementTree as ET<br>from pathlib import Path<br><br><br>PROJECT_ROOT = Path(__file__).resolve().parents[2]<br>TEST_FILE_PATH = PROJECT_ROOT / &quot;tests&quot; / &quot;test_planner_agent.py&quot;<br><br><br>def extract_case_id(test_name):<br> if &quot;[&quot; not in test_name or not test_name.endswith(&quot;]&quot;):<br>  return test_name<br><br> return test_name[test_name.rfind(&quot;[&quot;) + 1 : -1]<br><br><br>def build_pytest_command(workers, junit_xml_path, agent_module_name=None):<br> command = [<br>  sys.executable,<br>  &quot;-m&quot;,<br>  &quot;pytest&quot;,<br>  str(TEST_FILE_PATH),<br>  &quot;-n&quot;,<br>  str(workers),<br>  &quot;--junitxml&quot;,<br>  str(junit_xml_path),<br>  &quot;-q&quot;,<br> ]<br><br> if agent_module_name:<br>  command.extend([&quot;--agent-module&quot;, agent_module_name])<br><br> return command<br><br><br>def parse_junit_results(junit_xml_path):<br> root = ET.parse(junit_xml_path).getroot()<br> results = []<br><br> for testcase in root.iter(&quot;testcase&quot;):<br>  test_name = testcase.attrib.get(&quot;name&quot;, &quot;&quot;)<br>  case_id = extract_case_id(test_name)<br>  failure = testcase.find(&quot;failure&quot;)<br>  error = testcase.find(&quot;error&quot;)<br>  skipped = testcase.find(&quot;skipped&quot;)<br>  passed = failure is None and error is None and skipped is None<br><br>  results.append(<br>   {<br>    &quot;id&quot;: case_id,<br>    &quot;passed&quot;: passed,<br>    &quot;failure_text&quot;: (failure.text if failure is not None else None)<br>    or (error.text if error is not None else None)<br>    or (skipped.text if skipped is not None else None),<br>   }<br>  )<br><br> return results<br><br><br>def run_once(run_index, workers, agent_module_name=None):<br> with tempfile.TemporaryDirectory() as temp_dir:<br>  junit_xml_path = Path(temp_dir) / f&quot;planner-run-{run_index}.xml&quot;<br>  command = build_pytest_command(workers, junit_xml_path, agent_module_name)<br>  print(f&quot;[run {run_index}] starting: {&#39; &#39;.join(command)}&quot;, flush=True)<br>  start_time = time.perf_counter()<br>  completed = subprocess.run(<br>   command,<br>   cwd=PROJECT_ROOT,<br>   capture_output=True,<br>   text=True,<br>  )<br>  elapsed_seconds = time.perf_counter() - start_time<br>  print(<br>   f&quot;[run {run_index}] finished with exit code {completed.returncode} in {elapsed_seconds:.1f}s&quot;,<br>   flush=True,<br>  )<br><br>  if completed.returncode not in (0, 1):<br>   stderr = completed.stderr.strip()<br>   stdout = completed.stdout.strip()<br><br>   if &quot;unrecognized arguments: -n&quot; in stderr or &quot;No module named xdist&quot; in stderr:<br>    raise RuntimeError(<br>     &quot;pytest-xdist is required for parallel runs. Install it before running this experiment.&quot;<br>    )<br><br>   raise RuntimeError(<br>    f&quot;Pytest run {run_index} failed to execute.\nSTDOUT:\n{stdout}\n\nSTDERR:\n{stderr}&quot;<br>   )<br><br>  results = parse_junit_results(junit_xml_path)<br><br>  return {<br>   &quot;run&quot;: run_index,<br>   &quot;results&quot;: results,<br>   &quot;passed&quot;: sum(1 for item in results if item[&quot;passed&quot;]),<br>   &quot;failed&quot;: sum(1 for item in results if not item[&quot;passed&quot;]),<br>   &quot;stdout&quot;: completed.stdout,<br>   &quot;stderr&quot;: completed.stderr,<br>  }<br><br><br>def run_experiment(runs, workers, agent_module_name=None):<br> run_summaries = []<br> case_counts = {}<br> print(<br>  f&quot;Starting planner experiment: runs={runs}, workers={workers}, agent={agent_module_name or &#39;default test configuration&#39;}&quot;,<br>  flush=True,<br> )<br><br> for run_index in range(1, runs + 1):<br>  run_summary = run_once(run_index, workers, agent_module_name)<br>  run_summaries.append(run_summary)<br>  print(<br>   f&quot;[run {run_index}] summary: {run_summary[&#39;passed&#39;]} passed, {run_summary[&#39;failed&#39;]} failed&quot;,<br>   flush=True,<br>  )<br><br>  for outcome in run_summary[&quot;results&quot;]:<br>   counts = case_counts.setdefault(<br>    outcome[&quot;id&quot;],<br>    {&quot;passes&quot;: 0, &quot;failures&quot;: 0, &quot;last_failure&quot;: None},<br>   )<br><br>   if outcome[&quot;passed&quot;]:<br>    counts[&quot;passes&quot;] += 1<br>   else:<br>    counts[&quot;failures&quot;] += 1<br>    counts[&quot;last_failure&quot;] = outcome[&quot;failure_text&quot;]<br><br> return case_counts, run_summaries<br><br><br>def print_summary(case_counts, run_summaries, runs, workers, agent_module_name):<br> resolved_agent_module = agent_module_name or &quot;default test configuration&quot;<br> print(<br>  f&quot;Tool planner experiment finished: {runs} runs, workers={workers}, agent={resolved_agent_module}&quot;<br> )<br> print()<br> print(&quot;Per-run summary&quot;)<br> for summary in run_summaries:<br>  print(<br>   f&quot;- Run {summary[&#39;run&#39;]}: {summary[&#39;passed&#39;]} passed, {summary[&#39;failed&#39;]} failed&quot;<br>  )<br><br> print()<br> print(&quot;Failure counts by test case&quot;)<br> sorted_case_ids = sorted(<br>  case_counts,<br>  key=lambda case_id: (-case_counts[case_id][&quot;failures&quot;], case_id),<br> )<br><br> for case_id in sorted_case_ids:<br>  counts = case_counts[case_id]<br>  print(f&quot;- {case_id}: {counts[&#39;failures&#39;]} failures, {counts[&#39;passes&#39;]} passes&quot;)<br><br><br>def parse_args():<br> parser = argparse.ArgumentParser(<br>  description=&quot;Run the planner test suite repeatedly with parallel pytest workers and count failures per case.&quot;<br> )<br> parser.add_argument(<br>  &quot;--runs&quot;,<br>  type=int,<br>  default=5,<br>  help=&quot;Number of times to run the full planner test suite.&quot;,<br> )<br> parser.add_argument(<br>  &quot;--workers&quot;,<br>  default=&quot;auto&quot;,<br>  help=&quot;Value passed to pytest -n, for example auto, 2, or 4.&quot;,<br> )<br> parser.add_argument(<br>  &quot;--agent-module&quot;,<br>  default=None,<br>  help=&quot;Python module path for the agent under test, for example app.planner_agent_v1.&quot;,<br> )<br> return parser.parse_args()<br><br><br>def main():<br> args = parse_args()<br> case_counts, run_summaries = run_experiment(<br>  runs=args.runs,<br>  workers=args.workers,<br>  agent_module_name=args.agent_module,<br> )<br> print_summary(<br>  case_counts=case_counts,<br>  run_summaries=run_summaries,<br>  runs=args.runs,<br>  workers=args.workers,<br>  agent_module_name=args.agent_module,<br> )<br><br><br>if __name__ == &quot;__main__&quot;:<br> main()</pre><h3>Results</h3><p>Then I executed the tests, and I was able to pass them all after some debugging to address the input and output contract issues. Since the output shows 100% test clearance, which means the Planner agent was able to devise the correct execution plans and decline the requests when necessary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*KNXdDvNgrnPdM_hziQBlrg.png" /></figure><h3>Conclusion</h3><p>This is how I used LLM-as-a-Judge for the Planner agent to solve mathematical equations.</p><h4>Learnings:</h4><p>1. For LLM-as-a-Judge, the Agent as well as the Judge must use the same common language i.e the Response contracts must be the same. Otherwise we will be completely relying on the Judge LLM’s capability to understand the Agent output which can be unstructured, and spawn unnecessary data fields at the run time. We can use pydantic to address this, and fail fast.</p><p>2. LLM-as-a-Judge can be slow. If your CICD has even a few hundred LLM-as-a-Judge scenarios, it will become a bottleneck very fast, hence parallel test execution can be one good option to speedup the testing process.</p><p>But wait…that’s not it, we will have a walk through of more ways to evaluate your Agents in the future posts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=095fd0d46c56" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating Tool Calling in AI Agents with OpenEvals]]></title>
            <link>https://medium.com/@aditya-dawadikar/evaluating-tool-calling-in-ai-agents-with-openevals-d1125c925930?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1125c925930</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Fri, 29 May 2026 07:02:30 GMT</pubDate>
            <atom:updated>2026-05-29T07:02:30.429Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FqIxbA7yEbJfEGNM" /><figcaption>Photo by <a href="https://unsplash.com/@ryno_marais?utm_source=medium&amp;utm_medium=referral">Ryno Marais</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Goal: To ensure that our Agent selects the right tool for the task.</h3><p>An Agent needs tools to respond to the input. When the agent has a single tool i.e a single responsibility, its fairly straight forward for the agent act, and the outcome will be quite predictable. But what happens, when your Agent can use multiple tools? How can you ensure that your Agent knows which tool is the best fit for a given query?</p><p>To address this, we are going to incrementally build an Agent that can use simple tools to perform mathematical calculations. One of the challenges that an LLM faces is that it does not know the meaning of 2+2=4. For the LLM, either it memorizes the outcome of 2+2 which is 4, or it completely hallucinates. LLMs are not calculators. They can often answer arithmetic questions correctly, but there is no guarantee they will consistently perform calculations or map natural language requests to the correct operation without external tools. And today we are going to give our LLM a super power, i.e. math tools for various math operations.</p><p>In our case, we will build tools for operations like Add, Subtract, Multiply, Divide and Date Time. If a user asks a question like “what is 2.5 + 2.5?”, the LLM must answer “5” without hallucinating or guessing the correct answer.</p><p>There can be many variations for the same question.</p><ul><li>What is 2.5 + 2.5?</li><li>What is two point five + two point five?</li><li>What is two point five + 2.5?</li><li>What is 2.5 plus 2.5?</li><li>What do you get when you add 2.5 to 2.5?</li></ul><p>You see, how for a human brain each of these sentences mean the exact same thing? But our LLM struggles, because each scenario needs the LLM to understand that “2.5” and “Two point Five” are equivalent, similary “+” , “Plus” and “Add” are the exact same thing when it comes to this expression.</p><h3>Prerequisites</h3><ul><li>You must know how to deploy a local LLM using Ollama and preferably Docker</li><li>You must know how to use Pytest for testing traditional code</li><li>You must know basics of Langchain, Prompting and Agents</li></ul><h3>Tech Stack</h3><ul><li>Python 3.13.x</li><li>Langchain &gt; = 0.3.0</li><li>Langgraph &gt; = 0.2.0</li><li>Openevals &gt; = 0.2.0</li><li>Pytest &gt; = 9.0.3</li></ul><h3>Open Evals</h3><p><a href="https://open-evals.com/">Open Evals</a> is an opensource library for testing your Generative AI applications. It gives your a variety of evaluators, the one we are interested in for our use case is the <strong>JSON Match Evaluator</strong>. The JSON Match Evalutor <a href="https://github.com/langchain-ai/openevals/tree/main/python/openevals/json">`create_json_match_evaluator`</a> is the simplest kind of evaluator that one can use to compare structured output.</p><p>Irrespective of the Agent implementation, my goal is that the Agent must pass the following cases. This type of development is called Test Driven Development.</p><pre>[<br>  {<br>    &quot;id&quot;: &quot;add-basic&quot;,<br>    &quot;prompt&quot;: &quot;What is 14 plus 9?&quot;,<br>    &quot;expected_tool&quot;: &quot;add&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 14,<br>      &quot;b&quot;: 9<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;add-sum-language&quot;,<br>    &quot;prompt&quot;: &quot;Find the sum of 2.5 and 7.5.&quot;,<br>    &quot;expected_tool&quot;: &quot;add&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 2.5,<br>      &quot;b&quot;: 7.5<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;add-sum-language-2&quot;,<br>    &quot;prompt&quot;: &quot;Find the sum of twenty five and 7.&quot;,<br>    &quot;expected_tool&quot;: &quot;add&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 25,<br>      &quot;b&quot;: 7<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;subtract-basic&quot;,<br>    &quot;prompt&quot;: &quot;Subtract 18 from 45.&quot;,<br>    &quot;expected_tool&quot;: &quot;subtract&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 45,<br>      &quot;b&quot;: 18<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;subtract-difference-language&quot;,<br>    &quot;prompt&quot;: &quot;What is the difference between 100 and 63?&quot;,<br>    &quot;expected_tool&quot;: &quot;subtract&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 100,<br>      &quot;b&quot;: 63<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;multiply-basic&quot;,<br>    &quot;prompt&quot;: &quot;Multiply 6 by 7.&quot;,<br>    &quot;expected_tool&quot;: &quot;multiply&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 6,<br>      &quot;b&quot;: 7<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;multiply-product-language&quot;,<br>    &quot;prompt&quot;: &quot;What is the product of 3.5 and 8?&quot;,<br>    &quot;expected_tool&quot;: &quot;multiply&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 3.5,<br>      &quot;b&quot;: 8<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;divide-basic&quot;,<br>    &quot;prompt&quot;: &quot;Divide 144 by 12.&quot;,<br>    &quot;expected_tool&quot;: &quot;divide&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 144,<br>      &quot;b&quot;: 12<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;divide-quotient-language&quot;,<br>    &quot;prompt&quot;: &quot;What is the quotient when 81 is divided by 9?&quot;,<br>    &quot;expected_tool&quot;: &quot;divide&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;a&quot;: 81,<br>      &quot;b&quot;: 9<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;power-basic&quot;,<br>    &quot;prompt&quot;: &quot;What is 2 raised to the power of 5?&quot;,<br>    &quot;expected_tool&quot;: &quot;power&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;base&quot;: 2,<br>      &quot;exponent&quot;: 5<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;power-squared-language&quot;,<br>    &quot;prompt&quot;: &quot;Calculate 9 squared.&quot;,<br>    &quot;expected_tool&quot;: &quot;power&quot;,<br>    &quot;expected_arguments&quot;: {<br>      &quot;base&quot;: 9,<br>      &quot;exponent&quot;: 2<br>    }<br>  },<br>  {<br>    &quot;id&quot;: &quot;datetime-current-time&quot;,<br>    &quot;prompt&quot;: &quot;What time is it right now?&quot;,<br>    &quot;expected_tool&quot;: &quot;get_current_datetime&quot;,<br>    &quot;expected_arguments&quot;: {}<br>  },<br>  {<br>    &quot;id&quot;: &quot;datetime-current-date&quot;,<br>    &quot;prompt&quot;: &quot;Give me the current date and time.&quot;,<br>    &quot;expected_tool&quot;: &quot;get_current_datetime&quot;,<br>    &quot;expected_arguments&quot;: {}<br>  }<br>]</pre><p>This is the basic code that I wrote to test these scenarios.</p><pre># Some standard imports<br>import importlib<br>import json<br>import os<br>import sys<br>from pathlib import Path<br><br>import pytest<br>from langchain_core.messages import HumanMessage<br>from openevals.json import create_json_match_evaluator<br><br># Path Resolution for the files -&gt; You can ignore this<br>PROJECT_ROOT = Path(__file__).resolve().parents[1]<br>if str(PROJECT_ROOT) not in sys.path:<br>    sys.path.insert(0, str(PROJECT_ROOT))<br><br>TEST_CASES = json.loads(<br>    (Path(__file__).parent / &quot;data&quot; / &quot;tool_selection_cases.json&quot;).read_text(encoding=&quot;utf-8&quot;)<br>)<br><br># The actual test function<br>@pytest.mark.asyncio<br>@pytest.mark.parametrize(&quot;case&quot;, TEST_CASES, ids=[c[&quot;id&quot;] for c in TEST_CASES])<br>async def test_agent_tool_selection(case, agent_builder):<br>    <br>    # Build the Agent<br>    agent = agent_builder(with_memory=False)<br><br>    # Invoke the Agent<br>    result = await agent.ainvoke({&quot;messages&quot;: [HumanMessage(content=case[&quot;prompt&quot;])]})<br><br>    # Extract the output from the Agent response<br>    actual_tool_calls = normalize_tool_calls(<br>        extract_tool_calls(result),<br>        [<br>            {<br>                &quot;name&quot;: case[&quot;expected_tool&quot;],<br>                &quot;args&quot;: case[&quot;expected_arguments&quot;],<br>            }<br>        ],<br>    )<br><br>    # What is the expected output<br>    expected_tool_calls = [<br>        {<br>            &quot;name&quot;: case[&quot;expected_tool&quot;],<br>            &quot;args&quot;: case[&quot;expected_arguments&quot;]<br>        }<br>    ]<br><br>    # Evaluate the actual vs expected output<br>    eval_result = tool_eval(<br>        outputs=actual_tool_calls,<br>        reference_outputs=expected_tool_calls,<br>    )<br>    <br>    # We are simply asserting if the eval_result <br>    # has &quot;score&quot; field which is set to 1<br>    # The remaining dict is just an optional log<br>    # to verify the test and its data<br>    assert eval_result[0][&quot;score&quot;] == 1, {<br>        &quot;case&quot;: case[&quot;id&quot;],<br>        &quot;prompt&quot;: case[&quot;prompt&quot;],<br>        &quot;expected&quot;: expected_tool_calls,<br>        &quot;actual&quot;: actual_tool_calls,<br>        &quot;eval_result&quot;: eval_result,<br>    }<br></pre><p>It does not matter (at least for now) if you fully understand the code above or not, but what you must understand is that, we have a bunch of test cases that we stored as a JSON array, and then we use a single tester function to run each case one at a time.</p><p>Now, have a look at the basic Agent Implementation and the tool usage</p><pre>import os<br>from datetime import datetime<br><br>from langchain_core.tools import tool<br>from langchain_ollama import ChatOllama<br>from langgraph.checkpoint.memory import MemorySaver<br>from langgraph.prebuilt import create_react_agent<br><br><br>DEFAULT_SYSTEM_PROMPT = &quot;&quot;&quot;Use the available tools when needed.&quot;&quot;&quot;<br><br><br>@tool<br>def add(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Add two numeric values.<br><br>    Arguments must be named a and b.<br>    &quot;&quot;&quot;<br>    return a + b<br><br><br>@tool<br>def subtract(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Subtract b from a.<br><br>    Arguments must be named a and b, where the operation is a - b.<br>    &quot;&quot;&quot;<br>    return a - b<br><br><br>@tool<br>def multiply(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Multiply two numeric values.<br><br>    Arguments must be named a and b.<br>    &quot;&quot;&quot;<br>    return a * b<br><br><br>@tool<br>def divide(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Divide a by b.<br><br>    Arguments must be named a and b, where the operation is a / b.<br>    &quot;&quot;&quot;<br>    if b == 0:<br>        raise ValueError(&quot;Cannot divide by zero.&quot;)<br>    return a / b<br><br><br>@tool<br>def power(base: float, exponent: float) -&gt; float:<br>    &quot;&quot;&quot;Raise base to the power of exponent.<br><br>    Arguments must be named base and exponent. Do not use a and b for this tool.<br>    &quot;&quot;&quot;<br>    return base ** exponent<br><br><br>@tool<br>def get_current_datetime() -&gt; str:<br>    &quot;&quot;&quot;Return the current date and time.<br><br>    This tool takes no arguments.<br>    &quot;&quot;&quot;<br>    return datetime.now().strftime(&quot;%Y-%m-%d %H:%M:%S&quot;)<br><br><br>TOOLS = [add, subtract, multiply, divide, power, get_current_datetime]<br><br><br>def build_agent(with_memory: bool = False):<br>    llm = ChatOllama(<br>        model=os.getenv(&quot;OLLAMA_MODEL&quot;, &quot;llama3.2:1b&quot;),<br>        base_url=os.getenv(&quot;OLLAMA_BASE_URL&quot;, &quot;http://localhost:11434&quot;),<br>    )<br>    checkpointer = MemorySaver() if with_memory else None<br>    return create_react_agent(<br>        llm,<br>        TOOLS,<br>        prompt=DEFAULT_SYSTEM_PROMPT,<br>        checkpointer=checkpointer,<br>    )</pre><p>This is the simplest form of ReAct Agent that one can build with Langchain. We have created a bunch of tools using the @tool decorator and that tells our Agent that we can perform some action using the tool. The doc strings inside the tool tells the Agent what exactly the tool does. Consider it as the critical documentation that the Agent needs in order to effectively use the tool.</p><h3>Experimentation Setup</h3><ol><li>All tests were executed 5 times per version of the Agent, and we collect per test case pass/fail frequency</li></ol><h3>Version 1: Bare Minimum Agent Harness</h3><p>I started out with the smallest meaningful model that was deployable on my laptop -&gt; Llama-3.2–1B model. This is a tiny model of the Llama family with 1B parameters. I deployed it locally using Ollama inside a Docker container, you may choose to use remote hosted LLM endpoints if you have access to it.</p><p>When I executed the tests for the first time, I knew some of the test cases would fail. But what surprised me was the non-deterministic nature of the Agentic AI. Since the LLMs are stochastic in nature, the same test case may pass in some runs and fail in other.</p><p>So I decided to run the same experiment multiple times to understand the distribution, which cases are easy enough to get them right each time, vs which cases are the trickiest ones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*WHxnhyd-LIQG3hLW7Ddkbg.png" /><figcaption>Agent V1 Baseline Results</figcaption></figure><p>To my surprise, my LLM agent did not do well. It failed almost 45% of the times i.e Success rate ~55%. Interestingly, the Agent did not select the wrong tool for the test, but it did skip using the tool entirely, or passed incorrect input to the tool.</p><p>I pin pointed the exact cases where my agent failed miserably</p><ol><li>add-sum-language</li><li>multiply-product-language</li><li>subtract-difference-language</li></ol><p>This gave me an understanding that my Agent understands the task when I pass numbers directly in the input. But it struggles to translate the text representation of the same expression.</p><p>Second issue I noticed was some cases where most of the times Agent understood what to do, except when it did not. This was easier to fix.</p><p>So my fix? <strong>Give better system prompt and tool description.</strong></p><p>With this setting, I moved on to developing the V2 of my agent.</p><h3>Version 2: Improved System Prompt and Tool Description</h3><p>The Version 2 used the exact same LLM model. The only thing that changed was the Prompt. I added some few-shot examples, detailed rules for when to and when no to use a given tool, Input and Output contracts for the tools.</p><pre>import os<br>from datetime import datetime<br><br>from langchain_core.tools import tool<br>from langchain_ollama import ChatOllama<br>from langgraph.checkpoint.memory import MemorySaver<br>from langgraph.prebuilt import create_react_agent<br><br><br>DEFAULT_SYSTEM_PROMPT = &quot;&quot;&quot;You are a tool-using assistant.<br><br>You have access to these tools and should use them explicitly when the user asks for the matching operation:<br>- add(a, b): use for addition, plus, total, or sum requests.<br>- subtract(a, b): use for subtraction, minus, difference, or take-away requests. The operation is a - b.<br>- multiply(a, b): use for multiplication, times, or product requests.<br>- divide(a, b): use for division or quotient requests. The operation is a / b.<br>- power(base, exponent): use for exponentiation requests such as raised to the power, squared, or cubed.<br>- get_current_datetime(): use when the user asks for the current date, current time, or current date and time.<br><br>Tool-calling rules:<br>- For arithmetic or date/time requests, call the relevant tool instead of answering directly.<br>- Match the tool argument names exactly.<br>- For add, subtract, multiply, and divide, provide arguments named a and b.<br>- For power, provide arguments named base and exponent.<br>- Do not invent extra arguments.<br>- If a tool applies, make the tool call before giving a final answer.<br>&quot;&quot;&quot;<br><br><br>@tool<br>def add(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Add two numeric values.<br><br>    Use this for addition requests such as plus, add, total, or sum.<br>    Format the tool input as {&quot;a&quot;: &lt;first number&gt;, &quot;b&quot;: &lt;second number&gt;}.<br>    Arguments must be named a and b.<br>    &quot;&quot;&quot;<br>    return a + b<br><br><br>@tool<br>def subtract(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Subtract b from a.<br><br>    Use this for subtraction requests such as subtract, minus, difference, or take away.<br>    Format the tool input as {&quot;a&quot;: &lt;starting number&gt;, &quot;b&quot;: &lt;number to subtract&gt;}.<br>    Arguments must be named a and b, where the operation is a - b.<br>    &quot;&quot;&quot;<br>    return a - b<br><br><br>@tool<br>def multiply(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Multiply two numeric values.<br><br>    Use this for multiplication requests such as multiply, times, or product.<br>    Format the tool input as {&quot;a&quot;: &lt;first factor&gt;, &quot;b&quot;: &lt;second factor&gt;}.<br>    Arguments must be named a and b.<br>    &quot;&quot;&quot;<br>    return a * b<br><br><br>@tool<br>def divide(a: float, b: float) -&gt; float:<br>    &quot;&quot;&quot;Divide a by b.<br><br>    Use this for division requests such as divide or quotient.<br>    Format the tool input as {&quot;a&quot;: &lt;dividend&gt;, &quot;b&quot;: &lt;divisor&gt;}.<br>    Arguments must be named a and b, where the operation is a / b.<br>    &quot;&quot;&quot;<br>    if b == 0:<br>        raise ValueError(&quot;Cannot divide by zero.&quot;)<br>    return a / b<br><br><br>@tool<br>def power(base: float, exponent: float) -&gt; float:<br>    &quot;&quot;&quot;Raise base to the power of exponent.<br><br>    Use this for exponentiation requests such as power, raised to, squared, or cubed.<br>    Format the tool input as {&quot;base&quot;: &lt;base number&gt;, &quot;exponent&quot;: &lt;power&gt;}.<br>    Arguments must be named base and exponent. Do not use a and b for this tool.<br>    &quot;&quot;&quot;<br>    return base ** exponent<br><br><br>@tool<br>def get_current_datetime() -&gt; str:<br>    &quot;&quot;&quot;Return the current date and time.<br><br>    Use this when the user asks for the current date, current time, or both.<br>    Format the tool input as {} because this tool takes no arguments.<br>    This tool takes no arguments.<br>    &quot;&quot;&quot;<br>    return datetime.now().strftime(&quot;%Y-%m-%d %H:%M:%S&quot;)<br><br><br>TOOLS = [add, subtract, multiply, divide, power, get_current_datetime]<br><br><br>def build_agent(with_memory: bool = False):<br>    llm = ChatOllama(<br>        model=os.getenv(&quot;OLLAMA_MODEL&quot;, &quot;llama3.2:1b&quot;),<br>        base_url=os.getenv(&quot;OLLAMA_BASE_URL&quot;, &quot;http://localhost:11434&quot;),<br>    )<br>    checkpointer = MemorySaver() if with_memory else None<br>    return create_react_agent(<br>        llm,<br>        TOOLS,<br>        prompt=DEFAULT_SYSTEM_PROMPT,<br>        checkpointer=checkpointer,<br>    )</pre><p>And following are the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*lFPTWmZq7WNPuKV1TqRvbA.png" /><figcaption>Agent V2 Benchmarking Results</figcaption></figure><p>From the results it was clear that something did work. My tool usage accuracy went from 55% to roughly 65%.</p><p>This is interesting because, all we changed was the Prompts.</p><p>Lesson Learned !</p><ul><li>Write better prompts.</li><li>Your LLM output is only as good as the context.</li><li>Garbage In = Garbage Out.</li></ul><p>But we are still not done. I again analyzed the failure points.</p><p>The pain point is no longer the part where my Agent misses the tool usage, but its something more serious. My Agent was unable to parse the Query, understand the numbers and then effectively pass the arguments to my tools.</p><p>In other words, my LLM was not as semantically excellent as I though it was. I tried a few more times with varying details and examples in the prompts, but the needle did not move.</p><p>So I decided to upgrade my LLM.</p><h3>Version 3: Upgrading LLM (Llama-3.2–1B to Llama-3.1–8B)</h3><p>For Version 3, I used an 8B variant of the Llama model, hoping that it would capture the semantic meaning of the input query better than the 1B model.</p><p>And guess what happened…it worked. The only thing I changed in my code was the model I was using. Everything else remained the same i.e. I used the exact same prompt and tool description that I used for Agent V2.</p><pre>def build_agent(with_memory: bool = False):<br>    llm = ChatOllama(<br>        model=os.getenv(&quot;OLLAMA_MODEL&quot;, &quot;llama3.1:8b&quot;),<br>        base_url=os.getenv(&quot;OLLAMA_BASE_URL&quot;, &quot;http://localhost:11434&quot;),<br>    )<br>    checkpointer = MemorySaver() if with_memory else None<br>    return create_react_agent(<br>        llm,<br>        TOOLS,<br>        prompt=DEFAULT_SYSTEM_PROMPT,<br>        checkpointer=checkpointer,<br>    )</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*3yP1wNMzDC73yHqbug_tsA.png" /><figcaption>Agent V3 Benchmarking Results</figcaption></figure><p>As you can see in the above results, across all 5 runs, I had a tool selection success rate of 100%.</p><p>Interestingly, subtract-difference-language improved from 0/5 passes to 5/5 passes after prompt improvements, while add-sum-language and multiply-product-language remained at 0/5. This suggested that prompting alone was not enough and that model capability had become the primary bottleneck.</p><p>Lesson Learned!!!</p><ul><li>Bigger is not always better, but in our case it worked because our tests were failing due to poor semantic understanding by the model.</li><li>I would have not chosen a bigger model if it wasn’t for the above reason.</li></ul><h3>Conclusion</h3><p>So this is how I unit tested a very simple AI Agent. Took the tool selection accuracy from 55% to 100% by experimenting with my approach.</p><p>We went from V1 → V2: 55% → 65% (+10%) -&gt; By using Better Prompting technique.</p><p>We again went from V2 → V3: 65% → 100% (+35%) -&gt; By improving the model capability.</p><p>So we can say that, in our case</p><blockquote><strong>Prompting helped improve tool usage behavior, but model capability was the primary bottleneck for semantic understanding.</strong></blockquote><p>But wait, this is not all. There are many aspects that must be tested for a good production ready Agent, and I will be writing about that in the upcoming articles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1125c925930" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CellCraft-AI: The Final Important Bits]]></title>
            <link>https://medium.com/@aditya-dawadikar/cellcraft-ai-the-final-important-bits-fda9b5ad7da0?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/fda9b5ad7da0</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Mon, 07 Jul 2025 15:57:39 GMT</pubDate>
            <atom:updated>2025-07-07T15:57:39.553Z</atom:updated>
            <content:encoded><![CDATA[<p>In the previous blog, I talked about how I engineered the context for CellCraft-AI. In today’s blog I will discuss how LangChain’s introduction to the application future proofs CellCraft-AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eZ3yX93q3Ye2GREa" /><figcaption>Photo by <a href="https://unsplash.com/@mikezo?utm_source=medium&amp;utm_medium=referral">Mike Alonzo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Challenges</h3><ol><li>CellCraft-AI was based on Gemini-2.5-Flash, the sole reason being it provided a free tier. But in future, we want an easily swapable system, where we can plug-n-play with various LLM models.</li><li>Although we had a persistent condensed ground truth with <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai/blob/docs/docs/ContextCondensation.md">Checkpointing based Context Management</a> , yet there is a need for operational memory short term memory for coherent responses.</li><li>In future, we might want to integrate other tools like SQL Engines to CellCraft-AI, and we cannot build it from scratch for fast scaling.</li></ol><h3>The Solution</h3><p>To future proof CellCraft-AI I introduced LangChain — The framework that enables us to chain together various components of an Agentic AI ecosystem.</p><ol><li>One of the key benefits of using LangChain was that it provides an abstraction layer on various LLMs, so in future, my could would not have to commit to a single LLM model. Cherry on the cake is that, it provides a standardized way for prompts management and templating.</li><li>In some quick chat modes or for non transformative queries, we can still use LangChain’s memory eg: BufferWindowMemory to maintain short term flow without polluting the Checkpoint history.</li><li>LangChain provides integrations with third party Vector Stores and Databases, which can help us scale the application or add more features later on.</li></ol><p>With this vision, I integrated LangChain with CellCraft-AI. There are add-ons like <a href="https://www.langchain.com/langgraph">LangGraph</a> and <a href="https://www.langchain.com/langsmith">LangSmith</a>, which can helps us turn CellCraft-AI into a multi agent scalable product.</p><h3>The Implementation</h3><p>For now, I was simply going to use the Chat models abstraction and Prompt Templating feature.</p><p>These are the prompt templates I used for generating responses and summaries:</p><pre># langchain_prompt.py<br><br>from langchain_core.prompts import ChatPromptTemplate<br><br>transform_prompt = ChatPromptTemplate.from_messages([<br>    (&quot;system&quot;, &quot;&quot;&quot;<br>        You are a data cleaning assistant for CSV files.<br><br>        You operate in an isolated Python environment. You are allowed to:<br>        - Use ONLY the following libraries: pandas, numpy, matplotlib (as plt), seaborn (as sns)<br>        - Answer only questions about the data provided in the CSV<br>        - Politely decline unrelated questions and suggest asking questions about the dataset<br><br>        Always write code&#39;s output to a markdown or a readme file and save it<br>        When visualizing data (using matplotlib/seaborn), DO NOT show or display the plot. Instead, save the chart to a file using `plt.savefig(&quot;&lt;filename&gt;&quot;)`.<br>        When describing data (using pandas), DO NOT print or display the data. Instead, save the response as markdown or txt file.<br>                    <br>        always generate some text based code output for manual verification<br><br>        You must respond using one of the following JSON formats:<br><br>        If replying in plain text:<br>        {{<br>        &quot;mode&quot;: &quot;CHAT&quot;,<br>        &quot;response&quot;: &quot;&lt;your message&gt;&quot;<br>        }}<br><br>        If applying transformation:<br>        {{<br>        &quot;mode&quot;: &quot;CODE&quot;,<br>        &quot;key_steps&quot;: &quot;&lt;summary&gt;&quot;,<br>        &quot;executable_code&quot;: &quot;&lt;python pandas code&gt;&quot;,<br>        &quot;response&quot;: &quot;&lt;acknowledgement of transforms&gt;&quot;<br>        }}<br><br>        You can also manage which CSV version you&#39;re working on.<br>        If the user says &quot;undo&quot;, &quot;redo&quot;, or &quot;start over from commit XYZ&quot;, respond in CONTEXT mode.<br><br>        If changing HEAD:<br>        {{<br>        &quot;mode&quot;: &quot;CONTEXT&quot;,<br>        &quot;action&quot;: &quot;checkout&quot;,<br>        &quot;target_commit_id&quot;: &quot;&lt;commit_id&gt;&quot;,<br>        &quot;response&quot;: &quot;&lt;acknowledgement of context change&gt;&quot;<br>        }}<br><br>        If creating a new branch from an older commit:<br>        {{<br>        &quot;mode&quot;: &quot;CONTEXT&quot;,<br>        &quot;action&quot;: &quot;branch&quot;,<br>        &quot;target_commit_id&quot;: &quot;&lt;commit_id&gt;&quot;,<br>        &quot;response&quot;: &quot;&lt;acknowledgement of context change&gt;&quot;<br>        }}<br><br>        IMPORTANT:<br>        - All output must be a single valid JSON object.<br>        - Never include code outside the &quot;executable_code&quot; field.<br>        - Do not attempt to write files to directories other than the current working directory.<br><br>        When saving files like charts or summaries, use the variable commit_dir as the output folder. Example:<br>        plt.savefig(f&quot;{{commit_dir}}/myplot.png&quot;)<br>        or<br>        with open(f&quot;{{commit_dir}}/summary.md&quot;, &quot;w&quot;) as f: f.write(summary)<br><br>        You are provided the DataFrame as a variable named `df`. <br>        - Do NOT use `pd.read_csv()` or try to load &#39;df.csv&#39;.<br>        - The DataFrame is already available in memory.<br>        - Any cleaned or transformed result should overwrite the existing `df` variable.<br><br>        Return ONLY valid JSON.<br>    &quot;&quot;&quot;),<br>    (&quot;human&quot;, &quot;&quot;&quot;<br>        Data Preview:<br>        {preview}<br><br>        Previous Steps:<br>        {context}<br><br>        User Instruction:<br>        {query}<br>    &quot;&quot;&quot;)<br>])</pre><pre># langchain_summary_prompt.py<br><br>from langchain_core.prompts import ChatPromptTemplate<br><br>history_summary_prompt = ChatPromptTemplate.from_messages([<br>    (&quot;system&quot;, &quot;&quot;&quot;<br>        You are a summarization assistant for data transformation histories.<br><br>        Your task:<br>        - Summarize the following history into a concise, structured changelog using this exact YAML-like 3-line format:<br><br>        state: &quot;&lt;brief description of initial data condition or problem&gt;&quot;<br>        action: &quot;&lt;brief description of transformation attempted&gt;&quot;<br>        outcome: &quot;&lt;brief description of result OR error if failed&gt;&quot;<br><br>        Rules:<br>        - If the history indicates success: summarize the successful outcome.<br>        - If the history indicates an error: describe the error in the outcome field.<br>        - Do NOT write full sentences—keep each field short (max 10 words).<br>        - Never skip any of the 3 fields.<br><br>        Example (Success):<br>        state: &quot;null values in column A&quot;<br>        action: &quot;applied fillna() to column A&quot;<br>        outcome: &quot;missing values removed&quot;<br><br>        Example (Failure):<br>        state: &quot;attempted to plot histogram&quot;<br>        action: &quot;called plt.hist() with invalid data&quot;<br>        outcome: &quot;error: incompatible data type for plotting&quot;<br>    &quot;&quot;&quot;),<br>    (&quot;human&quot;, &quot;&quot;&quot;Here is the transformation history:<br>        {full_history}<br>    &quot;&quot;&quot;)<br>])</pre><p>With the templating in place, all I had to do was invoke the LLM that is wrapped using LangChain.</p><pre># langchain_llm.py<br><br>from langchain_google_genai import ChatGoogleGenerativeAI<br>import os<br>import dotenv<br>dotenv.load_dotenv()<br><br>GEMINI_API_KEY = os.getenv(&quot;GEMINI_API_KEY&quot;)<br><br>llm = ChatGoogleGenerativeAI(<br>    model=&quot;gemini-2.5-flash&quot;,<br>    google_api_key = GEMINI_API_KEY,<br>    temperature=0.3<br>)</pre><pre># langchain_chain.py<br><br>from services.langchain_prompt import transform_prompt<br>from services.langchain_summary_prompt import history_summary_prompt<br>from services.langchain_llm import llm<br><br>transform_chain = transform_prompt | llm<br>history_summary_chain = history_summary_prompt | llm</pre><pre># generating data transform responses<br><br>inputs = {<br>            &quot;preview&quot;: df_preview,<br>            &quot;context&quot;: key_step_changelog,<br>            &quot;query&quot;: query<br>        }<br><br>response = await transform_chain.ainvoke(inputs)<br>response_text = response.content</pre><pre># checkpoint summarization call<br><br>summary_response = await history_summary_chain.ainvoke({<br>  &quot;full_history&quot;: full_history<br>})<br>condensed_summary = summary_response.content.strip()</pre><h3>The Impact</h3><p>Now CellCraft-AI had the shape and feel of a real Agentic AI, similar to ones that are used in the industry in 2025. It had a simple Plug-n-Play modal for LLMs, clean templating for prompts and potential to grow into something bigger in future.</p><h3>Current Architecture</h3><p>Now the MVP was solid and extensible. It had all the elements necessary for scaling. Persistent data storage with MongoDB and S3 bucket, Caching with Redis, Agentic AI powered by LangChain, Python based Sandbox environment and a user friendly UI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*Jt8NSBZcCnX9chGc5cBAjQ.png" /><figcaption>CellCraft-AI Architecture</figcaption></figure><h3>Conclusion</h3><p>With these additions, I finally rest. CellCraft-AI — an Agentic AI was ready. Built with passion and tinkering habit. I learned valuable lessons around how important it is provide more in terms of context to the LLM with less in terms of tokens.</p><p>I don’t have immediate plans to extend CellCraft-AI. But I would love to know your ideas, how it can grow further. I’m making it open source and I’m open to collaborations. Thank you for your time.</p><p>Source Code: <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai">https://github.com/Aditya-Dawadikar/cell-craft-ai</a></p><p>Previous Blogs:</p><ol><li><a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-how-i-built-the-poc-5ee3d8ad029e">CellCraft-AI: How I built the POC</a></li><li><a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-from-poc-to-mvp-1a6d5404a67c">CellCraft-AI: From POC to MVP</a></li><li><a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-engineering-the-context-adf98328e81e">CellCraft-AI: Engineering the Context</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fda9b5ad7da0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CellCraft-AI: Engineering the Context]]></title>
            <link>https://medium.com/@aditya-dawadikar/cellcraft-ai-engineering-the-context-adf98328e81e?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/adf98328e81e</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[markov-decision-process]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Mon, 07 Jul 2025 02:35:42 GMT</pubDate>
            <atom:updated>2025-07-07T02:35:42.509Z</atom:updated>
            <content:encoded><![CDATA[<p>In the <a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-from-poc-to-mvp-1a6d5404a67c">last blog</a> I talked about how I built an MVP based on the POC. I ended the blog with a note on some challenges. In this blog, I will talk majorly about engineering the context for CellCraft-AI’s agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R-3j7Q9OMWcm9Xcx" /><figcaption>Photo by <a href="https://unsplash.com/@thisisengineering?utm_source=medium&amp;utm_medium=referral">ThisisEngineering</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Problem</h3><p>By our design, we needed the key_steps field in the commits data structure to generate new responses. key_steps when aggregated, would form the long term memory of our application. But what happens when we have 100s of commit documents, each with their own key_steps? It’s easy to imagine how long the context becomes very quickly.</p><h3>The Solution</h3><p>To find the solution, I took inspiration from how memory in human beings work. We have two types of memories — Short Term Memory and Long Term Memory. Short Term Memory is the memory that remembers the NOW. Imagine how you remember an OTP just long enough to type it. It has no value beyond that instance, so our brain forgets it. Where as the Long Term Memory is used when you want to remember something for a really long time, sometimes even the life time. For instance, the names of our loved ones.</p><p>I needed a similar design. Which means, I needed a summary of summaries. Something that is compact enough, that when it is expanded, it can help us get the gist of all the steps we performed so far. While we are working on the Agents, a very famous way to describe an Agent’s activity in Computer Science is using the Markov Decision Process (MDP).</p><p>An MDP is a mathematical framework for modelling the decision process of Agents that are partly random and partly deterministic, controlled by an Agent. An MDP is formally defined by a <strong>5-tuple</strong>:</p><p>MDP=(S,A,P,R,γ)</p><p>Where:</p><ol><li><strong>S</strong> → Finite set of <strong>states</strong></li><li><strong>A</strong> → Finite set of <strong>actions</strong></li><li><strong>P(s′∣s,a)</strong> → <strong>Transition probability function</strong>:<br> Probability of moving to state s′ when taking action a in state s</li><li><strong>R(s,a,s′)</strong> → <strong>Reward function</strong>:<br> Immediate reward received after moving from s to s’ via action a</li><li><strong>γ</strong> → <strong>Discount factor</strong> (0≤γ≤1):<br> Determines the present value of future rewards.</li></ol><p>Although, I don’t need the Transition Probability, Reward Function and the Discount Factor, I still need something that represents the existing state (steps), the action, and the outcome of the action.</p><p>So I devised a 3 tuple representation — Context=(s,a,o)</p><p>where,</p><p><strong>s — State</strong>: is the steps performed so far</p><p><strong>a — Action</strong>: is the action taking place right now</p><p><strong>o — Outcome</strong>: is the outcome of the action</p><p>I called it the condensed form of the key_steps from previous m commits.</p><p>Previously where we needed roughly 50–80 tokens to represent the steps for a commit, now we need only 15–20 tokens. That’s almost 1/4 of the previous token count.</p><h3>The Implementation</h3><p>Now that I had a way to extract the essence of my conversational history, I need some sort of checkpointing logic. This point will hold the conversational summary of all the commits occurring before this point in time. At any instance, when we want to converse with the application, we form context using the following formula</p><p>Working Context = Latest Checkpoint + Un-Checked Commits</p><p>So with this as the basic idea, I introduced a new data structure called the checkpoint . This will not be accessible to the user, and be internally used by the Agent to optimize the query.</p><pre>class Checkpoint(Document):<br>    checkpoint_id: str<br>    session_id: str<br>    timestamp: str<br>    parent_checkpoint: Optional[str] = None<br>    summary: str<br>    commit_ids: List[str]</pre><p>I developed a simple algorithm around this formula.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*dRv8wsUg1BPGF5eHjKPqDg.png" /><figcaption>Checkpoint based Context Management Algorithm</figcaption></figure><p>On a high level, we took the compressed summary coming from older chats, along with the uncompressed recent chats and combined it together to form the working context for the current query.</p><h3>The Impact</h3><p>This sped up the response time, while reducing the token count. I wish I could benchmark it, but due to lack of resources, I can simply estimate the gains after adopting this method.</p><p>To put things into perspective, in our earlier approach, each commit message used to consume around <strong>50–80 tokens</strong> on average. With <strong>100 commits</strong>, that could easily add up to <strong>8,000 tokens</strong> just to maintain the context — a number that’s clearly impractical for most language models. With the <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai/blob/docs/docs/ContextCondensation.md"><strong>Checkpoint based</strong> <strong>Context Management</strong></a><strong> </strong>mechanism where older commit messages are condensed to around <strong>20 tokens</strong> each, while retaining the <strong>last 10 commits in full</strong> for detailed context. This reduces the total to roughly <strong>3,500 tokens</strong> — a significant <strong>67.5% reduction </strong>(In Theory). This balance allows CellCraft-AI to stay informed about the entire transformation history without overwhelming the token limits of the LLM.</p><h3>Can we do better?</h3><p>Oh yes! absolutely. Although I have not introduced this yet, but we can easily limit the number of previous commits used to build a checkpoint. With the current logic, at some point the Checkpoint summary itself would become too large — and that’s when we can skip the ancient commit summaries to move on with the transformations.</p><p>If checkpoint based context management was not in place, we would be stuck with using the previous 10–20 commits as context, which might not have been enough for newer transformations. So, in a way, <strong>checkpointing has given us more breathing room to increase the depth and effectiveness of our context</strong>.</p><h3>Conclusion</h3><p>In this blog, I have covered the Context Engineering aspect of CellCraft-AI. My learnings were tremendous building this application, and I independently developed a new algorithm for condensing the summaries for long term memory of CellCraft-AI.</p><p>In the next blog, I will talk about, how I future proofed CellCraft-AI by using LangChain. Stay Tuned.</p><p>Source Code: <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai">https://github.com/Aditya-Dawadikar/cell-craft-ai</a></p><p>Previous Blogs:</p><ol><li><a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-how-i-built-the-poc-5ee3d8ad029e">CellCraft-AI: How I built the POC</a></li><li><a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-from-poc-to-mvp-1a6d5404a67c">CellCraft-AI: From POC to MVP</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adf98328e81e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CellCraft-AI: From POC to MVP]]></title>
            <link>https://medium.com/@aditya-dawadikar/cellcraft-ai-from-poc-to-mvp-1a6d5404a67c?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a6d5404a67c</guid>
            <category><![CDATA[system-design-project]]></category>
            <category><![CDATA[full-stack-development]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Sun, 06 Jul 2025 21:07:57 GMT</pubDate>
            <atom:updated>2025-07-06T21:07:57.054Z</atom:updated>
            <content:encoded><![CDATA[<p>In the <a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-how-i-built-the-poc-5ee3d8ad029e">last blog</a> , I talked about the basic idea and implementation approach for CellCraft-AI. With python’s exec() and data structures like sessions and commits I was able to perform data transforms and visualizations. In this blog, I will talk about how I turned the idea into a user friendly web application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q7Oup3Dcz0sMC0qd" /><figcaption>Photo by <a href="https://unsplash.com/@emilianovittoriosi?utm_source=medium&amp;utm_medium=referral">Emiliano Vittoriosi</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Features</h3><p>Before building any application, it’s important to have a clear set to feature expectations. For CellCraft-AI, I wanted a conversational window like ChatGPT and a Data Preview panel. Since I had introduced the versioning mechanism to the application, it only made sense to be able to show the version information to the user. So my feature list looked something like this:</p><ol><li>An interactive Chat Window</li><li>A Data Preview Panel capable of rendering the transformed csv files, as well as generated output files like matplotlib charts (.png) and text based code output (.md)</li><li>A commit DAG (Directed Acyclic Graph) to represent the version history</li><li>A navigation bar to select a chat session and button to create a new session</li><li>In the Data Preview panel, I can select the commit for which I want to view the output files, but by default the Data Preview would show data for the latest commit. Data Preview will be updated in real-time as the Agent responds</li></ol><p>With this I knew which APIs I had to build.</p><h3>REST API Design</h3><ol><li>POST /transform_csv : An api that will accept user’s input query and generate 3 types of responses — Chat, Code and Context change acknowledgement</li><li>GET /version_history : Api to fetch the commit thread, to render the commit DAG on the Data Preview Panel</li><li>GET /chat_history : Api to fetch entire conversation to render the Chat Panel</li><li>GET /commit_file_list : API to download all the generated files associated with the commit</li><li>And lastly, some CRUD APIs for Commit and Session</li></ol><p>Next, I wanted to decide the tech stack. Which starts with identifying the type of work load of my application. To read more about understanding data workloads, please check out this <a href="https://medium.com/@aditya-dawadikar/system-design-choosing-the-right-database-part-1-understanding-workloads-4c1d42cc7555">blog</a>.</p><h3>Understanding the CellCraft-AI’s Data Workload</h3><p>CellCraft-AI is a chat application, which performs data transformations. Which means it will have unstructured data. Specifically, the Chat will be more like a Document with flexible Schema, and the generated output files will be File Objects.</p><p>For the POC I was comfortable with storing the Chat conversation as JSON objects, but retrieval will become very slow as the Chat grows longer, or if we have multiple user sessions. Similarly, a static directory for the output files was sufficient. But for scaling the application, we needed something powerful. Especially because we want our application to be a stateless container on Kubernetes.</p><p>I had a couple of options, which provided free tier storage— Firebase and MongoDB for document storage, and Google Cloud Storage (GCS) or Amazon Simple Storage Service (S3). I have worked on GCP before, so for a change, I opted for S3 bucket. And I chose MongoDB over Firebase for document storage.</p><p>The rationale behind using MongoDB was, MongoDB is more widely used in the industry, making my skillset more relevant, at the same time Firestore has a limit on the number of indexes we can create. Although, you can request for increased number of indexes. CellCraft-AI is a hobby project, and won’t need as many indexes, yet it’s better to opt for a more powerful database considering the scalability in future. As the project will grow; migrating form Firebase to MongoDB will become a technical debt, so I opted for MongoDB from the very beginning.</p><h3>Developing the UI</h3><p>I am really comfortable building web apps with <a href="https://react.dev/">React</a>, so it is a go-to framework for me. It started out with simple state management with useState() and useEffect() hooks, but we needed a central store so that we can communicate between the Chat Panel and the Data Preview Panel. So I introduced a <a href="https://redux.js.org/api/store">Redux Store</a>. For simplicity, speed and elegance, I chose <a href="https://mui.com/material-ui/getting-started/">MUI</a> as my components library.</p><p>I would like to give a special mention to <a href="https://reactflow.dev/">ReactFlow</a>, because it made my life really easy to build the decent looking Commit DAG, which was an essential feature of CellCraft-AI to visualize the version history of the dataset. And a small shout out to <a href="https://uiwjs.github.io/react-split/">React-Split</a> to provide a resize-able wrapper component library to make smooth resizing for the Data Preview panels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*sE7DSsl5nL8H7UvFMZZ2Rg@2x.png" /></figure><p>Overall, I was really happy with how the application looked and feel.</p><h3>The Backend</h3><p>Finally, we needed a backend that can orchestrate the responses from the Agent, managing the generated files in the sandbox environment and maintaining the conversation history.</p><p>Our application is pretty basic, it won’t have concurrent users manipulating the same dataset. Although we can have teams later in the picture, we still won’t have 2 people talking to the same Agent. So no need of strong concurrency, which is why GoLang was out of picture. Other options were NodeJs and Python. Using python made more sense because, if I was to extend the application and add more AI code, I won’t have to build a separate Python service. So it was sensible to stick to Python.</p><p>But then which Python framework? my favorite one is <a href="https://fastapi.tiangolo.com/">FastAPI</a>. FastAPI has made it so easy to spin up web application server, especially if your main goal is not to write server logic, but to write business logic. Powered with <a href="https://fastapi.tiangolo.com/">Pydantic</a> for schema handling, and <a href="https://asgi.readthedocs.io/en/latest/">ASGI</a> for asynchronous processing, FastAPI was a very powerful tool for me to base my backend.</p><h3>Putting it all together</h3><p>With my clear feature expectation and well-thought-out architecture, CellCraft-AI was finally taking it’s form. There were no bloated services involved, this was a lean architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*1m_Exzx_RfW5r_CoYGxCaw@2x.png" /></figure><p>Checkout the <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai/blob/master/docs/CellCraftAI-Demo.gif">Demo on my Github</a>. Apologies, because I could not add a video in this blog.</p><h3>Conclusion</h3><p>CellCraft-AI’s MVP was ready. It was very exciting to see how the application took shape and it was actually generating output as expected. But the MVP still had room for improvement.</p><p>Next challenges:</p><ol><li>The input prompt was too big. Currently, the logic was to fetch all the commits in the chat, then stitch together the key_steps to form context for the LLM. However, the key_steps strings were too long, almost 10–50 words i.e 20–80 tokens per commit. This number will increase linearly as the chat grows big, which means, we needed some form of conversation compression logic.</li><li>Another challenge was that, we were unnecessarily generating Signed URLs with every /list_commit_files API call. While I was testing the application, I realized I was clicking on the Data Previews very often, which triggered URL generation redundantly. There was a need of a caching layer for the Data Preview.</li></ol><p>We will be discussing more about these challenges in the next blog.</p><p>Finally, just a few screenshots for you to look closely at the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*xwi4iSi2wjvBkn1zwFZ9rg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*sc3U_K1APq96qar5quniZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*SYT5hR2F9irPcZD4g8Rgqg.png" /><figcaption>Cell Craft AI’s Views</figcaption></figure><p>Thank you for sticking around!</p><p>Source Code: <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai">https://github.com/Aditya-Dawadikar/cell-craft-ai</a></p><p>Previous Blog: <a href="https://medium.com/@aditya-dawadikar/cellcraft-ai-how-i-built-the-poc-5ee3d8ad029e">CellCraft-AI: How I built the POC</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a6d5404a67c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CellCraft-AI: How I built the POC]]></title>
            <link>https://medium.com/@aditya-dawadikar/cellcraft-ai-how-i-built-the-poc-5ee3d8ad029e?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ee3d8ad029e</guid>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Sun, 06 Jul 2025 05:07:19 GMT</pubDate>
            <atom:updated>2025-07-06T05:07:19.162Z</atom:updated>
            <content:encoded><![CDATA[<p>While mentoring freshmen students who were learning data preprocessing for an ML course over the summer, I noticed how repetitive the preliminary data exploration and data cleaning tasks can be. For every dataset we worked on, there were always some erroneous values and data points out of place that make it difficult for the ML models to get meaningful insights.</p><p>Of course there are data transformation pipelines that the industry uses, but I’m pretty sure someone still has to sit down and inspect the sample of data before writing those transformation pipelines. This observation sparked the idea for some AI driven automation. And this is how I began building CellCraft-AI — The AI Powered Data Cleaning Assistant</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YeXbROO9Z7HIvSbi" /><figcaption>Photo by <a href="https://unsplash.com/@kommumikation?utm_source=medium&amp;utm_medium=referral">Mika Baumeister</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Idea</h3><p>The idea was simple, I don’t want to write the data exploration and data cleaning code. I will give input to an Agent in natural language, and the output will be a transformed dataset and visualizations.</p><h3>But How ?</h3><p>LLMs like ChatGPT are fully capable of writing small executable snippets. All I have to do is build a sandbox environment where I can execute the code. Well, that makes sense, but how can we build a sand box? The answer to this question starts from something very fundamental. We start by answering what is a “PROCESS”?</p><p>A process is something that takes some form of input, performs certain operations on the input, and gives us an output. For my system the Input is an AI generated code. If I manage to create a sandbox, I just need to inject the code in the environment, and execute a command to run that program file. Simple, right?</p><p>I explored my options for the Sandbox, and I learned that Python provides an inbuilt function — <a href="https://docs.python.org/3/library/functions.html#exec">exec()</a> that can execute any arbitrary code dynamically.</p><p>Formally, exec() takes 3 major types of inputs namely,</p><ul><li>globals is the optional dictionary representing the global namespace in which the code will run.</li><li>locals is the optional dictionary representing the local namespace.</li><li>code is the executable and valid python code, that can be interpreted and executed immediately.</li></ul><p>But running arbitrary code directly into the Python’s environment is RISKY! So we have to be cautious about it.</p><h3>A Better Way ?</h3><p>Kubernetes Jobs. Kubernetes Jobs execute in isolated environments, so this layer of virtualization can provide us the security we need. As a preemptive measure, we should also ensure that the LLM does not generate any risky code, which means prompt engineering is necessary.</p><h3>Scoping My Idea</h3><p>Now I knew how to generate executable code, I knew how to execute the generated code. But a conversation is incremental. For instance, once I start working on a dataset, I will apply transforms step by step, in an order. So I need some sorts of memory for my Agent.</p><p>Secondly, every time I apply a transform, a new version of my data is created, but what if I wish to roll back? Ummm… 🤔, that’s a big brainer. I realized, I need some sorts of commit style data structure that behaves like a memory.</p><p>Also, not every message exchange would result into a data transformation. Something I might just wish to ask questions about the data. So a major feature is also the Chat-like functionality where the responses are augmented with retrieval from the dataset, a.k.a RAG (Retrieval-Augmented Generation).</p><p>So this POC was slowly taking form. We were going to build:</p><ol><li>A sandbox environment</li><li>A data structure to log my data transformations</li><li>A conversation log</li></ol><h3>Data Structures</h3><p>I created a schema called Sessions. A session basically represented a project. For each session, user can upload a single CSV file, and then subsequent conversations will use a transformed version of this CSV file.</p><p>Each time a user has an exchange with the LLM, we create a Commit . But not all commits are alike — we can have 3 types of commits: CODE, CHAT, CONTEXT. Every commit will have a parent commit, unless it is the root commit.</p><p>A commit with type CODE, can have generated_files — a list of files generated as a result of AI generated code execution. A commit of type CONTEXT will tell us which branch of the code we are operating on right now.</p><p>I wanted to keep it a git style version control, except the version control is fully internally handled by the AI. Users can explicitly ask the AI to change the HEAD to a different commit or to create a new branch from any given point.</p><p>Initially, I planned to store everything locally. So a static folder for storing the output files, and a JSON file to track sessions and commits.</p><pre>class Commit(Document):<br>    commit_id: str<br>    session_id: str<br>    parent_commit: Optional[str] = None<br>    timestamp: str<br><br>    query: str<br>    mode: Literal[&quot;CODE&quot;,&quot;CHAT&quot;,&quot;CONTEXT&quot;]<br>    response: Optional[str] = None<br>    code: Optional[str] = None<br>    generated_files: List[GeneratedFile] = []<br><br>    success: bool<br>    error: Optional[str] = None<br><br>class GeneratedFile(BaseModel):<br>    type: str<br>    title: str<br>    url: str</pre><pre>class Session(Document):<br>    session_id: str<br>    session_name: str<br>    head: Optional[str] = None<br>    last_csv_path: Optional[str] = None<br>    history_path: Optional[str] = None<br>    session_dir: Optional[str] = None</pre><p>With this design, I had a pretty good foundation. But I was forgetting an important aspect of an Agentic AI. We need context.</p><p>But the number of input tokens are limited, so I couldnt pass the entire code responses from previous chats to generate new responses.</p><p>To solve this challenge, I introduced a field called key_steps .</p><p>key_steps was an interesting addition. I didn’t have to do much about it. After every transformation, I just asked the LLM to create a concise summary for me which I saved in this field. Whenever I was asking a new question, I could just stitch together these concise summaries, and give a new instruction.</p><p>I used the following prompt to get precisely the right kind of output as I wanted for my Agent to work.</p><pre>You are a data cleaning assistant. The user gave you this preview:<br>{df_preview}<br><br>Previous steps:<br>{key_step_changelog}<br><br>User says: &quot;{query}&quot;<br><br>Respond in JSON.<br><br>If replying in text only:<br>{{<br>  &quot;mode&quot;: &quot;CHAT&quot;,<br>  &quot;response&quot;: &quot;&lt;your message&gt;&quot;<br>}}<br><br>If applying transformation:<br>{{<br>  &quot;mode&quot;: &quot;CODE&quot;,<br>  &quot;key_steps&quot;: &quot;&lt;summary&gt;&quot;,<br>  &quot;executable_code&quot;: &quot;&lt;python pandas code&gt;&quot;<br>}}<br><br>You can also manage which CSV version you&#39;re working on.<br>If the user says &quot;undo&quot;, &quot;redo&quot;, or &quot;start over from commit XYZ&quot;, respond in CONTEXT mode.<br><br>If changing HEAD:<br>{{<br>  &quot;mode&quot;: &quot;CONTEXT&quot;,<br>  &quot;action&quot;: &quot;checkout&quot;,<br>  &quot;target_commit_id&quot;: &quot;&lt;commit_id&gt;&quot;,<br>  &quot;response&quot;: &quot;&lt;acknowledgement of context change&gt;&quot;<br>}}<br><br>If creating a new branch from an older commit:<br>{{<br>  &quot;mode&quot;: &quot;CONTEXT&quot;,<br>  &quot;action&quot;: &quot;branch&quot;,<br>  &quot;target_commit_id&quot;: &quot;&lt;commit_id&gt;&quot;,<br>  &quot;response&quot;: &quot;&lt;acknowledgement of context change&gt;&quot;<br>}}<br><br>IMPORTANT:<br>  - All output must be a single valid JSON object.<br>  - Never include code outside the &quot;executable_code&quot; field.<br>  - Do not attempt to write files to directories other than the current working directory.<br><br>When saving files like charts or summaries, use the variable commit_dir as the output folder. Example:<br>plt.savefig(f&quot;{{commit_dir}}/myplot.png&quot;)<br>or<br>with open(f&quot;{{commit_dir}}/summary.md&quot;, &quot;w&quot;) as f: f.write(summary)<br><br>You are provided the DataFrame as a variable named `df`. <br>  - Do NOT use `pd.read_csv()` or try to load &#39;df.csv&#39;.<br>  - The DataFrame is already available in memory.<br>  - Any cleaned or transformed result should overwrite the existing `df` variable.<br><br>Return ONLY valid JSON.</pre><p>This is a very verbose instruction. But I came up with it after a little bit of trial and error — and it worked for me.</p><p>Given that the instruction is huge, imagine how long it would become when we add dataset preview and previous conversation history to this prompt 😓.</p><p>With this I had a small working POC for myself. And now it was time to build an UI.</p><h3>Conclusion</h3><p>With this, the first blog in the series comes to an end. I was able to build a small functional POC. I was able to create a naive memory management system for my Agent, and a sandbox environment to execute my code. All the generated output files were saved in a temporary static directory, from which they can easily be accessed using REST APIs.</p><p>Few challenges awaited, which we will discuss in the next blogs. Thanks.</p><p>Checkout the Source Code: <a href="https://github.com/Aditya-Dawadikar/cell-craft-ai">CellCraft-AI</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ee3d8ad029e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I built a Neural Network from Scratch | Part 3: The Model Class and Training Loop]]></title>
            <link>https://medium.com/@aditya-dawadikar/how-i-built-a-neural-network-from-scratch-part-3-the-model-class-and-training-loop-09fc4e94db8e?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/09fc4e94db8e</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[sequential-model]]></category>
            <category><![CDATA[adam-optimizer]]></category>
            <category><![CDATA[early-stopping]]></category>
            <category><![CDATA[cpp]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Mon, 26 May 2025 17:37:54 GMT</pubDate>
            <atom:updated>2025-05-26T17:37:54.427Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*70ZAFLvRmES4tiQ5" /><figcaption>Photo by <a href="https://unsplash.com/@boliviainteligente?utm_source=medium&amp;utm_medium=referral">BoliviaInteligente</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Welcome back to my series on building a Neural Network from Scratch with C++, where we take a deep dive into the internal functioning of a Neural Network and draw a link between the theory and the code bolstering these artificial brains.</p><p>Previously we dove into the math and implementation of the <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b"><strong>Dense Layer</strong></a>, <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-2-activation-layers-and-loss-function-3d590dda5425"><strong>Activation Funtions (ReLU, Sigmoid) and the Loss Functions (MSE)</strong></a><strong>.</strong> Today we will uncover how to stack these layers together so that they can learn the patterns. We will use the classic example of XOR Gate which requires true Non-Linearity for producing the output. First we will cover the Model class, then build a training loop to train our model.</p><h3>The Model Class</h3><p>Libraries like <a href="https://pytorch.org/">Pytorch </a>and <a href="https://www.tensorflow.org/">Tensorflow </a>have their own ways of defining the model architecture. Most common and preliminary one is the Sequential paradigm.</p><h4>The Sequential Paradigm</h4><p>The sequential paradigm resembles a pipeline. Layers are stacked on top of each other. On the <em>forward()</em> call, the signal flows from the input layer, through the hidden layers and finally to the output layer. Likewise, the <em>backward()</em> call reverses the direction of signal flow.</p><pre>Model model;<br>model.add(new Dense(4, 8));<br>model.add(new ReLU());<br>model.add(new Dense(8, 1));<br>model.add(new Sigmoid());</pre><p>Both Tensorflow and Pytorch support Sequential paradigm.</p><ul><li><a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html">Pytorch — Sequential</a></li><li><a href="https://www.tensorflow.org/guide/keras/sequential_model">Keras — Sequential</a></li></ul><p>As the models grew more complex (eg: residual networks, attention, branching structures), the sequential model was not enough. Frameworks evolved, Pytorch embraced dynamic computation graphs using the <em>forward()</em> method in custom <a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html"><em>nn.Module</em></a> subclass, where as Tensorflow 2.x deprecated the old static-graph style in favor of eager execution and custom <a href="https://www.tensorflow.org/api_docs/python/tf/keras/Model">Model</a> classes.</p><p>To keep things simple, I went for the Sequential model paradigm, because this is sufficient for creating Multi Layer Perceptrons.</p><h4>Requirements for the Model Class</h4><p>For my implementation, I needed a dynamic datastructure that can tolerate extension, i.e. It can grow in size as I experiment with my model by adding/removing layers. So one such linear datastructure is LinkedList or a Vector. I went for the C++ STL Vector class for storing the layers. The implementation became easier, because all the layers (Dense, ReLU, Sigmoid) used the same base class — Layer. We also need a method — <em>add()</em> using which we can add the layers one by one to our datastructure.</p><p>We will trigger the <em>forward()</em> and <em>backward()</em> calls of each layers in a sequence, so we need a for loop that iterates through all the layers in the forward direction, each time calling the <em>forward()</em> on the current layer, and passing its output to the next layer. We reverse the order of traversal as we call the <em>backward()</em> on all layers.</p><pre>//model.hpp<br><br>#ifndef MODEL_HPP<br>#define MODEL_HPP<br><br>#include &lt;vector&gt;<br>#include &quot;matrix.hpp&quot;<br>#include &quot;layer.hpp&quot;<br>#include &quot;loss.hpp&quot;<br><br>class Model{<br>    private:<br>        std::vector&lt;Layer*&gt; layers;<br>    <br>    public:<br>        void add(Layer* layer);<br>        Matrix forward(const Matrix&amp; loss_grad);<br>        Matrix backward(const Matrix&amp; loss_grad);<br>        void update(double learning_rate);<br>};<br><br>#endif</pre><pre>//model.cpp<br><br>#include &quot;model.hpp&quot;<br>#include &quot;layer.hpp&quot;<br>#include &quot;loss.hpp&quot;<br>#include &lt;iomanip&gt;<br>#include &lt;iostream&gt;<br>#include &lt;limits&gt;<br><br>/// Adds a layer to the model<br>/// Layers are stored in a sequential order for forward and backward chaining<br>void Model::add(Layer* layer){<br>    layers.push_back(layer);<br>}<br><br>/// Performs the forward pass through all layers<br>///<br>/// Each layer applies a transformation:<br>///     x_{i+1} = layer_i.forward(x_i)<br>/// Final output is returned (typically used for loss computation)<br>Matrix Model::forward(const Matrix&amp; input){<br>    Matrix out = input;<br>    for(auto&amp; layer : layers){<br>        out = layer-&gt;forward(out);<br>    }<br>    return out;<br>}<br><br>/// Performs the backward pass (backpropagation) through all layers in reverse<br>///<br>/// Each layer receives the gradient from the layer above:<br>///     grad_{i} = layer_{i}.backward(grad_{i+1})<br>///<br>/// This propagates gradients from loss back to the first layer<br>Matrix Model::backward(const Matrix&amp; grad_output){<br>    Matrix grad = grad_output;<br>    for(auto it = layers.rbegin(); it!=layers.rend();++it){<br>        grad = (*it)-&gt;backward(grad);<br>    }<br>    return grad;<br>}<br><br>/// Updates all layers using their stored gradients and a learning rate<br>///<br>/// Typically used after `forward` + `backward` to perform one optimization step<br>void Model::update(double learning_rate){<br>    for(auto&amp; layer: layers){<br>        layer-&gt;update(learning_rate);<br>    }<br>}</pre><p>Using the above class, we can now create a sequential model, that looks like this:</p><pre>#include &lt;iostream&gt;<br>#include &lt;matrix.hpp&gt;<br>#include &lt;dense_layer.hpp&gt;<br>#include &lt;utils_random.hpp&gt;<br>#include &lt;activation_relu.hpp&gt;<br>#include &lt;activation_sigmoid.hpp&gt;<br>#include &lt;model.hpp&gt;<br><br>int main(){<br>    std::cout&lt;&lt;&quot;Neuronite Initialized!&quot;&lt;&lt;std::endl;<br><br>    set_random_seed(42);<br><br>    // Step 1: Input and Target (simple 2D XOR-style sample)<br>    Matrix input({<br>        {0.0, 0.0},<br>        {0.0, 1.0},<br>        {1.0, 0.0},<br>        {1.0, 1.0}<br>    });<br><br>    Matrix target({<br>        {0.0},<br>        {1.0},<br>        {1.0},<br>        {0.0}<br>    });<br><br>    // Step 2: Build model: 2 → 4 → 1<br>    Model model;<br>    model.add(new DenseLayer(2, 4));<br>    model.add(new ActivationReLU());<br>    model.add(new DenseLayer(4, 1));<br>    model.add(new ActivationSigmoid());<br><br>    return 0;<br>}</pre><p>Now we can clearly understand what the model architecture looks like. But we can help ourselves some trouble by providing an API to summarize the model. This API will be responsible to console log the model’s architecture and state the layer’s name and the number of parameters or the shape of the layer.</p><pre>// model.cpp<br><br>void Model::summarize(int input_dim){<br>    // Step 1: Run dummy forward<br>    Matrix dummy_input(1, input_dim);  // [1 × input_dim], batch size = 1<br>    for (auto&amp; row : dummy_input.data)<br>        std::fill(row.begin(), row.end(), 0.0);  // optional: fill with 0s<br><br>    this-&gt;forward(dummy_input);  // populates input/output shapes in layers<br><br>    // Step 2: Print header<br>    std::cout &lt;&lt; &quot;# Model Summary\n&quot;;<br>    std::cout &lt;&lt; &quot;──────────────────────────────────────────────────────────────\n&quot;;<br>    std::cout &lt;&lt; std::left<br>              &lt;&lt; std::setw(20) &lt;&lt; &quot;Layer (type)&quot;<br>              &lt;&lt; std::setw(18) &lt;&lt; &quot;Input Shape&quot;<br>              &lt;&lt; std::setw(18) &lt;&lt; &quot;Output Shape&quot;<br>              &lt;&lt; std::setw(10) &lt;&lt; &quot;Param #&quot; &lt;&lt; &quot;\n&quot;;<br>    std::cout &lt;&lt; &quot;==============================================================\n&quot;;<br><br>    int total_params = 0;<br><br>    for (const auto&amp; layer : layers) {<br>        auto in_shape = layer-&gt;get_input_shape();<br>        auto out_shape = layer-&gt;get_output_shape();<br>        int params = layer-&gt;param_count();<br><br>        std::string in_shape_str = &quot;[&quot; + std::to_string(in_shape.first) + &quot; × &quot; + std::to_string(in_shape.second) + &quot;]&quot;;<br>        std::string out_shape_str = &quot;[&quot; + std::to_string(out_shape.first) + &quot; × &quot; + std::to_string(out_shape.second) + &quot;]&quot;;<br><br>        std::cout &lt;&lt; std::left<br>                  &lt;&lt; std::setw(20) &lt;&lt; layer-&gt;get_name()<br>                  &lt;&lt; std::setw(18) &lt;&lt; in_shape_str<br>                  &lt;&lt; std::setw(18) &lt;&lt; out_shape_str<br>                  &lt;&lt; std::setw(10) &lt;&lt; params &lt;&lt; &quot;\n&quot;;<br><br>        total_params += params;<br>    }<br><br>    std::cout &lt;&lt; &quot;──────────────────────────────────────────────────────────────\n&quot;;<br>    std::cout &lt;&lt; &quot;Total Parameters: &quot; &lt;&lt; total_params &lt;&lt; &quot;\n&quot;;<br>    std::cout &lt;&lt; &quot;\n&quot;;<br>}</pre><p>Notice, how we used a dummy-input which is passed on to each layer during the forward pass. This is usually necessary for the layers to infer their input/output dimensions. Then I added some log formatting to get a pretty model summary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*FnpBJoGbMyvyWchB4pF02Q@2x.png" /><figcaption>Model summary sample output</figcaption></figure><p>we needed helper functions to fetch the input and output shapes, so I added those functions to all the Layers.</p><pre>//denser_layer.cpp<br><br>std::string DenseLayer:: get_name() const {<br>    return &quot;Dense(&quot;+std::to_string(weights.rows)+&quot; -&gt; &quot;+std::to_string(weights.cols)+&quot;)&quot;;<br>}<br><br>std::pair&lt;int,int&gt; DenseLayer::get_input_shape() const {<br>    return input_shape;<br>}<br><br>std::pair&lt;int,int&gt; DenseLayer::get_output_shape() const {<br>    return output_shape;<br>}<br><br>int DenseLayer::param_count() const{<br>    int input_dim = input_shape.second;<br>    int output_dim = output_shape.second;<br>    return input_dim*output_dim + output_dim;<br>}</pre><h3>The Training Loop</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BfaN4ZjRNSM1yq9S" /><figcaption>Photo by <a href="https://unsplash.com/@slelham?utm_source=medium&amp;utm_medium=referral">Steven Lelham</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Now that we have built the sequential model, its time to train our model. Usually models are trained for multiple <em>epochs</em>, one epoch means that the entirety of the training dataset was passed through the model once. For every entry of the training dataset, the model adapted it weights and biases. But, remember that our model is trying to learn, similar to how humans learn. Which means, we need to do the same thing over and over, until we get better at it. That is the whole point of training the model for multiple epochs. With every epoch, the model tries to reduce its loss and improve accuracy until the model converges i.e. it can no longer get meaningful insights from the model. Another important functionality of the training loop is to compute loss and initiate the backward pass.</p><h4>Building the Training Loop</h4><p>Our training loop will have some input parameters, like input, target, loss, and number of epoch. As the name suggest, the input is the input training data, and target is the expected output. Loss function computes the difference between the produced output vs the expected output, in our case, we will pass the instance of MSE loss. The number of epochs, describe how long the model trains for. After, every epoch, we will print the log and the accuracy achieved so far, so that we can see if the model is actually training.</p><pre>// model.cpp<br><br>void Model::train(const Matrix&amp; input,<br>                  const Matrix&amp; target,<br>                  Loss&amp; loss_fn,<br>                  int epochs) {<br>    <br>    double best_loss = std::numeric_limits&lt;double&gt;::infinity();<br><br>    for (int epoch = 0; epoch &lt; epochs; ++epoch) {<br>        // Forward pass<br>        Matrix prediction = this-&gt;forward(input);<br><br>        // Loss computation<br>        double loss = loss_fn.forward(prediction, target);<br><br>        // Backward pass<br>        Matrix grad = loss_fn.backward();<br>        this-&gt;backward(grad);<br><br>        // update weights<br>        this-&gt;update(learning_rate);<br><br>        // Logging<br>        double acc = Model::compute_accuracy(prediction, target);<br>        std::cout &lt;&lt; &quot;Epoch &quot; &lt;&lt; epoch<br>                &lt;&lt; &quot; | Loss: &quot; &lt;&lt; loss<br>                &lt;&lt; &quot; | Accuracy: &quot; &lt;&lt; std::fixed &lt;&lt; std::setprecision(4)<br>                &lt;&lt; acc * 100 &lt;&lt; &quot;%\n&quot;;<br><br>    }<br>}</pre><p>Finally, its time to test our code. I set the number of epochs to 10, and the learning rate to 0.1, and I was disappointed at the model’s performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/1*Kv6BehOs66Fv-ZvOwfuxLw@2x.png" /><figcaption>Initial model performance</figcaption></figure><p>My model was learning very slowly!!! The loss did decrease throughout the epochs, but it could not learn the underlying pattern within 10 epochs. The accuracy remained at 50%, which is terrible, given that we only have 4 combinations, and my model was wrong half the time. So I cranked up the number of epochs to 500, and this is what I observed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*2TOzmhgDy7Z2eGHr00R4Bw.png" /><figcaption>training logs with 500 epochs</figcaption></figure><p>Aha, something interesting happened. My model was able to be correct amost 75% of the times, and that is an improvement. Interestingly, the loss was also decreasing, which means my model has potential to be better!</p><p>This is when I thought, why not add an optimizer? An optimizer helps us adjusts the weights and biases based on the loss function.</p><h4>Adam Optimizer</h4><p>Adam optimizer is a well known optimizer. So I implemented an optimizer from scratch. I will quickly go over the math before jumping into the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*ycF5zdOoddW-IpDjKKCxpg.png" /><figcaption>Adam Optimizer</figcaption></figure><p>Implementing Adam was fairly easy, because we have already written code for computing gradients. I created two files, adam_optimizer.hpp and adam_optimizer.cpp.</p><pre>// adam_optimizer.hpp<br>#ifndef ADAM_OPTIMIZER_HPP<br>#define ADAM_OPTIMIZER_HPP<br><br>#include &quot;optimizer.hpp&quot;<br>#include &lt;unordered_map&gt;<br><br>class AdamOptimizer:public Optimizer{<br>    private:<br>        double lr;<br>        double beta1;<br>        double beta2;<br>        double epsilon;<br><br>        // t-th moment estimates for each layer pointer<br>        std::unordered_map&lt;Layer*, Matrix&gt; m_weights;<br>        std::unordered_map&lt;Layer*, Matrix&gt; v_weights;<br>        std::unordered_map&lt;Layer*, Matrix&gt; m_bias;<br>        std::unordered_map&lt;Layer*, Matrix&gt; v_bias;<br><br>    public:<br>        AdamOptimizer(double lr=0.001, double beta1=0.9, double beta2=0.999, double epsilon=1e-8);<br><br>        void step(Layer* layer, int t) override;<br>};<br><br>#endif</pre><pre>//adam_optimizer.cpp<br>#include &quot;adam_optimizer.hpp&quot;<br>#include &quot;dense_layer.hpp&quot;<br>#include &lt;cmath&gt;<br><br>AdamOptimizer::AdamOptimizer(double lr, double beta1, double beta2, double epsilon)<br>    : lr(lr), beta1(beta1), beta2(beta2), epsilon(epsilon) {}<br><br>/// Performs one step of the Adam optimization algorithm on a single layer<br>/// <br>/// Uses the layer&#39;s stored gradients (d_weights and d_bias) to compute updates.<br>/// Maintains per-layer first and second moment estimates for weights and bias.<br>///<br>/// Parameters:<br>/// - layer: pointer to the Layer (must be DenseLayer)<br>/// - t: current timestep (starting from 1), used for bias correction<br>void AdamOptimizer::step(Layer* layer, int t) {<br><br>    // Only apply Adam to Dense layers (skip ReLU/Sigmoid etc.)<br>    auto* dense = dynamic_cast&lt;DenseLayer*&gt;(layer);<br>    if (!dense) return;<br><br>    Matrix dw = dense-&gt;get_d_weights();<br>    Matrix db = dense-&gt;get_d_bias();<br><br>    // Initialize moment vectors if this is the first time seeing this layer<br>    if (m_weights.count(layer) == 0) {<br>        m_weights[layer] = Matrix(dw.rows, dw.cols);<br>        v_weights[layer] = Matrix(dw.rows, dw.cols);<br>        m_bias[layer] = Matrix(db.rows, db.cols);<br>        v_bias[layer] = Matrix(db.rows, db.cols);<br>    }<br><br>    // Aliases for convenience<br>    auto&amp; mw = m_weights[layer]; // First moment (mean) for weights<br>    auto&amp; vw = v_weights[layer]; // Second moment (variance) for weights<br>    auto&amp; mb = m_bias[layer];    // First moment (mean) for bias<br>    auto&amp; vb = v_bias[layer];    // Second moment (variance) for bias<br><br>    // Get current weights and biases to update them<br>    Matrix updated_weights = dense-&gt;get_weights();<br>    Matrix updated_bias = dense-&gt;get_bias();<br><br>    // ==== Update Weights ====<br>    for (int i = 0; i &lt; dw.rows; ++i) {<br>        for (int j = 0; j &lt; dw.cols; ++j) {<br>            // m ← β1·m + (1−β1)·∇θ<br>            mw.data[i][j] = beta1 * mw.data[i][j] + (1 - beta1) * dw.data[i][j];<br><br>            // v ← β2·v + (1−β2)·(∇θ)^2<br>            vw.data[i][j] = beta2 * vw.data[i][j] + (1 - beta2) * std::pow(dw.data[i][j], 2);<br><br>            // Bias-corrected moment estimates<br>            double m_hat = mw.data[i][j] / (1 - std::pow(beta1, t));<br>            double v_hat = vw.data[i][j] / (1 - std::pow(beta2, t));<br><br>            // θ ← θ − α·m̂ / (√v̂ + ε)<br>            updated_weights.data[i][j] -= lr * m_hat / (std::sqrt(v_hat) + epsilon);<br>        }<br>    }<br><br>    for (int j = 0; j &lt; db.cols; ++j) {<br>        mb.data[0][j] = beta1 * mb.data[0][j] + (1 - beta1) * db.data[0][j];<br>        vb.data[0][j] = beta2 * vb.data[0][j] + (1 - beta2) * std::pow(db.data[0][j], 2);<br><br>        double m_hat = mb.data[0][j] / (1 - std::pow(beta1, t));<br>        double v_hat = vb.data[0][j] / (1 - std::pow(beta2, t));<br><br>        updated_bias.data[0][j] -= lr * m_hat / (std::sqrt(v_hat) + epsilon);<br>    }<br><br>    // Apply updated parameters to the layer<br>    dense-&gt;apply_adam_update(updated_weights, updated_bias);<br>}</pre><p>I needed a function called <em>apply_adam_update()</em> in the Dense layer.</p><pre>// dense_layer.cpp<br><br>void DenseLayer::apply_adam_update(const Matrix&amp; new_weights, const Matrix&amp; new_bias) {<br>    weights = new_weights;<br>    bias = new_bias;<br>}</pre><p>Finally, with an optimizer in the picture, my training loop changed slightly.</p><pre>void Model::train(const Matrix&amp; input,<br>                  const Matrix&amp; target,<br>                  Loss&amp; loss_fn,<br>                  Optimizer&amp; optimizer,<br>                  int epochs) {<br>    <br>    double best_loss = std::numeric_limits&lt;double&gt;::infinity();<br>    int epochs_without_improvement = 0;<br><br>    for (int epoch = 0; epoch &lt; epochs; ++epoch) {<br>        // Forward pass<br>        Matrix prediction = this-&gt;forward(input);<br><br>        // Loss computation<br>        double loss = loss_fn.forward(prediction, target);<br><br>        // Backward pass<br>        Matrix grad = loss_fn.backward();<br>        this-&gt;backward(grad);<br><br>        // Optimizer step for each layer<br>        for (auto&amp; layer : layers) {<br>            optimizer.step(layer, epoch + 1);<br>        }<br>        // this-&gt;update(0.1); -&gt; no longer needed<br><br>        // Logging<br>        double acc = Model::compute_accuracy(prediction, target);<br>        std::cout &lt;&lt; &quot;Epoch &quot; &lt;&lt; epoch<br>                &lt;&lt; &quot; | Loss: &quot; &lt;&lt; loss<br>                &lt;&lt; &quot; | Accuracy: &quot; &lt;&lt; std::fixed &lt;&lt; std::setprecision(4)<br>                &lt;&lt; acc * 100 &lt;&lt; &quot;%\n&quot;;<br>    }<br>}</pre><p>I trained my model again for 500 epochs with adam optimizer, and learning rate of 0.01.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*B0-US5FyZ0XZwSQ5dvA5FA.png" /><figcaption>Adam Optimizer (learning_rate = 0.01)</figcaption></figure><p>Wow, with this I really surprised my self, we achieved an accuracy of 100% and loss as low as 0.0018!!! This is a win. I ramped up the learning rate to 0.1 and reduced the epochs from 500 to 100.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*M95MbTwYi3XqeVAi6EPgcQ.png" /><figcaption>Adam Optimizer (learning_rate=0.1)</figcaption></figure><p>Crazy progress. Within 100 epochs we were easily able to achieve an accuracy of 100%. But I noticed that we achieved accuracy of 100% way earlier. So I thought why not add an early stopping mechanism? So that we don’t waste our valuable compute? My logic for early stopping is simple, if the loss does not decrease for <em>k</em> consecutive epochs, we stop the training. This is a well known technique for model training. My training loop had minimal changes to implement this feature. This is what my final training loop looks like:</p><pre>void Model::train(const Matrix&amp; input,<br>                  const Matrix&amp; target,<br>                  Loss&amp; loss_fn,<br>                  Optimizer&amp; optimizer,<br>                  int epochs,<br>                  int patience) {<br>    <br>    double best_loss = std::numeric_limits&lt;double&gt;::infinity();<br>    int epochs_without_improvement = 0;<br><br>    for (int epoch = 0; epoch &lt; epochs; ++epoch) {<br>        // Forward pass<br>        Matrix prediction = this-&gt;forward(input);<br><br>        // Loss computation<br>        double loss = loss_fn.forward(prediction, target);<br><br>        // Backward pass<br>        Matrix grad = loss_fn.backward();<br>        this-&gt;backward(grad);<br><br>        // Optimizer step for each layer<br>        for (auto&amp; layer : layers) {<br>            optimizer.step(layer, epoch + 1);<br>        }<br>        // this-&gt;update(0.1);<br><br>        // Logging<br>        double acc = Model::compute_accuracy(prediction, target);<br>        std::cout &lt;&lt; &quot;Epoch &quot; &lt;&lt; epoch<br>                &lt;&lt; &quot; | Loss: &quot; &lt;&lt; loss<br>                &lt;&lt; &quot; | Accuracy: &quot; &lt;&lt; std::fixed &lt;&lt; std::setprecision(4)<br>                &lt;&lt; acc * 100 &lt;&lt; &quot;%\n&quot;;<br><br>        // Early stopping logic<br>        if (loss &lt; best_loss - 1e-6) {<br>            best_loss = loss;<br>            epochs_without_improvement = 0;<br>        } else {<br>            epochs_without_improvement++;<br>        }<br><br>        if (epochs_without_improvement &gt;= patience) {<br>            std::cout &lt;&lt; &quot;Early stopping at epoch &quot; &lt;&lt; epoch<br>                      &lt;&lt; &quot; (best loss = &quot; &lt;&lt; best_loss &lt;&lt; &quot;)\n&quot;;<br>            break;<br>        }<br>    }<br>}</pre><p>I introduced a new parameter called the <em>patience</em>, which tells us how long to observe the loss improvement before we pull the plug. And, this is what my final output looks like…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*JDGCrIznpUWGrQjceZux-w.png" /></figure><p>I set the patience to 20 epochs, learning rate to 0.1 and max_epochs to 500. Interestingly, the best accuracy we achieved was 75% and since the loss did not improve for 20 consecutive epochs we stopped training. Looks like we hit a local minima, which was only overcame when we had higher number of epochs. There were some instances of the early stopping, where it did not stop the training at all and let the model train for 500 epochs. Because of the stochastic nature of the Neural Network, we have less control over how the weights are updated, but this was fun.</p><h3>Conclusion</h3><p>With this blog, we come to the end of our series, where we built a Neural Network from scratch. On this journey, I learned a lot about the internal wirework of the Neural Networks, and evolved a Neural Network framework. The idea was simple, I wanted to build Neural Networks with C++, but I ended up make the code rich by adding utility functions to aid the development.</p><p>Not only was I able to dive deep into the math and link it with the code, but also to test things as they broke, dig into the literature to find ways around to make it work. Thanks for sticking around, I hope this blog had a different take on building the Neural Networks.</p><p>Source Code: <a href="https://github.com/Aditya-Dawadikar/Neuronite">https://github.com/Aditya-Dawadikar/Neuronite</a></p><p>Part 1: <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b">https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b</a></p><p>Part 2: <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-2-activation-layers-and-loss-function-3d590dda5425">https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-2-activation-layers-and-loss-function-3d590dda5425</a></p><p>…</p><p>PS: My github repo — <a href="https://github.com/Aditya-Dawadikar/Neuronite">Neuronite</a> is open for contributions, and feel free to reach out to me on <a href="https://linkedin.com/in/aditya-dawadikar">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=09fc4e94db8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I built a Neural Network from Scratch | Part 2: Activation Layers and Loss Function]]></title>
            <link>https://medium.com/@aditya-dawadikar/how-i-built-a-neural-network-from-scratch-part-2-activation-layers-and-loss-function-3d590dda5425?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d590dda5425</guid>
            <category><![CDATA[relu-function]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[mean-squared-error]]></category>
            <category><![CDATA[sigmoid-function]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Sat, 24 May 2025 18:17:01 GMT</pubDate>
            <atom:updated>2025-05-24T18:17:01.323Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GE7FtZvMJoy7tb6-" /><figcaption>Photo by <a href="https://unsplash.com/@bhautik_patel3?utm_source=medium&amp;utm_medium=referral">Bhautik Patel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Welcome back to the next part in my series — How I built a Neural Network from Scratch (with C++). In the <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b">previous post</a>, we dove into the Dense Layer, broke down its forward pass, backward pass and update phase and studied how the Gradient equations (based on Partial Derivatives) turn into matrix operations — the mathematical heartbeat of the Neural Networks. In this blog we will go a step further and by introducing the Activation Functions and the Loss Functions.</p><h3>Activation Function</h3><p>For my implementation, I kept it simple. I choose to support ReLU (Rectified Linear Unit) function and Sigmoid function — two most widely used activation function. You can also explore other Activations like Tanh, Leaky ReLU and SoftMax. Before diving into the math, let us understand why we even need an Activation function?</p><h4>The Barrier Analogy</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4TKf2ofK7YfxIMba" /><figcaption>Photo by <a href="https://unsplash.com/@komga?utm_source=medium&amp;utm_medium=referral">Gabriel Komorov</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Imagine an Activation Function like a barrier or a gatekeeper. If the signal crosses, a certain threshold, its allowed to pass, else it is blocked. For us, this barrier is a logical one. Let say that the input signal has a value of 0.8 and the barrier was of height 0.5, then this signal is strong enough to be passed on to the next neuron. When a signal cross this barrier, we say that the Neuron has <strong>Fired</strong> or been <strong>Activated</strong>. Now zoom out and imagine that we have thousands of neurons scattered on a plane. Each neuron evaluates the incoming signal, some fire, some stay quiet, we quickly realize, that it resembles a sieve. This sieve-like behavior introduces non-linearity, allowing neural networks to learn complex patterns.</p><h3>Loss Function</h3><p>Now that our Neuron fires and propagates signals, how can we ensure that our predictions are any good? Well, a loss function helps us quantify how far off our prediction is from the expected value i.e. the Error. This value acts as the feedback necessary for our Neural Network to adapt.</p><p>There are numerous loss functions like Mean Squared Error (MSE), Mean Absolute Error (MAE), Cross Entropy, Log Loss, etc. Now, each of these Loss functions have specific use cases. For simplicity, I used <strong>MSE</strong>.</p><h3>The Math</h3><h4>ReLU (Forward pass)</h4><p><a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html">ReLU</a> is one of the simplest activation function. A ReLU simply returns the maximum value between 0 and x, where x is the input to the function. So, if the input signal is negative, we discard it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*QLyLL3jjcHdnEt1Ik5FEXg@2x.png" /><figcaption>Mathematical Definition — ReLU</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*Ilimdo6IJ4GIYAoTIijV9Q.png" /><figcaption>Source: <a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html">Pytorch</a></figcaption></figure><h4>ReLU (Backward Pass)</h4><p>During the backward pass, we will propagate the gradient backwards for the active neurons. That means, we will need a way to cache which input neurons were active during the forward pass.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*_hZNGAT6G_ii5AGqeBDrOA@2x.png" /></figure><h4>Sigmoid (Forward pass)</h4><p>A <a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.Sigmoid.html">Sigmoid </a>function is essentially a scaling function. Irrespective of the scale of the input, it will bring it down to a range -1 to 1. Which means, large positive values will be closer to 1, and smallest negative values will be closer to -1, and 0 maps to 0.5 on our scale. This forms an S-shaped curve which makes it great for expressing confidence or probability or binary classification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*VeNbGinUDQovvaRoXapSIw@2x.png" /><figcaption>Mathematical Definition — Sigmoid</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*WrQdIE164XpS7N_PSWUiVQ.png" /><figcaption>Source: <a href="https://docs.pytorch.org/docs/stable/generated/torch.nn.Sigmoid.html">Pytorch</a></figcaption></figure><h4>Sigmoid (Backward pass)</h4><p>The backward pass of the Sigmoid is as not as straight forward as that of ReLU. Why? Have a look at following derivation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*8uQagDx0LX0tntIiv30QqQ@2x.png" /></figure><p>Using the chain rule, we learn that we need access to σ(x), i.e. the output of the forward pass, to compute the backward pass.</p><h4>Implementing ReLU and Sigmoid</h4><p>We will base both ReLU and Sigmoid on the <a href="https://github.com/Aditya-Dawadikar/Neuronite/blob/master/include/layer.hpp">Layer base class</a> we defined in the <a href="https://medium.com/@mail.aditya.dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b">previous blog</a>. I defined two header files <em>activation_relu.hpp</em> and <em>activation_sigmoid.hpp.</em></p><pre>// activation_relu.hpp<br>#ifndef ACTIVATION_RELU_HPP<br>#define ACTIVATION_RELU_HPP<br><br>#include &quot;layer.hpp&quot;<br><br>class ActivationReLU: public Layer{<br>    private:<br>        Matrix mask;<br><br>        std::pair&lt;int,int&gt; input_shape;<br>    <br>    public:<br>        Matrix forward(const Matrix&amp; input) override;<br>        Matrix backward(const Matrix&amp; grad_output) override;<br>        void update(double learning_rate) override;<br>};<br><br>#endif</pre><pre>// activation_sigmoid.hpp<br>#ifndef ACTIVATION_SIGMOID_HPP<br>#define ACTIVATION_SIGMOID_HPP<br><br>#include &quot;layer.hpp&quot;<br><br>class ActivationSigmoid: public Layer{<br>    private:<br>        Matrix mask;<br>        Matrix output_cache;<br><br>        std::pair&lt;int,int&gt; input_shape;<br>    <br>    public:<br>        static double sigmoid(double x);<br><br>        Matrix forward(const Matrix&amp; input) override;<br>        Matrix backward(const Matrix&amp; grad_output) override;<br>        void update(double learning_rate) override;<br>};<br><br>#endif</pre><p>Now that I have the skeleton of my code, I can move on to implementing the actual functions.</p><pre>//activation_relu.cpp<br>#include &lt;algorithm&gt;<br>#include &quot;matrix.hpp&quot;<br>#include &quot;activation_relu.hpp&quot;<br><br>/// Forward pass of ReLU activation<br>/// ReLU(x) = max(0, x)<br>///<br>/// For each element x in input:<br>///   y = max(0, x)<br>/// Also store a binary mask (1 if x &gt; 0, 0 otherwise) for use in backprop<br>Matrix ActivationReLU::forward(const Matrix&amp; input){<br>    Matrix output = Matrix(input.rows, input.cols);<br>    mask = Matrix(input.rows, input.cols);<br><br>    input_shape = {input.rows, input.cols};<br><br>    for(int i=0;i&lt;input.rows;++i){<br>        for(int j=0;j&lt;input.cols;++j){<br>            output.data[i][j] = std::max(0.0, input.data[i][j]);<br>            if(input.data[i][j] &gt; 0){<br>                mask.data[i][j] = 1;<br>            }else{<br>                mask.data[i][j] = 0;<br>            }<br>        }<br>    }<br><br>    return output;<br>}<br><br>/// Backward pass of ReLU<br>/// For upstream gradient dL/dy, we compute:<br>///   dL/dx = dL/dy * dReLU(x)/dx<br>///<br>/// Derivative of ReLU:<br>///   dReLU(x)/dx = 1 if x &gt; 0, else 0<br>/// So:<br>///   dL/dx = dL/dy if x &gt; 0, else 0<br>///<br>/// Uses the `mask` from forward pass.<br>Matrix ActivationReLU::backward(const Matrix&amp; grad_output){<br>    Matrix grad_input = Matrix(grad_output.rows, grad_output.cols);<br>    for(int i=0;i&lt;grad_output.rows;++i){<br>        for(int j=0;j&lt;grad_output.cols;++j){<br>            grad_input.data[i][j] = grad_output.data[i][j] * mask.data[i][j];<br>        }<br>    }<br>    return grad_input;<br>}<br><br>/// No-op for ReLU — it has no learnable parameters<br>void ActivationReLU::update(double learning_rate){<br>    return;<br>}</pre><p>Notice how the logic of the forward pass is straightforward where we simply computed the max(0, input). For the backward pass, since some of the neurons did not contribute we did not pass the gradient backwards for them. Thus ensuring that only the active neurons adapt based on the feedback.</p><p>To achieve this, we cached the firing neurons by creating a mask consisting of 0s and 1s — 1 for active neurons, 0 otherwise. In the backward pass, we use this mask to propagate the gradients backwards.</p><p>Also, since ReLU is not trainable, so we don’t need the update function.</p><pre>// activation_sigmoid.cpp<br>#include &lt;algorithm&gt;<br>#include &quot;matrix.hpp&quot;<br>#include &quot;activation_sigmoid.hpp&quot;<br>#include &lt;cmath&gt;<br><br>/// Static sigmoid function<br>/// σ(x) = 1 / (1 + e^(-x))<br>double ActivationSigmoid::sigmoid(double x){<br>    return 1.0 / (1.0 + std::exp(-x));<br>}<br><br>/// Forward pass for sigmoid activation<br>/// For each input x, computes:<br>///     y = σ(x) = 1 / (1 + e^(-x))<br>///<br>/// Also stores the output σ(x) in `output_cache`<br>/// for use in the backward pass.<br>Matrix ActivationSigmoid::forward(const Matrix&amp; input){<br>    Matrix output = Matrix(input.rows, input.cols);<br><br>    input_shape = {input.rows, input.cols};<br><br>    for(int i=0;i&lt;input.rows;++i){<br>        for(int j=0;j&lt;input.cols;++j){<br>            output.data[i][j] = ActivationSigmoid::sigmoid(input.data[i][j]);<br>        }<br>    }<br><br>    output_cache = output;<br><br>    return output;<br>}<br><br>/// Backward pass for sigmoid<br>/// Given upstream gradient dL/dy, computes:<br>///     dL/dx = dL/dy * σ(x) * (1 - σ(x))<br>///<br>/// This uses the derivative of the sigmoid function:<br>///     dσ/dx = σ(x) * (1 - σ(x))<br>Matrix ActivationSigmoid::backward(const Matrix&amp; grad_output){<br>    Matrix grad_input = Matrix(grad_output.rows, output_cache.cols);<br><br>    for(int i=0;i&lt;grad_input.rows;++i){<br>        for(int j=0;j&lt;grad_input.cols;++j){<br>            grad_input.data[i][j] = grad_output.data[i][j] * output_cache.data[i][j] * (1.0 - output_cache.data[i][j]);<br>        }<br>    }<br><br>    return grad_input;<br>}<br><br>/// No-op update - sigmoid has no learnable parameters<br>void ActivationSigmoid::update(double learning_rate){<br>    return;<br>}</pre><p>Notice how I created a helper function to compute the Sigmoid transform on the input signal. Then we cached the activation data in a variable called <em>output_cache </em>and used it later in the update phase.</p><h4>Mean Squared Error Loss (MSE)</h4><p>Finally, we implement the MSE, our metric of Loss. MSE is simple, in the sense that, we sum up the square of the difference between the mean of the signals and the individual signals. Then we divide it by the total number of signals to get a mean.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/1*ibJI2MZLgJCZxQsQMmyeqA@2x.png" /></figure><p>During the backpropagation, this loss will be used to compute the gradients with respect to the weights and biases of the Dense Layers.</p><h4>Implementing MSE</h4><p>I created a new base class called Loss (<a href="https://github.com/Aditya-Dawadikar/Neuronite/blob/master/include/loss.hpp">loss.hpp</a>), and extend this to create the LossMSE class (<a href="https://github.com/Aditya-Dawadikar/Neuronite/blob/master/include/loss_mse.hpp">loss_mse.hpp</a>).</p><pre>//loss.hpp<br>#ifndef LOSS_HPP<br>#define LOSS_HPP<br><br>#include &quot;matrix.hpp&quot;<br><br>class Loss{<br>    public:<br>        virtual double forward(const Matrix&amp; prediction, const Matrix&amp; target) = 0;<br>        virtual Matrix backward() = 0;<br>        virtual ~Loss() = default;<br>};<br><br>#endif</pre><pre>// loss_mse.hpp<br>#ifndef LOSS_MSE_HPP<br>#define LOSS_MSE_HPP<br><br>#include &quot;loss.hpp&quot;<br><br>class LossMSE: public Loss{<br>    private:<br>        Matrix prediction_cache;<br>        Matrix target_cache;<br>    <br>    public:<br>        double forward(const Matrix&amp; prediction, const Matrix&amp; target) override;<br>        Matrix backward() override;<br>};<br><br>#endif</pre><p>Using this base class, I implemented the <a href="https://github.com/Aditya-Dawadikar/Neuronite/blob/master/src/loss_mse.cpp">loss_mse.cpp</a> where I defined all the functions.</p><pre>#include &quot;matrix.hpp&quot;<br>#include &quot;loss_mse.hpp&quot;<br>#include &lt;cmath&gt;<br><br>/// Forward pass for Mean Squared Error (MSE) loss<br>///<br>/// MSE = (1 / n) * Σ (y_pred - y_true)^2<br>/// where:<br>///     y_pred = predicted output<br>///     y_true = ground truth (target)<br>///<br>/// Also caches the prediction and target for use in the backward pass.<br>double LossMSE::forward(const Matrix&amp; prediction, const Matrix&amp; target){<br><br>    if(prediction.rows!=target.rows || prediction.cols!=target.cols){<br>        throw std::invalid_argument(&quot;LossMSE::forward: Shape mismatch&quot;);<br>    }<br><br>    prediction_cache = prediction;<br>    target_cache = target;<br><br>    double loss = 0.0;<br><br>    for(int i=0;i&lt;prediction.rows;++i){<br>        for(int j=0;j&lt;target.cols;++j){<br>            loss += std::pow(prediction.data[i][j] - target.data[i][j],2);<br>        }<br>    }<br><br>    int total_elements = prediction.rows * prediction.cols;<br><br>    loss /= total_elements;<br><br>    return loss;<br>}<br><br>/// Backward pass for MSE loss<br>///<br>/// Gradient of MSE with respect to prediction:<br>///     dL/dy_pred = (2 / n) * (y_pred - y_true)<br>///<br>/// Returns matrix of gradients with the same shape as the prediction<br>Matrix LossMSE::backward(){<br>    Matrix grad_input = Matrix(prediction_cache.rows, prediction_cache.cols);<br><br>    int total_elements = prediction_cache.rows*prediction_cache.cols;<br><br>    for(int i=0;i&lt;grad_input.rows;++i){<br>        for(int j=0;j&lt;grad_input.cols;++j){<br>            grad_input.data[i][j] = (2.0/total_elements)*(prediction_cache.data[i][j]-target_cache.data[i][j]);<br>        }<br>    }<br><br>    return grad_input;<br>}</pre><h3>Wrapping Up</h3><p>Congratulations 🎉🎉🎉, we’ve now built all the fundamental building blocks required for a bare-minimal neural network from scratch.</p><p>So far, we have walked through</p><ul><li>The Dense Layer</li><li>Activation Functions — ReLU and Sigmoid</li><li>Loss Function — MSE</li></ul><p>I hope the blogs helped you understand both the math and the code-level implementation of each.</p><p>In the next blog, we will implement a Model class — a framework to stack these layers together. Then we will discuss how to trigger the forward, backward and update functions of these layers and see how each layer updates their gradients. Finally we will test our model to solve the classic problem: The XOR gate which requires true non-linearity to learn.</p><p>Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d590dda5425" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I built a Neural Network from Scratch | Part 1: Dense Layer]]></title>
            <link>https://medium.com/@aditya-dawadikar/how-i-built-a-neural-network-from-scratch-part-1-dense-layer-43d5b7f39e8b?source=rss-e5802f137581------2</link>
            <guid isPermaLink="false">https://medium.com/p/43d5b7f39e8b</guid>
            <category><![CDATA[mcculloch-pitts-neuron]]></category>
            <category><![CDATA[dense-layer]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[cpp]]></category>
            <dc:creator><![CDATA[Aditya Dawadikar]]></dc:creator>
            <pubDate>Thu, 22 May 2025 17:51:41 GMT</pubDate>
            <atom:updated>2025-05-22T17:51:41.830Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OIJ8PQIhwJpX4ZFi" /><figcaption>Photo by <a href="https://unsplash.com/@halacious?utm_source=medium&amp;utm_medium=referral">Hal Gatewood</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This is 3 Part series, where I will be taking a deep dive into the internal functioning of a Neural Network and implement it with C++ from scratch. Take a look at my Repo for your reference.</p><p><strong><em>Neuronite</em></strong> (<a href="https://github.com/Aditya-Dawadikar/Neuronite">https://github.com/Aditya-Dawadikar/Neuronite</a>).</p><h3>Why Neural Networks from Scratch?</h3><p>The short answer is, I got bored treating the ML libraries like a black box. I had read about MLPs, CNNs and Transformers — I thought I understood what was happening under the hood. I figured: how hard could it be? A dense layer, a loss function, activation functions like ReLU and Sigmoid…right? RIGHT?</p><p>Mathematically I knew the drill. Forward pass = input_signal*weight + base, Backward pass = loss, gradients, updates. On paper.</p><p>I thought I could write a Neuron class, plug them into a layer and wire them up. Easy. But soon I realized the math doesn’t map cleanly to the classical Object Oriented Design. I had to go bare metal– use primitives for a smaller memory footprint. Afterall, the language of my choice was C++, and I wanted speed.</p><h3>The Plan</h3><p>After some brainstorming, reading up on the internet, and a few logical deductions later, I figured why would they call it TensorFlow? I had my small eureka moment when I realized that the Tensor was not just to address the data, rather those tensors were the building blocks for implementing a Neural Net. If I know how to operate on matrices, I’m halfway there. For my use case, 2D matrices would be enough. But then came the real question — how to compute partial derivatives on 2D matrices?</p><p>Now I have a starting point, Matrices will be the foundation. I coded the basic operations on Matrices — Addition, Subtraction, Multiplication with Scalar, Transpose, Dot Product and a helper display utility.</p><pre>//matrix.hpp<br><br>#ifndef MATRIX_HPP<br>#define MATRIX_HPP<br><br>#include &lt;vector&gt;<br>#include &lt;iostream&gt;<br><br>class Matrix {<br><br>    public:<br>        int rows, cols;<br>        std::vector&lt;std::vector&lt;double&gt;&gt; data;<br><br>        Matrix();<br>        Matrix(int rows, int cols);<br>        Matrix(const std::vector&lt;std::vector&lt;double&gt;&gt;&amp; values);<br><br>        static Matrix dot(const Matrix&amp; A, const Matrix&amp; B);<br>        Matrix transpose() const;<br><br>        Matrix operator+(const Matrix&amp; other) const;<br>        Matrix operator-(const Matrix&amp; other) const;<br>        Matrix operator*(const Matrix&amp; other) const;<br>        Matrix operator*(double scalar) const;<br><br>        void print() const;<br>};<br><br>#endif</pre><p>Next created a base class for all my Layers — Dense Layer and Activation Layers. Each layer consists of dimensions (rows, columns) and implement a forward pass, backward pass and an update function. Also, we will base our Layer class from the Matrix Abstraction.</p><pre>// layer.hpp<br><br>#ifndef LAYER_HPP<br>#define LAYER_HPP<br><br>#include &quot;matrix.hpp&quot;<br><br>class Layer{<br>    public:<br>        virtual Matrix forward(const Matrix&amp; input) = 0;<br>        virtual Matrix backward(const Matrix&amp; grad_output) = 0;<br>        virtual void update(double learning_rate) = 0;<br>        virtual ~Layer() = default;<br>};<br><br>#endif</pre><p>For weight initialization, I added a small utility <strong><em>util_random.hpp</em></strong>, capable of generating random numbers and for reproducibility, setting the seed value.</p><pre>// util_random.hpp<br><br>#ifndef UTILS_RANDOM_HPP<br>#define UTILS_RANDOM_HPP<br><br>#include &quot;matrix.hpp&quot;<br><br>void set_random_seed(unsigned int seed);<br>void initialize_random(Matrix&amp; mat, double min=-1.0, double max = 1.0);<br><br>#endif</pre><h3>The Math of Neural Networks</h3><p>Now it was time to dig deeper into the math to implement a Dense Layer. A Dense Layer of a Multi Layer Perceptron consists of 2 things that change over the course of training — The Weights and the Biases. A neural network has a forward and backward phase.</p><h4>Forward Pass</h4><p>We take the raw input, pass it throw the layers, till it reach the output layer. This information flows in the form of signal that is influenced by the weight and the bias of a neuron. Mathematically, the layer’s output can be represented as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*wyrUiRhfomfA2A-2IBBr_A@2x.png" /></figure><h4>Backward Pass</h4><p>During this phase, our signals have already reached the last layer — output layer. This when we compute the loss, i.e how far is our predicted output from the expected output. There are various types of Loss functions, but for our use case, I relied on MSE (Mean Squared Error). Once using this loss, we will compute a gradient. A gradient tells us how much a small change in a parameter, affects the final loss. Mathematically, Gradient is represented as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*P7Jbk8R10ucBRm1H3uzMwA@2x.png" /></figure><h4>Update Weights and Bias</h4><p>Finally, use the gradient to update the weight and bias. The weight will be changed in the direction opposite to the gradient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*DQwzNzTGlbTEOS7Q5BN_Bw@2x.png" /></figure><h4>Translating the equations to Matrices</h4><p>Now we discuss the part where we compute the derivatives for real. The above equations boil down to the point where we can represent partial derivates in terms of dot product of two matrices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*EM2wyUrHzJIg6KE6OTRDvg@2x.png" /><figcaption>Why does the weight gradient become a dot product?</figcaption></figure><p>Ps: Apologies for the poor visibility, of the above image.</p><p>Now that we know why the partial derivatives can be represented as simple matrix dot product, we will implement this. I designed a class called the DenseLayer that is inherited from the abstract class Layer.</p><pre>#ifndef DENSELAYER_HPP<br>#define DENSELAYER_HPP<br><br>#include &lt;matrix.hpp&gt;<br>#include &quot;layer.hpp&quot;<br><br>class DenseLayer: public Layer{<br>    private:<br><br>        Matrix input_cache;<br>        Matrix d_weights;<br>        Matrix d_bias;<br><br>        std::pair&lt;int,int&gt; input_shape;<br>        std::pair&lt;int,int&gt; output_shape;<br>    <br>    public:<br>        Matrix weights;<br>        Matrix bias;<br><br>        DenseLayer(int input_dim, int output_dim);<br><br>        Matrix forward(const Matrix&amp; input) override;<br>        Matrix backward(const Matrix&amp; grad_output) override;<br>        void update(double learning_rate) override;<br>};<br><br>#endif</pre><p>Then I implemented the functions in the dense_layer.cpp file</p><pre>// denser_layer.cpp<br><br>#include &quot;matrix.hpp&quot;<br>#include &quot;dense_layer.hpp&quot;<br>#include &quot;utils_random.hpp&quot;<br>#include &lt;cmath&gt;<br><br>// DenseLayer constructor<br>// Initializes weights and biases, and allocates memory for gradients<br>// weight shape: (input_dim × output_dim)<br>// bias shape:   (1 × output_dim)<br>// d_weights:    (input_dim × output_dim)<br>// d_bias:       (1 × output_dim)<br>DenseLayer:: DenseLayer(int input_dim, int output_dim)<br>    : weights(input_dim, output_dim),<br>        bias(1, output_dim),<br>        d_weights(input_dim, output_dim),<br>        d_bias(1, output_dim){<br>    // dont worry about this line, just needed for keeping the random number<br>    // distribution tight<br>    double limit = std::sqrt(6.0 / (input_dim + output_dim));<br>    initialize_random(weights, -1*limit, 1*limit);<br>}<br><br>// Forward pass of the dense layer<br>// input shape:  (batch_size × input_dim)<br>// output shape: (batch_size × output_dim)<br>// Computes: Z = X · W + b<br>Matrix DenseLayer:: forward(const Matrix&amp; input){<br>    // Cache input for use in backward pass<br>    input_cache = input;<br>    input_shape = {input.rows, input.cols};<br><br>    // Matrix multiplication: (batch_size × input_dim) · (input_dim × output_dim)<br>    Matrix output = Matrix::dot(input, weights);<br><br>    // Broadcast and add bias: bias is (1 × output_dim)<br>    output = output + bias;<br><br>    output_shape = {output.rows, output.cols};<br><br>    return output;<br>}<br><br>// Backward pass of the dense layer<br>// grad_output = ∂L/∂Z (gradient of loss w.r.t. layer output)<br>// grad_output shape: (batch_size × output_dim)<br>// Returns: ∂L/∂X = grad_input, shape: (batch_size × input_dim)<br>//<br>// Computes:<br>// d_weights = Xᵗ · ∂L/∂Z       (input_dim × output_dim)<br>// d_bias    = sum_rows(∂L/∂Z)  (1 × output_dim)<br>// grad_input = ∂L/∂Z · Wᵗ      (batch_size × input_dim)<br>Matrix DenseLayer:: backward(const Matrix&amp; grad_output){<br>    // ∂L/∂W = inputᵗ · grad_output<br>    d_weights = Matrix::dot(input_cache.transpose(),grad_output);<br><br>    // ∂L/∂b = row-wise sum of grad_output<br>    d_bias = grad_output.row_wise_sum();<br><br>    // ∂L/∂X = grad_output · weightsᵗ<br>    Matrix grad_input = Matrix::dot(grad_output, weights.transpose());<br><br>    return grad_input;<br>}<br><br>// Update step: performs SGD on weights and bias<br>// W := W - η ∂L/∂W<br>// b := b - η ∂L/∂b<br>void DenseLayer:: update(double learning_rate){<br>    // apply gradient updates<br>    Matrix scaled_weights = d_weights*learning_rate;<br>    Matrix scaled_bias = d_bias*learning_rate;<br><br>    weights = weights - scaled_weights;<br>    bias = bias - scaled_bias;<br>}</pre><p>I know this was heavy. But with that, we wrap up the first part of our journey — building a fully functional <strong>Dense Layer</strong> grounded in the foundational ideas of the <strong>McCulloch-Pitts model</strong>. From matrix multiplications to backpropagated gradients, we’ve taken a close look at how a single layer of a neural network processes and learns from data.</p><p>But this is just the beginning. In the next blog, we’ll peel back the layers even further — exploring <strong>activation functions</strong> that bring non-linearity to our networks and the <strong>loss functions</strong> that guide learning. Stay tuned, it only gets more exciting from here. 🚀🚀🚀</p><p>PS:</p><p>Find the implementation at <a href="https://github.com/Aditya-Dawadikar/Neuronite">Neuronite</a>. I named it Neuronite because of two reasons, first it sounds similar to Neural Net, secondly its a combination of two words Neuron + Lite, “lite” because its a bare metal implementation of Neural Networks, stripped down to the essentials. And as a bonus, the suffix <strong>“-ite”</strong> gives it the feel of being a modular or foundational component of something larger. So yeah, I guess I just came up with a really cool name for my project.</p><p>It was very difficult to add equations to the Medium blog, so I had to type it in LaTex and then add the screenshots to this blog. Thanks for your understanding.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43d5b7f39e8b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>