<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amey Borkar on Medium]]></title>
        <description><![CDATA[Stories by Amey Borkar on Medium]]></description>
        <link>https://medium.com/@ameyborkar771?source=rss-3dc41494fe21------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vUOOceR6vMVgX3uo9cpx3A.jpeg</url>
            <title>Stories by Amey Borkar on Medium</title>
            <link>https://medium.com/@ameyborkar771?source=rss-3dc41494fe21------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 20:59:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ameyborkar771/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building an AI Intelligence Layer: From Data to Decision Support]]></title>
            <link>https://ameyborkar771.medium.com/building-an-ai-intelligence-layer-from-data-to-decision-support-77ce303ca399?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/77ce303ca399</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 16:09:45 GMT</pubDate>
            <atom:updated>2026-04-30T16:09:45.560Z</atom:updated>
            <content:encoded><![CDATA[<p>Lessons from building applied AI systems that turn messy business data into predictions, recommendations, anomaly detection, LLM-powered explanations, and actionable decisions.</p><p>One of the biggest lessons I’ve learned while working on applied AI and machine learning systems is this:</p><p><strong>The model is rarely the hardest part.</strong></p><p>The harder challenge is turning messy, underused operational data into intelligence that people can actually trust, understand, and act on.</p><p>Over the past several months, I’ve been working on designing and building an AI/ML intelligence layer on top of a real-world operational system. At a high level, the goal has not been to “just build models.”</p><p>The goal has been to take complex operational data and transform it into something more valuable:</p><ul><li>predictions</li><li>prioritization</li><li>recommendations</li><li>anomaly detection</li><li>decision support</li><li>explainable business signals</li><li>natural-language interaction through LLMs</li></ul><p>This experience has reinforced something I now strongly believe:</p><p><strong>Real-world AI is not about flashy demos. It is about building useful intelligence on top of imperfect systems, noisy data, and practical business constraints.</strong></p><h3>The Problem Is Bigger Than Modeling</h3><p>In many business systems, a lot of value is hidden inside everyday operational data.</p><p>That data may include things like:</p><ul><li>historical activity</li><li>product movement</li><li>customer or account behavior</li><li>timing patterns</li><li>inventory-related signals</li><li>regional or category-level trends</li><li>exception cases</li><li>relationships between entities</li></ul><p>But raw data alone does not create value.</p><p>Reports alone do not always create value either.</p><p>The real value comes from helping people answer better questions.</p><p>Instead of only asking:</p><blockquote><em>What happened last month?</em></blockquote><p>An intelligence layer should help answer:</p><blockquote><em>What is likely to happen next?<br>What looks unusual?<br>What should be prioritized?<br>What should be recommended?<br>What hidden relationship is worth paying attention to?<br>How can a business user understand this without reading through raw tables or model outputs?</em></blockquote><p>That is where AI becomes more than analysis.</p><p>It becomes decision support.</p><h3>What I Mean by an Intelligence Layer</h3><p>When building AI for a business system, I have learned that it is better to avoid thinking of it as a simple “model project.”</p><p>A model project usually sounds like this:</p><blockquote><em>Train a model, evaluate it, deploy it.</em></blockquote><p>But real applied AI usually requires more than that.</p><p>A better framing is:</p><blockquote><em>Build an intelligence layer that sits on top of an existing workflow and makes the workflow smarter.</em></blockquote><p>That layer should not disrupt how the business already runs.</p><p>It should enhance it.</p><p>It should help teams make better decisions using the data they already generate every day.</p><p>In practice, that intelligence layer may include capabilities such as:</p><ul><li>predictive scoring</li><li>anomaly and exception detection</li><li>recommendation systems</li><li>ranking and prioritization</li><li>relationship discovery</li><li>explainability</li><li>APIs and services</li><li>LLM-powered interfaces</li><li>decision-support dashboards or workflows</li></ul><p>The important shift is this:</p><p><strong>You are not only building a model. You are building a system that converts raw operational data into useful business action.</strong></p><h3>The System Flow</h3><p>A simplified version of the architecture looks like this:</p><pre>Raw Operational Data<br>        ↓<br>Clean Analytical Layer<br>        ↓<br>Feature Engineering<br>        ↓<br>Prediction / Ranking / Recommendation / Anomaly Detection<br>        ↓<br>LLM Layer: Explanation / Query / Interaction<br>        ↓<br>APIs and Services<br>        ↓<br>Decision-Support Interface<br>        ↓<br>Business Action</pre><p>Each layer matters.</p><p>If the raw data is misunderstood, the features will be weak.</p><p>If the features are weak, the model output will be unreliable.</p><p>If the model output is not explainable, users may not trust it.</p><p>If the output does not fit into the workflow, it may never be adopted.</p><p>That is why real-world AI is a systems problem.</p><p>It is not only about choosing the best algorithm.</p><p>It is about designing the entire path from data to decision.</p><h3>Why Operational Data Is Hard</h3><p>In theory, machine learning starts with a clean dataset.</p><p>In practice, applied AI often starts with:</p><ul><li>fragmented tables</li><li>inconsistent naming conventions</li><li>missing values</li><li>business rules hidden in legacy workflows</li><li>historical exceptions</li><li>unclear relationships between entities</li><li>noisy labels</li><li>changing behavior over time</li><li>data designed for operations, not analytics</li></ul><p>This is one reason applied AI work can be challenging.</p><p>The source system may be excellent at running the business process, but that does not mean it is immediately ready for machine learning.</p><p>A lot of value comes from creating the right intermediate layers:</p><ul><li>cleaned tables</li><li>analytical views</li><li>entity-level aggregations</li><li>time-based features</li><li>behavior summaries</li><li>relationship mappings</li><li>scoring-ready datasets</li></ul><p>This work may not always look glamorous, but it is often where the foundation of the AI system is built.</p><p>Before prediction, recommendation, or anomaly detection can be useful, the system needs a reliable understanding of the data.</p><p>That usually means asking questions like:</p><ul><li>What does each entity represent?</li><li>Which tables define the core workflow?</li><li>Which fields are reliable?</li><li>Which fields are noisy or incomplete?</li><li>What business rules affect interpretation?</li><li>What historical behavior should be treated as normal?</li><li>What should be considered unusual?</li></ul><p>These questions are not separate from machine learning.</p><p>They are part of machine learning in the real world.</p><h3>Core Capabilities of the Intelligence Layer</h3><p>An intelligence layer is usually not one model or one dashboard.</p><p>It is a combination of different capabilities working together.</p><h3>1. Predictive Signals</h3><p>Prediction is not just about forecasting numbers.</p><p>It is about identifying useful future signals before they become obvious.</p><p>For example:</p><ul><li>Which entities are likely to take action soon?</li><li>Which accounts, records, or locations may need follow-up?</li><li>Which activity patterns suggest future demand?</li><li>Which behaviors indicate a meaningful change?</li></ul><p>The value of prediction is not the probability score itself.</p><p>The value is what the score enables someone to do.</p><p>A good predictive system should help a team act earlier, prioritize better, and reduce guesswork.</p><p>Instead of saying:</p><blockquote><em>This item has a score of 0.82.</em></blockquote><p>A more useful system should help answer:</p><blockquote><em>Why is this item ranked highly, and what action should be considered next?</em></blockquote><p>That difference matters.</p><h3>2. Anomaly and Exception Intelligence</h3><p>Operational systems often contain unusual patterns that are easy to miss in standard reporting.</p><p>An anomaly detection layer can help identify:</p><ul><li>abnormal combinations</li><li>unexpected activity</li><li>unusual timing</li><li>outlier behavior</li><li>inconsistent patterns</li><li>operational exceptions</li></ul><p>But anomaly detection is only useful when it is understandable.</p><p>Flagging something as “anomalous” is not enough.</p><p>The system should also help explain why something looks unusual.</p><p>That explanation may come from:</p><ul><li>feature contributions</li><li>comparison against historical behavior</li><li>rule-based context</li><li>similar past cases</li><li>business-friendly reason codes</li><li>LLM-generated summaries</li></ul><p>Without explanation, anomaly detection becomes noise.</p><p>With explanation, it becomes operational intelligence.</p><h3>3. Recommendation Support</h3><p>Recommendations are powerful because many business systems contain repeated behavior and hidden relationships.</p><p>Recommendation logic can help answer questions like:</p><ul><li>What is the next-best action?</li><li>Which item is most relevant for this entity?</li><li>Which products, services, or options are commonly associated?</li><li>Which entities behave similarly?</li><li>What should be suggested based on historical patterns?</li></ul><p>This can involve multiple approaches:</p><ul><li>collaborative filtering</li><li>content-based recommendation</li><li>association rules</li><li>similarity-based methods</li><li>ranking models</li><li>graph-based relationship discovery</li></ul><p>The important part is not only recommending something.</p><p>The important part is recommending something relevant, timely, and explainable.</p><p>A recommendation should not feel random.</p><p>It should feel grounded in behavior, context, and relationships.</p><h3>4. Prioritization and Ranking</h3><p>In many workflows, the problem is not that teams lack data.</p><p>The problem is that everything looks equally important.</p><p>A ranking layer helps solve that.</p><p>Instead of treating every record, account, order, product, or event the same way, the system can help prioritize based on signals such as:</p><ul><li>likelihood</li><li>urgency</li><li>business value</li><li>risk</li><li>frequency</li><li>historical behavior</li><li>similarity</li><li>operational impact</li></ul><p>This is one of the most practical uses of AI in business systems.</p><p>The output does not always need to be complicated.</p><p>Sometimes, the most valuable output is simply:</p><blockquote><em>Here are the top things worth looking at first, and here is why.</em></blockquote><p>That kind of prioritization can save time, focus attention, and make decision-making more consistent.</p><h3>5. LLM-Powered Explanation and Interaction</h3><p>More recently, I have also been thinking about the role of LLMs in this type of system.</p><p>Traditional ML models can score, rank, classify, recommend, or detect anomalies.</p><p>But business users often need something more:</p><ul><li>explanation</li><li>context</li><li>summaries</li><li>natural-language access</li><li>conversational exploration</li><li>easier interpretation of model outputs</li></ul><p>This is where an LLM layer can be useful.</p><p>An LLM should not replace the analytical or ML layer.</p><p>Instead, it can sit on top of structured outputs and help make the intelligence more accessible.</p><p>For example, an LLM layer can help users ask:</p><blockquote><em>Why was this item flagged?<br> What are the top reasons behind this recommendation?<br> Summarize the key changes in this account’s behavior.<br> Which records should I prioritize today and why?<br> Explain this anomaly in simple business language.</em></blockquote><p>This can bridge the gap between technical outputs and business understanding.</p><p>However, the LLM layer needs guardrails.</p><p>It should be grounded in actual system outputs, structured data, and retrieved context.</p><p>The goal is not to generate impressive language.</p><p>The goal is to make reliable intelligence easier to understand and use.</p><h3>The Importance of Hidden Relationships 🔍</h3><p>One of the most exciting parts of building an intelligence layer is discovering that business value often lives in relationships that are not obvious in flat reporting.</p><p>For example, useful signals may come from relationships between:</p><ul><li>entities and their historical behavior</li><li>products and co-occurrence patterns</li><li>time gaps and recurring cycles</li><li>operational exceptions and downstream outcomes</li><li>similar entities with similar behavior</li><li>region, frequency, volume, and consistency patterns</li></ul><p>Traditional dashboards are useful, but they often summarize what is already visible.</p><p>Relationship-driven intelligence can reveal what is not immediately visible.</p><p>That is why I have become increasingly interested in areas like:</p><ul><li>recommendation systems</li><li>ranking strategies</li><li>graph-based modeling</li><li>behavioral similarity</li><li>pattern mining</li><li>explainable predictive systems</li></ul><p>In many cases, the next level of value does not come from more reporting.</p><p>It comes from better relationship understanding.</p><h3>Accuracy Is Not the End Goal</h3><p>One of the biggest mindset shifts for me has been this:</p><p><strong>Accuracy is a metric. Usefulness is the outcome.</strong></p><p>A model can perform well technically and still fail in practice.</p><p>It may have good evaluation metrics but still not be useful to the people who need to act on it.</p><p>For applied AI to create value, it needs to answer questions like:</p><ul><li>Can someone act on this output?</li><li>Does it improve prioritization?</li><li>Does it save time?</li><li>Does it surface hidden risk?</li><li>Does it help identify opportunities?</li><li>Does it improve decision quality?</li><li>Can stakeholders understand and trust it?</li><li>Can it fit into the current workflow?</li></ul><p>This is where engineering, product thinking, and business understanding come together.</p><p>A model score is not the final product.</p><p>The final product is a better decision.</p><h3>Lessons This Work Has Reinforced</h3><p>This project has shaped how I think about applied AI in several important ways.</p><h3>1. Start With the Decision</h3><p>Before choosing a model, it is important to understand the decision the system is supposed to improve.</p><p>Who will use the output?</p><p>What will they do with it?</p><p>What does a good recommendation look like?</p><p>What does a false positive cost?</p><p>What does a missed opportunity cost?</p><p>How will success be measured?</p><p>These questions are just as important as model selection.</p><h3>2. Build Systems, Not Isolated Notebooks</h3><p>Notebooks are useful for exploration.</p><p>But production value comes from systems.</p><p>A real AI system needs:</p><ul><li>repeatable data pipelines</li><li>reliable feature generation</li><li>scoring logic</li><li>APIs or integration points</li><li>monitoring</li><li>explainability</li><li>user-facing delivery</li><li>feedback loops</li></ul><p>The model is one component.</p><p>The system is the product.</p><h3>3. Treat Explainability as a Requirement</h3><p>Explainability should not be an afterthought.</p><p>If a system recommends something, ranks something, or flags something, users naturally want to know why.</p><p>That does not always require complex explainable AI methods.</p><p>Sometimes it means providing simple, useful context:</p><ul><li>top contributing factors</li><li>historical comparison</li><li>similar past behavior</li><li>rule-based reasons</li><li>confidence level</li><li>supporting evidence</li><li>natural-language summaries</li></ul><p>The goal is to help users build trust.</p><h3>4. Make the Output Actionable</h3><p>A dashboard can show information.</p><p>An intelligence layer should support action.</p><p>There is a big difference between saying:</p><blockquote><em>Here is a chart.</em></blockquote><p>And saying:</p><blockquote><em>Here are the top priorities, the reason they matter, and the action you may want to consider.</em></blockquote><p>That difference is where AI becomes more useful.</p><h3>5. Think Beyond One Model</h3><p>Business systems are rarely one-dimensional.</p><p>They contain:</p><ul><li>behavioral patterns</li><li>temporal patterns</li><li>entity relationships</li><li>operational dependencies</li><li>exceptions</li><li>repeated actions</li><li>changing trends</li></ul><p>Because of that, one model is rarely enough.</p><p>A strong intelligence layer may combine:</p><ul><li>supervised ML for prediction</li><li>unsupervised methods for anomaly detection</li><li>ranking logic for prioritization</li><li>recommendation systems for next-best actions</li><li>graph-oriented methods for hidden relationships</li><li>rules and business logic for guardrails</li><li>LLMs for explanation and interaction</li><li>APIs and interfaces for delivery</li></ul><p>The combination matters more than any single technique.</p><h3>What This Is Teaching Me as an Engineer</h3><p>This work has strengthened how I think as an AI/ML engineer.</p><p>It has reminded me that strong applied AI is usually a combination of:</p><pre>Business Understanding<br>        +<br>Data Design<br>        +<br>System Architecture<br>        +<br>Modeling<br>        +<br>Explainability<br>        +<br>Usable Delivery</pre><p>Not just one of those things.</p><p>The most valuable AI systems are not always the flashiest.</p><p>Sometimes the biggest impact comes from quietly improving a decision that happens every day across many records, users, or transactions.</p><p>That kind of AI may not always look dramatic from the outside.</p><p>But inside a business workflow, it can create real leverage.</p><h3>Final Thought 🚀</h3><p>Applied AI is as much about clarity and usefulness as it is about intelligence.</p><p>Anyone can train a model.</p><p>The harder and more valuable challenge is building an AI system that fits into a real workflow, solves a real problem, and creates real business value.</p><p>That is the kind of AI I am most interested in building:</p><p><strong>AI that is technically strong, operationally useful, explainable, trusted, scalable and aligned with how businesses actually work.</strong></p><p>The future of AI in business will not only be about better models.</p><p>It will be about better intelligence layers — systems that connect data, models, LLMs, workflows, and human decisions in a way that creates measurable value.</p><p>That is the kind of work I want to keep building.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DSErS99Ns-EttsO9JkwLkg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77ce303ca399" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When My AI Demo Worked Perfectly, and I Still Didn’t Trust It ]]></title>
            <link>https://ameyborkar771.medium.com/when-my-ai-demo-worked-perfectly-and-i-still-didnt-trust-it-4aa4de7b6cdf?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/4aa4de7b6cdf</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-products]]></category>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Wed, 28 Jan 2026 15:03:52 GMT</pubDate>
            <atom:updated>2026-01-28T15:03:52.321Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why shipping AI isn’t about intelligence, but about trust, evals, and explainable systems</h3><p>I used to think shipping AI was about getting the demo right.</p><p>You know the feeling.</p><p>✅ The response is clean.<br>✅ The prompt is clever.<br>✅ The UI looks polished.<br>✅ Someone says, <em>“This is insane.”</em></p><p>And for a moment, you believe you’ve built something real.</p><p>Then one day… you stop believing your own system.</p><p>Not because it crashed.<br>Not because it failed loudly.</p><p>Because it answered <strong>confidently</strong> — and you couldn’t explain why.</p><p>That moment is quiet, but it changes everything.</p><h3>When confidence becomes the problem ⚠️</h3><p>It wasn’t one bug.<br>It was the pattern.</p><p>The system worked beautifully in the examples I tried — and unpredictably everywhere else.</p><p>Small prompt tweaks fixed one case and broke three others.<br>Latency was “fine” until real usage showed up.<br>Costs crept up slowly, then all at once.<br>Failures didn’t point to root causes — they sparked opinions.</p><p>And the most uncomfortable part?</p><p>When something went wrong, I had <strong>opinions</strong>, not evidence.</p><p>That’s the part no one posts screenshots of.</p><h3>The real disappointment 😔</h3><p>The disappointment wasn’t that the model wasn’t smart.</p><p>It was realizing:</p><p><strong>This isn’t scalable.</strong></p><p>Because what I had built wasn’t a system.<br>It was a performance, convincing as long as conditions stayed perfect.</p><p>And production doesn’t care about perfect conditions.</p><h3>The warning I wish I had taken seriously 🧱</h3><p>If you don’t build <strong>evaluation and observability</strong> into an AI system,<br>you don’t have a product.</p><p>You have a demo that just hasn’t met reality yet.</p><p>This isn’t about being “over-engineered.”<br>It’s about being honest with yourself about what you’re shipping.</p><h3>Relief didn’t come from a better prompt 😌</h3><p>Relief came when I stopped treating quality like a feeling<br>and started treating it like engineering.</p><p>That shift changed everything.</p><p>Not because the system became flawless,<br>but because it became <strong>explainable</strong>.</p><h3>What actually made the system trustworthy 🔍</h3><p>I didn’t start big. I started boring.</p><h3>1. A small golden set of real inputs</h3><p>Not cherry-picked examples.<br>Messy, ambiguous, real ones — the kind users actually send.</p><p>This became my anchor.<br>If the system got worse here, something was wrong — no debate needed.</p><h3>2. A rubric that matched the job</h3><p>Instead of asking <em>“Is this good?”</em>, I asked:</p><ul><li>Is it accurate?</li><li>Is it complete?</li><li>Is it safe?</li><li>Does it follow instructions?</li><li>Is the format usable?</li></ul><p>Quality stopped being subjective.</p><h3>3. Regression checks on every meaningful change</h3><p>Prompt changes.<br>Model swaps.<br>Retrieval tweaks.</p><p>If it affected output, it got evaluated.</p><p>Iteration stopped feeling risky.</p><h3>4. Traces that showed the entire chain</h3><p>For any response, I wanted to see:</p><ul><li>the user input</li><li>the retrieved context</li><li>the tool calls</li><li>the model output</li><li>latency and cost</li></ul><p>Once the system could explain itself, debugging became mechanical instead of emotional.</p><h3>5. Fallbacks for when reality misbehaves</h3><p>Because tools time out.<br>Models fail.<br>Context goes missing.</p><p>Failing safely is part of shipping responsibly.</p><h3>Confidence came back — quietly 🌱</h3><p>Not because the system became perfect.</p><p>But because when it failed, I could answer:</p><ul><li>how often it fails</li><li>where it fails</li><li>why it fails</li></ul><p>That’s when scaling becomes possible.</p><p>Not through hype.<br>Through control.</p><h3>The part that matters most 💡</h3><p>AI systems speak with confidence.</p><p>Users trust that confidence, even when it’s misplaced.</p><p>If you can’t measure, trace, and explain your system’s behavior,<br> you’re not shipping intelligence.</p><p>You’re shipping <strong>trust debt</strong>.</p><p>And eventually, someone pays for it.</p><p>If you’re building AI systems today, the hardest part usually isn’t modeling.</p><p>It’s earning the right to trust what you’ve built.</p><p>— <strong>Amey</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5S4zBryTMUwG2StZsKqF5A.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4aa4de7b6cdf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Minerals to Model Tokens: Chips, GPUs, AI Stack, Power — and Where Quantum Fits]]></title>
            <link>https://ameyborkar771.medium.com/from-minerals-to-model-tokens-chips-gpus-ai-stack-power-and-where-quantum-fits-bab7db7d14e7?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/bab7db7d14e7</guid>
            <category><![CDATA[technology-trends]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai-ethics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Thu, 02 Oct 2025 12:46:26 GMT</pubDate>
            <atom:updated>2025-10-02T12:46:26.123Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Minerals to Model Tokens: Chips, GPUs, AI Stack, Power — and Where Quantum Fits</h3><p><strong>TL;DR:</strong> Modern AI sits on a long chain: <strong>critical minerals</strong> feed <strong>chipmaking</strong>, chips become <strong>GPUs/AI accelerators</strong>, those run <strong>LLMs</strong>, and the whole thing leans on <strong>electricity and cooling</strong>. Supply jolts upstream or grid constraints downstream can slow the whole system. In the near term we’ll see more <strong>renewables + firm power (geothermal/SMRs)</strong>, better <strong>cooling</strong>, and <strong>heterogeneous silicon</strong> (GPUs + custom ASICs). Over the longer run, <strong>silicon photonics, analog/compute-in-memory, and quantum</strong> could bend the cost/energy curve — quantum most of all for specific problem classes — once engineering catches up.</p><h3>1) Minerals → Chips: what’s actually “rare” here</h3><p>Chips are mostly <strong>silicon</strong>. The “rare” part shows up around the <strong>manufacturing ecosystem</strong>: polishing compounds (e.g., cerium oxide), <strong>high-performance magnets</strong> in precision motors, lasers/optics in lithography, and — importantly — <strong>critical minerals</strong> like <strong>gallium</strong> and <strong>germanium</strong> for compound semiconductors and photonics. When a major supplier tightens export rules on these inputs, it pushes cost and lead-time shocks through the chip supply chain. China’s 2023–2025 controls on <strong>gallium and germanium</strong> are the canonical example.</p><p><strong>What that means:</strong> If upstream inputs wobble, fab tools arrive later, yields get pricier, and downstream <strong>accelerators</strong> (GPUs, HBM memory, networking) get scarcer or more expensive.</p><h3>2) Chips → GPUs: why accelerators are the workhorses</h3><p>Large language models are basically <strong>linear algebra at scale</strong>: gigantic matrix multiplies plus a ton of memory movement. <strong>GPUs</strong> won because they deliver massive parallel math and have a mature software stack (CUDA, PyTorch). The other choke point is <strong>HBM (high-bandwidth memory)</strong>; when HBM is sold out, GPU shipments slip, period. (Custom ASICs like TPUs/Trainium are growing — but GPUs remain the default target in most stacks.)</p><p><strong>So the chain so far looks like:</strong> minerals &amp; materials → <strong>fabs</strong> → <strong>HBM + GPUs</strong> → clusters.</p><h3>3) GPUs → LLMs → energy &amp; cooling</h3><p>As clusters scale, they pull hard on the grid. The <strong>IEA</strong> projects global <strong>data-centre electricity use to roughly double by 2030 to ~945 TWh</strong>, with AI the biggest driver of that growth. That’s more than the current electricity consumption of Japan. Power availability, transmission queues, and siting now decide <strong>where</strong> big AI gets built.</p><p>Cooling is the other hard limit. Air is giving way to <strong>direct liquid cooling</strong> and now <strong>on-chip microfluidics</strong> — tiny channels etched into the silicon so coolant flows where heat is born. Microsoft recently reported <strong>up to 3× better heat removal than cold plates</strong> in tests, which is a big deal for density, reliability, and cost.</p><p><strong>How one thing leads to another:</strong> scarce inputs → fewer chips → costlier GPUs → smaller/slower LLM runs → more incentive to squeeze efficiency; or, plentiful GPUs without enough power/cooling still stall deployment, so operators chase <strong>clean, firm power</strong> and better thermals.</p><h3>4) Where the electricity will actually come from</h3><p><strong>Near term:</strong> lots of <strong>wind/solar</strong> plus <strong>grid-scale batteries</strong> for daily balancing, <strong>but</strong> siting and transmission are the bottlenecks. At the same time, operators are seeking <strong>24/7 firm low-carbon</strong> to anchor campuses: <strong>advanced nuclear (SMRs)</strong> and <strong>geothermal</strong> are the leading candidates.</p><p><strong>Moonshots:</strong></p><ul><li><strong>Fusion</strong> made real scientific progress — NIF achieved <strong>fusion ignition</strong> (more energy out of the target than the laser energy into it) and private players signed first-of-a-kind power deals (e.g., <strong>Microsoft–Helion 50 MW PPA</strong>, targeting late-decade). It’s still <strong>pre-commercial</strong>, but it’s no longer sci-fi.</li><li><strong>Space-based solar</strong> is still experimental; interesting for constant sunlight, but far from grid-scale economics.</li></ul><h3>5) Add Quantum: how it relates, and what it could change</h3><p><strong>What quantum is (in our context):</strong> A <strong>co-processor</strong> that uses qubits to explore huge combinatorial spaces or simulate quantum systems more naturally than classical machines. It’s not a drop-in replacement for GPUs; think <strong>hybrid workflows</strong> where GPUs do tensor math, while a quantum system tackles optimization subproblems, sampling, chemistry/materials simulation, or cryptography-relevant tasks.</p><p><strong>Why it matters to the chain:</strong></p><ul><li>If quantum offloads <strong>hard optimization</strong> (routing, scheduling, model/training graph partitioning), you can get <strong>better utilization</strong> out of classical clusters — potentially fewer GPUs for the same quality or faster time-to-train.</li><li>In <strong>materials discovery</strong>, quantum can accelerate finding <strong>new compounds</strong> for chips, batteries, or even <strong>cooling and photonics</strong> — feeding improvements back <strong>upstream</strong> into the mineral/material side and <strong>downstream</strong> into datacenter efficiency.</li><li><strong>Energy planning</strong> itself is an optimization problem: grid dispatch, siting, transmission routing. Quantum-enhanced solvers could improve <strong>how we use</strong> electricity while we work on <strong>how we make</strong> it.</li></ul><p><strong>Where we are right now:</strong> Roadmaps from major players point to <strong>bigger systems, better error mitigation, and first examples of quantum advantage in specific domains</strong> in the 2026–2029 window — still early, but moving. Governments and industry are investing heavily (e.g., IBM’s public roadmap; large new facilities for photonic-based quantum).</p><p><strong>How one thing leads to another with quantum:</strong> progress in quantum → better optimization/materials → more efficient chips/cooling/power use → cheaper/bigger AI — looping benefits back into the entire chain.</p><h3>6) What else is coming (not fully mature, but promising)</h3><ul><li><strong>Silicon photonics &amp; co-packaged optics</strong>: pushing light closer to the die slashes I/O power and boosts bandwidth; already moving from lab to product roadmaps for AI networking. Over time, some <strong>compute</strong> may also move into the optical domain.</li><li><strong>Analog / compute-in-memory</strong>: doing math <strong>inside memory arrays</strong> to avoid data shuttling — especially potent for <strong>inference</strong>. Tooling and calibration are improving.</li><li><strong>Custom AI ASICs</strong>: TPUs/Trainium/other datacenter chips continue to rise, reducing reliance on a single GPU vendor and improving performance per watt.</li><li><strong>Better system software</strong>: sparsity, lower precision (FP8/FP4), smarter compilers/schedulers — these cut <strong>watts per token</strong> without new physics.</li></ul><h3>7) What to watch (signals that the landscape is shifting)</h3><ul><li><strong>Policy on critical minerals</strong> (gallium/germanium): if rules loosen or tighten, chip pricing and lead times will feel it within months.</li><li><strong>Photonic adoption</strong>: real deployments of <strong>co-packaged optics</strong> in AI switches/accelerators.</li><li><strong>Cooling roadmaps</strong>: production-grade <strong>microfluidic</strong> deployments beyond pilots.</li><li><strong>Quantum milestones</strong>: credible demonstrations of problem-specific advantage, plus early hybrid pipelines in optimization or chemistry.</li><li><strong>Power deals</strong>: long-term PPAs for geothermal/SMRs and serious utility-scale plans for AI-heavy regions.</li></ul><h3>8) The big picture</h3><p>We don’t “run out of elements” so much as we run into <strong>concentration, coordination, and thermodynamics</strong>. Upstream concentration (minerals, materials) can pinch chipmaking; downstream, the grid and heat removal can pinch deployment. That’s why the next decade is about <strong>redundancy and efficiency</strong>: diversify inputs, shift more bits with <strong>photons</strong>, move some math <strong>into memory</strong>, cool <strong>inside the die</strong>, and — where it makes sense — offload a few very hard subproblems to <strong>quantum</strong>.</p><p>If those pieces advance together, the chain from <strong>rocks to tokens</strong> gets sturdier — and a lot more affordable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lQL7narOdrP9sbdGWb--Jg.png" /><figcaption>The <strong>AI stack</strong> from end-to-end: <strong>minerals</strong>, <strong>chips</strong>, <strong>GPUs</strong>, <strong>LLMs</strong>, and <strong>data center power</strong>. Visualizing the critical dependencies, <strong>cooling innovations</strong>, and the transformative potential of <strong>quantum AI</strong> in building scalable and sustainable <strong>artificial intelligence</strong>.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bab7db7d14e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI & Graph Neural Networks: Smarter Inventory in Logistics]]></title>
            <link>https://ameyborkar771.medium.com/ai-graph-neural-networks-smarter-inventory-in-logistics-03000dd13ae6?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/03000dd13ae6</guid>
            <category><![CDATA[graph-neural-networks]]></category>
            <category><![CDATA[inventory-management]]></category>
            <category><![CDATA[supply-chain-management]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Mon, 29 Sep 2025 15:17:30 GMT</pubDate>
            <atom:updated>2025-09-29T15:17:30.930Z</atom:updated>
            <content:encoded><![CDATA[<p>Inventory management has always been a balancing act — too little stock and you lose sales, too much and warehouses overflow. But today, a new AI approach is changing the game: <strong>Graph Neural Networks (GNNs).</strong></p><p>Think of a supply chain as a web of suppliers, warehouses, and stores. Traditional models often treat these pieces separately. GNNs, however, are built to understand <strong>relationships</strong> across the entire network. They don’t just predict what one warehouse needs — they see how a delay at a port, a surge in regional demand, or a supplier slowdown can ripple across the whole system.</p><p>Here’s how GNNs make inventory smarter:</p><ul><li><strong>Better forecasting:</strong> By looking at the whole network, GNNs catch demand trends early and reduce the dreaded <em>bullwhip effect</em>.</li><li><strong>Real-time adaptability:</strong> GNNs update as new data arrives, helping businesses reroute stock or adjust safety levels before shelves run empty.</li><li><strong>Holistic optimization:</strong> Instead of siloed decisions, GNNs balance stock across the entire chain, cutting costs while keeping service levels high.</li><li><strong>Risk prevention:</strong> By spotting unusual patterns, they flag potential shortages and suggest proactive fixes.</li></ul><p>The result? Leaner inventories, fewer surprises, and supply chains that can absorb shocks instead of breaking under pressure.</p><p>For logistics professionals, this means fewer late-night crises. For tech experts, it’s proof that AI can finally match the complexity of real-world networks. And for everyday customers — it means the product you need is more likely to be on the shelf when you want it.</p><p><strong>Graph Neural Networks are turning supply chain chaos into competitive advantage.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2-mhAasXuWdJBqZw73r8SA.png" /><figcaption>How Graph Neural Networks (GNNs) optimize logistics: smarter forecasting, real-time adaptability, holistic inventory management, and proactive risk prevention across the supply chain.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=03000dd13ae6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Predicting the Future to Creating It: My Journey Through the Generative AI Shift]]></title>
            <link>https://ameyborkar771.medium.com/from-predicting-the-future-to-creating-it-my-journey-through-the-generative-ai-shift-d7bfeee8f102?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7bfeee8f102</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[graduate-school]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Tue, 25 Mar 2025 12:02:39 GMT</pubDate>
            <atom:updated>2025-03-25T12:02:39.522Z</atom:updated>
            <content:encoded><![CDATA[<p>A couple of years ago, I was a freshly enrolled grad student, juggling coursework, deadlines, and late-night experiments with scikit-learn. Back then, my days started with a strong cup of coffee and a notebook full of questions — mostly about data cleaning, model overfitting, and why my code wasn’t converging.</p><p>Machine learning, to me, was a fascinating puzzle. I was deep into building models that could predict customer churn or flag anomalies. It was all about precision — feature engineering, tuning hyperparameters, and squeezing every bit of accuracy out of classical models. Predictive modeling felt like magic at the time.</p><p>Fast forward to today, and everything feels… different. Now, I find myself working with language models that can write, reason, and sometimes surprise me. I no longer spend all my time building models from scratch — instead, I’m designing prompts, integrating APIs, and exploring how these generative models can assist, enhance, and even inspire.</p><p>The shift from predictive to generative AI wasn’t something I expected when I started my program. But it’s everywhere. During a recent hackathon, my team built a bot that mimicked patient responses to help train healthcare professionals. We used a RAG (Retrieval-Augmented Generation) pipeline — and watching the model simulate symptoms, emotions, and dialogue was a surreal moment. That project made me realize something big: we’re no longer just training models. We’re crafting experiences.</p><p>Of course, the technical depth still matters. As a student, I’ve had to understand fine-tuning, inference optimization, and ways to minimize hallucinations. But the real game-changer has been learning to approach AI as a collaborator, not just a tool.</p><p>It’s been humbling. And honestly, fun.</p><p>This new wave of AI is messy and unpredictable. It doesn’t always follow textbook logic. But there’s a creative spark in building with it — an energy that I didn’t feel when grinding through cross-validation runs at 2 AM. I’ve started to embrace the experimentation. The playfulness. The uncomfortable moments when the model surprises me — and teaches me something in the process.</p><p>If there’s one takeaway from my journey so far, it’s this: staying relevant in AI isn’t about mastering every new model architecture. It’s about curiosity. Adaptability. Letting go of old rules when the landscape shifts. And knowing when to lean in.</p><p>Being a grad student during this AI revolution has been a wild ride. But it’s also been the most exciting learning experience of my life.</p><p>Because now, I’m not just studying intelligent systems — I’m working with them. And that changes everything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9zMqI4n-XvpmK8XeKF1EMg.jpeg" /></figure><p>#GenerativeAI #MachineLearning #StudentLife #AIJourney #PromptEngineering #TechTransformation #AIInAcademia #GradStudentChronicles #FutureOfWork #HumanInTheLoop</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7bfeee8f102" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Secret Weapon for Tabular Data: Why TabNet Will Change How You Use Machine Learning!]]></title>
            <link>https://ameyborkar771.medium.com/the-secret-weapon-for-tabular-data-why-tabnet-will-change-how-you-use-machine-learning-dd8a1d96722c?source=rss-3dc41494fe21------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd8a1d96722c</guid>
            <dc:creator><![CDATA[Amey Borkar]]></dc:creator>
            <pubDate>Sat, 19 Oct 2024 05:33:38 GMT</pubDate>
            <atom:updated>2024-10-19T05:33:38.723Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine a world where machines make decisions that affect your life, but you have no idea how or why they made them. Sounds unsettling, right? That’s the reality we face when using many machine learning models today. They make predictions, but what goes on under the hood remains a mystery — a black box.</p><p>But what if I told you there’s a new, game-changing model that’s set to disrupt the way we handle data? Meet <strong>TabNet</strong>: the neural network that’s here to revolutionize machine learning, specifically for <strong>tabular data</strong>. If you’re curious about the future of data analysis, you’ll want to keep reading.</p><h3>So, What Exactly is TabNet?</h3><p>Before your eyes glaze over at the mention of another new machine learning model, let’s get one thing straight: <strong>TabNet isn’t just another algorithm — it’s a breakthrough</strong>. Why? Because while traditional models like Random Forests and XGBoost have been the gold standard for tabular data (think spreadsheets and databases), they’ve always struggled with one major flaw: they’re hard to understand.</p><p>TabNet solves this problem by offering a unique mix of <strong>performance</strong> and <strong>transparency</strong>. Unlike the complex, hidden processes of other deep learning models, TabNet shows you <strong>exactly how it reaches a decision</strong>. This is huge because explainability is the missing piece in many machine learning applications today.</p><h3>Why Should You Care About Tabular Data?</h3><p>You might be wondering: <strong>What’s the big deal about tabular data?</strong></p><p>Tabular data is <strong>everywhere</strong>. From banking transactions to medical records, e-commerce activity to insurance claims, it’s the backbone of most industries. Companies, hospitals, and governments use this kind of data to make crucial decisions every day. But here’s the catch — while deep learning is phenomenal for images and text, it has struggled to master tabular data. Enter TabNet, the model that finally bridges this gap.</p><h3>The Game-Changing Technology Behind TabNet</h3><p>You’ve heard of attention mechanisms in deep learning, right? If not, here’s the simple version: they allow the model to <strong>focus</strong> on the most important information instead of treating everything equally. And guess what? That’s exactly what TabNet does. But instead of using it for images or text, it’s tailored to <strong>tabular data</strong>.</p><p>TabNet works step-by-step, processing features in a <strong>smart, adaptive</strong> way. It’s like having a detective sift through a pile of evidence, picking out only the clues that matter while ignoring the noise. Unlike traditional neural networks that blindly process all features at once, TabNet has the intelligence to figure out what’s relevant and what’s not.</p><h3>The Big Question: Why Does TabNet Matter to You?</h3><p>Here’s the deal — most machine learning models today are a bit like magic tricks. You put in some data, and voilà! A prediction appears. But how did it get there? Even experienced data scientists often don’t know. That’s why <strong>interpretability</strong> is the next frontier of machine learning, and TabNet delivers it in spades.</p><p>Imagine you’re in finance, and a model predicts that a loan applicant is a high risk. You need to know <strong>why</strong> it made that call. Was it their credit history? Their employment status? Without that transparency, you’re flying blind. TabNet doesn’t just give you an answer — it shows you the entire reasoning process behind the prediction.</p><h3>But Wait — There’s More!</h3><p>Beyond its clear explanations, TabNet also offers <strong>top-tier performance</strong>. Yes, it can go head-to-head with traditional models like XGBoost in terms of accuracy, but with the added bonus of being more efficient. <strong>No more struggling with endless data preprocessing!</strong> TabNet can handle raw data like a pro, simplifying your workflow.</p><p>Oh, and did we mention TabNet’s ability to handle <strong>imbalanced datasets</strong>? It’s a common headache in machine learning when one category dominates the others, like when detecting fraud, which is rare but critical. TabNet is exceptionally skilled at zooming in on these minority classes, making it a powerful tool for problems where getting every detail right matters.</p><h3>Real-Life Superpowers of TabNet</h3><p>Now, let’s talk about where TabNet really shines. Here are some industries where it’s already making waves:</p><ul><li><strong>Healthcare</strong>: Predicting diseases or patient outcomes is incredibly complex, but also critically important. Doctors need models they can trust and understand, and TabNet provides exactly that — clear, understandable predictions based on key medical factors.</li><li><strong>Finance</strong>: From fraud detection to risk scoring, finance professionals need to be able to explain their models’ decisions to regulators and stakeholders. With TabNet, you get not just the prediction, but also the “why” behind it.</li><li><strong>E-commerce</strong>: Imagine predicting customer churn or recommending products with a model that can tell you exactly what behaviors influenced the decision. This isn’t just useful — it’s a game changer for businesses looking to get personal with their customers.</li></ul><h3>The Future is Here: Why TabNet is a Must-Know</h3><p>Machine learning isn’t going away. In fact, it’s becoming more important by the day. But the truth is, we’ve hit a ceiling with traditional models — they’re powerful, but they don’t tell us what’s going on inside. <strong>TabNet breaks through that ceiling</strong>, combining the brainpower of deep learning with the transparency and simplicity we’ve been craving.</p><p>Think about it: in a world that’s increasingly driven by data, you can’t afford to rely on models you don’t fully understand. Whether you’re in finance, healthcare, or any field dealing with data, TabNet gives you the power to <strong>see inside the black box</strong> — without sacrificing accuracy.</p><h3>Ready to Join the Revolution?</h3><p>If you’re working with tabular data, TabNet could be the game-changer you’ve been waiting for. It’s not just another shiny new tool — it’s a smarter, clearer, and more efficient way to work with the data that matters most. Imagine the possibilities: accurate predictions <strong>you can trust</strong> and understand.</p><p>In a world where machine learning is often mysterious, TabNet brings a breath of fresh air. It’s time to leave behind the old ways of blindly trusting models and step into the future with a tool that’s as transparent as it is powerful.</p><p><strong>Curious? Intrigued?</strong> You should be. The next time you’re tackling a dataset, give TabNet a try and experience the future of machine learning for yourself. You’ll wonder how you ever lived without it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpzLFomTzyZb5Yl_iEpXkQ.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd8a1d96722c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>