<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Muhammad Azam Mehr Ghulam on Medium]]></title>
        <description><![CDATA[Stories by Muhammad Azam Mehr Ghulam on Medium]]></description>
        <link>https://medium.com/@azamalis?source=rss-a7a2221df1be------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Oa9Sr-egHLftfTvQ</url>
            <title>Stories by Muhammad Azam Mehr Ghulam on Medium</title>
            <link>https://medium.com/@azamalis?source=rss-a7a2221df1be------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 22:39:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@azamalis/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Your AI Agents Need an Operating System: Harnesses, Orchestration, and the Permission Model]]></title>
            <link>https://medium.com/version-1/your-ai-agents-need-an-operating-system-harnesses-orchestration-and-the-permission-model-7c1c140590b1?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c1c140590b1</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai-agent-harness]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Mon, 27 Apr 2026 08:04:35 GMT</pubDate>
            <atom:updated>2026-04-27T08:04:35.151Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building agents is easy. Running them safely in production requires harness engineering, smart orchestration, and a layered permission model.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kA3GRFQ4bhmmQ5movu9eJg.jpeg" /></figure><p><em>This is Part 1 of a two-part series, building on my previous articles on single-agent design and MCP.</em></p><p>To see why all three matter, picture a corporate assistant where an HR agent handles employee records, a finance agent queries budgets, and an IT agent manages tickets. The moment these agents start working together, four problems show up almost immediately, and none of them have clean answers in today’s frameworks:</p><ol><li><strong>Identity</strong>: What identity is each agent using to access its data source?</li><li><strong>Access <em>(agent boundary)</em></strong>: If the finance agent needs employee context to process an expense, what stops it from reading salary records it has no business seeing?</li><li><strong>Enforcement <em>(user boundary)</em></strong>: If a user can only view their own department’s records, does the agent respect that limit, or does it query everything via a shared service account?</li><li><strong>Continuity</strong>: As context is compacted and handed off between agents, how do you ensure permission decisions and user identity are never lost along the way?</li></ol><p>Modern agent frameworks can verify who is making a request. Whether that request should be allowed (which tool, which data, which action) is left entirely to you.</p><p>Getting an large language model (LLM) to execute a tool is a solved problem. The engineering challenge lies in the <strong>harness</strong> that manages its lifecycle, the <strong>orchestration</strong> that coordinates multiple agents, and the <strong>permission model </strong>that keeps each one within its boundaries. This article covers all three.</p><blockquote><a href="https://www.version1.com/ai-strategy-and-implementation/">AI strategy and implementation — Version 1</a></blockquote><h3>What Is an Agent Harness?</h3><p>If the LLM is the CPU and the context window is RAM, the <strong>harness is the operating system</strong>. It manages everything the model cannot manage itself: session state, memory, tool execution, recovery from crashes, and the audit trail that proves what happened and why.</p><p>In practice, a production harness brings together eight distinct responsibilities. Each one addresses a failure mode that has bitten real teams in production:</p><figure><img alt="Harness components" src="https://cdn-images-1.medium.com/max/777/1*PGiEBijTQvNDH13t4BR3UQ.jpeg" /></figure><blockquote>Build the harness before you build the second agent.</blockquote><p>Most teams discover they need a harness only after their first production incident: an agent that loops forever, leaks context across users, or quietly fails halfway through a transaction with no record of what it touched.</p><h3>Orchestration Patterns: How Agents Work Together</h3><p>A robust harness keeps a single agent alive. The harder problem starts when several agents need to work together, and that is where orchestration comes in.</p><p>In a hospital A&amp;E department, a triage nurse evaluates each patient and routes them to the right specialist. The nurse does not perform surgery, but without the nurse, the surgeon would be overwhelmed with minor complaints.</p><p>That is orchestration. To do it right, you must distinguish between <strong>workflows </strong>(static sequences defined in code) and <strong>agents</strong> (dynamic behaviour decided by the LLM). Successful orchestration uses deterministic workflows to control non-deterministic agents.</p><p>Four orchestration patterns dominate production systems today. They differ in where the routing decision lives and how much autonomy the LLM has over the flow.</p><h4>Pattern 1: The Supervisor (Orchestrator as Classifier)</h4><p>A central orchestrator agent receives all user requests, classifies the intent, and routes to the appropriate specialist.</p><p><strong><em>Best for (agent-driven):</em></strong><em> Enterprise workflows with distinct data domains like HR or Finance.</em></p><h4>Pattern 2: Routing Workflow (Conditional Branching)</h4><p>User input is processed by a simple function or keyword matcher. If certain keywords appear, processing branches to a specific specialist.</p><p><strong><em>Best for (deterministic workflow):</em></strong><em> Simple intent-based branching where the routing logic is deterministic.</em></p><h4><strong>Pattern 3: Hierarchical Delegation (Agent-as-Tool)</strong></h4><p>Agents can invoke other agents as if they were tools. A “research agent” might call a “web search agent” as a sub-task.</p><p><strong><em>Best for (agent-driven): </em></strong><em>Tasks that decompose naturally into sub-tasks with clear outputs.</em></p><h4>Pattern 4: Evaluator-Generator (Verification Loop, hybrid: deterministic loop, agent-driven content)</h4><p>One agent generates output while a second agent evaluates it. If it fails, the evaluator sends feedback for a retry.</p><p><strong><em>Best for (hybrid): </em></strong><em>Quality-critical outputs where “close enough” is not acceptable.</em></p><h3>A 4-Layer Permission Model</h3><p>Routing requests to the right specialist is only half the multi-agent challenge. The other half is making sure each specialist stays inside its mandate once the request arrives.</p><p>A solicitor acts on behalf of a client, but cannot exceed the client’s authority. They cannot sign contracts the client did not approve, transfer money the client did not authorise, or share information the client did not consent to release. An agent is no different. The Open Worldwide Application Security Project (OWASP) Top 10 for Agentic Applications (released December 2025) formally catalogues risks such as <strong>ASI03: Identity &amp; Privilege Abuse</strong>. When an agent acts on behalf of a user, it inherits that user’s risks. To solve this, I recommend a layered approach.</p><h4>Layer 1: Traditional RBAC</h4><p>Role-Based Access Control (RBAC) assigns roles like “Manager” or “Employee.” While necessary, it is insufficient on its own. A coding agent with a “Manager” role might still drop a production database because it was the most efficient way to “clean up the system.”</p><h4>Layer 2: Agent-Specific Scope</h4><p>Each specialist agent should have a defined boundary. This restricts the agent itself, regardless of who is asking. An IT support agent should never have the ability to query the salary table in the HR database. This is enforced through tool allowlists and a “default-deny” policy.</p><h4>Layer 3: User-Context Propagation</h4><p>This is the hardest layer to get right, and the place where most multi-agent systems quietly fail. The user’s identity and access context must flow through the entire delegation chain, and almost no framework gives you this for free.</p><p>Consider a typical scenario. A user with regional access asks the orchestrator a question. The orchestrator delegates to a specialist agent. The specialist queries a database using a service account that can see every record in the system. Somewhere between the user logging in and the SQL hitting the database, the user’s identity has quietly evaporated. The agent now returns data the user was never authorised to see, and no audit log will catch it, because from the database’s perspective the service account was perfectly entitled to read it.</p><p>Solving this requires three commitments. First, the user’s identity and access context must travel with every request as it crosses agent boundaries, not sit implicitly inside conversation state. Second, the harness must persist that identity outside the LLM context window, so context compaction cannot strip it during summarisation. Third, the tools themselves must apply the user’s restrictions at the data layer, not trust the agent to filter results after the fact.</p><p>An agent that forgets who it is acting for is an agent that has just become a privilege escalation vector, and one that can no longer answer the Identity or Enforcement questions you asked at the start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_470_cUXy295tp72pKmDfg.png" /></figure><h4><strong>Layer 4: Runtime Guardrails</strong></h4><p>This layer catches unexpected behaviour or malicious attempts at goal hijacking (ASI01 in the OWASP Agentic Top 10), where an agent is manipulated by instructions embedded in the data it reads. Layer 4 includes:</p><ul><li><strong>Circuit breakers</strong>: Halting the agent if it enters an infinite loop.</li><li><strong>Execution budgets</strong>: Capping tool calls and token usage per session.</li><li><strong>Human-in-the-loop gates:</strong> Requiring manual approval for high-impact actions like processing a payment.</li></ul><h3>The 2026 Landscape: Frameworks vs Governance</h3><p>Building the harness, orchestration, and permission layers yourself is the hard path. The industry is rapidly building tooling that addresses all three.</p><p>The choice of framework often dictates how you implement these patterns. <strong>LangGraph</strong> remains the leading open-source choice for production due to its explicit state management. <strong>CrewAI </strong>offers an intuitive role-based model that gets teams to a working prototype quickly, though many migrate to LangGraph when state management complexity increases, while Google ADK provides native <strong>Agent-to-Agent</strong> (A2A) protocol support.</p><p>However, we are now seeing a shift toward dedicated governance layers that sit <strong>above </strong>the framework. In early April 2026, Microsoft released the <strong>Agent Governance Toolkit</strong>: an open-source, framework-agnostic system that provides deterministic policy enforcement and cryptographic agent identity. Similarly, <strong>Amazon Bedrock</strong> AgentCore’s Policy feature allows developers to define tool access using the Cedar policy language.</p><p>These tools move us away from “probabilistic security” (hoping the agent follows instructions) toward “deterministic security” (the gateway denying the tool call before it executes). If you are building a custom stack, none of this comes for free. The four-layer model above is where to start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PvzEtBqOrwRBDpWUfhpFnQ.png" /></figure><h3>Looking Ahead: Building OpsAssist</h3><p>In Part 2, we move from theory to working code. We will build <strong>OpsAssist,</strong> a multi-agent corporate assistant with HR, Finance, and IT specialists, and you will learn how to:</p><ul><li>Wrap a LangGraph agent in a custom Python harness that manages session state and audit logging</li><li>Implement a Supervisor that routes requests to the right specialist without leaking context between domains</li><li>Build a ‘PermissionGuard’ class that enforces all four layers, including user identity and access context that survives compaction and agent handoff</li><li>Add a deterministic policy enforcement layer using the Microsoft Agent Governance Toolkit, and see how it compares to your custom PermissionGuard</li><li>Deploy the whole stack with Docker, with a public GitHub repo to fork</li></ul><p>Building an agentic system is no longer just about the prompts. It is about building the operating system that ensures those prompts stay within the lines.</p><h4>About the author</h4><p>Muhammad Azam Mehr Ghulam is an AI Technical Lead here at <a href="https://www.version1.com/">Version 1</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c1c140590b1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/your-ai-agents-need-an-operating-system-harnesses-orchestration-and-the-permission-model-7c1c140590b1">Your AI Agents Need an Operating System: Harnesses, Orchestration, and the Permission Model</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing AI Projects in the Real World: Beyond Traditional SDLC]]></title>
            <link>https://medium.com/version-1/managing-ai-projects-in-the-real-world-beyond-traditional-sdlc-b1edd5a2353e?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1edd5a2353e</guid>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Thu, 11 Dec 2025 11:19:52 GMT</pubDate>
            <atom:updated>2025-12-11T11:19:52.658Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Why generative AI, LLMs, and RAG demand a new approach to data, testing, costs, and maintenance.</em></h4><figure><img alt="From deterministic world to probabilistic world" src="https://cdn-images-1.medium.com/max/1024/1*14sTj-s27SOyfFMIRtEc1Q.jpeg" /></figure><p>For decades, the Software Development Life Cycle (SDLC) has been our safety net. Whether using <strong>Waterfall </strong>or <strong>Agile</strong>, the fundamental premise has remained consistent: humans write logic, machines execute it, and if the code is bug-free, the output is guaranteed.</p><p>But as organisations pivot toward Artificial Intelligence, especially Generative AI and Large Language Models (LLMs), that predictability dissolves. “Done” becomes a moving target. <strong><em>Deadlines blur</em></strong>. This isn’t because your team is failing; it’s because the medium has changed. This is because we are moving from the <strong><em>deterministic world</em></strong> of traditional software to the <strong><em>probabilistic world </em></strong><em>of AI</em>. While a traditional app follows <em>if X → Y</em>, in AI the model suggests <em>given X, Y is 85% likely</em>.</p><p>Understanding this shift, often framed as a distinct Machine Learning Development Life Cycle (MDLC), is the difference between a project that delivers real ROI and one that remains stuck in perpetual research.</p><blockquote><a href="https://www.version1.com/blog/the-other-side-of-roi-in-ai-managing-mistakes/">The other side of ROI in AI: Managing Mistakes — Version 1</a></blockquote><h3>The Definition Gap: It’s Not Just “Software”</h3><p>Let’s be clear:</p><p><em>An AI project isn’t just a standard web app with a chatbot glued to the side.</em></p><p>A <strong><em>traditional software project</em></strong><em> </em>focuses on UI, APIs, and workflow engines. While the UI in an AI app might be predictable, the “brain” behind it is not.</p><p>This article focuses on the unique challenges of AI-driven initiatives, particularly those involving Generative AI and Retrieval-Augmented Generation (RAG), which currently dominate enterprise adoption.</p><h3>Phase 1: The Pre-Work (The Data Foundation)</h3><blockquote>In traditional development, a database is often just a storage bin.<br>In AI, <strong>data is the product</strong>.</blockquote><h4>The Knowledge Base and Governance</h4><p>Generative models are powerful, but they are generalists. For an AI system to solve a specific business problem, it needs a curated <strong>Knowledge Base (KB)</strong>, a structured collection of your internal PDFs, wikis, logs, and documents.</p><p>If this data is messy, outdated, or contradictory, the model won’t just fail; it will <strong>confidently lie to you. </strong>The old adage <strong><em>garbage in → garbage out</em></strong> is absolute law here.</p><p>This makes Data Governance a survival requirement, not just a compliance checkbox. You need to answer:</p><ul><li>Who actually owns this data?</li><li>Is it legally cleared for AI ingestion?</li><li>Who validates that the wiki page from 2019 is still accurate?</li></ul><p>Without these foundations, you’re building a skyscraper on quicksand.</p><h4>The Ground Truth Strategy</h4><p>Before writing a single line of application code, you must define what “good” looks like. In traditional coding, a unit test passes or fails. In AI, the answer is graded on a curve. You need a <strong>Ground Truth dataset</strong>: a collection of inputs with verified “correct” outputs, usually labelled by human experts. Without this golden standard, you are flying blind, unable to tell if your latest prompt tweak improved the model or made it hallucinate more.</p><h3>Phase 2: The Economics of Inference (ROI &amp; Hidden Costs)</h3><blockquote>The biggest shock to traditional project managers is the cost model.</blockquote><p>In the old world, once you provisioned servers, your marginal cost per user was negligible.</p><p>In AI, costs are typically usage-based when you rely on hosted inference APIs. You can avoid per-token pricing by deploying foundation models yourself, but doing so introduces significant infrastructure and maintenance costs, since these models are extremely resource-hungry.</p><h4>The “Token” Tax</h4><p>If you are using third party hosted inference APIs then every time a user interacts with an LLM, you <strong>pay for inference</strong>, the computational power required to generate an output. This is billed per token (roughly 3–4 characters)</p><p>Imagine a customer service bot handling 10,000 users a day, with each conversation burning 1,000 tokens. Your daily burn rate can spike unexpectedly, terrifying your finance department.</p><h4>The Hidden Costs</h4><p>Beyond inference, AI introduces additional ongoing costs:</p><ul><li><strong>Vector database storage</strong> (for embeddings and Knowledge Base)</li><li><strong>Logging &amp; audit trails</strong> for regulated industries</li><li><strong>Human review</strong> for safety and quality assurance</li></ul><h4>Model Arbitration</h4><p>To manage costs and performance, architects use <strong>Model Arbitration. </strong>You don’t need GPT-4 to reset a password.</p><ul><li>Route simple queries (“How do I log in?”) to cheaper, faster models (like GPT-4o-mini or Haiku).</li><li>Route complex reasoning tasks to the heavy hitters.</li></ul><p>Balancing latency, intelligence, and cost is the new project management triangle.</p><h3>Phase 3: The Development Cycle (The Experimental Loop)</h3><blockquote>Once coding begins, the linear path of <strong>Requirements → Build → Test → Deploy</strong> dies. It is replaced by a rapid, frustrating loop: <strong>Data → Experiment → Eval → Refine → Repeat</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*1TBAB-cmfABXq3BOrPiWvA.png" /></figure><h4>Evals Over Unit Tests</h4><p>You cannot “unit test” a conversation. Instead, you build an <strong>Evaluation Pipeline (Evals)</strong>.<br>Every time you update your prompt or knowledge base, you run your <strong>Ground Truth</strong> questions through the model.</p><p>Example:<br> You update the Knowledge Base and see:</p><ul><li>Summarisation accuracy improves (80% → 90%)</li><li>Tone consistency drops (85% → 60%)</li></ul><p>Your Eval pipeline catches this regression instantly. These evals can be run by both humans and using other LLMs specialised for evaluations.</p><h4>Shadow Mode &amp; A/B Testing</h4><p>Because AI is probabilistic, you rarely deploy straight to production.</p><ol><li><strong>Shadow Mode:</strong> The AI generates answers in the background without showing the user. You review the logs to see if it <em>would</em> have messed up.</li><li><strong>A/B Testing:</strong> Once stable, you test different prompts against real traffic to see which performs better.</li></ol><h3>Phase 4: Strategic Architecture (RAG vs Fine-Tuning)</h3><blockquote>A crucial decision is how the model gains knowledge. Think of it like a student taking an exam.</blockquote><h4>RAG (Retrieval-Augmented Generation)</h4><p><em>The Open-Book Exam.</em><br>You keep the model as-is but allow it to look up your documents during the test.</p><ul><li><strong>Pros:</strong> Cheaper, reduces hallucinations, and allows for real-time updates (just update the document, not the model).</li></ul><h4>Fine-Tuning</h4><p><em>Studying for the Exam.</em><br>You retrain the model’s brain on your data.</p><ul><li><strong>Pros:</strong> Great for teaching a specific <em>style</em>, format, or distinct behaviour (like speaking in JSON or medical shorthand).</li><li><strong>Cons:</strong> Expensive, slow, and the knowledge is frozen in time.</li></ul><h4>The Modular Approach</h4><p>The AI field moves at breakneck speed. The “<strong>state of the art</strong>” model you select today may be obsolete in six months.</p><p>Resilient architectures use <strong>model abstraction layers</strong>, allowing you to swap out the underlying model without rewriting your entire application.</p><h3>Phase 5: The “Living” Project (Maintenance &amp; Drift)</h3><blockquote>In traditional software, code doesn’t generally rot. If you leave a calculator app alone for five years, 2 + 2 still equals 4.</blockquote><p>AI is different. It decays in silence.</p><ul><li><strong>Data Drift:</strong> User behaviour changes. A fraud model trained on 2023 scams will fail against 2025 tactics.</li><li><strong>Concept Drift:</strong> The definition of “correct” evolves. Policy criteria valid last year might be compliance violations today.</li></ul><p>Maintenance isn’t about bug fixes; it’s about constant gardening. You are perpetually monitoring performance, updating knowledge bases, and re-evaluating prompts.</p><h4>Explainability &amp; Auditability</h4><p>In regulated industries, you must explain <em>why</em> the AI gave a specific answer.<br>This requires <strong>logging</strong>, <strong>traceability</strong>, and <strong>versioning </strong>far beyond standard error logs.</p><h3>Summary: The Paradigm Shift</h3><p>A quick comparison:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*_hAR96B_EUnvwyG-skcjeQ.png" /></figure><h3>The Checklist before starting any AI project</h3><p>Before approving your next AI initiative, ask these hard questions:</p><ul><li><strong>Use-Case Validity:</strong> Does this truly require AI, or could a rules engine solve it?</li><li><strong>Ground Truth:</strong> Do we have a verified dataset to evaluate against?</li><li><strong>Data Hygiene:</strong> Is our Knowledge Base clean, accurate, and governed?</li><li><strong>Inference Budget:</strong> Have we estimated cost-per-token <em>at scale</em>?</li><li><strong>Human-in-the-Loop:</strong> Do we have a workflow for low-confidence predictions?</li></ul><p>By respecting the probabilistic nature of AI and treating data as your product, you can harness the power of these systems without being crushed by their complexity.</p><h4>About the author</h4><p>Muhammad Azam Mehr Ghulalm is a Java Technical Lead here at <a href="https://www.version1.com/">Version 1</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1edd5a2353e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/managing-ai-projects-in-the-real-world-beyond-traditional-sdlc-b1edd5a2353e">Managing AI Projects in the Real World: Beyond Traditional SDLC</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Greedy to Genius: Understanding Decoding Strategies in Large Language Models]]></title>
            <link>https://medium.com/version-1/from-greedy-to-genius-understanding-decoding-strategies-in-large-language-models-93be0c036b9a?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/93be0c036b9a</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[llm-agent]]></category>
            <category><![CDATA[llms-utilization]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Tue, 21 Oct 2025 07:53:55 GMT</pubDate>
            <atom:updated>2025-10-21T07:53:55.947Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Large Language Models decide what to say next and how temperature, top-p, and other decoding parameters let you tune their creativity and control.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5x6ri3I0NljzslAsVaCmcg.png" /></figure><blockquote>Every time an LLM generates text, it doesn’t write sentences; it predicts one token at a time. The magic lies in how it chooses that next token.<br> This piece explains the decoding strategies behind text generation, from greedy decoding to sampling, and how tweaking parameters such as temperature, top-k and top-p can transform your model’s personality.</blockquote><p>If you have ever tried generating text with a Large Language Model (LLM), whether through an API, a chatbot, or an in-house deployment, you have probably noticed something curious.<br> Sometimes the model sounds brilliant and almost human. At other times, it rambles, repeats itself, or becomes strangely mechanical.</p><p>You might adjust parameters such as temperature or top-p, hoping to make it more creative, but what is actually happening beneath the surface when you do that?</p><p>This article explores that question. It explains how a Large Language Model decides what to say next, what decoding strategies really mean, and how you can fine-tune these parameters to balance creativity and control, transforming raw predictions into purposeful, human-like writing.</p><blockquote><a href="https://www.version1.com/media/blog/">More on AI — Version 1</a></blockquote><h3>How an LLM Thinks (One Token at a Time)</h3><p>Before we explore the knobs and levers, let us look more closely at the generation process itself.</p><p>A large language model does not think in full sentences; it generates text one token at a time.</p><p>At each step, it:</p><ul><li>Reads your input and its own previous outputs (the context).</li><li>Predicts which token might come next, based on probabilities.</li><li>Assigns a score to every possible token in its vocabulary.</li><li>Selects one token to output.</li></ul><p>The final step, deciding which token to choose, is what decoding strategies control. Think of it as the model’s decision-making layer, where probabilities become actual words.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*e100obcXtWCKGPUj3JrfLA.png" /></figure><h3>Decoding Strategies: Predictable vs. Creative</h3><p>There are two main philosophies behind decoding; one values <strong>certainty</strong>, the other <strong>variety</strong>.</p><h4>1. Greedy Decoding — The Predictable Path</h4><p>Greedy decoding is straightforward: at each step, the model chooses the token with the <strong>highest probability</strong>.<br> It’s simple, deterministic, and fast. Run it twice, and you’ll always get the same result.</p><p><strong>The trade-off</strong>? It’s often dull and repetitive.<br>Greedy decoding is so focused on the “most likely” next word that it forgets the broader picture — the flow and creativity of natural language.</p><blockquote><em>Example: “The sky blue.”<br>What you probably wanted: “The sky is cloudy”</em></blockquote><h4>2. Sampling — The Creative Route</h4><p>Sampling gives the model some freedom. Instead of always taking the top token, it <strong>samples</strong> from the entire probability distribution, meaning less likely words sometimes get chosen.</p><p>This adds <strong>randomness</strong> and <strong>diversity</strong>, producing more engaging, natural text. But it also means results are <em>non-deterministic</em>: the same input can yield slightly different outputs.</p><p>In most frameworks, this is enabled with do_sample=True.</p><p>Sampling by itself is only the start. To control it effectively, you use several parameters that shape how much freedom or discipline the model has.</p><h3>The Four Knobs of Text Generation</h3><p>Here’s where the engineering craft comes in.<br>Each of the following parameters affects how the model “thinks” when sampling its next token.</p><h4>1. Temperature: Controlling the Chaos</h4><p>Temperature adjusts how random or focused the output is.</p><ul><li><strong>Low temperature (≈ 0.1–0.5):</strong> Conservative, deterministic text. Ideal for structured or factual responses.</li><li><strong>High temperature (≈ 0.8–1.2):</strong> Adds flair and unpredictability — great for brainstorming or creative writing.</li><li><strong>Temperature = 0:</strong> Equivalent to greedy decoding.</li></ul><blockquote>Think of temperature as like a dial that changes the shape of probability distribution by rescaling these probabilities and in this way influencing which token is chosen.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*D9qB--BrJOXbmT65GXJ1ew.png" /></figure><h4>2. Top-K Sampling: Limiting the Options</h4><p>Here, the model considers only the <strong>K most likely tokens</strong> before picking one at random.</p><ul><li><strong>Small K (e.g., 10):</strong> Predictable, focused outputs.</li><li><strong>Large K (e.g., 100+):</strong> Broader vocabulary, more creativity.</li></ul><p>Top-K acts like a shortlist — it doesn’t change the ranking of tokens, just the number of options on the table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*IUHk_vdq1AhNeLb3J26-rg.png" /></figure><h4>3. Top-P (Nucleus) Sampling: Context-Sensitive Filtering</h4><p>Top-P, or <em>nucleus sampling</em>, takes a more adaptive approach.<br>Instead of using a fixed number of tokens, it includes as many as needed to reach a <strong>cumulative probability threshold (p)</strong>.</p><p>If <em>p = 0.8</em>, the model considers just enough tokens whose combined probability sums to 80%.</p><p>When the model is confident, that might be a handful of tokens; when it’s uncertain, the pool expands.</p><blockquote>Top-P adapts to context, producing text that’s often smoother and more contextually aware than fixed Top-K.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*soqyJeuRZpuXMC0KS4YSvQ.png" /></figure><h4>4. Repetition Penalty: Keeping Things Fresh</h4><p>LLMs can sometimes fall into loops — “The sky is blue. The sky is blue. The sky is blue.” To prevent that, we apply a <strong>repetition penalty</strong>, which discourages the reuse of previously generated tokens.</p><ul><li><strong>Typical value:</strong> 1.2</li><li><strong>Higher values (1.5–2.0):</strong> Reduce repetition but can make phrasing less natural.</li></ul><p>It’s a small tweak that dramatically improves flow and readability.</p><h4>Top-K vs. Top-P: What’s the Real Difference?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*3wodMwf1G_LfhgiHp7Y9fw.png" /></figure><p>Top-K sets a fixed cut-off regardless of context, while Top-P adjusts dynamically based on confidence, making it more nuanced and context aware.</p><h3>How to Tune These Knobs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*rWz4gP3ll7fN0q8DiLpY-A.png" /></figure><blockquote>There’s no universal setting. The best models are tuned by experimentation, testing what “feels right” for your task. These parameters are what shape the <strong>model’s personality</strong>.</blockquote><blockquote>- Turn them down and you get a careful, precise assistant.</blockquote><blockquote>- Turn them up, and you get something more imaginative and expressive.</blockquote><p>When we talk about Large Language Models being creative or predictable, we are really talking about how they decode and how they choose each word. Once you understand these <strong>decoding strategies</strong>, you move from using a model to shaping it.<br>You begin to see the output not as random chance but as something you can design, adjust, and guide. Large Language Models are not simply text predictors; <em>they are probabilistic storytellers</em>. The art of working with them lies in <strong>tuning the balance</strong> between <em>predictability </em>and <em>creativity</em>, depending on what you are building.</p><h4>About the author</h4><p>Muhammad Azam Mehr Ghulalm is a Java Technical Lead here at <a href="https://www.version1.com/">Version 1</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93be0c036b9a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/from-greedy-to-genius-understanding-decoding-strategies-in-large-language-models-93be0c036b9a">From Greedy to Genius: Understanding Decoding Strategies in Large Language Models</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting the Most Out of AI Coding Agents: Key Lessons from Using Windsurf and Cursor]]></title>
            <link>https://medium.com/version-1/getting-the-most-out-of-ai-coding-agents-key-lessons-from-using-windsurf-and-cursor-ce666421fdd0?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce666421fdd0</guid>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[prototyping]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[windsurf]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Mon, 04 Aug 2025 08:44:59 GMT</pubDate>
            <atom:updated>2025-08-04T08:44:59.766Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1zTvLfR1v7NIryuJU-HYRA.jpeg" /></figure><p>AI-powered coding tools like Windsurf and Cursor are quickly becoming essential for developers who want to streamline their workflow and automate repetitive tasks. After successfully building a Proof-of-Concept (POC) for a client, I’ve gathered a set of practical guidelines that can help developers use these AI tools effectively and safely.</p><blockquote><a href="https://www.version1.com/data-ai/">Data x AI | Version 1</a></blockquote><p>Here’s what I learned:</p><h3>1. Start Small and Keep It Simple</h3><p>Always begin with manageable, clearly defined tasks:</p><ul><li>Pick tasks small enough to finish in a single commit.</li><li>Clearly define what ‘done’ means, like passing tests or resolving merge conflicts.</li><li>Smaller tasks make it easier for the AI to get things right, and simpler for you to check the results.</li></ul><h3>2. Automate the Boring Bits</h3><p>AI excels at repetitive tasks that most developers dread:</p><ul><li>Delegate merge conflict resolutions, as AI typically handles these very accurately.</li><li>Automate responses to pull request comments, small bug fixes, and linting issues.</li><li>Let the AI manage routine infrastructure tasks, such as configuration tweaks and database migrations.</li></ul><h3>3. Be Crystal Clear in Your Instructions</h3><p>Explicit instructions greatly improve AI performance:</p><ul><li>Specify not just what you want, but also how it should be done (e.g., use Test-Driven Development).</li><li>Clearly identify frameworks, files, or functions the agent should target.</li><li>More clarity means faster, more accurate coding from the AI.</li></ul><h3>4. Embrace Rapid Prototyping — “Code is Cheap”</h3><p>AI makes coding quick and disposable:</p><ul><li>Prototype rapidly and don’t hesitate to discard generated code if it doesn’t fit the bill.</li><li>Rapid iteration helps build better solutions faster.</li></ul><h3>5. Trust But Always Verify</h3><p>Never skip reviewing the AI-generated code:</p><ul><li>Always have a human check and validate the code before merging to avoid technical debt.</li><li>Run the generated code in temporary or local environments to double-check correctness.</li><li>Remember, as a developer, you’re ultimately responsible for all merged code.</li></ul><h3>6. Prioritise Quality Early On</h3><p>Adopt a “sculptor’s approach” for shaping initial AI outputs:</p><ul><li>Tools like Sculptor help identify and fix issues early.</li><li>Insist the AI plans carefully before it starts coding.</li><li>Treat documentation and specifications as critical parts of your development workflow.</li><li>Enforce strict coding and style guidelines specifically tailored for AI to reduce common mistakes.</li></ul><h3>7. Robustly Handle Errors and Issues</h3><p>Use automation and testing proactively:</p><ul><li>Regularly run linters (like Ruff or Mypy) for minor error fixes.</li><li>Encourage AI-generated tests, but retain only those that are genuinely useful. AI agents can create large number of tests quickly, so focus on ones that cover key functionality and important tasks, while still maintaining strong overall test coverage.</li><li>Write code in a functional style to simplify testing.</li><li>Focus particularly on integration tests that verify functionality from the user’s perspective.</li><li>Regularly use AI to check your code against style guidelines, architecture, and test coverage.</li></ul><h3>8. Understand What AI Does Well (And Doesn’t)</h3><p>AI tools excel at the detailed tasks of writing and debugging code:</p><ul><li>They’re great at coding loops — quick iterations and improvements.</li><li>But they’re not suited to big-picture thinking, empathy-driven tasks, or strategic architecture planning. Human developers still need to handle these aspects.</li></ul><h3>9. Keep Things Safe with Sandboxing</h3><p>AI agents need careful handling to avoid mishaps:</p><ul><li>Always run agents within secure environments, like Docker containers, to prevent unintended consequences.</li><li>Limit API credentials and permissions to the absolute minimum required.</li></ul><h3>10. Perfect for Internal Tools and New Projects</h3><p>AI agents are particularly good for specific project scenarios:</p><ul><li>Excellent for quickly building internal tools or new projects where heavy human oversight isn’t critical.</li><li>Facilitates “vibe coding” — fast, intuitive coding sessions perfect for rapid internal development.</li></ul><h3>Final Thoughts</h3><p>Using AI coding tools like Windsurf and Cursor effectively means balancing automation with careful human oversight. Follow these practices and you’ll harness the full potential of AI, ensuring your code stays reliable, secure, and aligned with your project’s overall goals.</p><h3>Recommended Resources:</h3><ul><li><a href="https://www.youtube.com/watch?v=o_hhkJtlbSs">Software Development Agents</a></li><li><a href="https://www.youtube.com/watch?v=x_1EumTaXeE">Beyond the Prototype</a></li></ul><p>Adopting these guidelines will make AI a trusted partner in your coding workflow, rather than just another tool in your kit.</p><h4>About the author</h4><p>Muhammad Azam Mehr Ghulalm is a Java Technical Lead here at <a href="https://www.version1.com/">Version 1</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce666421fdd0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/getting-the-most-out-of-ai-coding-agents-key-lessons-from-using-windsurf-and-cursor-ce666421fdd0">Getting the Most Out of AI Coding Agents: Key Lessons from Using Windsurf and Cursor</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a JIRA AI Agent Using Spring AI and MCP]]></title>
            <link>https://medium.com/version-1/building-a-jira-ai-agent-using-spring-ai-and-mcp-7b522235ebf7?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b522235ebf7</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[spring-ai]]></category>
            <category><![CDATA[jira]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Fri, 02 May 2025 12:01:01 GMT</pubDate>
            <atom:updated>2025-05-04T10:29:14.580Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Automate Ticket Creation using Spring AI and Model-Context-Protocol</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nxPuS473IT2NafLS" /><figcaption>Photo by <a href="https://unsplash.com/@benjaminsharpe?utm_source=medium&amp;utm_medium=referral">Benjamin Sharpe</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>AI is no longer just about answering questions; it’s about planning, acting, and achieving goals. Today, we’re witnessing the rise of agentic systems — AI agents that can think ahead, make decisions, and carry out real-world actions autonomously.</p><p>This article is the first in a two-part series, where I’ll introduce the key concepts behind building an AI agent that automates JIRA ticket creation using Spring AI and the Model Context Protocol (MCP).<br>We’ll explore what agentic systems are, how MCP works, and where these tools fit into real-world applications.</p><p>In Part 2, we’ll move into the practical implementation, with a full code walkthrough and setup guide.<br>Even if you’re completely new to MCP or agentic systems, don’t worry — everything will be explained clearly and simply.</p><h3>What is an Agentic System?</h3><p>Think of a traditional chatbot as a calculator: it only provides an answer when asked. In contrast, an agentic system is more like a personal assistant, with the following traits:</p><ul><li>It understands the goal,</li><li>Plans multiple steps,</li><li>Uses external tools when needed,</li><li>And completes the task with little or no further input.</li></ul><p>Agentic systems are powered by <strong>augmented large language models (LLMs)</strong>, which go beyond simple text generation by incorporating external tools, memory, reasoning capabilities, and access to APIs.</p><p>In practice, an augmented LLM can not only understand and produce text, but also remember past interactions, plan and adapt its actions, retrieve real-time data, and communicate with external services to carry out complex tasks autonomously.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*s4d1Qw-WGq5Y0tl8jU2ofw.jpeg" /><figcaption><strong><em>Figure 1: AI Agentic System</em></strong></figcaption></figure><h3>Why External Tools Matter: The Power of Augmented LLMs</h3><p>A standard large language model (LLM) can generate text, but it cannot, on its own, create a JIRA ticket or query a database.<br> This is where tool integration becomes essential. A core component of an agentic system is an augmented LLM, which can:</p><ul><li><strong>Recall past conversations</strong>, thanks to built-in memory modules.</li><li><strong>Call APIs, query databases</strong>, or trigger external systems by making use of integrated tools.</li><li><strong>Plan and carry out complex tasks</strong>, as many models are fine-tuned on instructional datasets that teach them to reason step-by-step (a method known as <em>chain-of-thought prompting</em>), and to use memory and tool access to iterate towards a goal.</li></ul><p>This added capability transforms language models from mere text generators into genuine problem solvers.</p><h3>Enter the Model Context Protocol (MCP)</h3><p><strong>Model Context Protocol (MCP)</strong> is a standardised approach that enables AI models to communicate with external systems.</p><p>Introduced by Anthropic in late 2024, MCP provides:</p><ul><li>A consistent protocol for interaction between models and tools</li><li>A client–server architecture where:<br> — <strong>Clients</strong> (AI agents) request services<br> — <strong>Servers</strong> (MCP servers) expose specific capabilities, such as “create a<br> JIRA ticket”</li></ul><p>With MCP, developers can easily add, remove, or update tools without retraining or altering the AI agent itself.</p><p><strong>Spring AI</strong>, part of the broader Spring ecosystem, simplifies the development of AI-powered applications for Java developers. It provides:</p><ul><li>Straightforward integration with MCP (as both client and server)</li><li>Seamless connection to LLMs like OpenAI’s GPT-4</li><li>Spring Boot-style ease: configure, wire up, and you’re ready to go</li></ul><p>In short: Spring AI and MCP together make it simple to build real-world, Java-based AI agents.</p><h3>Behind the Scenes: How the Agent Works with the MCP Server (e.g., BA-Copilot)</h3><p>Building an intelligent agent like BA-Copilot requires seamless interaction between the user, the agent, the MCP server, and external systems such as the JIRA API. Let’s walk through the complete journey, step by step.</p><h4>Step 1: User Input</h4><p>The user begins by providing a task description, for example:</p><blockquote><em>“Create all tickets required to build a To-Do application.”</em></blockquote><p>This query is passed to the agent for interpretation and planning.</p><h4>Step 2: Agent Reasoning</h4><p>The agent, powered by an augmented LLM, analyses the input and works out the actions required to fulfil the request. In this example, it recognises that multiple JIRA tickets need to be created.</p><h4>Step 3: Tool Selection</h4><p>The agent selects the most appropriate tool based on the capabilities published by the MCP server. Here, it chooses the JIRA tool provided by the MCP server to handle ticket creation.</p><h4>Step 4: MCP Server Communication</h4><p>The MCP server plays a vital role in the process. It is responsible for:</p><ul><li>Publishing the available tools and their descriptions to the agent</li><li>Receiving task-specific requests from the agent</li><li>Forwarding those requests to the appropriate external systems, such as the JIRA API</li></ul><h4>Step 5: External Tool Execution</h4><p>The external system, such as the JIRA API, receives a complete request from the MCP server and proceeds to carry out the task. For example</p><blockquote><em>“Create ticket XYZ in the JIRA”</em></blockquote><p>Once completed, it returns the result to the MCP server.</p><h4>Step 6: Confirmation and Final Response</h4><p>The MCP server returns the outcome to the agent.<br> The agent then processes this information and generates a clear, user-friendly response to confirm that the requested actions have been successfully completed.</p><p>The following diagram visually summarises this workflow, highlighting how the agent, MCP server, and external systems interact throughout the process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*d6JYLktKJTmlIBvbaDxGHQ.jpeg" /><figcaption><strong><em>Figure 2: Workflow of the BA-Copilot Agent interacting with the MCP Server and External Systems.</em></strong></figcaption></figure><h3>Why This Matters for Developers</h3><p>With <strong>Spring AI</strong> and <strong>MCP</strong>, Java developers can:</p><ul><li>Quickly build <strong>AI agents</strong> that interact with any service.</li><li><strong>Wrap existing APIs</strong> easily as MCP servers.</li><li>Enable <strong>multi-tool</strong>, <strong>multi-agent</strong>, and <strong>autonomous workflows</strong>.</li><li>Build <strong>future-proof</strong> applications with modular, pluggable components.</li></ul><p><strong>The best part?</strong><br>You don’t have to reinvent your entire architecture — just extend your existing services using MCP and Spring AI!</p><h3>Conclusion</h3><p>The combination of <strong>Model Context Protocol</strong> and <strong>Spring AI</strong> unlocks the next generation of AI applications.<br>Whether you want to automate internal workflows (like JIRA ticket management) or enable dynamic, autonomous systems, the path is now clear.</p><p>If you’re a Java developer ready to explore <strong>AI-driven agents</strong>, <strong>Spring AI</strong> and <strong>MCP</strong> offer a simple, scalable, and powerful toolkit.</p><p>🚀 The future isn’t just reactive — it’s agentic.</p><h3>Further Reading</h3><ul><li><a href="https://docs.spring.io/spring-ai/reference/">Spring AI Official Documentation</a></li><li><a href="https://docs.spring.io/spring-ai/reference/api/mcp/mcp-overview.html">MCP Protocol Overview by Anthropic</a></li></ul><blockquote><a href="https://www.version1.com/data-ai/">Data x AI | Version 1</a></blockquote><h4>🚀 <strong>Stay Tuned!</strong></h4><p>In the next part of this series — <br><strong>Building a JIRA AI Agent with Spring AI and MCP (Part 2: Full Code Walkthrough)</strong> — <br> we’ll dive into the complete code behind the BA-Copilot project.</p><p>You’ll learn how to:</p><ul><li>Set up the MCP server to expose JIRA operations,</li><li>Build the AI agent client using Spring AI,</li><li>Connect the agent to the MCP server and automate JIRA ticket creation dynamically.</li></ul><p>🛠️ <strong>Coming Soon:</strong><br>Hands-on tutorials, real-world integration examples, and step-by-step guidance to help you build your own intelligent JIRA agents — from scratch to deployment. Don’t miss it!</p><blockquote>Follow me to get notified when Part 2 drops! 👨‍💻✨</blockquote><h4>About the author</h4><p>Muhammad Azam Mehr Ghulalm is a Java Technical Lead here at <a href="https://www.version1.com/">Version 1.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b522235ebf7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/building-a-jira-ai-agent-using-spring-ai-and-mcp-7b522235ebf7">Building a JIRA AI Agent Using Spring AI and MCP</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enhance Your AI Agent’s Capabilities Using Functions in Spring AI]]></title>
            <link>https://medium.com/version-1/enhance-your-ai-agents-capabilities-using-functions-in-spring-ai-c73ba7596739?source=rss-a7a2221df1be------2</link>
            <guid isPermaLink="false">https://medium.com/p/c73ba7596739</guid>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-ai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Muhammad Azam Mehr Ghulam]]></dc:creator>
            <pubDate>Fri, 07 Mar 2025 09:10:25 GMT</pubDate>
            <atom:updated>2025-03-08T13:35:48.373Z</atom:updated>
            <content:encoded><![CDATA[<p>AI Agents are programs that use “tools” to support AI models to perform specific tasks. These tools allow agents to execute client-side functions and interact with external tools and APIs in real time. By leveraging real-time data, AI models can generate more relevant and dynamic responses. This is especially useful for applications that require up-to-date information, such as weather updates or stock prices.</p><p>Spring AI has an integrated function API that defines specific tools and makes them available to AI models. In this article, we’ll explore how API functions can be used alongside LLMs to call external APIs (e.g., a weather API) and provide real-time weather information to users.</p><h3>Spring AI and function calling</h3><p>Spring AI is a framework designed to simplify the incorporation of AI functionalities into Java applications. Spring AI’s “Function calling API” gives generative AI apps capability to interact with APIs and external tools to enhance their capabilities.</p><h3>How Function Calling Works in LLMs</h3><p>To enable an LLM to use functions, the process follows these steps:</p><h4>1. Define the Function</h4><p>The first step is to define the function that the LLM will use. For example, we may create a function to fetch weather updates from an API.</p><h4>2. Provide Function Details to the LLM</h4><p>Next, we provide the LLM with a list of available functions along with their descriptions. These details are included in the prompt sent to the LLM, enabling it to recognise and use the functions appropriately.</p><h4>3. LLM Identifies the Required Function</h4><p>When the LLM processes a user prompt, it determines which function to call and provides the necessary parameters for execution.</p><h4>4. Execute the Function</h4><p>The function name is extracted from the LLM’s response, and the function is executed using the provided parameters.</p><h4>5. Return the Function Result to the LLM</h4><p>The function’s result is then sent back to the LLM along with the context of the previous interaction.</p><h4>6. Generate the Final Response</h4><p>Using the function’s output, the LLM generates a response to the original user query.</p><p>This entire process can either be handled using custom code or streamlined with frameworks that simplify function execution and integration.</p><blockquote><a href="https://www.version1.com/data-ai/">Data x AI | Version 1</a></blockquote><h3>How Function Calling Works in Spring AI</h3><p>Framework “Spring AI” provides an API to define and manage tool interactions, making it easier to integrate external functions into AI applications. It also handles structured responses and error management, which improve reliability.</p><p>Following steps are involved in defining the functions:</p><ul><li>Developers define functions as <strong>Spring Beans</strong>, specifying function names, descriptions, and argument schemas to guide AI-triggered execution.</li><li>Spring AI simplifies implementation by wrapping functions with adapter code, reducing boilerplate and handling function invocation efficiently.</li></ul><p>This approach enables agents to have seamless interactions with external tools and APIs.</p><p>The following diagram (from Spring AI’s official documentation) illustrates the sequence of actions in the function-calling process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*l4tZ-9XYIm0uaZzB9gsGsg.png" /></figure><h3>Setting Up the Project — Real-Time Weather Updates (With Code Examples)</h3><p>First, we need to select an LLM model capable of recognising and utilising tools (functions) to successfully complete tasks. For this project, we have chosen Llama 3.1 to build a live weather agent using Spring AI.</p><p>This project will use the preview version <strong>1.0.0-M5</strong> of Spring AI along with the<a href="https://www.weatherapi.com/api-explorer.aspx"> Weather API</a> to fetch real-time weather updates for a specific city or location. A simple REST API will be implemented, allowing users to retrieve weather information via a GET request.</p><h4>Running AI Models Locally with Ollama</h4><p>Ollama can also be run in a docker container using the official container <a href="https://hub.docker.com/r/ollama/ollama">https://hub.docker.com/r/ollama/ollama</a> but for this article we will be installing it locally.</p><p>To run AI models locally, you can use<a href="https://ollama.ai/"> Ollama</a>, an easy-to-use framework that enables running various LLMs on your own machine. You can Install Ollama for your OS from this link <a href="https://ollama.com/download">https://ollama.com/download</a>.</p><p>After installation, you can pull and run a llama (8b) model locally:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*B28Y82zgc2YyEqlv_nnqrA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*9z-yb53e7c6fg_R7Ph6Oqg.png" /></figure><p>If we want to deploy a Model “llama” for <strong><em>production</em></strong>, AWS bedrock (serverless architecture) can be one option. Amazon Bedrock is a fully managed service that provides access to high-performing foundation models (FMs) from top AI companies (A121 labs, Anthropic, Cohere, Meta, Luma AI, Mistral, Stability AI) through a unified API. It also includes robust security and compliance measures, such as guardrails for filtering harmful content and protecting sensitive data.</p><p>Now, let’s integrate AI into a Java application using Spring AI.</p><h4>1. Initialise a Spring Boot Application</h4><p>Start by creating a new Spring Boot project using <a href="https://start.spring.io/">Spring Initializr</a>. Select Gradle as the build tool and Java (version 21) as the language. Add dependencies such as Ollama and Spring Web. Generate and download the project. When you open the project in your favourite IDE, the build.gradle file will define the dependencies as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*UO3y0mVZed5db-VYtM0dJg.png" /><figcaption>build.gradle</figcaption></figure><h4>2. Configure Local Model Access</h4><p>Configure the application to interact with a locally running Llama model via Ollama’s API.</p><p>Modify your application.properties or application.yml file to include:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*GLdneZx-el5XM18Vmix-3A.png" /></figure><p>We also need to register at <a href="https://www.weatherapi.com/api-explorer.aspx">WeatherAPI</a> and get an API key to fetch weather data for any city. Set the API key as an environment variable for secure access.</p><h3>Implementing Function Calling</h3><p>In this example, we’ll implement a function to retrieve current weather details by:</p><ol><li><strong>Creating </strong>a record to store the weather API configuration.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*h30MbJZl7QItTyKnkjJILQ.png" /></figure><p>2. <strong>Implementing </strong>a service class, CurrentWeatherService, to call the weather API using a functional interface. A REST client will be used to make API calls. In this service, a <strong>record object</strong> is used to parse specific fields (Location<strong>, </strong>Current<strong>,</strong> and Condition) from the response received from the weather API. These records help in structuring the API response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9If1zbtskh0SJyZw-cgyA.png" /></figure><p>3. <strong>Defining </strong>a configuration class to register CurrentWeatherService as a <strong>function </strong>using the @Bean annotation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*DjjUI8jLXnpLDgJK4XAriQ.png" /></figure><p>4. <strong>Creating </strong>a REST controller to handle user requests and test the weather API. This REST API can be tested using the Curl command or any REST API client like Postman.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lT8Iz_o1lrvccxyMS12hZw.png" /></figure><h3>Testing the Agent</h3><p>After running the application, you can test the agent by sending requests to the <strong>/weather</strong> endpoint.</p><h4>Response from LLM model without function.</h4><p>When the model was asked, “What is the current weather in Cardiff?” without being provided with a function for the weather API, it responded, “<em>This functionality is not yet available; however, I can suggest a source where you can find the current weather for Cardiff — BBC weather or Accuweather website</em>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*_d7WaGCRiNMBZ3N-LeLqEQ.png" /></figure><h4>Response from LLM when Weather Function is provided:</h4><p>When the model was asked, “<em>What is the current weather in Cardiff</em>” and provided with information about the weather API function, it responded: “<em>The current weather in Cardiff is overcast with a temperature of 39.7°F and winds of 8.1 mph.</em>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*flyTCH9XkXT5cUeL-dUaXQ.png" /></figure><h3>Conclusion</h3><p>By leveraging Llama for local model execution and Spring AI with a Llama model, Java developers can create agentic AI applications capable of autonomous decision-making and function execution. This integration enhances the interactivity and intelligence of applications, opening new possibilities in various domains.</p><p>For more detailed information, refer to the<a href="https://docs.spring.io/spring-ai/reference/"> Spring AI Reference Documentation</a>. Code can be found on this GitHub repository: <a href="https://github.com/AzamMehr/springAi-agent">https://github.com/AzamMehr/springAi-agent</a>.</p><h4>About the author</h4><p>Muhammad Azam Mehr Ghulalm is Java Technical Lead here at <a href="https://www.version1.com/">Version 1</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c73ba7596739" width="1" height="1" alt=""><hr><p><a href="https://medium.com/version-1/enhance-your-ai-agents-capabilities-using-functions-in-spring-ai-c73ba7596739">Enhance Your AI Agent’s Capabilities Using Functions in Spring AI</a> was originally published in <a href="https://medium.com/version-1">Version 1</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>