<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Saurav Samantray on Medium]]></title>
        <description><![CDATA[Stories by Saurav Samantray on Medium]]></description>
        <link>https://medium.com/@saurav-samantray?source=rss-2795532fc0f2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*q_r5nHyLJz7ANOzM6NWocg.png</url>
            <title>Stories by Saurav Samantray on Medium</title>
            <link>https://medium.com/@saurav-samantray?source=rss-2795532fc0f2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:23:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@saurav-samantray/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Your AI Agent Refunded a Customer Twice. Here’s How to Make Tool Calls Run Exactly Once]]></title>
            <link>https://saurav-samantray.medium.com/your-ai-agent-refunded-a-customer-twice-heres-how-to-make-tool-calls-run-exactly-once-7d147d8e3758?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d147d8e3758</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 02:24:04 GMT</pubDate>
            <atom:updated>2026-11-10T06:28:03.275Z</atom:updated>
            <content:encoded><![CDATA[<h4>A hands-on guide to idempotency keys, a reserve-first ledger and an honest “unknown”. Build it in under 50 lines of Python and SQLite, then run it in an open-source agent runtime. No API key needed.</h4><figure><img alt="Algen.AI header: “Run it exactly once.” A refund times out. Retry, and you pay twice. Don’t, and you may not pay at all. Idempotent tool calls for AI agents · a 50-line SQLite ledger, then an open-source runtime. On the right, issue_credit(acct-1001, 2500) times out and goes to the ledger instead of a retry: a tool-execution ledger card with key run_7f3a:call_2, tool billing.issue_credit, attempts 1, status completed, and three steps: reserved before it ran (started), timed out after the request" src="https://cdn-images-1.medium.com/max/1024/1*D6mXug9sTUfAjus48W1oGA.png" /></figure><p>An agent issues a $25 refund. The request goes out, and the connection times out before the answer comes back. Did the customer get their money?</p><p>Nobody knows. Retry, and they might be refunded twice. Don’t, and they might not be refunded at all. Most agent frameworks pick one of those answers without telling you which, and the usual pick is to retry, because that’s what you do with timeouts.</p><p>For a read, that’s fine. For a refund, an email or a deleted record, a second run means a second refund, a second email, or an error on a record that’s already gone.</p><p>This is a hands-on guide to fixing it. You don’t need an API key, a cloud account or a database server. In about fifteen minutes you’ll:</p><ol><li><strong>Build a reserve-first ledger</strong> in under 50 lines of Python and SQLite, and watch it refuse to pay twice.</li><li><strong>Run the same pattern in an open-source agent runtime</strong>, through four failure scenarios, offline.</li><li><strong>Put the rule in CI</strong> with a test that fails a tool that retries refunds.</li></ol><h3>Why agents hit this harder than ordinary services</h3><p>A web service retries a timeout now and then. An agent has four separate ways to run the same tool twice:</p><ul><li><strong>The framework retries a timeout.</strong> A generic retry policy can’t tell a refund from a search. Three attempts with backoff is a good default for reads and a bad one for writes.</li><li><strong>The process restarts mid-step.</strong> The run is recovered from its last checkpoint, which was saved before the tool ran. Replaying the step replays the call.</li><li><strong>Two processes pick up the same run.</strong> A deploy overlaps old and new workers, or recovery starts on a second machine while the first is still alive.</li><li><strong>The model calls it again.</strong> The tool returned an error or a result the model didn’t expect, so it tries once more, perhaps with slightly different arguments. To the model, that’s diligence.</li></ul><p>Agent runs also pause for hours and resume on other processes, which gives all four more chances to happen.</p><figure><img alt="Algen.AI diagram titled “Four ways an agent runs a tool twice”, four cards: “The framework retries a timeout” (a clock with a retry arrow, “retry ×3”), “The process restarts mid-step” (a crashed window and a restart marker), “Two processes pick up one run” (old and new workers both pointing at one run) and “The model calls it again” (a speech bubble, “didn’t seem to work… call again”). The first three are labelled “The runtime’s job”, the last “Partly the prompt’s job”." src="https://cdn-images-1.medium.com/max/1024/1*2bPc4EDaivtfs3Yj1Z16jg.png" /><figcaption>Four ways an agent runs a tool twice. The first three are the runtime’s job.</figcaption></figure><h3>“Exactly once” doesn’t exist. Here’s what does</h3><p>Distributed systems people will tell you, correctly, that exactly-once <em>delivery</em> is impossible. A message is either sent until it’s acknowledged, which can mean twice, or sent once, which can mean never.</p><p>What you can have is an <em>effect</em> that happens once: send the request as many times as you need to, and make the receiver recognise a repeat. That’s what an idempotency key is for. Stripe has accepted an Idempotency-Key header on payment requests for years for exactly this reason: send the same key twice and the second request gets the first one&#39;s answer instead of a second charge.</p><p>So “run it exactly once” asks three things of whatever runs your agent’s tools:</p><ol><li>Don’t send a request again when you already know the answer.</li><li>When you do have to send it again, send the same key.</li><li>When you can’t know, say so, and don’t guess.</li></ol><h3>Part 1: A ledger in under 50 lines</h3><p>Here’s the whole idea in plain Python, with SQLite from the standard library. Create an empty folder and save this as ledger.py:</p><pre>import json<br>import sqlite3</pre><pre>db = sqlite3.connect(&quot;ledger.db&quot;)<br>db.execute(&quot;&quot;&quot;CREATE TABLE IF NOT EXISTS tool_calls (<br>    key    TEXT PRIMARY KEY,   -- &lt;run_id&gt;:&lt;tool_call_id&gt;<br>    status TEXT NOT NULL,      -- started | completed | indeterminate<br>    result TEXT<br>)&quot;&quot;&quot;)<br></pre><pre>class Unknown(Exception):<br>    &quot;&quot;&quot;The call may or may not have taken effect. Don&#39;t run it again; look it up.&quot;&quot;&quot;<br></pre><pre>def run_once(key, tool, *args):<br>    # 1. Reserve the key before doing anything. Only one caller can insert it.<br>    with db:<br>        reserved = db.execute(<br>            &quot;INSERT INTO tool_calls (key, status) VALUES (?, &#39;started&#39;)&quot;<br>            &quot; ON CONFLICT DO NOTHING&quot;,<br>            (key,),<br>        ).rowcount == 1<br>    if not reserved:<br>        status, result = db.execute(<br>            &quot;SELECT status, result FROM tool_calls WHERE key = ?&quot;, (key,)<br>        ).fetchone()<br>        if status == &quot;completed&quot;:<br>            return json.loads(result)  # a replay reads the ledger<br>        raise Unknown(f&quot;{key} is {status}&quot;)  # never guess</pre><pre>    # 2. Run it once, passing the key downstream.<br>    try:<br>        result = tool(*args, idempotency_key=key)<br>    except TimeoutError:<br>        with db:<br>            db.execute(<br>                &quot;UPDATE tool_calls SET status = &#39;indeterminate&#39; WHERE key = ?&quot;, (key,)<br>            )<br>        raise Unknown(f&quot;{key} timed out after the request left&quot;)<br>    with db:<br>        db.execute(<br>            &quot;UPDATE tool_calls SET status = &#39;completed&#39;, result = ? WHERE key = ?&quot;,<br>            (json.dumps(result), key),<br>        )<br>    return result</pre><p>Three things are doing the work:</p><ul><li><strong>The key is reserved before the tool runs.</strong> The INSERT … ON CONFLICT DO NOTHING is atomic, so if two processes race for the same call, only one of them gets rowcount == 1 and runs it.</li><li><strong>A completed key returns its recorded result.</strong> A replay after a crash reads the ledger instead of calling the tool again.</li><li><strong>A timeout is recorded as </strong><strong>indeterminate, not </strong><strong>failed.</strong> The request left; it may have landed. run_once refuses to run that key again and raises Unknown instead. So does a call whose process died mid-run: it&#39;s still started, and that&#39;s unknown too.</li></ul><p>Now a fake payments API that times out on one order. Save it as demo.py:</p><pre>from ledger import Unknown, run_once</pre><pre>sent = []<br></pre><pre>def refund(order_id, cents, idempotency_key):<br>    sent.append(idempotency_key)<br>    if order_id == &quot;ord-2&quot;:<br>        raise TimeoutError(&quot;payments API timed out&quot;)<br>    return {&quot;refund_id&quot;: f&quot;rf-{len(sent)}&quot;, &quot;order_id&quot;: order_id, &quot;cents&quot;: cents}<br></pre><pre>print(run_once(&quot;run-1:call-1&quot;, refund, &quot;ord-1&quot;, 2500))<br>print(run_once(&quot;run-1:call-1&quot;, refund, &quot;ord-1&quot;, 2500))   # replay after a crash</pre><pre>for attempt in (1, 2):<br>    try:<br>        run_once(&quot;run-1:call-2&quot;, refund, &quot;ord-2&quot;, 4000)<br>    except Unknown as e:<br>        print(f&quot;attempt {attempt}: unknown -&gt; {e}&quot;)</pre><pre>print(f&quot;requests sent to payments: {len(sent)}&quot;)</pre><p>Run it:</p><pre>python demo.py</pre><p>You should see:</p><pre>{&#39;refund_id&#39;: &#39;rf-1&#39;, &#39;order_id&#39;: &#39;ord-1&#39;, &#39;cents&#39;: 2500}<br>{&#39;refund_id&#39;: &#39;rf-1&#39;, &#39;order_id&#39;: &#39;ord-1&#39;, &#39;cents&#39;: 2500}<br>attempt 1: unknown -&gt; run-1:call-2 timed out after the request left<br>attempt 2: unknown -&gt; run-1:call-2 is indeterminate<br>requests sent to payments: 2</pre><p>Four calls to run_once, two requests to the payments API. The replay of run-1:call-1 came back from the ledger with the same refund_id, and the second attempt at the timed-out refund was refused without sending anything. Open the database to see what&#39;s left behind:</p><pre>sqlite3 ledger.db &quot;select key, status from tool_calls&quot;</pre><pre>run-1:call-1|completed<br>run-1:call-2|indeterminate</pre><p>Notice what the key is made of: the run and the tool call, &lt;run_id&gt;:&lt;tool_call_id&gt;. It is <em>not</em> a hash of the arguments. Two separate $25 refunds that a customer really is owed have identical arguments and are still two effects; a hash would merge them. And a key that changed whenever the model reworded an argument wouldn&#39;t catch a replay at all.</p><p>The same SQL works on PostgreSQL with %s placeholders, which is where a production ledger belongs: next to your run checkpoints.</p><figure><img alt="Diagram “One tool call in the ledger”. Reserve: record the key &lt;run_id&gt;:&lt;tool_call_id&gt; as started. It ends Completed (it ran; a replay returns the recorded result), Failed (it didn’t happen; nothing took effect, so it may run again) or Unknown (nobody knows; started or indeterminate; never run again by itself). Unknown goes to Settle it: ask the tool (the reconcile hook looks the key up; found → completed, not found → failed), ask a person (an operator resolves it, with a reason), or undo it (co" src="https://cdn-images-1.medium.com/max/1024/1*wPYvB7aIkwKT5TUnVOgnxA.png" /><figcaption>One tool call in the ledger: completed, failed, or honestly unknown, and three ways to settle an unknown.</figcaption></figure><p>What these 50 lines don’t do is the hard half: settle an indeterminate call. Somebody has to find out whether the refund landed, the run has to wait for that answer instead of failing, and the model has to be told the truth afterwards. That&#39;s what Part 2 is about.</p><h3>Part 2: The same pattern in an agent runtime</h3><p>We build <a href="https://github.com/AlgenAI/algen-agent-runtime">Algen Agent Runtime</a>, an open-source Python runtime for governed agents, and this ledger is built into how it runs tools. Its reconciliation example runs offline with a scripted mock model and an in-process fake billing system. In a fresh virtual environment:</p><pre>python -m pip install --pre algen-agent-runtime<br>python -m examples.pattern_side_effect_reconciliation.app</pre><p>It finishes in about a second and plays four scenarios:</p><figure><img alt="Terminal recording: python -m examples.pattern_side_effect_reconciliation.app prints four scenarios. 1: the credit was applied then timed out; the run pauses in awaiting_reconciliation, the reconcile hook finds cr-0001 and the run completes. 2: the request timed out before it was applied; GET /v1/tool-executions without the scope returns 403, reconcile says unknown, an operator resolves it failed with a reason and the run completes. 3: an operator compensates the first credit and cr-0001 is void" src="https://cdn-images-1.medium.com/max/1000/1*jMZ7HNAAwn70okHONm1ulg.png" /><figcaption>A real run of the reconciliation example on algen-agent-runtime 0.1.0a23: four scenarios, offline, in about a second.</figcaption></figure><h3>1. The credit landed, then the response timed out</h3><pre>run status: awaiting_reconciliation<br>pause: kind=tool_reconciliation tool=billing.issue_credit key=&lt;run_id&gt;:&lt;tool_call_id&gt;<br>operator lists unknown calls: [(&#39;billing.issue_credit&#39;, &#39;indeterminate&#39;)]<br>reconcile hook: outcome=completed run_resumed=True result_reference=cr-0001<br>run status: completed; output: Credited acct-1001 with 25.00.<br>events: [&#39;tool.reconciliation_required&#39;, &#39;tool.reconciled&#39;, &#39;tool.completed&#39;]</pre><p>The run doesn’t fail and doesn’t retry. It pauses in awaiting_reconciliation, and the ledger lists the call as indeterminate. The tool&#39;s reconcile hook then asks the billing system whether a credit exists under that idempotency key. It does (cr-0001), so the run resumes with the real result as if the timeout never happened.</p><h3>2. The request timed out before it landed</h3><pre>run status: awaiting_reconciliation<br>GET /v1/tool-executions without tools:reconcile -&gt; 403<br>GET /v1/tool-executions -&gt; billing.issue_credit indeterminate (error ToolExecutionError, has_result False)<br>POST .../reconcile -&gt; outcome=unknown<br>POST .../resolve -&gt; outcome=failed run_resumed=True actor=ops-maria reason=&#39;billing support confirms no credit for acct-2002&#39;<br>run status: completed; output: The credit did not go through; nothing was charged.</pre><p>This time the hook can’t find anything, and “not found yet” doesn’t prove “never arrived”, so it answers unknown. An operator with the tools:reconcile scope (a viewer gets a 403) checks with billing and resolves the call failed, with a reason. The run resumes and the model is told the credit didn&#39;t go through.</p><h3>3. Undo, only when someone asks</h3><pre>compensate: outcome=compensated status=compensated<br>ledger decisions: [&#39;reconcile:completed&#39;, &#39;compensate:compensating&#39;, &#39;compensate:compensated&#39;]<br>billing voided credits: [&#39;cr-0001&#39;]</pre><p>Compensation exists, but the runtime never runs it by itself. It can’t tell whether a failure later in a run makes an earlier effect wrong, and compensating a call that never happened can be a side effect of its own. An operator decides.</p><h3>4. Refused before anything was sent</h3><pre>run status: completed<br>ledger: billing.issue_credit failed (error ToolNotAttemptedError)<br>requests sent to billing: 0<br>left to reconcile: 0</pre><p>The account is locked, so the tool raises ToolNotAttemptedError before calling billing. Nothing left the building, so the call is recorded failed, not indeterminate, and there&#39;s nothing to reconcile. Only a call that may have reached the other system becomes unknown. That distinction saves a lot of reconciliation work.</p><h3>The tool that makes it work</h3><p>Open examples/pattern_side_effect_reconciliation/tools.py in the installed package. The tool declares its side effect, passes the key downstream, and brings two hooks:</p><pre>async def find_credit(record: ToolExecutionRecord, context: ToolContext) -&gt; ReconcileOutcome:<br>    &quot;&quot;&quot;Ask the billing system whether the call took effect. Read-only: it may run often.&quot;&quot;&quot;<br>    system: FakeBillingSystem = await billing.client()<br>    credit = system.find_credit(record.idempotency_key)<br>    if credit is None:<br>        # Absent now does not prove it never arrived (it may still be in flight upstream), so<br>        # say unknown and let an operator decide.<br>        return ReconcileOutcome(status=&quot;unknown&quot;, detail=&quot;no credit under this key yet&quot;)<br>    return ReconcileOutcome(<br>        status=&quot;completed&quot;,<br>        value=credit.model_dump(),<br>        result_reference=credit.credit_id,<br>        detail=&quot;credit found by idempotency key&quot;,<br>    )<br></pre><pre>async def void_credit(result: ToolResult, context: ToolContext) -&gt; None:<br>    &quot;&quot;&quot;Undo the credit. Idempotent by key; copes with a credit that never happened.&quot;&quot;&quot;<br>    system: FakeBillingSystem = await billing.client()<br>    system.void_credit(context.idempotency_key)<br></pre><pre>@billing.tool(external(timeout=10), reconcile=find_credit, compensation=void_credit)<br>async def issue_credit(account_id: str, amount_cents: int, context: ToolContext) -&gt; Credit:<br>    &quot;&quot;&quot;Credit an account, in cents. The billing system deduplicates by idempotency key.&quot;&quot;&quot;<br>    system: FakeBillingSystem = await billing.client()<br>    if account_id in system.locked_accounts:<br>        # Refused before anything was sent: the ledger records the call failed, not<br>        # indeterminate, so nobody has to reconcile it.<br>        raise ToolNotAttemptedError(f&quot;account {account_id} is locked; nothing was sent&quot;)<br>    return system.apply_credit(context.idempotency_key, account_id, amount_cents)</pre><p>external(...) means the tool acts outside the system, so it gets exactly one attempt. context.idempotency_key is &lt;run_id&gt;:&lt;tool_call_id&gt;, stable across every replay of the call. The reconcile hook only reads, so it&#39;s safe to run as often as needed, including when a recovered run replays a call that was in flight when the process died.</p><p>If the API you’re calling doesn’t honour idempotency keys, store the key on the record you create (a reference, metadata or description field) so a reconcile hook can at least look it up later.</p><h3>One line of config to pause instead of fail</h3><pre>agents:<br>  - name: billing-assistant<br>    tool_policy:<br>      on_indeterminate: pause<br>      on_error: return_to_model</pre><p>With on_indeterminate: pause, a run whose call is unknown waits in awaiting_reconciliation, and settling the call resumes it. Without it, an unknown call is handled like any failed call.</p><h3>The operator API</h3><p>Everything an operator did above is also plain HTTP:</p><ul><li>GET /v1/tool-executions lists a tenant&#39;s unknown calls, longest-waiting first.</li><li>POST /v1/tool-executions/{key}/reconcile asks the tool&#39;s hook.</li><li>POST /v1/tool-executions/{key}/resolve records completed or failed, with a required reason.</li><li>POST /v1/tool-executions/{key}/compensate runs the tool&#39;s undo.</li></ul><p>Every decision is compare-and-set on the ledger status, so if two operators act at once, exactly one wins, and it’s recorded with the operator’s id. A call that finally returns after someone settled it can’t overwrite their decision.</p><h3>Part 3: Put the rule in CI</h3><p>The retry rule is simple enough to fit in a table, and simple enough to forget in a code review:</p><figure><img alt="Algen.AI table titled “Retry reads, not writes.” read(), look up an order: up to 3 attempts, a timeout means try again. write(), change an account’s owner: 1 attempt, indeterminate. external(), send an email or issue a credit: 1 attempt, indeterminate. destructive(), delete a record: 1 attempt, indeterminate. Footer: Checked in CI by assert_tool_contract." src="https://cdn-images-1.medium.com/max/1024/1*y1mTZ0ziilqdhU0DTMZtGA.png" /><figcaption>The retry rule: a tool’s declared side effect sets how many attempts it gets and what a timeout means.</figcaption></figure><p>So check it in a test. The runtime ships assert_tool_contract, which fails any tool that writes, sends or deletes and is allowed more than one attempt or has no idempotency key. Here&#39;s a &quot;helpful&quot; refund tool that retries three times, next to the billing tool from Part 2. Save as test_tool_contract.py:</p><pre>from algen_agent_runtime.tools.authoring import Toolkit, external<br>from algen_agent_runtime.tools.contracts import ToolContext<br>from algen_agent_runtime.tools.testing import assert_tool_contract<br>from pydantic import BaseModel</pre><pre>from examples.pattern_side_effect_reconciliation.tools import issue_credit</pre><pre>payments = Toolkit(&quot;payments&quot;)<br></pre><pre>class Refund(BaseModel):<br>    refund_id: str<br></pre><pre># A &quot;helpful&quot; retry policy: three attempts at a refund.<br>@payments.tool(external().replace(max_attempts=3))<br>async def refund(order_id: str, amount_cents: int, context: ToolContext) -&gt; Refund:<br>    &quot;&quot;&quot;Refund an order, in cents.&quot;&quot;&quot;<br>    ...<br></pre><pre>def test_issue_credit_keeps_the_contract():<br>    assert_tool_contract(issue_credit)<br></pre><pre>def test_refund_keeps_the_contract():<br>    assert_tool_contract(refund)</pre><pre>python -m pip install pytest<br>python -m pytest -q test_tool_contract.py</pre><pre>...<br>E           AssertionError: tool &#39;payments.refund&#39; breaks the tool contract:<br>E           - a external tool is retried 3 times; a retry after a timeout could repeat the change, so allow one attempt<br>FAILED test_tool_contract.py::test_refund_keeps_the_contract - AssertionError...<br>1 failed, 1 passed in 0.23s</pre><p>Delete .replace(max_attempts=3) and both tests pass. The rule now lives in CI instead of in a wiki page.</p><h3>What this doesn’t solve</h3><p><strong>The other system has to cooperate.</strong> The key makes a repeat recognisable; only the receiver can refuse it. Against an API with no idempotency support and nowhere to store the key, the ledger still stops <em>you</em> from repeating yourself, but an unknown call can only be settled by a person checking by hand.</p><p><strong>A new call from the model is a new key.</strong> If the model hears that a call failed and decides to make it again, that’s a different tool call. on_indeterminate: pause exists for this reason: the model doesn&#39;t hear &quot;failed&quot; while the honest answer is &quot;unknown&quot;.</p><p><strong>Workflow retries are at-least-once.</strong> A workflow node set to recovery_policy: retry starts a fresh agent run with its own keys. Use it only for nodes whose tools are read-only or have a reconcile hook.</p><p><strong>It’s early.</strong> At the time of writing the runtime is 0.1.0a23, a pre-release hardened for a single node. Coordinated recovery across several processes is still open work.</p><h3>Five things worth copying</h3><p>Whatever you build on:</p><ul><li><strong>Key every side effect by call, not by arguments,</strong> and pass the key to the system you’re changing.</li><li><strong>Record the call before you make it.</strong> A replay should read the ledger, not rerun the tool.</li><li><strong>Give writes one attempt.</strong> Retries are for reads.</li><li><strong>Treat “unknown” as an answer.</strong> Record it, pause on it and look it up. Never resolve it by trying again.</li><li><strong>Undo by decision, not by reflex.</strong> Compensation is an operator’s call.</li></ul><p>An agent that does something twice is worse than one that does nothing, because nobody notices until the customer does.</p><p>If you’d rather break something than build it, play <a href="https://algen.ai/arcade/heist-the-agent">Heist the Agent</a>. Its fourth trick is a double payout: Penny the refund bot pays you, crashes before recording it, and pays you again on restart.</p><p><em>The design behind this guide, including how we SIGKILL the runtime mid-call in our tests to check it, is in the original post, </em><a href="https://algen.ai/blog/run-it-exactly-once"><em>Run It Exactly Once: Idempotent Tool Calls for AI Agents</em></a><em>. The runtime is open source on </em><a href="https://github.com/AlgenAI/algen-agent-runtime"><em>GitHub</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d147d8e3758" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Approvals Belong in the Runtime, Not the Prompt]]></title>
            <link>https://saurav-samantray.medium.com/approvals-belong-in-the-runtime-not-the-prompt-e616b4410d2c?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e616b4410d2c</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 03 Nov 2026 13:53:57 GMT</pubDate>
            <atom:updated>2026-11-03T13:53:57.386Z</atom:updated>
            <content:encoded><![CDATA[<h4>“Always ask before sending an email” is a suggestion, not a control. Four ways prompt-based approvals fail, and what an approval in the execution path needs.</h4><figure><img alt="Algen.AI card: “Approvals belong in the runtime, not the prompt.” The prompt line “Always ask the user before sending an email.” is struck through as a suggestion; below it a runtime approval request shows the exact call update_owner with its arguments, schema and precheck checks, Approve / Modify / Reject buttons and an expiry. Footer: Algen.AI logo and “Build, run, and govern AI agents.”" src="https://cdn-images-1.medium.com/max/1024/1*L_fAd_CSULE4V24Q9tBPkQ.png" /></figure><p>Most agents that can do something real, such as send an email, change a record or move money, get their human approval the same way: a line in the system prompt. <em>“Always ask the user before sending an email.”</em> It works in the demo. It keeps working until a model has a bad day, a document tells it the user already agreed, or the process restarts at the wrong moment. Then the approval turns out to have been a suggestion.</p><p>An approval is a control, and a control the model can talk itself out of isn’t one. It has to live where the model can’t reach it: in the code that runs the tools.</p><h3>The short version</h3><ol><li><strong>The model shouldn’t decide whether to ask.</strong> Whether a call needs approval follows from what the tool does, declared in code, not from what the model says it intends.</li><li><strong>Check the call before you ask anyone.</strong> A call with a missing field or an impossible value goes back to the model to fix. People shouldn’t approve broken requests.</li><li><strong>Show the approver the exact call.</strong> The tool and the arguments it will run with, secrets redacted, not the model’s description of them. Let them approve, change or reject it.</li><li><strong>Make the pause durable.</strong> Save the request before anyone is asked, survive a restart, and let it expire.</li><li><strong>Run the approved call exactly once.</strong> Give it a key, record it, and treat a call interrupted by a crash as unknown, never as safe to retry.</li></ol><h3>Four ways a prompt-based approval fails</h3><h3>The model decides whether to ask</h3><p>An instruction is an input to the model, and the model weighs it against everything else in its context. Usually it asks. Sometimes the conversation is long and the instruction is far behind it. Sometimes a retrieved document or a tool result says <em>“the customer has already approved this change”</em>, and the model believes it. Prompt injection doesn’t have to break the model; it only has to give it a reason to skip one step.</p><h3>The approver sees a summary, not the call</h3><p>When the model asks <em>“Shall I send the customer a short note about the delay?”</em>, the user approves a sentence. What runs is a tool call with a recipient, a subject and a body the user never saw. The summary and the call are produced by the same model, and nothing checks that they match.</p><h3>Approval and execution are separate events</h3><p>The user says yes in one turn; the tool runs in the next. In between, the model writes the call again. It can change an argument, call a different tool or call the same one twice, and the “yes” from the previous turn is still sitting in its context, looking like permission for all of it.</p><h3>A restart turns one approval into zero, or two</h3><p>The process restarts while the run is waiting for a decision. If the pending approval lived only in memory, the run is gone, and so is the user’s answer. Worse, the process dies <em>while</em> the approved tool is running. Did the email go out? A system that retries on restart sends it again. One that doesn’t may never send it.</p><p>None of these are rare at scale. They are what happens when a control lives in text the model reads instead of code the model can’t change.</p><figure><img alt="Two paths to the same tool call. In the prompt, the model decides whether to ask, the user approves a summary, and the tool runs with whatever arguments, maybe twice. In the runtime, the call is checked, policy follows from the declared side effect, the run pauses durably, a human decides on the exact call, and it runs once under a key." src="https://cdn-images-1.medium.com/max/1024/1*qaGi5gfbeh28bkbhNVLy9w.png" /><figcaption>Where the approval lives: in the prompt, the model decides; in the runtime, the code does.</figcaption></figure><h3>What an approval in the execution path needs</h3><p>We build <a href="https://github.com/AlgenAI/algen-agent-runtime">Algen Agent Runtime</a>, an open-source Python runtime for governed agents, and approvals are one of the places where it’s strictest. Here is what we think an approval needs, and how the runtime does it.</p><h3>Declared by the tool, not decided by the model</h3><p>Every tool declares its side effect: read-only, write, external or destructive. The approval rule follows from that declaration.</p><pre>@crm.tool(write(approval=True))<br>async def update_owner(account_id: str, owner: str, context: ToolContext) -&gt; Receipt:<br>    &quot;&quot;&quot;Change an account&#39;s owner.&quot;&quot;&quot;<br>    client = await crm.client()<br>    return await client.set_owner(account_id, owner, key=context.idempotency_key)</pre><p>approval=True means every call to this tool waits for a human, whatever the agent&#39;s own policy says. An agent&#39;s policy can add approvals but never waive this one. By default an agent asks before every write, external and destructive call anyway. The tool executor enforces it too: it refuses to run such a call without a reference to the approval that covers it, so code that calls the tool directly can&#39;t skip it either.</p><p>Policies can also come from a central, signed bundle that no agent can opt out of, and a <em>deny</em> from any policy wins over another policy’s request for approval. A human can’t approve their way past a rule that says no.</p><h3>Checked before anyone is asked</h3><p>Before a request reaches a person, the call is validated against the tool’s schema, and an optional precheck judges it against the current state. A change with a missing field, or one that conflicts with another call in the same reply, goes back to the model as an error to fix. Approval queues fill up fast enough without requests that would fail anyway.</p><h3>The exact call, redacted</h3><p>The approval request holds the tool, its side effect and the parameters it will run with. Secrets such as tokens, passwords and API keys are redacted <em>before</em> the request is stored. The approver can approve, reject, or modify the parameters, and a modified call is validated against the same schema before it runs. A rejection can be told to the model, with a reason, so the turn carries on and the model explains what it didn’t do instead of failing.</p><h3>A pause that survives</h3><p>A run that needs a decision is checkpointed with status awaiting_approval <em>before</em> anyone is asked. It survives a restart, and a version check stops two processes from both resuming it. Every pending request in a tenant can be listed from one approval inbox, so approvers don&#39;t need to poll runs. Requests expire: an approval that sat for a week was given for a world that has changed.</p><h3>Exactly once, or knowingly unknown</h3><p>When the approved call runs, it carries an idempotency key unique to its run and call, and it’s recorded in a tool-execution ledger. Writes are tried once. If the process dies mid-call, the call is marked <em>indeterminate</em>, not failed and not retried. An operator settles it through an audited reconciliation API: mark it completed or failed, ask the tool’s own system whether it took effect (by that idempotency key), or run its compensation. The approval and the execution are both in the audit trail: who decided, when, and on exactly which arguments.</p><h3>See it in a minute</h3><p>No API key and no network: a deterministic mock model stands in for a real one.</p><pre>python -m pip install --pre algen-agent-runtime<br>python -m examples.quickstart_approval.app</pre><p>The workflow proposes a change and pauses at an approval node. Type approve and it applies the change once, under an idempotency key. Type reject and nothing is written.</p><figure><img alt="Terminal recording: the approval quickstart pauses, the user types approve, and the run prints the approval decision and the applied result with its idempotency key" src="https://cdn-images-1.medium.com/max/1024/1*6Ag5B2tmJTH6GwVKww9aZA.png" /><figcaption>The approval quickstart: the run pauses, you type approve, and the change is applied once under an idempotency key.</figcaption></figure><p>In a multi-step workflow, the approval is a node of its own, with the review fields, the schema for changed parameters and an expiry declared next to it:</p><pre>- id: approve<br>  kind: approval<br>  depends_on: [propose]<br>  approval:<br>    prompt: Review this synthetic change before it is applied.<br>    review_from: [proposal.change]<br>    parameters_from: proposal.parameters<br>    allow_modification: true<br>    rejection_policy: skip_dependents<br>    expires_seconds: 1800</pre><h3>What this doesn’t solve</h3><p><strong>A person can still approve a bad change.</strong> The runtime makes sure they see the real call and that their answer is the one that counts. It can’t make them read it. Ask for approval where it matters, on writes, external messages and anything destructive, and keep reads fast. An approver who sees fifty requests a day stops reading them.</p><p><strong>Approvals are not unique to us.</strong> LangGraph, the OpenAI Agents SDK, Pydantic AI and others can all pause for a human now. What we care about is what’s coupled to the pause: the declared side effect, the precheck, the redaction, the idempotent ledger, tenant and user identity on every record, and an audit trail, all in one run lifecycle that survives a restart.</p><p><strong>It’s early.</strong> At the time of writing the runtime is 0.1.0a21, a pre-release hardened for a single node. Multi-worker execution is still open work.</p><h3>Five things worth copying</h3><p>Whatever you build on:</p><ul><li><strong>Derive approval from the tool, not the prompt.</strong> Declare each tool’s side effect in code and attach the rule to the declaration.</li><li><strong>Validate before you ask.</strong> Don’t put a call in front of a person until it would actually run.</li><li><strong>Show the arguments, not the summary,</strong> with secrets redacted before they’re stored, and let the approver change them.</li><li><strong>Save the pause before you ask, and let it expire.</strong> A decision that only lived in memory was never made.</li><li><strong>Key every side effect.</strong> An interrupted call is unknown until someone checks, never safe to retry.</li></ul><p>The model is very good at deciding what to do. It shouldn’t also be the one deciding whether it’s allowed to.</p><p><em>Originally published on the </em><a href="https://algen.ai/blog/approvals-belong-in-the-runtime"><em>Algen.AI blog</em></a><em>. Algen Agent Runtime is open source on </em><a href="https://github.com/AlgenAI/algen-agent-runtime"><em>GitHub</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e616b4410d2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reasoning vs. Standard LLMs: When Is It Worth Paying the “Thinking Tax”?]]></title>
            <link>https://saurav-samantray.medium.com/reasoning-vs-standard-llms-when-is-it-worth-paying-the-thinking-tax-bb61d44c61ea?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb61d44c61ea</guid>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[tech-strategy]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Wed, 05 Aug 2026 09:04:50 GMT</pubDate>
            <atom:updated>2026-08-05T09:04:50.312Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ZQjPJwcwEPpM31xCCOtxw.png" /></figure><p>Imagine sitting down with two different developers to design a solution:</p><ul><li><strong>Developer A</strong> starts typing code the exact second you finish speaking.</li><li><strong>Developer B</strong> sits back, grabs a whiteboard, scribbles down edge cases and rollback strategies, and <em>then</em> writes the code.</li></ul><p>That is the fundamentally different user experience between a <strong>Standard LLM</strong> and a <strong>Reasoning (Thinking) LLM</strong>.</p><p>In this hands-on breakdown, we’ll run a local side-by-side benchmark using two popular models via Ollama to explore when that extra thinking time pays off — and when it’s just wasted compute.</p><h3>The Core Concept: Allocate Thinking Time to Task Risk</h3><p>The main question when picking a model isn’t <em>“Which model is smarter?”</em></p><p>It’s <em>“How much thinking time does this specific problem actually deserve?”</em></p><pre><br>+--------------------------------------------------------------------------------------------+<br>|                                   TASK RISK &amp; COMPLEXITY                                   |<br>+--------------------------------------------------------------------------------------------+<br>| Low Risk -----------------------------------------------------&gt; High Risk                  |<br>| [ Standard Mode ]                                      [ Reasoning Mode ]                  |<br>| • Quick definitions &amp; summaries                        • Multi-step math &amp; logic puzzles   |<br>| • Simple code snippets                                 • Distributed systems design        |<br>| • Copywriting &amp; text polishing                         • Multi-step debugging &amp; edge cases |<br>+--------------------------------------------------------------------------------------------+<br><br></pre><blockquote><strong><em>The Analogy:</em></strong><em> Asking a reasoning model to draft a basic status update email is like calling a private detective to find your car keys when they were already in your pocket.</em></blockquote><ul><li><strong>Standard Mode:</strong> Blazing fast, direct responses. Great when the cost of a slight oversight is low.</li><li><strong>Reasoning Mode:</strong> Deliberate, structured planning. Crucial when a task contains subtle failure modes, structural dependencies, or high blast-radius risks.</li></ul><h3>The Trade-off at a Glance</h3><pre><br>+-------------------+--------------------------------------+-------------------------------------------------------------+<br>|      Factor       |      Standard LLM (llama3.2:1b)      |                  Reasoning LLM (qwen3:4b)                   |<br>+-------------------+--------------------------------------+-------------------------------------------------------------+<br>|                   |                                      |                                                             |<br>| Response Latency  | Under 5 seconds                      | 2+ minutes                                                  |<br>|                   |                                      |                                                             |<br>| Output Volume     | ~300 tokens (Concise)                | &quot;~5500+ tokens (Extensive trace)&quot;                           |<br>|                   |                                      |                                                             |<br>| Compute Footprint | Extremely low                        | High GPU/CPU utilization                                    |<br>|                   |                                      |                                                             |<br>| Engineering Value | Rapid prototyping &amp; baseline scripts | &quot;Structural edge cases rollback strategies safety handling&quot; |<br>+-------------------+--------------------------------------+-------------------------------------------------------------+</pre><h3>Practical Benchmark: Writing a Python File Renamer</h3><p>To make this concrete, let’s look at how both models handle a standard DevOps task: writing a file-renaming script.</p><h3>1. The Setup</h3><p>Make sure Ollama is running on your machine and fetch both lightweight models:</p><pre>ollama pull llama3.2:1b<br>ollama pull qwen3:4b<br>ollama serve</pre><h3>2. The Execution &amp; Prompts</h3><p>We gave each model a prompt suited to its native design pattern:</p><ul><li><strong>Standard LLM Prompt:</strong></li></ul><pre>&quot;Create a Python script that renames files in a directory.&quot;</pre><ul><li><strong>Reasoning LLM Prompt:</strong></li></ul><pre>&quot;Create a Python script that renames files in a directory. Before coding: <br>1. Identify edge cases. <br>2. Consider rollback strategy. <br>3. Consider filename collisions. <br>4. Generate test cases. <br>5. Write the code.&quot;</pre><h3>3. Metric Comparison</h3><pre>┌─────────────────────────┬──────────────┬──────────────────┐<br>│ Model                   │ Latency      │ Token Count      │<br>├─────────────────────────┼──────────────┼──────────────────┤<br>│ Standard (llama3.2:1b)  │ 4.49s        │ 308 tokens       │<br>│ Reasoning (qwen3:4b)    │ 135.77s      │ 5,542 tokens     │<br>└─────────────────────────┴──────────────┴──────────────────┘</pre><h3>Verdict &amp; Code Quality Analysis</h3><h4>The Standard LLM (llama3.2:1b)</h4><ul><li><strong>Speed:</strong> Near-instant output (~4.5s).</li><li><strong>Quality:</strong> Generated a standard os.rename() loop. Fine for quick scripts on your local system, but lacks protection against overwriting files if name collisions occur.</li></ul><h3>The Reasoning Model (qwen3:4b)</h3><ul><li><strong>Speed:</strong> Substantially slower (~135s thinking tax).</li><li><strong>Quality:</strong> Spent thousands of tokens reasoning through edge cases <em>before</em> emitting code. It automatically structured collision checks, created dry-run modes, and built a logging state to allow rollbacks if an error occurred mid-execution.</li></ul><blockquote><em>“A standard model gives you working code. A reasoning model gives you engineered code.”</em></blockquote><h3>The Fine Print: Real-World Limitations</h3><ul><li><strong>Reasoning is not immunity to hallucinations:</strong> A thinking model can still talk itself into a logical circle and be confidently incorrect.</li><li><strong>Model size still sets the ceiling:</strong> A compact reasoning model like qwen3:4b punches above its weight, but it won&#39;t replace full-scale frontier models (like DeepSeek-R1 or OpenAI&#39;s o1) on complex system architecture.</li><li><strong>Trace parsing:</strong> Depending on your client implementation, you may need to parse out or ignore the inner &lt;think&gt;...&lt;/think&gt; block to isolate the final deliverable.</li></ul><h3>Practical Rule of Thumb</h3><p>Default to <strong>Standard Mode</strong> for 80% of daily tasks. Switch to <strong>Reasoning Mode</strong> only when the task is multi-step, high-stakes, or where fixing a subtle bug later will cost significantly more time than waiting two minutes for the initial response.</p><p>Spend your compute tokens where they actually buy you peace of mind.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb61d44c61ea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Token Optimization vs Context Loss across Data Formats— JSON vs YAML vs CSV vs TOON]]></title>
            <link>https://saurav-samantray.medium.com/token-optimization-vs-context-loss-across-data-formats-json-vs-yaml-vs-csv-vs-toon-b2b145e06510?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2b145e06510</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[data-formats]]></category>
            <category><![CDATA[token-optimization]]></category>
            <category><![CDATA[llm-tokens]]></category>
            <category><![CDATA[context-loss]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Wed, 19 Nov 2025 13:26:05 GMT</pubDate>
            <atom:updated>2025-11-19T13:26:05.133Z</atom:updated>
            <content:encoded><![CDATA[<p>LLMs have changed how we think about data. The more we send, the more we pay — and suddenly every brace, comma, and indentation matters. This has sparked a growing debate across engineering teams: <em>Is there a “silver bullet” format that keeps token costs down without sacrificing structure?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*cvx_wi0DtNWQjg8H0N9AJg.png" /></figure><h3>The Format Contenders</h3><ul><li><strong>JSON</strong> — the default choice: reliable, readable, and incredibly chatty.</li><li><strong>YAML</strong> — easier on the eyes and surprisingly efficient, but unforgiving with whitespace.</li><li><strong>CSV</strong> — brutally simple and unbeatable for rows and columns, at the cost of hierarchy.</li><li><strong>TOON</strong> — the new kid built for token efficiency, though still growing its tooling ecosystem.</li></ul><h3>Token Analysis: Three Data Categories</h3><p>I conducted a systematic analysis of format efficiency across four distinct data patterns that represent real-world scenarios:</p><h4>Category 1: Tabular Data (Product Inventory)</h4><p><strong>Data Structure</strong>: Simple rows and columns with 3 products</p><p><strong>Results</strong>:</p><pre>| Format | Size (Bytes) | Tokens | Token Efficiency |<br>|--------|-------------|--------|-------------------|<br>| JSON   | 441         | 134    | 0.304 tokens/byte |<br>| YAML   | 296         | 99     | 0.351 tokens/byte |<br>| CSV    | 141         | 48     | 0.342 tokens/byte |<br>| TOON   | 216         | 74     | 0.337 tokens/byte |</pre><p><strong>Conclusion:</strong> When data is truly row-based, CSV outperforms the others. Its simplicity becomes a difference maker — no braces, no indentation, no wasted tokens.</p><h4>Category 2: Simple Nested Data (User Profile with Projects)</h4><p><strong>Data Structure</strong>: User object with nested profile, preferences, plus 8 projects and 8 tasks</p><p><strong>Results</strong>:</p><pre>| Format | Size (Bytes) | Tokens | Token Efficiency |<br>|--------|-------------|--------|-------------------|<br>| JSON   | 2070        | 623    | 0.300 tokens/byte |<br>| YAML   | 1889        | 463    | 0.245 tokens/byte |<br>| CSV    | 1149        | 152    | 0.132 tokens/byte |<br>| TOON   | 1151        | 245    | 0.162 tokens/byte |</pre><p><strong>Conclusion:</strong> CSV stays the most compact, but only if you’re willing to flatten everything. YAML and TOON become more interesting here: they keep the hierarchy intact while using far fewer tokens than JSON.</p><h4>Category 3: Deeply Nested Data (Organization Hierarchy)</h4><p><strong>Data Structure</strong>: Multi-level organization with departments, teams, employees, and projects — 6 levels of nesting</p><p><strong>Results</strong>:</p><pre>| Format | Size (Bytes) | Tokens | Token Efficiency |<br>|--------|-------------|--------|------------------|<br>| JSON   | 8340        | 2512   | 0.301 tokens/byte |<br>| YAML   | 7220        | 1845   | 0.255 tokens/byte |<br>| CSV    | 6150        | 948    | 0.154 tokens/byte |<br>| TOON   | 6890        | 1423   | 0.206 tokens/byte |</pre><p><strong>Conclusion:</strong> As nesting grows, YAML’s efficiency really shows. It maintains a full hierarchy with noticeably fewer tokens than JSON. TOON lands between YAML and CSV, offering structure without the full weight of JSON.</p><h3>Choosing the Right Format</h3><ul><li><strong>CSV</strong> shines when your data is genuinely tabular and you need the smallest possible payload.</li><li><strong>YAML</strong> is the sweet spot for structured data that still needs to be readable and token-efficient.</li><li><strong>JSON</strong> remains the universal fallback when speed, compatibility, or ecosystem tooling matter more than token count.</li><li><strong>TOON</strong> is a strong emerging option if you want CSV-like efficiency without giving up nested structure.</li></ul><h3>The Reality</h3><p>The initial intent of the blog was to do a token comparison between the four formats and conclude on a framework that can be used to determine the best one to use based on the problem at hand.</p><blockquote>However, I thought, optimizing the token used based on the format is just considering quantity, what about quality!</blockquote><h3>Context Loss across formats</h3><p>I ran another test to also compute the context loss across all four formats for the same three data categories(tabular, nested, and deeply nested).</p><p>The average results</p><pre>┌─────────┬──────────────┬────────────┬────────────────┬──────────────┐<br>│ Format  │ Avg Size(KB) │ Avg Tokens │ Avg Ctx Loss % │ Avg Tok/Out  │<br>├─────────┼──────────────┼────────────┼────────────────┼──────────────┤<br>│ YAML    │     1.59     │    466     │     57.90%     │     6.33     │<br>│ JSON    │     2.86     │    674     │     65.33%     │     9.57     │<br>│ CSV     │     1.52     │    505     │     66.33%     │     8.38     │<br>│ TOON    │     1.17     │    390     │     65.40%     │     6.87     │<br>└─────────┴──────────────┴────────────┴────────────────┴──────────────┘</pre><h3>Conclusion</h3><p>While CSV does help a great deal in squeezing the data using the least tokens, but for data which is not truely tabular, it comes at a cost of losing quite a bit of context.</p><p>YAML and TOON seem to be the middle ground, giving a good trade-off between context loss and token optimization. The bad numbers around TOON’s context loss in the above table could be because I used locally running llama3, which might not be aware of the few format.</p><p>JSON format wastes a lot of tokens, and when the size of the data grows, that also leads to a lot of context loss because of the context window limit of the LLM.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2b145e06510" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supercharge Your Local AI: Connecting Ollama Agents to Real-Time Data and Tools using MCP]]></title>
            <link>https://saurav-samantray.medium.com/supercharge-your-local-ai-connecting-ollama-agents-to-real-time-data-and-tools-using-mcp-0b4773fc3496?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/0b4773fc3496</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <category><![CDATA[ai-tools]]></category>
            <category><![CDATA[ollama]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 01 Jul 2025 03:32:51 GMT</pubDate>
            <atom:updated>2025-07-01T03:32:51.789Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OloHcNuK7M3xGH-M06921A.png" /></figure><p>The world of AI is buzzing with the power of local Large Language Models (LLMs). With tools like Ollama, anyone can run powerful models like Llama 3, Mistral, or DeepSeek right on their personal computer. This is a monumental leap for privacy, cost-effectiveness, and offline capability.</p><p>However, these local models have an inherent limitation: their knowledge is frozen in time. They can’t tell you today’s weather, the current stock price of Apple, or even perform a simple, real-time calculation. They are brilliant but isolated brains.</p><p>So, how do we bridge this gap? How do we give our local AI agents access to the live, dynamic world? The answer lies in a powerful architectural pattern: <strong>decoupling the agent’s reasoning from its tools.</strong></p><p>In this post, we’ll walk through a project that does exactly that. We’ll explore a system with two core components:</p><ol><li><strong>A Local AI Agent:</strong> A smart, command-line “brain” powered by an Ollama LLM and built with Python and LangChain.</li><li><strong>A Remote MCP Server:</strong> A “Multi-tool Control Protocol” server that provides a suite of tools (weather, currency exchange, etc.) via a simple API.</li></ol><p>This architecture allows our local agent to “phone a friend” whenever it needs information or a capability it doesn’t possess, transforming it from a simple chatbot into a true problem-solver.</p><h3>The Architecture: A Brain and Its Toolbox</h3><p>The key to this system is separating the “thinking” from the “doing”.</p><h3>The AI Agent (The Brain)</h3><p>Our agent is a Python application that runs locally. Its job is to understand the user’s request and figure out a plan. It uses a reasoning framework called <strong>ReAct (Reasoning and Acting)</strong>, which allows the LLM to “think” step-by-step, decide if a tool is needed, use it, observe the result, and continue this loop until it has a final answer.</p><h3>The MCP Server (The Toolbox)</h3><p>The MCP server is a lightweight FastAPI application that can run anywhere — on the same machine, on your local network, or in the cloud. It exposes a set of tools through a standard REST API.</p><p>The most crucial feature of this server is its <strong>discovery endpoint</strong>. When the agent starts, it first asks the server, “What tools do you have?” The server responds with a structured list of all available tools, including their names, what they do, and what parameters they need.</p><p>Let’s look at the metadata for the get_stock_price tool on our server:</p><p><strong>File: </strong><strong>mcp_server/server.py</strong></p><pre>ToolMetadata(<br>    name=&quot;get_stock_price&quot;,<br>    description=&quot;Fetches the latest price of a stock using its ticker symbol. Returns a number representing the price.&quot;,<br>    parameters=[<br>        ToolParameter(name=&quot;ticker_symbol&quot;, type=&quot;string&quot;, description=&quot;The stock ticker symbol, e.g., &#39;AAPL&#39; for Apple or &#39;GOOGL&#39; for Google.&quot;, required=True)<br>    ]<br>)</pre><p>This simple, descriptive metadata is pure gold for the AI agent. The description tells the LLM <em>when</em> to use the tool and what to expect, and the parameters section tells it <em>how</em> to use it.</p><h3>A Step-by-Step Walkthrough: From Question to Answer</h3><p>Let’s trace a user’s request through the system to see how it all connects.</p><p><strong>User Query:</strong> What is the price of Apple stock in Euros?</p><h3>Step 1: The Agent Learns Its Capabilities</h3><p>When our agent starts, it immediately contacts the MCP server’s /tools endpoint.</p><p><strong>File: </strong><strong>ai_agent/agent.py</strong></p><pre>class MCPClient:<br>    # ...<br>    def fetch_tools(self) -&gt; list[Tool]:<br>        &quot;&quot;&quot;Fetches tool definitions from the server and creates LangChain Tool objects.&quot;&quot;&quot;<br>        try:<br>            response = requests.get(f&quot;{self.server_url}/tools&quot;, timeout=5)<br>            response.raise_for_status()<br>            tool_definitions = response.json()<br>            logging.info(f&quot;Successfully fetched {len(tool_definitions)} tool definitions from the server.&quot;)<br><br>            tools = []<br>            for tool_def in tool_definitions:<br>                # ... logic to create a callable LangChain Tool object ...<br>                langchain_tool = Tool(<br>                    name=tool_def[&#39;name&#39;],<br>                    func=tool_func_wrapper,<br>                    description=tool_def[&#39;description&#39;]<br>                )<br>                tools.append(langchain_tool)<br>            return tools<br>        # ...</pre><p>The agent parses the JSON response and dynamically creates a list of Tool objects that LangChain can work with. It now knows it has access to a weather tool, a stock tool, a currency converter, and a calculator.</p><h3>Step 2: The Agent Thinks (The ReAct Loop)</h3><p>The user’s query is fed into the agent’s brain — the Ollama LLM — guided by a carefully crafted prompt. This prompt instructs the model to think in a specific format: <strong>Thought, Action, Action Input, Observation.</strong></p><p>Here’s a simplified look at the agent’s internal monologue, which we can see in the logs:</p><pre>&gt; Entering new AgentExecutor chain...<br><br>Thought: To answer the question, I first need to find the price of Apple stock in USD. Then I need to find the exchange rate between USD and EUR. Finally, I will use the calculator to multiply the stock price by the exchange rate. I will start by getting the stock price.<br>Action: get_stock_price<br>Action Input: ticker_symbol=&#39;AAPL&#39;</pre><p>The agent correctly deduces that it needs the stock price first and identifies the get_stock_price tool as the right one for the job.</p><h3>Step 3: The Agent Acts and Observes</h3><p>The agent’s MCPClient now makes an API call to the server: GET <a href="http://127.0.0.1:8000/tools/get_stock_price?ticker_symbol=AAPL.">http://127.0.0.1:8000/tools/get_stock_price?ticker_symbol=AAPL.</a></p><p>The server executes the corresponding function and returns the result.</p><p><strong>File: </strong><strong>mcp_server/server.py</strong></p><pre>@app.get(&quot;/tools/get_stock_price&quot;, summary=&quot;Get Stock Price&quot;)<br>async def get_stock_price(ticker_symbol: str = Query(..., description=&quot;The stock ticker symbol.&quot;)):<br>    &quot;&quot;&quot;Provides a mock stock price.&quot;&quot;&quot;<br>    logging.info(f&quot;Executing tool &#39;get_stock_price&#39; for ticker: {ticker_symbol}&quot;)<br>    # Mock data<br>    prices = {&quot;AAPL&quot;: 175.21, &quot;GOOGL&quot;: 140.55, &quot;MSFT&quot;: 330.89}<br>    price = prices.get(ticker_symbol.upper())<br>    if price:<br>        return {&quot;result&quot;: price}<br>    # ...</pre><p>The agent receives the result and logs it as an <strong>Observation</strong>.</p><pre>Observation: 175.21</pre><h3>Step 4: The Loop Continues</h3><p>The observation 175.21 is added to the agent&#39;s &quot;scratchpad&quot; and fed back into the LLM for the next step.</p><pre>Thought: I have the price of Apple stock in USD, which is 175.21. Now I need to get the exchange rate from USD to EUR.<br>Action: get_currency_exchange_rate<br>Action Input: from_currency=&#39;USD&#39;, to_currency=&#39;EUR&#39;</pre><p>The agent again correctly identifies the next tool. It makes the API call, and the server returns the mock rate.</p><pre>Observation: 0.92</pre><p>The agent is almost there. One final loop.</p><pre>Thought: I have the stock price in USD (175.21) and the USD to EUR exchange rate (0.92). I need to multiply these two numbers to get the final answer.<br>Action: calculator<br>Action Input: 175.21 * 0.92</pre><h3>Step 5: The Final Answer</h3><p>The calculator tool is called, and the final piece of information is observed.</p><pre>Observation: 161.1932</pre><p>Now, the agent has everything it needs. It concludes its thought process.</p><pre>Thought: I now know the final answer.<br>Final Answer: The price of Apple stock is approximately 161.19 EUR.</pre><p>The agent prints this final, synthesized answer to the user, having successfully used multiple remote tools to solve a multi-step problem.</p><h3>Why This Architecture Is So Powerful</h3><p>This decoupled agent-server model is more than just a neat trick; it’s a robust and scalable pattern for building sophisticated AI systems.</p><ul><li><strong>Extensibility:</strong> Want to add a new tool to search Wikipedia? Simply add a new function and its metadata to server.py. The agent will automatically discover and learn to use it on its next run—<strong>no changes needed in the agent&#39;s code.</strong></li><li><strong>Maintainability:</strong> The logic for interacting with complex external APIs is neatly contained within the server. The agent’s code remains clean and focused on reasoning.</li><li><strong>Flexibility:</strong> You can swap out the Ollama model in the agent with any other model (or even a proprietary one like GPT-4) with minimal changes. The tools remain the same.</li><li><strong>Scalability:</strong> The tool server can be a high-powered, independently scaled service used by dozens of different agents.</li></ul><p>By connecting local LLMs to a dynamic, remote toolbox, we unlock their true potential. They are no longer just repositories of static knowledge but can become active participants in the digital world, capable of accessing information and taking action on our behalf.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0b4773fc3496" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting your hands dirty with Jules —  Google’s new AI-powered coding assistant]]></title>
            <link>https://saurav-samantray.medium.com/getting-your-hands-dirty-with-jules-googles-new-ai-powered-coding-assistant-d6547b729c87?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d6547b729c87</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[google-jules]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[coding-agents]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 03 Jun 2025 03:00:03 GMT</pubDate>
            <atom:updated>2025-06-03T03:00:03.653Z</atom:updated>
            <content:encoded><![CDATA[<h3>Getting your hands dirty with Jules — Google’s new AI-powered coding assistant</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wNoFS8GDm4ck7mLJuweTag.png" /></figure><p>“We’re at a turning point: agentic development is shifting from prototype to product and quickly becoming central to how software gets built,” Kathy Koravec, director of product management at Google Labs, wrote in a company <a href="https://blog.google/technology/google-labs/jules/">blog post</a> published a couple of weeks back.</p><p>The competition to launch AI agents is intensifying. During its annual I/O developer conference <strong>on May 20, 2025, Google revealed that Jules, its new AI-powered coding assistant, is now globally accessible in public beta.</strong></p><p>The model operates asynchronously, meaning, in programming terms, it can initiate and work on multiple tasks simultaneously without waiting for each one to complete.</p><p>Once done, it delivers a detailed summary of the changes it made to the user’s code, offering insight into its decision-making process.</p><h3>Getting our hands dirty</h3><p>I signed up for Jules and linked my GitHub account to put it to the test.</p><p>To start, I chose a time-consuming but common task: a <strong>version upgrade</strong>. I picked my public GitHub repository, <strong>custom-auth-service</strong>, which hosts a Dockerized Keycloak version 21 instance customized with a welcome page, login screen, and admin console.</p><p>It’s a relatively straightforward setup — but with just enough complexity to see whether Jules can handle it.</p><h3>Use Case — Upgrade a repository to support Keycloak 22</h3><p>I prompted Jules to upgrade the repository to support Keycloak 22. It responded with a detailed upgrade plan and paused for my confirmation before proceeding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*VcwbgeU8Fy-aGAHUhLwqvw.png" /></figure><p>Once I approved, Jules conducted a thorough analysis of my repository and the official Keycloak 22 release notes. It identified potential breaking changes that needed to be addressed before proceeding with the upgrade.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*Jd1UQ-Jj0RcghG2TOt59eg.png" /></figure><p>Again prompted for confirmation, created a new feature branch, ran tests, and prepared a commit to the new feature branch for my approval, and all under 15mins!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*0Zc906pHKHsYGZK4CJ8v_Q.png" /></figure><h3>Key Takeaways</h3><ul><li><strong>Faster Iterations</strong>: The asynchronous nature allows it to begin multiple tasks at once, reducing wait time compared to traditional sequential agents.</li><li><strong>Great for Simple Tasks</strong>: Performs basic coding operations (e.g., refactoring, adding comments, small bug fixes) reliably and with minimal guidance.</li><li><strong>Clear Explanations</strong>: It provides a transparent summary of code changes, making it easier to review and trust the output.</li><li><strong>Improves Developer Flow</strong>: Enables developers to offload repetitive tasks and focus on higher-level logic or design decisions.</li><li><strong>Still Needs Guardrails</strong>: While useful, it occasionally needs human oversight to catch edge cases or ensure changes align with context-specific requirements.</li><li><strong>Not Yet Autonomous for Complex Logic</strong>: The agent struggles with multi-file logic, deep architectural changes, or tasks that require nuanced understanding.</li><li><strong>Great Companion, Not a Replacement</strong>: Best used as a collaborative tool rather than a hands-off solution.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6547b729c87" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Intelligent Systems: Why Event-Driven Agents are the Future (and How We Built One!)]]></title>
            <link>https://saurav-samantray.medium.com/building-intelligent-systems-why-event-driven-agents-are-the-future-and-how-we-built-one-8c69af01f7a1?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c69af01f7a1</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[multi-agent-systems]]></category>
            <category><![CDATA[agent-communication]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 20 May 2025 04:04:59 GMT</pubDate>
            <atom:updated>2025-05-20T04:04:59.110Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GKpEgzOUcE4DKTPGUj3fPg.png" /></figure><p>In the rapidly evolving world of AI, building sophisticated applications often means orchestrating multiple specialized “<strong>agents</strong>” that can collaborate to solve complex problems. But how do these agents talk to each other effectively? How do they work together without getting tangled up or stepping on each other’s toes? The answer, increasingly, lies in <strong>event-based communication</strong>.</p><p>This post dives into why event-driven architectures are crucial for multi-agent systems and walks you through a practical example: an Ollama-powered multi-LLM agent application that uses Kafka for its communication backbone.</p><h3>Why Event-Based Communication is Essential for Agents</h3><p>Imagine trying to coordinate a team of experts to complete a project. You could have them all in one giant conference call, constantly interrupting each other (synchronous communication), or you could let them work on their specific tasks and report back when they have results, picking up new tasks when they’re ready (asynchronous, event-driven communication). The latter is almost always more efficient and scalable.</p><p>For AI agents, event-based communication offers several key advantages:</p><ul><li><strong>Decoupling:</strong> Agents publish events (e.g., “task completed”) to a central message bus without needing to know others’ status or location. Subscribers react as needed, enabling a modular system where agents can fail or update with minimal disruption.</li><li><strong>Scalability:</strong> New agents can be added without reconfiguring existing ones. With a message bus like Kafka, the system scales horizontally to handle high event volumes.</li><li><strong>Resilience &amp; Fault Tolerance:</strong> Failed tasks remain in the queue for later reprocessing, preventing data loss and ensuring system robustness.</li><li><strong>Asynchronous Processing:</strong> Agents work in parallel, letting workers handle tasks independently, crucial for performance in time-consuming operations like LLM calls or API requests.</li><li><strong>Flexibility &amp; Extensibility:</strong> New events or agents can be added without disrupting existing workflows, making the system easy to evolve.</li></ul><p>In essence, event-driven architectures allow us to build complex, distributed AI systems that are more robust, scalable, and easier to manage and evolve.</p><h3>Application Architecture: Ollama Agents &amp; Kafka</h3><p>Let’s look at the multi-agent system I’ve built. It’s designed to take a user’s prompt, break it into manageable tasks, have specialized agents work on these tasks, and synthesize a final response. The complete code is <a href="https://github.com/saurav-samantray/multi-agent-event-communication.git">here</a>.</p><h4><strong>Core Components:</strong></h4><ol><li><strong>User Interface (CLI — </strong><strong>main_cli.py)</strong>: The user&#39;s entry point. It sends prompts to Kafka and listens for the final answer.</li></ol><p>2. <strong>Kafka Message Broker</strong>: The heart of our event-driven system. It hosts several topics:</p><ul><li>user_prompts_topic: For new requests from the user.</li><li>agent_tasks_researcher / agent_tasks_writer: For tasks assigned to specific worker agents.</li><li>agent_results_topic: For results coming back from worker agents.</li><li>final_output_topic: For the final, synthesized response to be picked up by the CLI.</li></ul><p>3. <strong>Master Agent (</strong><strong>agents/master_agent.py)</strong>: The orchestrator.</p><ul><li>Listens for user prompts.</li><li>Uses an Ollama LLM to create a task plan.</li><li>Dispatches tasks to worker agents via Kafka.</li><li>Collects results from workers.</li><li>Uses Ollama again to synthesize the final response.</li></ul><p>4. <strong>Worker Agents (</strong><strong>agents/research_agent.py, </strong><strong>agents/writing_agent.py)</strong>:</p><ul><li>ResearchAgent: Listens for research tasks, uses Ollama to find information, and publishes results.</li><li>WritingAgent: Listens for writing tasks, uses Ollama (and any provided context) to compose text, and publishes results.</li></ul><p>5. <strong>Ollama LLM Service</strong>: Provides local, open-source LLM capabilities to all agents.</p><p>6. <strong>Docker Environment</strong>: Hosts Kafka and Ollama, ensuring a consistent environment.</p><h3>Local Setup and Testing</h3><p>Follow the <a href="https://github.com/saurav-samantray/multi-agent-event-communication/blob/main/README.md">README.md</a> file in the <a href="https://github.com/saurav-samantray/multi-agent-event-communication/blob/main/README.md">GitHub repository</a> to set up the multi-agent application locally!</p><h3>Conclusion</h3><p>Event-driven architectures, powered by message brokers like Kafka, are a powerful paradigm for building scalable, resilient, and flexible multi-agent AI systems. By decoupling agents and enabling asynchronous communication, we can create sophisticated applications that leverage the strengths of specialized LLM-powered components. Our example, using Ollama for local LLM capabilities, demonstrates these principles in action and provides a launchpad for even more advanced intelligent systems.</p><p>The journey of building intelligent agents is just beginning, and the way they communicate will be fundamental to their success!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c69af01f7a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlock the Power of Local AI: Build Your First Agent with Ollama, RAG, and LangChain]]></title>
            <link>https://saurav-samantray.medium.com/unlock-the-power-of-local-ai-build-your-first-agent-with-ollama-rag-and-langchain-f8d3bb171ea1?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8d3bb171ea1</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Wed, 07 May 2025 04:29:09 GMT</pubDate>
            <atom:updated>2025-05-09T06:23:09.736Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Unlock the Power of Local AI: Build Your First Agent with Ollama, RAG, LangChain and Tools</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gCSQhXfUF2810RGE5ViXtA.png" /></figure><p>Curious how to create an AI that runs on your computer, understands your documents, and performs custom actions? With the right tools, it’s easier than you think!<br> This post explores the <strong>Simple Ollama RAG + Tools Example</strong> (GitHub: <a href="https://github.com/saurav-samantray/ollama-llm-rag-tools-example.git">Link</a>), breaking down what it does, key concepts, and a step-by-step code walkthrough.</p><h3>What You’ll Learn: Core AI Concepts</h3><p>By exploring this application, you’ll get hands-on insight into:</p><ul><li><strong>Local LLMs with Ollama</strong>: Run models like Llama 3 locally for private, cloud-free AI.</li><li><strong>Retrieval-Augmented Generation (RAG)</strong>: Make LLMs smarter by pulling relevant data from your documents.</li><li><strong>LangChain Agents (ReAct)</strong>: Build agents that reason and pick the right tools or sources to answer questions.</li><li><strong>Custom Tools</strong>: Add new skills by connecting Python functions to your agents.</li><li><strong>LangChain Framework</strong>: Use LangChain to seamlessly connect models, data, tools, and logic.</li></ul><h3><strong>The Use Case: A Smart, Local Assistant</strong></h3><ul><li><strong>Knows Your Projects</strong>: Answers questions using your project documents (e.g., goals, timelines).</li><li><strong>Has Utilities</strong>: Provides date/time info or calculates days to a future date.</li><li><strong>Works Offline</strong>: Runs fully local with Ollama, keeping data private.</li></ul><p>A compact example of a private, customizable assistant for personal or team use.</p><h3>Diving into the Code: A Step-by-Step Guide</h3><p>Let’s clone the code <a href="https://github.com/saurav-samantray/ollama-llm-rag-tools-example.git">here</a> and explore its anatomy.</p><p><strong>(You can find setup instructions for Ollama and Python dependencies in the </strong><a href="https://github.com/saurav-samantray/ollama-llm-rag-tools-example/blob/main/Readme.md"><strong>Readme.md</strong></a></p><h4>1. Configuration &amp; LLM Setup</h4><pre># --- Configuration ---<br>OLLAMA_MODEL = &quot;llama3:8b&quot;<br>OLLAMA_BASE_URL = &quot;http://localhost:11434&quot; # Default Ollama URL<br>DOCS_FOLDER = &quot;docs_to_load&quot; # Folder to load documents from<br><br># --- 1. Set up the LLM ---<br>llm = ChatOllama(model=OLLAMA_MODEL, base_url=OLLAMA_BASE_URL, temperature=0)</pre><p><strong>Configuration:</strong> The script starts by defining key parameters:</p><ul><li>OLLAMA_MODEL: Specifies which Ollama model to use (e.g., llama3:8b).</li><li>OLLAMA_BASE_URL: The address where your Ollama service is running.</li><li>DOCS_FOLDER: The directory where your custom documents (for RAG) are stored.</li></ul><p><strong>LLM Initialization:</strong> ChatOllama from langchain_ollama is used to connect to your local Ollama instance.</p><ul><li><strong>Note:</strong> temperature=0 is key for agents — it ensures predictable, structured outputs, while higher values add randomness, better suited for creative tasks.</li></ul><h4>2. Setting up RAG (Retrieval-Augmented Generation)</h4><p>This is where we give the LLM access to custom knowledge.</p><pre># a) Load documents from the specified folder<br>loader = DirectoryLoader(DOCS_FOLDER, glob=&quot;**/*.txt&quot;, show_progress=True)<br>loaded_docs = loader.load()<br><br># b) Create embeddings<br>embeddings = OllamaEmbeddings(model=OLLAMA_MODEL, base_url=OLLAMA_BASE_URL)<br><br># c) Create a vector store and retriever<br>vector_store = FAISS.from_documents(loaded_docs, embeddings)<br>retriever = vector_store.as_retriever(search_kwargs={&quot;k&quot;: 2})</pre><p><strong>a) Load Documents:</strong> DirectoryLoader (from langchain_community.document_loaders) is used to load all .txt files from the DOCS_FOLDER. LangChain offers many other loaders for different file types (PDFs, CSVs, webpages, etc.).</p><p><strong>b) Create Embeddings:</strong> To make documents searchable by meaning (not just keywords), we need to convert them into numerical representations called “embeddings.” OllamaEmbeddings uses your local Ollama model to generate these.</p><p><strong>c) Create Vector Store &amp; Retriever:</strong></p><ul><li>FAISS (from langchain_community.vectorstores) is an efficient library for storing and searching these embeddings. The loaded_docs and their embeddings are used to create an in-memory vector store.</li><li>The vector_store.as_retriever() method creates a retriever object. This retriever is what we&#39;ll use to find the most relevant document chunks based on a user&#39;s query. search_kwargs={&quot;k&quot;: 2} means it will retrieve the top 2 most relevant chunks.</li></ul><h4>3. Creating Custom Tools</h4><p>Tools are functions that the AI agent can decide to use.</p><pre>from langchain.tools import tool<br><br>@tool<br>def get_current_date(arg: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    Returns the current date in YYYY-MM-DD format.<br>    Use this tool specifically for getting the date. It does not provide the time.<br>    Ignores any input argument.<br>    &quot;&quot;&quot;<br>    return date.today().isoformat()<br><br>@tool<br>def get_current_time(arg: str) -&gt; str:<br>    &quot;&quot;&quot;Returns the current time in HH:MM:SS format. Ignores any input argument.&quot;&quot;&quot;<br>    return datetime.now().strftime(&quot;%H:%M:%S&quot;)<br><br>@tool<br>def calculate_days_until(future_date_str: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    Calculates the number of days from today until a specified future date.<br>    Expects the future date as a string in &#39;YYYY-MM-DD&#39; format.<br>    Returns a string describing the number of days remaining or an error message.<br>    &quot;&quot;&quot;<br>    # ... (implementation with input cleaning and date logic) ...<br>    print(f&quot;DEBUG: calculate_days_until received raw input: &#39;{future_date_str}&#39; (length: {len(future_date_str)})&quot;)<br>    try:<br>        cleaned_date_str = future_date_str.strip().strip(&quot;&#39;\&quot;&quot;)<br>        # ... rest of the logic ...</pre><ul><li>The @tool decorator from langchain.tools is a convenient way to make any Python function available to a LangChain agent.</li><li><strong>Docstrings are Key:</strong> The docstring of each tool function (e.g., for get_current_date) is very important! The LLM uses these descriptions to understand what each tool does and when to use it. Clear, concise docstrings are essential for reliable tool usage.</li><li>The calculate_days_until tool includes some input string cleaning (.strip().strip(&quot;&#39;\&quot;&quot;)) which is a good practice, as LLMs might sometimes include extra quotes or whitespace in their generated Action Input.</li></ul><h4>4. Building the LangChain Agent (ReAct)</h4><p>This is the “brain” of our application.</p><pre>tools = [get_current_date, get_current_time, calculate_days_until]<br><br>custom_prompt_template = &quot;&quot;&quot;<br>You have been provided with the following context information retrieved from relevant documents:<br>&lt;RAG_CONTEXT_START&gt;<br>{rag_context}<br>&lt;RAG_CONTEXT_END&gt;<br><br>Answer the following question. First, carefully review the RAG_CONTEXT...<br>... (rest of the prompt template) ...<br>Thought:{agent_scratchpad}<br>&quot;&quot;&quot;<br><br>prompt = PromptTemplate(<br>    input_variables=[&quot;agent_scratchpad&quot;, &quot;input&quot;, &quot;tool_names&quot;, &quot;tools&quot;, &quot;rag_context&quot;],<br>    template=custom_prompt_template,<br>)<br><br>agent = create_react_agent(llm, tools, prompt)<br>agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=False)</pre><ol><li><strong>tools : </strong>A list containing all the tool functions we defined.</li><li><strong>custom_prompt_template:</strong> This is the heart of the ReAct (Reasoning and Acting) agent. It&#39;s a carefully crafted set of instructions that tells the LLM:<br> — About the rag_context that will be provided.<br> — What tools it has access to (LangChain automatically fills in {tools} and {tool_names}).<br> — The <strong>exact format</strong> it MUST use for its internal reasoning steps:<br>— — — Question: The user&#39;s input. <br> — — — Thought: The LLM&#39;s reasoning about what to do next. <br>— — — Action: The tool to use, or &quot;Final Answer&quot;. <br>— — — Action Input: The input for the chosen tool. <br>— — — Observation: The result from executing the tool.<br> — — — This cycle can repeat multiple times.<br> — It includes specific instructions on how to behave after receiving an Observation to avoid loops and ensure it moves towards a Final Answer.<br> — The {agent_scratchpad} variable is where LangChain stores the history of Thought/Action/Observation steps for the current query.</li><li><strong>PromptTemplate:</strong> Wraps the string template for use with LangChain.</li><li><strong>create_react_agent:</strong> This LangChain function takes the LLM, the list of tools, and the prompt to construct the core agent logic.</li><li><strong>AgentExecutor:</strong> This is what runs the agent. <br>— verbose=True is incredibly helpful for debugging, as it prints out the agent&#39;s internal &quot;thoughts&quot; and actions. <br>— handle_parsing_errors=False (or a custom error handler) is recommended during development with local models. If the LLM doesn&#39;t perfectly adhere to the prompt&#39;s format, this will raise an error showing the malformed output, making it easier to debug the prompt or the LLM&#39;s behavior. If set to True, it might try to recover, but can sometimes lead to unexpected behavior or loops.</li></ol><h4>5. Running the Application (The Main Loop)</h4><p>This section handles user interaction and orchestrates the RAG + Agent flow.</p><pre>while True:<br>    try:<br>        user_input = input(&quot;\nYour question: &quot;)<br>        if user_input.lower() == &#39;quit&#39;:<br>            break<br><br>        # 1. Perform RAG retrieval first<br>        retrieved_docs = retriever.invoke(user_input)<br>        rag_context_str = &quot;\n\n&quot;.join([doc.page_content for doc in retrieved_docs])<br>        if not rag_context_str:<br>            rag_context_str = &quot;No specific context found in documents for this query.&quot;<br><br>        # 2. Invoke the agent with the user input and the retrieved RAG context<br>        agent_input = {&quot;input&quot;: user_input, &quot;rag_context&quot;: rag_context_str}<br>        response = agent_executor.invoke(agent_input)<br><br>        print(&quot;\nAgent Response:&quot;)<br>        print(response[&#39;output&#39;])<br>    # ... (error handling) ...</pre><p>The loop continuously prompts the user for questions.</p><ol><li><strong>Perform RAG Retrieval First:</strong> For <em>every</em> user question, the retriever.invoke(user_input) is called. This fetches document chunks relevant to the current question. This context (rag_context_str) is then passed directly to the agent. This design ensures the agent <em>always</em> has potentially relevant document information at its disposal before it starts its reasoning process.</li><li><strong>Invoke the Agent:</strong> The agent_executor.invoke() method is called with a dictionary containing the input (user&#39;s question) and the rag_context we just fetched.</li><li>The agent then goes through its ReAct cycle (Thought, Action, Observation) and eventually returns a response containing the output.</li></ol><h3>Putting It All Together: The Flow</h3><ol><li><strong>User Asks:</strong> You type a question like, “When is the next meeting for Project Alpha, and how many days are left?”</li><li><strong>RAG Kicks In:</strong> The retriever searches the docs_to_load for text related to &quot;Project Alpha meeting.&quot; Let&#39;s say it finds &quot;Project Alpha&#39;s next team sync is scheduled for 2026-05-13.&quot; This becomes the rag_context.</li><li><strong>Agent Receives Input:</strong> The LLM (via the agent prompt) gets your question AND the rag_context.</li><li><strong>Agent Thinks (Thought #1):</strong> “The RAG context says the meeting is 2026–05–13. The user also wants to know how many days are left. I need to use the calculate_days_until tool for that.&quot;</li><li><strong>Agent Acts (Action #1): <br>— </strong>Action: calculate_days_until <br>— Action Input: 2026-05-13</li><li><strong>Tool Executes:</strong> The calculate_days_until Python function runs with &quot;2026-05-13&quot;.</li><li><strong>Agent Observes (Observation #1):</strong> The tool returns (for example) “There are 372 days remaining until 2026–05–13.”</li><li><strong>Agent Thinks (Thought #2):</strong> “I have the meeting date from RAG (2026–05–13) and the number of days remaining from the tool (372 days). I now have all the information needed to answer the user’s question.”</li><li><strong>Agent Gives Final Answer: <br>— </strong>Final Answer: The next meeting for Project Alpha is scheduled for 2026-05-13, and there are 372 days remaining until the meeting.</li></ol><h3>Conclusion: Your Journey into Local AI Agents</h3><p>The <strong>Simple Ollama RAG + Tools Example</strong> is a gateway to building smart, private AI assistants by combining local LLMs, RAG, tools, and LangChain.</p><p>The beauty of this example lies in its clarity and modularity. Each component can be swapped out or expanded upon, paving the way for more complex and powerful applications.</p><h3><strong>Next Concepts to Explore</strong></h3><ul><li><strong>Advanced Document Loading</strong>: Load data from PDFs, CSVs, websites, etc.</li><li><strong>Persistent Vector Stores</strong>: Use ChromaDB or Weaviate to avoid recalculating embeddings.</li><li><strong>API-Based Tools</strong>: Create tools that interact with external APIs.</li><li><strong>Different Agent Types</strong>: Explore LangChain’s various agent types.</li><li><strong>Error Handling, Streaming, UI</strong>: Improve your app’s reliability and user experience.</li></ul><p>Happy building! We hope this breakdown helps you on your journey to creating amazing AI-powered applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8d3bb171ea1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Handbook for Tech Leaders on Using Agentic AI and RAG to Address Technical Debt]]></title>
            <link>https://saurav-samantray.medium.com/a-handbook-for-tech-leaders-on-using-agentic-ai-and-rag-to-address-technical-debt-4015cdf64ac8?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/4015cdf64ac8</guid>
            <category><![CDATA[technical-debt]]></category>
            <category><![CDATA[tech-debt]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[genarative-ai]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Tue, 29 Apr 2025 03:57:49 GMT</pubDate>
            <atom:updated>2025-04-29T03:57:49.491Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rcWKzaPZacMayNu-kISblA.png" /></figure><p>Every engineering manager and tech leader understands the importance of technical debt, yet it frequently gets lost in the shuffle of feature requests, deadline pressure, and production fire drills. Technical debt, however, subtly reduces velocity, raises errors, and irritates your team if it is not addressed.<br>This article will teach you how to recognize technical debt, comprehend it, and — above all — develop a long-term strategy to manage and minimize it without causing your roadmap to veer.</p><h4>Technical Debt: What Is It?</h4><p>The projected cost of future rework resulting from selecting a quicker, less-than-ideal solution now is known as technical debt. It’s not always bad — sometimes you have to move quickly — but like any debt, it gradually accrues interest.. That interest can manifest as:</p><ul><li>Slower development velocity</li><li>Increased bugs or outages</li><li>Fragile or hard-to-change code</li><li>Decreased developer morale</li></ul><h3>Types of Technical Debt</h3><ol><li><strong>Deliberate debt</strong> — “We know this isn’t ideal, but we’ll fix it later.”</li><li><strong>Accidental debt</strong> — Poor practices due to a lack of experience or context.</li><li><strong>Aging debt</strong> — Once-good code no longer fits the current architecture or scale.</li></ol><h3>Step 1: Use LLMs to Auto-Detect and Categorize Debt</h3><p>LLMs (Large Language Models) can analyze your codebase and documentation to <strong>spot code smells, legacy patterns, and anti-patterns</strong> across modules.</p><h4>How to apply it:</h4><p>Use AI tools (e.g., GitHub Copilot, CodiumAI, Tabnine) to highlight maintainability issues directly in your IDE.</p><ul><li>Run periodic batch scans of your repo using AI-powered code analyzers to <strong>generate a “tech debt report.”</strong></li><li>Categorize debt into themes: performance, readability, test coverage, outdated dependencies, etc.</li></ul><blockquote><em>Bonus: Concerned with potential code exposure and secrets leakage when using public LLMs? Try out an opensource foundational Agentic AI + RAG project I’m working which uses local Ollama to analyze, detect and categorize tech debt. </em><a href="https://github.com/saurav-samantray/tech-debt-analyzer-agent.git">https://github.com/saurav-samantray/tech-debt-analyzer-agent.git</a></blockquote><h3>Step 2: Prioritize Debt with AI-Driven Impact Analysis</h3><p>Rather than relying solely on intuition, use <strong>Agentic AI</strong> systems to simulate and estimate the <strong>real impact of tech debt on team velocity and product quality</strong>.</p><h4>Examples:</h4><ul><li>Use a local RAG-based AI agent that pulls from your project’s history, architecture docs, Jira tickets, and incident data to <strong>prioritize tech debt items</strong> by impact.</li><li>Run “what-if” simulations: How much effort would a rewrite take? What modules are most volatile or bug-prone?</li></ul><blockquote><em>Tools like </em><a href="https://codescene.com/"><em>CodeScene</em></a><em> and </em><a href="https://stepsize.com/"><em>Stepsize AI</em></a><em> are already moving in this direction.</em></blockquote><h3>Step 3: Leverage Generative AI for Auto-Refactoring</h3><p>Now that you’ve identified and prioritized debt, let generative AI suggest or even perform the fix.</p><h4>How to use it:</h4><ul><li>Feed code snippets into LLMs (ChatGPT, Claude, Gemini) and prompt them:<br> &quot;Refactor this to improve readability and testability, and reduce coupling.&quot;</li><li>Use GitHub Copilot or IDE plugins to generate test cases and refactor large functions into cleaner abstractions.</li><li>Combine GenAI with version control: Have the AI create a pull request, include a description of the fix, and tag reviewers.</li></ul><blockquote><em>Pro tip: Layer LLMs with linters and static analyzers to verify safety before merging.</em></blockquote><h3>Step 4: Build Agentic RAG Pipelines to Solve Recurring Debt</h3><p>Imagine autonomous AI agents that continuously analyze, reason, and fix small chunks of tech debt over time.</p><h4>Set up an Agentic RAG workflow like this:</h4><ol><li><strong>RAG Layer</strong> retrieves relevant docs, code history, and design patterns from your repo.</li><li><strong>LLM Layer</strong> analyzes it and proposes a change.</li><li><strong>Agent Layer</strong> opens a PR, runs tests, waits for human feedback, and iterates.</li></ol><p>You can even assign an agent per domain (e.g., “Frontend Tech Debt Agent” or “API Quality Bot”).</p><blockquote><em>Tools like LangChain + OpenAI + GitHub Actions + Your CI/CD can make this real today.</em></blockquote><h3>Step 5: Use AI for Continuous Monitoring and Reporting</h3><p>AI doesn’t just fix code — it can report on the evolving state of your codebase in real-time.</p><h4>Set up:</h4><ul><li>Weekly Slack summaries from your AI agent: “Here are 5 areas of growing complexity.”</li><li>AI-generated dashboards that track metrics like cyclomatic complexity, PR review lag, or fragile modules.</li><li>Natural language interfaces for engineering leaders: “Show me all modules that have high churn and low test coverage.”</li></ul><blockquote><em>Combine with observability tools like Datadog, Sentry, and Jira for a full picture.</em></blockquote><h3>Step 6: Coach the Team with AI-Augmented Learning</h3><p>Many tech debt issues stem from knowledge gaps. Use AI to <strong>upskill your team</strong> where needed.</p><h3>Tips:</h3><ul><li>Use internal LLM-powered assistants to explain legacy code or design decisions.</li><li>Integrate AI learning bots in PR reviews to explain why certain patterns are discouraged.</li><li>Auto-suggest documentation or training links based on the code someone is working on.</li></ul><h3>Final Thoughts: AI Is a Force Multiplier, Not a Silver Bullet</h3><p>AI won’t magically make technical debt disappear — but it <strong>amplifies your ability to understand, manage, and resolve it</strong>.</p><p>As a tech leader, your new role is to:</p><ul><li><strong>Orchestrate humans + AI</strong> for continuous codebase health</li><li><strong>Invest in internal AI tooling</strong> tailored to your architecture</li><li><strong>Democratize debt visibility</strong> across the organization</li></ul><p>If done right, AI will not only reduce technical debt but also create a culture of code quality that scales with your team and product.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4015cdf64ac8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Securing REST API with Role-based access control(RBAC) using Keycloak — Part I (NodeJS)]]></title>
            <link>https://saurav-samantray.medium.com/securing-rest-api-with-role-based-access-control-rbac-using-keycloak-part-i-nodejs-8f59be925a42?source=rss-2795532fc0f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f59be925a42</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[expressjs]]></category>
            <category><![CDATA[keycloak]]></category>
            <category><![CDATA[rbac]]></category>
            <category><![CDATA[rest-api]]></category>
            <dc:creator><![CDATA[Saurav Samantray]]></dc:creator>
            <pubDate>Wed, 29 May 2024 13:44:15 GMT</pubDate>
            <atom:updated>2024-05-29T13:44:15.907Z</atom:updated>
            <content:encoded><![CDATA[<h3>Securing Nodejs(Express) REST API with Role-based access control(RBAC) using Keycloak</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A5vBqlgV7LBXp-8Ccvs-aw.png" /></figure><p>REST API could contain sensitive data(user information, financial details, or proprietary business data) and needs to be protected from unauthorized access. In this article, we will explore steps to secure a Nodejs(Express) REST API using RBAC(Role-based access control) setup in Keycloak.</p><h3>Pre-requisite</h3><p><strong>Spin up Keycloak server </strong>— If you have Docker setup in your local system, you can use one of the existing GitHub repositories I created to spin up the Keycloak server — <a href="https://github.com/saurav-samantray/custom-auth-service">https://github.com/saurav-samantray/custom-auth-service</a></p><h3>Setup Keycloak</h3><h4>Step1: Access the Local Keycloak Server</h4><p>Access the local Keycloak server setup in the previous setup at <a href="http://localhost:8080/">http://localhost:8080</a></p><h4><strong>Step2:</strong> Create a new client</h4><ul><li>Go to Clients in the left navigation.</li><li>Click on Import Client</li><li>Upload the file -&gt; <a href="https://raw.githubusercontent.com/saurav-samantray/secure-express-service/main/keycloak/secure-express-service.json"><em>secure-express-service.json</em></a></li><li>Click on save</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jmEpiEGuFEeWQFuz3entw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q0QCC1NLBTD2yYRui3RStg.png" /></figure><h4><strong>Step3:</strong> Create Realm Roles</h4><ul><li>Navigate to the <strong>Realm roles </strong>in the left navigation menu and click on <strong>Create role</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2nIeRPLeigTEF5bHbtTjvg.png" /></figure><ul><li>Create user role — <strong>express-user</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4iebI7TxIOIafuJtXCdFnQ.png" /></figure><ul><li>Repeat the steps to create an admin role — <strong>express-admin</strong></li></ul><h4><strong>Step4:</strong> Create Users</h4><p>Navigate to the <strong>Users</strong> section from the left navigation menu. Fill out the new user form and click on Create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLtjFnvLtRFPe8kUZzYgEw.png" /></figure><ul><li>Navigate to the details of the newly created user — <strong>user@nerdcore.com, </strong>go to the <strong>Credentials</strong> tab and Click on <strong>Set Password</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NHY27iEv7IknjU2JaEUUBA.png" /></figure><ul><li>Fill out the Set password form with the appropriate password. Make sure to mark the Temporary toggle button Off</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*wcMX5WCzZFu8h2XbZ5g02Q.png" /></figure><ul><li>Create another user <strong>admin@nerdcode.com</strong></li></ul><h4><strong>Step5:</strong> Assign roles to users</h4><p>Navigate to the <strong>Role Mapping</strong> Tab on the user details page and click on the <strong>Assign Role</strong> button</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8PlXR-tS_JM2TTcNzH516g.png" /></figure><ul><li>Select <strong>express-user</strong> and click on the <strong>Assign</strong> button</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yOfiPfuN9t1TycGQGwygSQ.png" /></figure><ul><li>Similarly, assign the <strong>express-admin</strong> role to user <strong>admin@nerdcore.com</strong></li></ul><h3>Basic Express JS application to serve REST endpoints</h3><h4><strong>Initialize a Node JS application</strong></h4><p>run the below set of commands to initialize a NodeJS application as our express server will reside inside the Node application.</p><pre>mkdir secure-express-service<br>cd secure-express-service<br>npm init</pre><p>The npm init command will initiate a series of prompts to create the base package.json. Sample input below</p><pre>package name: (secure-express-service)<br>version: (1.0.0)                                                                                                                                                 <br>description: Express JS based REST APIs secured using Keycloak auth server<br>entry point: (index.js) app.js                                                                                                                                   <br>test command:                                                                                                                                                    <br>git repository:                                                                                                                                                  <br>keywords:                                                                                                                                                        <br>author: Saurav Samantray                                                                                                                                         <br>license: (ISC)                                   </pre><h4>Install dependencies</h4><pre>npm install express express-session keycloak-connect nodemon</pre><h4>Define main execution file — app.js</h4><p>Add a file named app.js at the root level of your secure-express-service project.</p><p>Define imports and express app</p><pre>// file - app.js<br><br>const express = require(&#39;express&#39;);<br>const session = require(&quot;express-session&quot;);<br>const Keycloak = require(&quot;keycloak-connect&quot;);<br><br>const app = express();<br>const PORT = 3000;<br><br>...</pre><p>Setup Keycloak Middleware</p><pre>// file - app.js<br><br>...<br><br>const USER_ROLE = process.env.USER_ROLE || &#39;express-user&#39;;<br>const ADMIN_ROLE = process.env.ADMIN_ROLE || &#39;express-admin&#39;;<br><br>const kcConfig = {<br>    clientId: process.env.AUTH_CLIENT_ID || &#39;secure-express-service&#39;,<br>    bearerOnly: true,<br>    serverUrl: process.env.AUTH_SERVER || &#39;http://localhost:8080&#39;,<br>    realm: process.env.AUTH_REALM || &#39;master&#39;<br>};<br><br>const memoryStore = new session.MemoryStore();<br><br>Keycloak.prototype.accessDenied = function (request, response) {<br>    response.status(401)<br>    response.setHeader(&#39;Content-Type&#39;, &#39;application/json&#39;)<br>    response.end(JSON.stringify({ status: 401, message: &#39;Unauthorized/Forbidden&#39;, result: { errorCode: &#39;ERR-401&#39;, errorMessage: &#39;Unauthorized/Forbidden&#39; } }))<br>}<br><br>const keycloak = new Keycloak({ store: memoryStore }, kcConfig);<br><br>function adminOnly(token, request) {<br>    return token.hasRole(`realm:${ADMIN_ROLE}`);<br>}<br><br>function isAuthenticated(token, request) {<br>    return token.hasRole(`realm:${ADMIN_ROLE}`) || token.hasRole(`realm:${USER_ROLE}`);<br>}<br><br><br>app.use(session({<br>    secret: process.env.APP_SECRET || &#39;BV&amp;%R*BD66JH&#39;,<br>    resave: false,<br>    saveUninitialized: true,<br>    store: memoryStore<br>}));<br>  <br>app.use( keycloak.middleware() );<br><br>...</pre><p>Setup REST endpoint and server</p><pre>app.get(&#39;/public&#39;, (req, res) =&gt; {<br>    res.status(200).send({<br>        &#39;message&#39;: &quot;This is a public enpoint which can be accessed by anonymous users&quot;,<br>    });<br>})<br><br>app.get(&#39;/secured&#39;, [keycloak.protect(isAuthenticated)], (req, res) =&gt; {<br>    res.status(200).send({<br>        &#39;message&#39;: &quot;This is a secured enpoint which can be accessed by any authenticated user&quot;,<br>    });<br>})<br><br>app.get(&#39;/secured-admin&#39;, [keycloak.protect(adminOnly)], (req, res) =&gt; {<br>    res.status(200).send({<br>        &#39;message&#39;: &quot;This is a secured enpoint which can be accessed only by any authenticated user with role admin&quot;,<br>    });<br>})<br><br>app.listen(PORT, (error) =&gt; {<br>    if (!error) {<br>        console.log(&quot;Server is Successfully Running,  and App is listening on port &quot; + PORT)<br>    }<br>    else {<br>        console.log(&quot;Error occurred, server can&#39;t start&quot;, error);<br>    }<br>}<br>);</pre><h4>Update package.json scripts to run server</h4><ul><li>Update the content in the <strong>scripts </strong>section of <strong>package.json</strong> as below. It will help you run your express server in both development and production mode</li></ul><pre>  &quot;scripts&quot;: {<br>    &quot;start&quot;: &quot;node app.js&quot;,<br>    &quot;dev&quot;: &quot;nodemon app.js&quot;<br>  },</pre><ul><li>Start your express server in development mode(auto-reload on file change) using the below command</li></ul><pre>npm run dev</pre><h3>Testing your REST APIs</h3><h4>Public Endpoint— <a href="http://localhost:3000/public">http://localhost:3000/public</a></h4><ul><li>Even without an authentication token, this endpoint will fetch a response.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xjvkOCdQJ5VIolwBtu-TWg.png" /></figure><h4>Secured Endpoint — <a href="http://localhost:3000/public">http://localhost:3000/s</a>ecured</h4><ul><li>Without a token, you should get an authentication error</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rSANl9_J1BRma6p1UjgLPg.png" /></figure><ul><li>Let’s set up the <strong>Authorization</strong> tab to generate a token using the <strong>Configure New Token </strong>section. You can use the values below.</li></ul><pre>Token Name - secure-express-service<br>Grant type - Password Credentials<br>Access Token URL - http://localhost:8080/realms/master/protocol/openid-connect/token<br>Client ID - secure-express-service<br>Username - user@nerdcore.com<br>Password - <br>Client Authentication - Send as basic auth header</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*AEhl6BHxVPT5J6X6wKnxGg.png" /></figure><ul><li>Click on <strong>Get New Access Token</strong>, then P<strong>roceed</strong>, then <strong>Use Token</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*t7pdQRYhqsnhK_NDBSD19g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*MwMrf1npagIRpGmXEfOWcw.png" /></figure><ul><li>Now fire the <strong>/secured</strong> rest endpoint again, and you will get a successful response.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xsq6E7TWbMeTHXzNEQMD1Q.png" /></figure><h4>Secured Admin Endpoint — <a href="http://localhost:3000/public">http://localhost:3000/s</a>ecured-admin</h4><ul><li>If you try to access the endpoint without any token — Auth error</li><li>If you try to access the endpoint with the token from user@nerdcore.com — Auth error</li><li>If you try to access the endpoint with the token from admin@nerdcore.com — Success</li></ul><p>You can find the GitHub repository <a href="https://github.com/saurav-samantray/secure-express-service">here</a> for reference.</p><p>Happy learning and happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f59be925a42" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>