<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vikram on Medium]]></title>
        <description><![CDATA[Stories by Vikram on Medium]]></description>
        <link>https://medium.com/@vikram-ai?source=rss-90840cdc6a37------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Izy2rULCrzlC3mQwQE1V0w.jpeg</url>
            <title>Stories by Vikram on Medium</title>
            <link>https://medium.com/@vikram-ai?source=rss-90840cdc6a37------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 19:45:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vikram-ai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cutting Through the AI Noise: My 2025 Productivity Stack for non-coding tasks]]></title>
            <link>https://vikram-ai.medium.com/cutting-through-the-ai-noise-my-2025-productivity-stack-for-non-coding-tasks-bca7f17ebe0d?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/bca7f17ebe0d</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Sun, 22 Jun 2025 22:03:11 GMT</pubDate>
            <atom:updated>2025-06-22T22:03:11.912Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Choosing tools shouldn’t feel like scrolling an endless “best-of” list. I’ve test-driven dozens of AI products and settled on six that quietly handle the heavy lifting in my day-to-day work. Here’s how they fit together, with real-world examples you can copy-and-paste into your own routine.</em></p><h3>1. Ask Smarter, Learn Faster with Perplexity</h3><p>Perplexity is my “search with superpowers.” When I bump into a term like <em>prime rate vs. overnight rate</em> or want a neutral rundown of the latest Israel-Iran developments, I open Perplexity instead of a search engine. It delivers an answer in plain language, then shows the sources so I can check the facts myself. The new <strong>Pro Perks</strong> program even throws in partner benefits (discounted subscriptions, premium data sources) for paid users, which sweetens the deal for heavy researchers. <a href="https://www.perplexity.ai/hub/blog/more-value-in-every-answer-new-benefits-for-every-level-of-perplexity-user?utm_source=chatgpt.com">perplexity.ai</a></p><p><em>Try this:</em> Paste a confusing paragraph from a white paper and ask, “Summarize this in two sentences for a ninth-grader.” Instant clarity.</p><h3>2. Automate the Mundane with n8n</h3><p>If Perplexity is my brain, n8n is my hands. The open-source workflow builder lets me string together apps without writing code:</p><ul><li><strong>Newsletter engine:</strong> A weekly “jobs in AI” mailer scrapes new listings via a RapidAPI endpoint, filters them in Google Sheets, and sends a polished email. All triggered by a single n8n workflow. <a href="https://n8n.io/workflows/4470-jobs-newsletter-automation-system-n8n-boltnew-rapidapi-mailsso-and-chatgpt/?utm_source=chatgpt.com">n8n.io</a></li><li><strong>Bank-spend tagging:</strong> Every time a transaction hits my email, n8n pushes the details to a Notion database and auto-labels categories (“groceries,” “tools,” “subscriptions”) so I can budget in minutes.</li></ul><p>Because you host it yourself (cloud or local), privacy and cost stay under your control.</p><h3>3. Write Like a Pro with ChatGPT, Claude 3 &amp; Gemini</h3><p>Generative models are the Swiss-army knife of writing:</p><ul><li><strong>ChatGPT</strong> drafts first-pass outlines, marketing copy, or quick code snippets.</li><li><strong>Claude 3 Opus</strong> excels at long, structured output. Think 2-page project briefs or JSON data pulls — thanks to its improved reasoning and formatting skills. <a href="https://www.anthropic.com/news/claude-3-family?utm_source=chatgpt.com">anthropic.com</a></li><li><strong>Gemini</strong> slips into my Google Docs and Gmail, suggesting headlines or rewriting paragraphs without leaving the page. <a href="https://gemini.google.com/?utm_source=chatgpt.com">gemini.google.com</a></li></ul><p>I keep a simple rule: one paragraph from me, one from AI, then I polish. It halves my writing time while keeping the voice authentic.</p><h3>4. Prototype Ideas Quickly with Replit</h3><p>Every “What if we built…?” conversation now lives in Replit. The browser-based IDE spins up a live sandbox in seconds, and <strong>Replit AI</strong> autocompletes functions, fixes bugs, and explains unfamiliar code. Even the free tier includes AI assistance, while Pro unlocks faster GPUs for heavier apps. <a href="https://blog.replit.com/ai4all?ref=taaft&amp;utm_source=chatgpt.com">blog.replit.com</a></p><p><em>Example:</em> I mocked up a Flask micro-service during a client call, shared the live URL, and collected feedback before the meeting ended, no local setup, no DevOps headache.</p><h3>5. Tell Visual Stories with Canva + Magic Studio</h3><p>When text isn’t enough, Canva’s <strong>Magic Media</strong> transforms prompts like “10-second kinetic typography intro with calm jazz” into editable video clips. A fresh integration with Google’s Veo 3 model even adds AI-generated audio, so my social reels look agency-made without the agency bill.</p><p>I keep brand templates in one folder; Magic Media respects them automatically, which means faster consistency across decks, posts, and ads.</p><h3>6. Capture Knowledge with NoteGPT</h3><p>Webinars and 90-minute conference talks turn into bite-sized insights: paste the YouTube link into NoteGPT and get a bullet summary, key quotes, and a timestamped outline. Perfect for revisiting during project research or dropping highlights into Notion. <a href="https://notegpt.io/youtube-video-summarizer?utm_source=chatgpt.com">notegpt.io</a></p><h3>Staying Organized (So the Stack Doesn’t Become Stacked Against You)</h3><ol><li><strong>One inbox for ideas:</strong> I pipe outputs from Perplexity, NoteGPT, and Claude into a “/inbox” page in Notion. If it lands there, it gets processed — research, reference, or reject.</li><li><strong>Name workflows, not tasks:</strong> In n8n, prefixes like fin-, content-, ops- keep similar automations grouped and searchable.</li><li><strong>Weekly tool audit:</strong> Every Friday I ask myself, <em>Did this tool save more minutes than it cost?</em> Anything that flunks gets trimmed or replaced.</li></ol><h3>A Day in the Life of the Stack</h3><ol><li><strong>Morning curiosity:</strong> Perplexity explains a new industry term in 100 words.</li><li><strong>Before lunch:</strong> n8n files overnight bank alerts and schedules a newsletter send.</li><li><strong>Early afternoon:</strong> ChatGPT and Claude tag-team a client proposal; Gemini polishes the email pitch.</li><li><strong>Late afternoon:</strong> Replit hosts a quick proof-of-concept web app.</li><li><strong>Evening:</strong> Canva turns that day’s insights into a LinkedIn carousel; NoteGPT summarizes a webinar while I cook.</li></ol><p>Total “deep-work” hours preserved: <strong>~2</strong>.</p><h3>Final Thoughts</h3><p>AI products pop up (and fizzle out) weekly. Instead of chasing every headline, pick one tool in each category — research, automation, writing, prototyping, visuals, and knowledge capture. Give it 30 days, measure the time saved, then decide whether it earns a permanent slot in your stack. Start small, stay curious, and let the robots handle the busywork so you can focus on the work that matters.</p><p><em>Have a favourite tool I missed? Drop it in the comments — because in 2025, the best AI advice still comes from humans sharing what works.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bca7f17ebe0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is ChatGPT making people dumb? Not really!]]></title>
            <link>https://vikram-ai.medium.com/misinterpreted-research-on-chatgpt-9290ffbfb838?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/9290ffbfb838</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Sun, 22 Jun 2025 12:26:45 GMT</pubDate>
            <atom:updated>2025-06-22T12:37:44.498Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Is ChatGPT Frying Our Brains? MIT’s EEG Study*, Explained</strong></p><p>So, MIT plugged students into EEG machines while they wrote essays, and suddenly everyone’s screaming: <em>“ChatGPT is frying our brains!”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7CFDakbJgE1H40cse8r37g.png" /><figcaption>Prime example of clickbait journalism</figcaption></figure><p>Hold your panic.</p><p>Yes, MIT Media Lab just published a fascinating study. They discovered that students relying heavily on ChatGPT for quick-and-easy essays showed lower brain activity, weaker memory recall, and virtually zero sense of ownership over their writing. Their peers using just their own neurons? Their brains were glowing like Christmas lights. And the Google-assisted group? Comfortably somewhere in between.</p><p>Cue the dramatic headlines: <em>“AI Makes Us Stupid!”</em> or <em>“ChatGPT Could Destroy Our Brains!”</em> Panic-mode journalism at its finest.</p><h3>What MIT Actually Did</h3><p>Let’s get the facts straight first. Researchers had 54 students write timed SAT-style essays under three distinct conditions: using only their brains, using Google search, and using ChatGPT assistance. After several rounds, the researchers mixed things up — students initially dependent on ChatGPT were suddenly left to their own devices. And guess what? Their brains remained pretty sluggish. Conversely, students who started writing manually and then shifted to AI-assistance saw a boost in brain activity, possibly from actively comparing their ideas with AI-generated suggestions.</p><p>This clearly indicates something deeper than just “AI good” or “AI bad.” It’s about how and when AI is used.</p><h3>Why the Media Got It Wrong</h3><p>The real problem isn’t the careful MIT study — it’s sensationalist journalism spinning nuanced findings into apocalypse-level scenarios. A quick 20-minute “write this essay for me” scenario isn’t representative of how most of us interact with AI in real-world scenarios. Typically, AI is part of a more complex process: brainstorming, drafting, editing, revising, and collaborating. Not just blindly copy-pasting.</p><p>Yet, sensational headlines grab attention more than nuanced discussion, turning a thoughtful, context-rich study into panic-inducing clickbait.</p><h3>Real Insights from the Study</h3><p>Still, let’s not dismiss the real insights here. The MIT study does highlight a genuine risk: if young learners, still developing crucial learning skills, habitually offload cognitive tasks onto AI, it might genuinely stunt their cognitive growth. Critical thinking, creativity, and independent reasoning could indeed suffer if students overly rely on AI without engaging their minds actively.</p><p>Especially for young students still mastering “how to learn,” AI shortcuts could severely limit their cognitive development. We might face a generation adept at getting tasks done quickly but lacking the deeper cognitive abilities essential for real-world challenges.</p><h3>What Broader Research Should Look Like</h3><p>To genuinely explore whether AI negatively impacts learning, we need far more comprehensive research:</p><ul><li>Diverse tasks (creative writing, mathematics, coding, analytical reasoning)</li><li>Long-term observation across weeks, months, or semesters</li><li>Multiple cognitive aids (AI tools, traditional search, human tutoring)</li><li>Richer neurological and cognitive measurements</li><li>Real-world classroom dynamics and collaborative settings</li><li>Diverse student demographics and educational backgrounds</li></ul><p>Only with such rigorous studies can we truly understand AI’s educational impact.</p><h3>AI is Not the Enemy — Passive Learning Is</h3><p>This study isn’t an indictment against AI itself; it’s a cautionary tale about passive and lazy usage. When used thoughtfully, starting with your own ideas and refining them using AI, these tools can significantly enhance learning and creativity. It’s passive dependency — “write it for me, so I don’t have to think” — that we should worry about.</p><h3>Let’s Finish with a Fun Metaphor</h3><p>Imagine a cat deciding knitting is too exhausting, so it lets a robot knit a sweater. Sure, the cat ends up cozy — but did it learn to knit? Did it even try? Of course not. Treating AI purely as your personal knitting robot might keep you comfy, but you won’t grow.</p><p>Keep your brain curious. Use AI to enrich your thinking — not replace it. Remember, ChatGPT isn’t here to write your essays entirely; it’s here to help your own ideas shine brighter.</p><p>In short: don’t panic — just stay smart.</p><p>*<a href="https://www.media.mit.edu/publications/your-brain-on-chatgpt/">https://www.media.mit.edu/publications/your-brain-on-chatgpt/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9290ffbfb838" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Inbox Chaos to Curated Clarity: An AI-Powered Newsletter Digest with n8n]]></title>
            <link>https://vikram-ai.medium.com/from-inbox-chaos-to-curated-clarity-an-ai-powered-newsletter-digest-with-n8n-0655fa6ac502?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/0655fa6ac502</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Fri, 13 Jun 2025 11:41:57 GMT</pubDate>
            <atom:updated>2025-06-13T11:41:57.134Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mawGHBDrLqjdJCGE" /></figure><p>I love newsletters, but after subscribing to dozens, my inbox became an unruly beast. Instead of skimming headlines over coffee, I was doom-scrolling unread emails. So, I decided to automate myself out of the problem.</p><p>Below is the bite-sized story of how I turned a mountain of newsletter emails into one polished weekly digest using <strong>n8n</strong>, GPT-4o, and Google Sheets. Feel free to copy, remix, and make it yours.</p><h3>The Problem</h3><p><em>💡 Too many great newsletters, not enough time to read them.</em><br> Diverse formats, varying quality, and a scattered arrival schedule made it hard to spot patterns or act on insights. I needed a single, concise briefing and no manual copy-pasting or Sunday-night guilt.</p><h3>The Solution (In 5 Automations)</h3><ol><li><strong>Listen &amp; Filter</strong></li></ol><ul><li><strong>Gmail Trigger</strong> polls every hour.</li><li>An <strong>IF node</strong> captures mails with keywords like <em>newsletter</em>, <em>roundup</em>, or senders such as TLDR.</li></ul><p><strong>2. Instant Summaries with GPT-4o</strong></p><ul><li>Each email is fed to an <strong>Email Summarizer</strong> (LangChain agent + GPT-4o).</li><li>The model returns a JSON payload: { title, summary }.</li></ul><p><strong>3. Structured Storage</strong></p><ul><li>A <strong>Google Sheets node</strong> appends title, summary, and ISO week to a “NewsletterSummaries” sheet.</li><li>This builds a rolling knowledge base with zero maintenance.</li></ul><p><strong>4. Weekly Digest Generation</strong></p><ul><li>A <strong>Schedule Trigger</strong> fires every Friday at 09:00.</li><li>It fetches that week’s rows, aggregates them, and hands the bundle to an <strong>AI Agent</strong> that crafts a reader-friendly HTML report (executive summary, grouped topics, insights, action items, emoji-sprinkled headings).</li></ul><p><strong>5. Delivery &amp; Audit Trail</strong></p><ul><li>The freshly minted digest lands in my inbox (or Slack, Teams, your pick).</li><li>A second Sheets tab logs send metadata for peace-of-mind auditing.</li></ul><h3>Impact</h3><ul><li>📉 <strong>Inbox load</strong>: -90% newsletter noise.</li><li>🕒 <strong>Time saved</strong>: ~2 hours/week previously spent skimming.</li><li>📈 <strong>Insight quality</strong>: Trends leap out when content is grouped and synthesized.</li></ul><h3>How You Can Adapt It</h3><pre>| Need              | Quick Tweak                                                              |<br>| ----------------- | ------------------------------------------------------------------------ |<br>| Different source  | Swap Gmail Trigger for RSS, Slack, or a webhook                          |<br>| Other AI models   | Point LangChain at Gemini, Claude, Mistral—whatever fits budget &amp; policy |<br>| Alternate storage | Use Notion, Airtable, Postgres, or a Markdown file in Git                |<br>| Custom cadence    | Run daily, monthly, or on-demand via an HTTP call                        |<br>| New use-cases     | Summarize customer tickets, product feedback, sales leads                |</pre><h3>Why n8n?</h3><ul><li><strong>No-code canvas</strong> for rapid experimentation</li><li><strong>First-class AI integrations</strong> via LangChain nodes</li><li><strong>Self-host optional: </strong>keep data where you trust it</li><li><strong>Extensible: </strong>drop to JavaScript functions when needed</li></ul><h3>Parting Thoughts</h3><p>Automation shouldn’t just eliminate drudgery; it should amplify insight. By chaining smart filters, large language models, and lightweight data storage, I turned inbox overload into a weekly “aha!” moment that I actually look forward to reading.</p><p>If your inbox feels like mine once did, fork the JSON, tweak a few nodes, and let the bots do the boring bits. Your future Friday self will thank you.</p><p><em>Questions, ideas, or war stories? Drop them in the comments. Let’s keep each other out of newsletter purgatory.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0655fa6ac502" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A No‑BS Guide to Building LLM Agents]]></title>
            <link>https://vikram-ai.medium.com/a-no-bs-guide-to-building-llm-agents-84da93bf3b45?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/84da93bf3b45</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Thu, 29 May 2025 00:11:39 GMT</pubDate>
            <atom:updated>2025-05-29T00:16:39.924Z</atom:updated>
            <content:encoded><![CDATA[<p>Most “AI agent” tutorials leap straight into orchestration diagrams and swarms of autonomous workers. That’s overkill for most products. <strong>This article is a sanity check.</strong> It lays out a staircase: start with the simplest retrieval‑augmented chatbot and add exactly <em>one</em> new capability only when the last step stops meeting user needs. Follow it to avoid premature complexity, runaway cloud bills, and week‑long debugging sessions.</p><h3>TL;DR</h3><p>Advance through <strong>five maturity levels</strong> only when a clear pain point appears. Each level introduces exactly one new capability — Tools, Memory, Reflection, Orchestration — paired with the leanest framework you actually need.</p><h3>Level 0 — One‑Shot Q‑A (Static LLM + Retrieval)</h3><p><strong>Pain solved:</strong> People just want an answer — but it must be grounded in your own documents, so the model doesn’t hallucinate.</p><p><strong>Typical use cases</strong>:</p><ul><li>A workplace HR chatbot answering policy questions from the employee handbook.</li><li>An IT help‑desk bot surfacing step‑by‑step fixes from the internal KB.</li><li>A university info kiosk answering course prerequisite queries using the course catalogue.</li></ul><p><em>Frameworks to reach for</em> — Fin‑style hosted KB bots, Zendesk AI, Freshchat, or DIY with LangChain RAG templates, LlamaIndex, or Pinecone Serverless.</p><p><em>Important</em> — Designing a robust RAG pipeline (chunking, embedding, ranking, freshness) is its own discipline and outside the scope of this guide; we focus on what to add <strong>after</strong> retrieval is reliable.</p><p><em>When to stop here</em> — Docs cover the questions, latency &lt; 1 s, no personalization or tool calls needed.</p><h3>Level 1 — Add Tools When the Answer Lives Outside the Model</h3><p><strong>Pain solved:</strong> The agent needs real‑time or privileged data.</p><p><strong>Typical use cases</strong></p><ul><li>An e‑commerce order tracker that pulls live shipping status from a logistics API.</li><li>A banking assistant fetching current account balances and recent transactions.</li><li>A flight‑status bot retrieving gate changes and delays in real time.</li></ul><p><em>Frameworks to reach for</em> — LangChain Function Calling, Neon MCP action endpoints, OpenAI Tools. Expose one HTTPS action per business verb; keep the planner dumb — <em>intent → call</em>.</p><h3>Level 2 — Bolt On Memory for Multi‑Turn Personalization</h3><p><strong>Pain solved:</strong> The agent must remember user quirks across sessions.</p><p><strong>Typical use cases</strong></p><ul><li>A personal‑finance coach that recalls monthly budgets and spending goals.</li><li>A fitness planner remembering past injuries and available gym equipment.</li><li>A language tutor tracking recurring grammar mistakes across lessons.</li></ul><p><em>Frameworks to reach for</em> — Letta (structured preferences), Mem0 (vector‑store memories), LangGraph Memory Agent template. Summarise or vector‑store after ~20 turns to curb latency.</p><h3>Level 3 — Insert Reflection to Police Quality &amp; Compliance</h3><p><strong>Pain solved:</strong> Wrong answers now cost real money or legal risk.</p><p><strong>Typical use cases</strong> * A contract reviewer that flags missing indemnity clauses and rewrites the draft. * A coding assistant that lints and self‑critiques generated code before showing it. * A clinical triage bot that double‑checks symptom interpretation before advice.</p><p><em>Frameworks to reach for</em> — CriticGPT, DeepEval, TruLens. Generator‑critic loops with rubrics and dashboards let you measure and enforce quality. Run the critic only on high‑stakes outputs; throttle retries.</p><h3>Level 4 — Orchestrate When Tasks Multiply &amp; Need Parallelism</h3><p><strong>Pain solved:</strong> A single request explodes into many subtasks — or you need specialised roles working together.</p><p><strong>Typical use cases</strong></p><ul><li>A market‑analysis pipeline that scrapes competitor sites, clusters trends, and drafts an executive briefing.</li><li>A product‑launch agent coordinating copywriting, image generation, and publishing schedules. *</li><li>A literature‑review assistant searching academic databases, summarising findings, and drafting a report.</li></ul><p><em>Frameworks to reach for</em> — LangGraph for deterministic DAGs, CrewAI for role‑playing swarms, AutoGen or AgentStack for message‑passing agents. Layer in tracing (LangSmith) and feedback loops (TruLens) to see what each worker is doing.</p><h3>Upgrade Checklist</h3><ol><li><strong>Validate the pain first.</strong> Don’t add Memory if users never return.</li><li><strong>Swap components, don’t stack endlessly.</strong> If a framework bundles two layers, use it — don’t bolt on another.</li><li><strong>Log everything</strong> — tool calls, memory hits, critic votes — so you can debug quickly.</li><li><strong>Budget every second and cent.</strong> Each added capability roughly doubles latency and cost; plan capacity early.</li><li><strong>Keep a human referee.</strong> Even a light reviewer loop still beats full autonomy on safety‑critical tasks.</li></ol><h3>Final Word</h3><p>Think stair‑steps, not rocket stages. One‑shot Q‑A still answers more than half of workplace questions. Climb only when the new complexity directly fixes measurable pain.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84da93bf3b45" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When to Use ML, LLMs, and Agents (and When to Panic) ]]></title>
            <link>https://vikram-ai.medium.com/when-to-use-ml-llms-and-agents-and-when-to-panic-e2fa1f111b66?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2fa1f111b66</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Sun, 11 May 2025 21:30:01 GMT</pubDate>
            <atom:updated>2025-05-12T00:34:48.004Z</atom:updated>
            <content:encoded><![CDATA[<p>Choosing the right AI/ML approach is like outfitting your kitchen: you wouldn’t use a blender to toast bread, and you <em>definitely</em> shouldn’t use a Large Language Model (LLM) to predict next quarter’s sales with pinpoint accuracy (unless you enjoy giving your CFO heart palpitations). Here’s a handy guide for software engineers and business users to decide which AI/ML approach to use — and when to seriously reconsider your choices or call for backup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HlrjTk2wlsj6q3g5gRLBiA.png" /></figure><p>Especially with the current hype cycle, it can sound like every problem needs a complex, multi-agentic workflow. This is demonstrably untrue. For many challenges, more established, reliable, and cost-effective solutions already exist. If you’re a software engineer or a business leader, identifying the right tool for the job can save you significant time, resources, and sanity. Here’s my expanded guide:</p><h3>ML/Deep Learning: The Reliable Toaster 🍞</h3><p><em>The workhorse. Does specific tasks exceptionally well with the right preparation.</em></p><p>Machine Learning/AI used to be an umbrella term used for most pre-transformer era models. These include Linear Regression, Random Forests, SVMs, Clustering algorithms, Recommendation Engines (collaborative filtering), Neural Nets, LSTMs and even small language models. They are amazing at solving specialized tasks when trained on that exact data. The training is relatively easy (a few hours to days on consumer GPUs) and unlike LLMs anyone can build these models from scratch.</p><p><strong>When to use:</strong></p><ol><li><strong>Predictable, Structured Input:</strong> Your data is primarily numerical, categorical, tabular (think spreadsheets, database records), or even images/audio <em>when the task is well-defined within that domain</em> (e.g., “is this a cat?” not “write a poem about this cat image”). The input features and their distributions are relatively known.</li><li><strong>Abundant, Labeled Training Data:</strong> You have a substantial dataset where the “answers” (labels or target variables) are already known. Supervised learning thrives here. The more high-quality, relevant data, the better the model’s performance.</li><li><strong>Specific, Well-Defined Tasks:</strong></li></ol><ul><li><strong>Classification:</strong> “Is this email spam or not spam?”, “Is this transaction fraudulent?”, “What type of customer is this?”</li><li><strong>Regression:</strong> “How many pizzas will we sell next Friday?”, “What will be the price of this stock tomorrow?”, “What is the optimal dosage for this patient?”</li><li><strong>Clustering:</strong> “Group these customers into distinct segments based on purchasing behavior,” “Identify natural groupings in sensor data.”</li><li><strong>Anomaly Detection:</strong> “Is this network activity unusual?”, “Is this machine reading outside normal parameters?”</li></ul><p><strong>4. High Accuracy &amp; Explainability (Sometimes) Required:</strong> For critical applications (e.g., medical diagnosis from scans, credit scoring), you often need very high precision and recall. Some ML models (like decision trees or linear regression) offer better interpretability than others (deep neural networks). <em>While 99%+ is a nice goal, the “required” accuracy is highly domain dependent. Sometimes 80% is great, other times 99.99% is essential.</em></p><p><strong>5. Efficiency and Cost-Effectiveness for Specific Problems:</strong> Once trained, many ML models are computationally efficient for inference, making them suitable for high-volume, low-latency applications.</p><p><strong>Examples:</strong></p><ul><li><strong>Predicting house prices:</strong> Based on square footage, location, number of bathrooms, age, school district ratings, historical sales data in the area. (Feed it a philosophical essay on “home,” and it’ll error out.)</li><li><strong>Classifying images:</strong> Identifying specific objects (cars, pedestrians, traffic signs in autonomous driving), detecting defects in manufacturing, categorizing medical scans (e.g., presence of tumors).</li><li><strong>Forecasting sales:</strong> Using historical sales data, seasonality, promotional events, economic indicators, and even competitor actions (if quantifiable).</li><li><strong>Customer Churn Prediction:</strong> Analyzing customer demographics, usage patterns, support ticket history, and engagement metrics to predict who is likely to leave.</li><li><strong>Recommendation Engines:</strong> Suggesting products on an e-commerce site based on past purchases and browsing history of the user and similar users (collaborative filtering) or based on product attributes (content-based filtering).</li></ul><p><strong>When to Panic (or at least pause and rethink):</strong></p><ul><li>If your data is messy, scarce, or unlabeled, and you have no resources to clean/label it.</li><li>If the problem requires understanding nuanced human language, generating creative text, or common-sense reasoning beyond statistical patterns.</li><li>If the underlying patterns in your data change very rapidly and unpredictably, requiring constant retraining that isn’t feasible.</li><li>If you’re trying to force a square peg (unstructured data) into a round hole (an ML model expecting structured features) without proper feature engineering.</li></ul><h3>LLM (Large Language Model): The Chatty Blender 🗣️🥤</h3><p><em>Versatile with language, can mix up novel outputs, but sometimes makes a mess or gives you something unexpected.</em></p><p>LLMs are several billion parameter models that have been fine-tuned on several trillion tokens of data on industry grade GPUs for several months with millions of dollars spent. GPT/Llama/Claude/Gemini are some popular examples. It is possible to fine-tune these on specific tasks but often won’t improve the results significantly. The models are meant to be general purpose. Architecturally speaking, these use the transformer architecture used by most small language models but scaled by100–1000x.</p><p><strong>When to use:</strong></p><ol><li><strong>Unstructured Input is Key:</strong> The primary input is natural language text (user queries, documents, articles, conversations), code, or even images/audio if the task is about <em>describing or reasoning about them in language</em>.</li><li><strong>Leveraging Pre-trained Knowledge:</strong> When you lack specific training data for a task, but the task relies on general world knowledge, language understanding, or common sense that a human with broad knowledge could perform.</li><li><strong>Generative and Transformative Language Tasks:</strong></li></ol><ul><li>Summarization, translation, paraphrasing.</li><li>Content creation (drafting emails, articles, marketing copy, code).</li><li>Question-answering over provided context or general knowledge.</li><li>Chatbots and conversational interfaces.</li></ul><p><strong>4. Acceptable “Good Enough” Accuracy / Human-in-the-Loop:</strong> When perfect factual accuracy isn’t strictly required for all outputs, or when a human can easily review and correct. LLMs can hallucinate or be subtly wrong. <em>The “accuracy” here is often about fluency, coherence, and relevance rather than a single quantifiable metric.</em></p><p><em>5. </em><strong>Rapid Prototyping for Language Tasks:</strong> Quickly building a proof-of-concept for a language-based feature before investing in more complex solutions.</p><p><strong>Examples:</strong></p><ul><li><strong>Summarizing a 50-page contract:</strong> Extracting key clauses, obligations, and dates into a digestible summary. (It might miss a subtle legal nuance, so lawyer review is still key!)</li><li><strong>Chatbots for customer queries:</strong> Handling FAQs, guiding users through simple processes, understanding intent from varied phrasing. (But not for executing financial transactions without safeguards).</li><li><strong>Classifying input data into patterns at scale:</strong> Sentiment analysis of customer reviews, topic modeling of news articles, intent recognition from user support requests.</li><li><strong>Drafting initial content:</strong> Generating blog post outlines, product descriptions, email responses, or even boilerplate code snippets.</li><li><strong>Semantic Search:</strong> Understanding the <em>meaning</em> behind a search query, not just keywords, to find more relevant documents.</li></ul><p><strong>When to Panic:</strong></p><ul><li>If you need 100% factual accuracy for critical decisions without verification (e.g., medical advice, financial reporting). LLMs <em>will</em> hallucinate.</li><li>If your task involves complex, multi-step reasoning or mathematical calculations where precision is paramount (LLMs are notoriously bad at math without tools).</li><li>If you’re feeding it highly sensitive or private data without understanding the data privacy implications of the specific LLM service.</li><li>If you expect it to have real-time, up-to-the-second information about the world (most have knowledge cut-off dates).</li><li>If you need deep domain expertise that wasn’t prevalent in its training data.</li></ul><h3>Agents: The Adventurous Sous-Chef (LLMs with Tools &amp; Goals) 🕵️‍♂️🍳</h3><p><em>The LLM that can now chop, sauté, and fetch ingredients from the pantry (APIs, databases) based on a recipe (plan).</em></p><p><strong>When to use:</strong></p><ol><li><strong>LLM Knowledge Isn’t Enough:</strong> The task requires information beyond the LLM’s training data, such as:</li></ol><ul><li><strong>Real-time information:</strong> Stock prices, weather, current news.</li><li><strong>Proprietary/Private Data:</strong> Accessing a user’s calendar, a company’s internal knowledge base, customer order status from a database.</li><li><strong>Specific Computations:</strong> Using a calculator for math, a code interpreter for execution.</li></ul><p><strong>2. Need to Take Actions in External Systems:</strong> The AI needs to do more than just generate text; it needs to interact with other software or systems (e.g., send an email, book a flight, update a CRM).</p><p><strong>3. Goal-Oriented Reasoning and Planning (Light):</strong> You want the AI to break down a high-level goal into smaller, executable steps, choose appropriate tools for those steps, and potentially adapt its plan based on tool outputs.</p><p><strong>4. Dynamic Interaction:</strong> The task involves a sequence of interactions where the AI needs to gather information, act, observe the result, and then decide the next step.</p><p><strong>Examples:</strong></p><ol><li><strong>AI assistant for meeting management:</strong> Reads emails to identify meeting requests, checks your calendar availability (via API), proposes times to attendees (via email API), and books the meeting once confirmed.</li></ol><p><strong>2. Travel bot:</strong> User: “Find me a flight to Paris next Tuesday, a hotel near the Eiffel Tower for 3 nights, and a good vegan restaurant.” The agent would:</p><ul><li>Query flight APIs.</li><li>Query hotel booking APIs with location filters.</li><li>Query a restaurant review API or perform a web search for vegan restaurants.</li><li>Present options to the user and await confirmation before booking.</li></ul><p><strong>3. Automated customer support (Tier 1–2):</strong> Fetches order status from a database, initiates a return process via an e-commerce API, updates customer records in a CRM, and escalates to a human if the query is too complex or requires empathy.</p><p><strong>4. Simple research assistant:</strong> Given a topic, it could use a search engine tool to find relevant articles, then use the LLM’s summarization capability on the retrieved content.</p><p><strong>When to Panic:</strong></p><ul><li>If you grant an agent powerful tools (e.g., ability to delete files, spend money, send mass emails) without robust safeguards, confirmation steps, and monitoring.</li><li>If the agent’s planning or error-handling capabilities are not sophisticated enough for the complexity of the task, leading to it getting stuck, making mistakes, or performing unintended actions.</li><li>If the cost of LLM calls and tool usage escalates uncontrollably due to inefficient planning or loops.</li><li>If the security of the tools and APIs it interacts with is compromised.</li></ul><h3>Multi-Agent Workflows: The AI Kitchen Brigade 👩‍🍳👨‍🍳🤖</h3><p><em>A team of specialized AI agents (and potentially humans) collaborating, each with distinct roles and expertise, orchestrated to achieve a complex, overarching goal.</em></p><p><strong>When to use:</strong></p><ol><li><strong>Highly Complex Problems Requiring Diverse Expertise:</strong> The problem is too large or multifaceted for a single agent (even with tools) to handle effectively. It benefits from a “divide and conquer” strategy where different agents specialize in sub-tasks.</li><li><strong>No Single, Well-Defined Workflow; Emergent Solutions:</strong> The path to the solution isn’t clear from the outset and may require dynamic collaboration, negotiation, or iterative refinement between agents. The solution “emerges” from their interactions.</li><li><strong>Tasks Benefiting from Specialization and Parallelization:</strong> Breaking down a task allows different agents to work on parts simultaneously or sequentially, with each agent optimized for its specific role (e.g., a “researcher” agent, a “writer” agent, a “validator” agent).</li><li><strong>Need for Robustness through Redundancy or Different Perspectives:</strong> Multiple agents might tackle the same sub-problem using different methods, and their results can be compared or combined for a more robust outcome (e.g., one agent analyzes data, another critiques the analysis).</li><li><strong>Simulating Complex Systems or Human Team Dynamics:</strong> Useful for modeling scenarios where multiple autonomous entities interact (e.g., economic simulations, wargaming, organizational behavior studies).</li></ol><p><strong>Examples:</strong></p><p><strong>Automated scientific research:</strong></p><ul><li><em>Agent 1 (Literature Reviewer):</em> Scans academic databases for relevant papers based on a hypothesis.</li><li><em>Agent 2 (Data Extractor):</em> Parses retrieved papers/datasets for specific data points.</li><li><em>Agent 3 (Hypothesis Tester/Simulator):</em> Runs simulations or statistical analyses on extracted data.</li><li><em>Agent 4 (Summarizer/Report Writer):</em> Compiles findings into a coherent report.</li><li><em>Agent 5 (Critique Agent):</em> Reviews the report for flaws, biases, or gaps.</li></ul><p><strong>Advanced Coding Co-pilot System:</strong></p><ul><li><em>Agent 1 (Requirements Analyst):</em> Clarifies user requirements for a new software feature.</li><li><em>Agent 2 (Architect):</em> Designs the high-level structure and API contracts.</li><li><em>Agent 3 (Coder):</em> Writes the code for individual modules based on specs.</li><li><em>Agent 4 (Tester):</em> Generates and runs unit/integration tests.</li><li><em>Agent 5 (Refactor/Optimizer):</em> Suggests improvements to code quality or performance.</li></ul><p><strong>Sophisticated Personalized Shopping Assistant:</strong></p><ul><li><em>Agent 1 (Style Profiler):</em> Understands user’s preferences, past purchases, and stated needs.</li><li><em>Agent 2 (Product Scout):</em> Scans multiple e-commerce sites for items matching the profile.</li><li><em>Agent 3 (Review Analyzer):</em> Summarizes reviews for shortlisted products, looking for specific pros/cons relevant to the user.</li><li><em>Agent 4 (Price Negotiator/Deal Finder — future concept):</em> Looks for best prices or available discounts.</li><li><em>Agent 5 (Outfit Coordinator):</em> Suggests how new items could fit with existing wardrobe items (if data is available).</li><li><strong>Complex Financial Analysis &amp; Reporting:</strong> One agent gathers market data, another performs technical analysis, a third focuses on fundamental analysis, a fourth drafts a report, and a fifth ensures compliance checks.</li></ul><p><strong>When to Panic:</strong></p><ul><li>If the orchestration and communication overhead between agents becomes more complex than the problem itself.</li><li>If debugging becomes a nightmare because it’s hard to trace which agent caused an error or how their interactions led to an undesirable outcome.</li><li>If the cost of running multiple, potentially sophisticated agents simultaneously become prohibitive.</li><li>If there’s no clear mechanism for resolving conflicts or disagreements between agents, leading to deadlock or suboptimal results.</li><li>If emergent behaviors arise that are harmful or counterproductive, and the system lacks control mechanisms.</li></ul><h3>TL; DR Table: Choosing Your AI Appliance</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatawrapper.dwcdn.net%2FlBlgk%2F1%2F&amp;display_name=Datawrapper&amp;url=https%3A%2F%2Fdatawrapper.dwcdn.net%2FlBlgk%2F1%2F&amp;image=https%3A%2F%2Fdatawrapper.dwcdn.net%2FlBlgk%2Fplain-s.png%3Fv%3D1&amp;type=text%2Fhtml&amp;schema=dwcdn" width="600" height="762" frameborder="0" scrolling="no"><a href="https://medium.com/media/86bf4263c6a169dc546dce1672bb3b68/href">https://medium.com/media/86bf4263c6a169dc546dce1672bb3b68/href</a></iframe><h3>Key Considerations for ALL Approaches:</h3><p>Before diving in with any AI/ML solution, always consider:</p><ol><li><strong>Problem Definition:</strong> Is the problem clearly defined? What does success look like? Can it be broken down?</li><li><strong>Data Availability &amp; Quality:</strong> Do you have the right data? Is it clean, representative, and unbiased?</li><li><strong>Ethical Implications &amp; Bias:</strong> Could this system perpetuate or amplify biases? What is the fairness, accountability, and transparency (FAT) considerations?</li><li><strong>Cost vs. Benefit:</strong> What is the development, deployment, and maintenance costs? Do the potential benefits justify these costs? Consider compute, API calls, and human oversight.</li><li><strong>Scalability &amp; Maintainability:</strong> Can the solution scale with your needs? How easy will it be to update, retrain, and maintain?</li><li><strong>Explainability &amp; Interpretability:</strong> How important is it to understand <em>why</em> the AI made a particular decision or prediction? This varies greatly by application.</li><li><strong>Security &amp; Privacy:</strong> How will you protect sensitive data used by or generated by the AI? If using agents, how do you secure tool access?</li></ol><h3>Final Thoughts 💡</h3><p>When in doubt, ask yourself these refined questions:</p><ol><li><strong>Is my core challenge about finding patterns or making predictions from well-structured, historical data?</strong> (Likely ML 🍞)</li><li><strong>Is my core challenge about understanding, generating, or transforming human language or code, based on broad existing knowledge?</strong> (Likely LLM 🗣️)</li><li><strong>Does my LLM-based task need to access real-time/private data or perform actions in other systems to achieve a specific goal?</strong> (Likely Agent 🕵️‍♂️)</li><li><strong>Is my problem incredibly complex, requiring diverse specialized skills, dynamic collaboration, and the orchestration of multiple, distinct processes to achieve an overarching objective?</strong> (Likely Multi-Agent 👩‍🍳)</li></ol><p>And if all else fails, remember the golden rule: <strong>Start simple.</strong> Don’t jump to a multi-agent brigade when a reliable toaster (or even a good old-fashioned if-else statement) will do the job. And <em>never</em> let an unverified LLM order your lunch if you have dietary restrictions — you might end up with a “philosophically intriguing but entirely inedible quantum sandwich.” 🥪</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2fa1f111b66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating LLM-Based Applications: Fail Fast, Test Hard]]></title>
            <link>https://vikram-ai.medium.com/evaluating-llm-based-applications-fail-fast-test-hard-7b4a39bc5dab?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b4a39bc5dab</guid>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Mon, 05 May 2025 11:36:36 GMT</pubDate>
            <atom:updated>2025-05-05T11:36:36.394Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*yx5Mw4FrxszlWzphWN7yZQ.png" /></figure><p>“Sure, your AI demo looks great in PowerPoint. So does my dog playing fetch. He’s still unemployed.”</p><p>Whipping up an AI demo? Piece of cake. A teenager with GPT access and some TikTok tutorials can nail it in a weekend. But turning that slick prototype into something that <em>doesn’t</em> crash and burn in the real world? Ah, that’s where the real fun (and occasional facepalming) starts.</p><p>Here’s a no-BS guide to brutally testing your LLM apps. Find the fails now, <em>before</em> your customers do.</p><h3>1. Reality Check: Real Users = Real Mess</h3><p><strong>Why:</strong> Your perfect demo data? It’s fake. Real users hit you with typos, slang, weird jargon, and requests that make zero sense.</p><p><strong>Real Talk:</strong> That fancy bank chatbot aced the canned questions but choked on, “Why did you charge me twice, you thieves?!” It took digging through tons of angry customer tickets just to make it <em>slightly</em> less clueless.</p><p><strong>How:</strong></p><ul><li>Talk to <em>actual humans</em>. Ask them for their weirdest, most annoying problems. Spend time curating your test dataset. It is worth spending the time on understanding what the real environment for the service is going to look like.</li><li>Remember the 80/20 rule: It takes only 20% of the effort for the service to solve 80% of the problems. Ask yourself, what level of threshold are you looking for. If you are looking for 99%, LLMs may not be the answer (yet) but if you are looking for 90%, you might already be there.</li><li>Above all, find a way to measure how good your LLM system is. Without a metric (even a very approximate one) you are flying blind and building by going with the vibe.</li></ul><h3>2. Automate Testing or Drown Trying</h3><p><strong>Why:</strong> If you’re testing manually, you’re already behind. It’s basically just crossing your fingers and hoping for the best.</p><p><strong>Real Talk:</strong> Alaska Airlines automated tests for thousands of flight searches. Good thing, too — before that, it nearly suggested phantom flights. Imagine <em>that</em> customer service call.</p><p><strong>How:</strong></p><ul><li>Hook up testing tools (promptfoo, DeepEval, Inspect etc.) right into your deployment pipeline (CI/CD).</li><li>Write tests that automatically scream “WRONG!” when the LLM spits out garbage. You can use LLMs generously to generate such test cases at scale.</li></ul><h3>3. Feedback Loops: You Don’t Know Everything</h3><p><strong>Why:</strong> Your tests catch some stuff. Reality finds new ways to break things.</p><p><strong>Real Talk:</strong> HumanLoop let support agents literally click a “That’s dumb” button in Slack when the AI messed up. Surprise! The AI got smarter.</p><p><strong>How:</strong></p><ul><li>Log it all: the prompt, the response, the user’s pain.</li><li>Set up alerts. Know <em>immediately</em> when the AI starts talking crazy again.</li></ul><h3>4. Structured Output: Free Text is the Enemy</h3><p><strong>Why:</strong> Letting the LLM ramble freely sounds flexible. It’s actually a nightmare waiting to happen when you need to <em>use</em> that output reliably.</p><p><strong>Real Talk:</strong> A news app cut errors way down (like 90%!) just by forcing the LLM to stick to predefined categories. Suddenly, “Other” was everyone’s favorite.</p><p><strong>How:</strong></p><ul><li>Force structured output (like JSON). Use schemas.</li><li>Reject <em>anything</em> that doesn’t fit the mold. Be ruthless.</li><li>Add validations on the outputs.</li></ul><h3>5. Human Review: Keep AI Honest</h3><p><strong>Why:</strong> Humans catch the subtle stuff — the “technically right but totally wrong” answers that need common sense.</p><p><strong>Real Talk:</strong> A telco had people review 5% of chatbot answers. They caught gems like telling someone to “unplug and re-plug” their mobile data. Genius.</p><p><strong>How:</strong></p><ul><li>Send a slice of AI responses for human review regularly. Call it quality control (or an AI roast).</li><li>Use their feedback to make your <em>automated</em> tests smarter.</li></ul><h3>6. Use an LLM to Judge Another LLM (Seriously)</h3><p><strong>Why:</strong> Because sometimes, fighting fire with fire works. Having one LLM check another’s work can catch things humans miss at scale.</p><p><strong>Real Talk:</strong> An e-commerce team had a second LLM rate product suggestions from their main AI. It flagged brilliant ideas like suggesting batteries for bananas.</p><p><strong>How:</strong></p><ul><li>Use frameworks built for this (like Phoenix LLM Evals, Opik etc.).</li><li>But <em>always</em> benchmark the judge LLM against human ratings first. Don’t trust AI blindly.</li></ul><h3>7. Attack Your Own App: Adversarial &amp; Fuzz Testing</h3><p><strong>Why:</strong> Hackers aren’t nice. They <em>try</em> to break things. You need to beat them to it.</p><p><strong>Real Talk:</strong> Dropbox found hackers using sneaky backspace tricks to bypass security. Testing for weird attacks like this stopped them cold.</p><p><strong>How:</strong></p><ul><li>Throw weird stuff at your LLM constantly (weird prompts, code snippets, pure gibberish).</li><li>Use tools (like OpenAI’s Red Team Toolkit) to automate the attacks.</li></ul><h3>8. Know When to Fold ’Em: Sometimes AI Isn’t the Answer</h3><p><strong>Why:</strong> Maybe the LLM just isn’t working for this problem. No amount of tweaking will fix it. Maybe you need better data, a simpler approach, or… <em>no AI at all</em>.</p><p><strong>Real Talk:</strong> It’s okay to admit defeat. Fix the root cause, find better data, or just ditch the AI part. Your sanity (and budget) might thank you.</p><h3>Conclusion: Fail Hard, Learn Fast</h3><p>LLMs <em>can</em> be amazing, but they’re headache machines unless you test them relentlessly. Most production fails aren’t fancy AI problems; they’re wishful thinking or cutting corners on testing.</p><p>Bottom line: If you need rock-solid reliability, LLMs alone won’t cut it. Test hard, learn from the fails (try to laugh?), and find problems fast. It’s way cheaper and less embarrassing that way.</p><p>Happy testing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b4a39bc5dab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM architecture to solve 90% of your problems]]></title>
            <link>https://vikram-ai.medium.com/llm-architecture-to-solve-90-of-your-problems-983969135f42?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/983969135f42</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Wed, 30 Apr 2025 12:57:59 GMT</pubDate>
            <atom:updated>2025-05-02T12:42:02.566Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s face it: most business LLM applications don’t need a cast of thousands of agents. If you’re a software engineer looking to augment your pipeline with an LLM, you want a reliable, repeatable workflow that won’t make your data engineers weep. You might think that’s obvious — apparently, it isn’t. I keep seeing teams over-engineer their stack with multi-agent frameworks when a single call (or none at all) would do. Welcome to 2025: “the year of agents,” right? Let me simplify things.</p><p>After wrestling with half-baked docs, brittle APIs, and the occasional existential crisis, I’ve learned that 90% of use cases boil down to the same four ingredients. It’s so straightforward you’ll wonder why you ever started building a digital Rube-Goldberg machine in the first place.</p><h4>Here’s the secret sauce:</h4><h4>1. Breaking Things Down into Smaller Pieces</h4><blockquote>“How do you eat an elephant? One bite at a time. How do you solve an LLM task? One subtask at a time.”</blockquote><p>Before you dump a 200-page PDF on an LLM and pray for coherence, ask yourself: how would you delegate this to a human, deterministically? (Hint: “Just summarize it, please” isn’t the answer.)</p><ol><li><strong>Extract structure first</strong></li></ol><p>Pull out the table of contents, headings or metadata with a lightweight parser (e.g., Apache Tika, PyPDF2). Identify logical chunks — chapters, sections, or slides.</p><p><strong>2. Summarize hierarchically</strong></p><pre># Example: hierarchical summarization with LangChain <br>from langchain.document_loaders <br>import PyPDFLoader <br>from langchain.text_splitter <br>import RecursiveCharacterTextSplitter <br>from langchain.chains.summarize <br>import load_summarize_chain <br>from langchain.llms <br>import OpenAI  <br><br>loader = PyPDFLoader(&quot;big_report.pdf&quot;) <br>docs = loader.load() <br>splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200) <br>chunks = splitter.split_documents(docs)  # First pass: summarize each chunk <br>summaries = [] <br><br>for chunk in chunks:     <br>    summary = OpenAI().summarize(chunk.page_content)     <br>    summaries.append(summary)  # Second pass: aggregate summaries <br>    final_summary = load_summarize_chain(OpenAI(), chain_type=&quot;map_reduce&quot;).run(summaries) <br>print(final_summary)</pre><p>These are just a couple of ideas. Use your domain expertise and get creative about how you can make things easy and predictable for the LLM. By breaking it down you get predictable, manageable inputs — and save tokens.</p><p><strong>2. Use Structured Outputs</strong></p><p>LLMs are like interns: they do better with clear instructions and templates. Define JSON schemas while calling your APIs (<a href="https://ai.google.dev/gemini-api/docs/structured-output?lang=python">Generate structured output with the Gemini API | Google AI for Developers</a> is a great starting point). Some LLM providers like Anthropic may not support this and you might want to use a third-party library that handles the structured outputs for you. You might want to use either langchain, pydantic-ai or simpler libraries like instructor (<a href="https://github.com/jxnl/instructor">https://github.com/jxnl/instructor</a>) to get outputs you can actually use, not just admire.</p><p>This also has the added benefit of reducing hallunications by a huge margin.</p><p><strong>Example</strong></p><p>If you are trying to get information on an image, you might want to use a schema like,</p><pre>LIST_OF_CLASSES = [&quot;cat&quot;, &quot;dog&quot;, &quot;car&quot;, &quot;tree&quot;]<br><br>class ImageSummary(BaseModel):<br>  title: str = Field(..., description=&quot;Title of the image&quot;)<br>  description: str =Field(..., description=&quot;Description of the image&quot;)<br>  classification: Literal[*LIST_OF_CLASSES] = Field(..., description=&quot;Label&quot;)<br>  confidence: int = Field(..., description=&quot;Confidence score between 1-100&quot;)</pre><p>This schema will let you access all the fields in the schema for each output generated. You can build your business logic on top of this and make your data pipelines predictable.</p><h4>3. Tool Calling</h4><p>Sometimes your LLM needs a library card — and a smartphone.</p><p><strong>Up-to-date data</strong>: let the model call your internal REST API rather than hoping it remembers yesterday’s prices.</p><p><strong>Web search</strong>: integrate a search API or Bing wrapper to pull live facts.</p><p><strong>Deterministic transforms</strong>: math, date calculations, or database lookups should happen outside the LLM.</p><pre>import openai<br><br>response = openai.ChatCompletion.create(<br>    model=&quot;gpt-4o-mini&quot;,<br>    messages=[{&quot;role&quot;:&quot;user&quot;,&quot;content&quot;:&quot;What&#39;s my portfolio value?&quot;}],<br>    functions=[{<br>        &quot;name&quot;:&quot;get_portfolio_value&quot;,<br>        &quot;description&quot;:&quot;Fetch the user&#39;s current portfolio value&quot;,<br>        &quot;parameters&quot;: {<br>            &quot;type&quot;:&quot;object&quot;,<br>            &quot;properties&quot;:{&quot;user_id&quot;:{&quot;type&quot;:&quot;string&quot;}},<br>            &quot;required&quot;:[&quot;user_id&quot;]<br>        }<br>    }]<br>)<br>if response.choices[0].message.get(&quot;function_call&quot;):<br>    args = json.loads(response.choices[0].message[&quot;function_call&quot;][&quot;arguments&quot;])<br>    portfolio = get_portfolio_value(user_id=args[&quot;user_id&quot;])<br>    # Feed back into the conversation…</pre><h4><strong>4. Reflection/Validation</strong></h4><p>Trust, but verify. Add validation steps to catch those “creative” LLM outputs before they hit production. For example, check if the numerical values make sense.</p><p>Do this both during processing (e.g., data validation) and during build</p><ul><li>Python unit tests with [deepeval](<a href="https://github.com/confident-ai/deepeval">https://github.com/confident-ai/deepeval</a>)</li><li>For language agnostic testing: [promptfoo](<a href="https://github.com/jackd/Promptfoo">https://github.com/jackd/Promptfoo</a>)</li><li>or [inspect](<a href="https://github.com/inspect-ai/inspect">https://github.com/inspect-ai/inspect</a>))</li></ul><p><strong>Example</strong>:<br>- Validate that all required fields are present<br>- Check that scores are within expected ranges<br>- Build up a suite of test cases to catch regressions</p><h4><strong>When NOT to Use Multi-Agent Architectures</strong></h4><p>Multi-agent systems are cool… until you’re debugging why Agent #7 is arguing with Agent #3 about the meaning of “summary.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MIlNHCP2FQBM_I1aLlyHPg.png" /></figure><p>Unless you’re building the next AI-powered Mars rover, keep it simple. Most business problems just need the below workflow with the above ingredients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3QpXjqssH-1rfTZ-f3LDMQ.png" /></figure><h4><strong>Conclusion</strong></h4><p>You don’t need a multi-agent circus for the vast majority of LLM use cases. Start by:</p><ol><li><strong>Decomposing</strong> the problem.</li><li><strong>Structuring</strong> your outputs.</li><li><strong>Calling</strong> tools for facts and logic.</li><li><strong>Validating</strong> everything before it goes live.</li></ol><p>Keep it simple. Keep it reliable. And if you ever catch yourself debugging a dozen arguing agents, you’ve probably taken a wrong turn at “simple.” Now go forth and build!</p><p>— -</p><p>Now go forth and build! And remember: if you ever find yourself debugging a conversation between 12 agents, you might have taken a wrong turn at “simple.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=983969135f42" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying LLM Apps on AWS Lambda]]></title>
            <link>https://vikram-ai.medium.com/deploying-llm-apps-on-aws-lambda-3247943dad51?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/3247943dad51</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[production]]></category>
            <category><![CDATA[llm-applications]]></category>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Mon, 21 Apr 2025 01:05:50 GMT</pubDate>
            <atom:updated>2025-05-09T00:42:17.881Z</atom:updated>
            <content:encoded><![CDATA[<h4>Integrating LLMs into Enterprise Infrastructure with AWS Lambda</h4><p>In today’s rapidly evolving AI landscape, integrating Large Language Models (LLMs) into existing enterprise infrastructure presents a significant challenge. Organizations face a dilemma: they need the cutting-edge capabilities of modern LLMs, but their existing systems may not easily accommodate the Python ecosystem where most LLM advancements occur.</p><p>This article explores a practical solution to this challenge: using AWS Lambda functions as a bridge between your existing infrastructure and Python-based LLM capabilities. We’ll cover the technical details of building, deploying, and consuming a serverless LLM API that can process various document types while minimizing impact on your current systems.</p><h4>The Integration Challenge</h4><p>Enterprise systems often run on established platforms like Java, .NET, or legacy systems that may not easily integrate with Python-based LLM libraries. However, Python has become the de facto standard for AI development due to:</p><p>1. Rapid innovation: Most cutting-edge LLM libraries appear first in Python<br>2. Rich ecosystem: Tools like LangChain, LlamaIndex, and instructor-embedding are Python-first<br>3. Direct model access: Python SDKs for OpenAI, Anthropic, and other providers are more mature<br>4. Community support: The AI community primarily shares solutions in Python</p><p>Rather than rewriting existing systems or waiting for LLM libraries to mature in other languages, AWS Lambda offers an elegant solution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZH_l7m29AlNr1c_MmdIQQ.png" /></figure><h4>Building a Serverless LLM Document Processor</h4><p>Let’s create a Python-based LLM service on AWS Lambda that can:</p><p>1. Process various document types (JSON, images, PDFs)<br>2. Apply LLM capabilities for analysis<br>3. Return structured responses<br>4. Offer a secure, scalable HTTP API endpoint</p><h4>Project Structure</h4><p>```<br>.<br>├── src/<br>│ ├── lambda_functions/ # AWS Lambda function implementations<br>│ │ └── document_processor/ # Document processing function<br>│ ├── utils/ # Shared utility modules<br>│ └── config/ # Configuration modules<br>├── scripts/ # Deployment and utility scripts<br>├── terraform/ # Infrastructure as code<br>├── Dockerfile # Container definition for Lambda<br>└── lambda_entry.py # Local Lambda testing script<br>```</p><h4>Setting Up the FastAPI App</h4><p>We’ll use FastAPI to create a web service that AWS Lambda can run. This approach gives us the best of both worlds:</p><ol><li>Local development using standard web frameworks</li><li>Serverless deployment through AWS Lambda</li></ol><pre>```python<br>from fastapi import FastAPI<br>from mangum import Mangum<br>app = FastAPI()<br>@app.post(&quot;/process&quot;)<br>async def process_document(request: dict):<br> # … LLM processing logic …<br> return {&quot;result&quot;: &quot;Processed!&quot;}<br>handler = Mangum(app)<br>```</pre><p>This sets up a FastAPI application with a `/process` endpoint that accepts document data and returns structured LLM analysis.</p><h4>Containerizing the Lambda Function</h4><p>Using Docker containers for Lambda deployment simplifies dependency management. Here’s the essential part of the `Dockerfile`:</p><pre>```dockerfile<br>FROM public.ecr.aws/lambda/python:3.10<br>COPY src/ ${LAMBDA_TASK_ROOT}/src/<br>COPY requirements.txt lambda_entry.py ${LAMBDA_TASK_ROOT}/<br>RUN pip install - no-cache-dir uv<br>RUN uv venv /opt/venv<br>ENV VIRTUAL_ENV=/opt/venv<br>ENV PATH=&quot;/opt/venv/bin:$PATH&quot;<br>RUN uv pip install -r ${LAMBDA_TASK_ROOT}/requirements.txt<br>ENV PYTHONPATH=${LAMBDA_TASK_ROOT}<br>CMD [ &quot;src.lambda_functions.document_processor.app.handler&quot; ]</pre><h4>Deploying the Solution</h4><p>A single Bash script handles secrets, builds, pushes, and deploys:</p><p><a href="https://github.com/Vikramardham/lambdas/blob/main/scripts/deploy_and_secrets.sh">lambdas/scripts/deploy_and_secrets.sh at main · Vikramardham/lambdas</a></p><p>This script will:<br> — Set up AWS Secrets Manager with your API keys (from environment variables)<br> — Build and push the Docker image to ECR<br> — Deploy the Lambda/API Gateway using Terraform</p><h4>Deployment Process with Terraform</h4><p>Terraform is used to provision all AWS resources in a repeatable, infrastructure-as-code manner. The key resources defined include:</p><p>- <strong>aws_lambda_function</strong>: Deploys your Docker image as a Lambda function.<br>- <strong>aws_apigatewayv2_api</strong>: Creates an HTTP API Gateway to expose your Lambda as a REST endpoint.<br>- <strong>aws_secretsmanager_secret</strong>: Stores your API keys securely.<br>- <strong>IAM roles and policies</strong>: Grant the Lambda function permissions to access secrets and write logs.</p><p>Check the terraform files here: <a href="https://github.com/Vikramardham/lambdas/tree/main/terraform">lambdas/terraform at main · Vikramardham/lambdas</a></p><p>With this setup, you can easily update, destroy, or redeploy your infrastructure with a single command:</p><pre>```bash<br>cd terraform<br>terraform apply<br>```</pre><p>For the full code and deployment scripts, see the [GitHub repository](<a href="https://github.com/Vikramardham/lambdas">https://github.com/Vikramardham/lambdas</a>).</p><h4>Testing and Using the API</h4><p>After deployment, get your API endpoint from the AWS Console or Terraform output. Then, test your Lambda-powered LLM API with:</p><pre>```bash<br>curl -X POST &#39;&lt;your-api-endpoint&gt;/process&#39; \<br> -H &#39;Content-Type: application/json&#39; \<br> -d &#39;{&quot;document_data&quot;: &quot;{\&quot;test\&quot;: \&quot;data\&quot;}&quot;, &quot;document_type&quot;: &quot;json&quot;, &quot;instructions&quot;: &quot;Summarize the test data&quot;}&#39;<br>```</pre><p>You can also use Postman or any HTTP client to interact with your endpoint.</p><h4>Benefits of This Approach</h4><p>1. Clean separation: Your existing infrastructure remains untouched<br>2. Python ecosystem access: Use the latest LLM libraries and techniques<br>3. Scalability: AWS Lambda auto-scales based on usage<br>4. Cost-effectiveness: Pay only for what you use<br>5. Security: API keys are stored securely in AWS Secrets Manager<br>6. Infrastructure as code: Terraform ensures consistent deployments<br>7. Containerization: Docker simplifies dependency management</p><p>— -</p><h4><strong>Integration with Existing Systems</strong></h4><p>Existing applications can easily consume the API in any language. For example, in Java:</p><pre>```java<br>HttpRequest request = HttpRequest.newBuilder()<br> .uri(URI.create(apiUrl + &quot;/process&quot;))<br> .header(&quot;Content-Type&quot;, &quot;application/json&quot;)<br> .POST(HttpRequest.BodyPublishers.ofString(payload))<br> .build();<br>```</pre><h4>Conclusion</h4><p>This approach lets you harness the power of Python-based LLMs in any enterprise environment through AWS Lambda functions. By creating a clean API boundary, you decouple your LLM implementation from existing systems while providing a stable interface for integration.</p><p>— -</p><p>By leveraging the serverless paradigm, we can introduce powerful LLM capabilities to legacy systems without disrupting existing workflows or requiring major rewrites. This pragmatic approach enables organizations to benefit from the latest AI advancements regardless of their current technology stack.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3247943dad51" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM Applications in Java with LangChain4J]]></title>
            <link>https://vikram-ai.medium.com/llm-application-in-java-with-langchain4j-9a74c3012896?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a74c3012896</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[production]]></category>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Mon, 21 Apr 2025 00:51:02 GMT</pubDate>
            <atom:updated>2025-04-21T00:51:20.231Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Introduction</strong></p><p>Integrating Large Language Models (LLMs) into existing enterprise infrastructure presents significant challenges, particularly when dealing with Python runtimes. While Python is the go-to language for AI development, it often creates friction when integrating with established Java or .NET environments that many enterprises rely on.</p><p>In this article, I’ll walk through how to build a Java-based service that extracts structured data from PDFs using LLMs. This approach allows for seamless integration with existing Java ecosystems while leveraging the power of modern AI models.</p><p><strong>The Challenge with Python Runtimes</strong></p><p>Many organizations face these challenges when trying to integrate Python-based LLM solutions:</p><p>1. **Deployment complexity**: Managing Python dependencies in production environments<br>2. **Performance overhead**: Python’s runtime performance compared to compiled languages<br>3. **Integration barriers**: Connecting Python services with Java/JVM-based systems<br>4. **Security considerations**: Adding new runtime environments increases attack surface<br>5. **Maintenance burden**: Supporting multiple language ecosystems</p><p><strong>Our Solution: A Java-Based LLM Service</strong></p><p>By building our solution in Java, we create a service that:</p><p>- Integrates natively with existing Java infrastructure<br>- Provides stronger type safety<br>- Scales efficiently within existing deployment pipelines<br>- Reduces operational complexity</p><p>Let’s dive into how we built this system.</p><p><strong>Architecture Overview</strong></p><p>Our service extracts structured data from PDFs (and potentially other document types) using LLMs. Here’s the high-level architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yK-n5yrKqlzILZmUG9RD7g.png" /></figure><h4><strong>Implementation Details</strong></h4><p><strong>Core Components</strong></p><p>Our service consists of these main components:</p><p>1. **<strong>Document Processing Service</strong>**: Handles document uploads and preprocessing<br>2. **<strong>LLM Service</strong>**: Manages interactions with different LLM providers<br>3. **<strong>Schema Repository</strong>**: Defines the structure for extracted data<br>4. **<strong>Storage Service</strong>**: Manages saving and retrieving extracted information</p><p><strong>Technology Stack</strong></p><p>- **<strong>Java 11+</strong>**: Core language<br>- **<strong>Spring Boot</strong>**: Web framework and dependency injection<br>- **<strong>LangChain4J</strong>**: Java library for LLM interactions<br>- **<strong>Jackson</strong>**: JSON parsing and processing<br>- **<strong>JUnit</strong>**: Testing framework</p><p><strong>The LLM Service Implementation</strong></p><p>The LLM service is the heart of our application. It abstracts away the details of different LLM providers, presenting a unified interface for document processing.</p><pre>public class LLMService {<br> private final ChatLanguageModel chatModel;<br> private final ObjectMapper objectMapper;<br> private static final String OUTPUT_DIR = &quot;outputs&quot;;<br> private static final String SUMMARY_PROMPT = &quot;Analyze this PDF document and extract all relevant information…&quot;;<br>public enum InputType {<br> PDF, JSON, TEXT, IMAGE<br> }<br>public LLMService(String apiKey, String modelType) {<br> this.objectMapper = new ObjectMapper();<br> this.chatModel = createChatModel(apiKey, modelType);<br> createOutputDirectory();<br> }<br>private ChatLanguageModel createChatModel(String apiKey, String modelType) {<br> switch (modelType.toLowerCase()) {<br> case &quot;gemini&quot;:<br> return GoogleAiGeminiChatModel.builder()<br> .apiKey(apiKey)<br> .modelName(&quot;gemini-2.0-flash&quot;)<br> .temperature(0.0)<br> .build();<br> case &quot;anthropic&quot;:<br> return AnthropicChatModel.builder()<br> .apiKey(apiKey)<br> .modelName(&quot;claude-3–7-sonnet-20250224&quot;)<br> .temperature(0.0)<br> .build();<br> case &quot;openai&quot;:<br> return OpenAiChatModel.builder()<br> .apiKey(apiKey)<br> .modelName(&quot;gpt-4o&quot;)<br> .temperature(0.0)<br> .build();<br> default:<br> throw new IllegalArgumentException(&quot;Unsupported model type: &quot; + modelType);<br> }<br> }<br>public String processDocument(Path inputPath, InputType inputType, String schemaType) {<br> try {<br> if (inputType != InputType.PDF) {<br> throw new IllegalArgumentException(&quot;Only PDF files are supported in this version&quot;);<br> }<br>UserMessage userMessage = createPdfMessage(inputPath);<br>// Create chat request with JSON response format and schema<br> JsonSchema schema;<br> if (schemaType.equals(&quot;inspection&quot;)) {<br> schema = JsonSchemas.jsonSchemaFrom(InspectionReport.class).get();<br> } else if (schemaType.equals(&quot;new&quot;)) {<br> schema = JsonSchemas.jsonSchemaFrom(NewReport.class).get();<br> } else {<br> throw new IllegalArgumentException(&quot;Unsupported schema type: &quot; + schemaType);<br> }<br>ResponseFormat responseFormat = ResponseFormat.builder()<br> .type(ResponseFormatType.JSON)<br> .jsonSchema(schema)<br> .build();<br>ChatRequest chatRequest = ChatRequest.builder()<br> .messages(userMessage)<br> .responseFormat(responseFormat)<br> .build();<br>// Get response from model<br> ChatResponse response = chatModel.chat(chatRequest);<br> String jsonResponse = response.aiMessage().text();<br>// Save the JSON response to a file<br> saveJsonResponse(jsonResponse, inputPath);<br>return jsonResponse;<br>} catch (Exception e) {<br> throw new RuntimeException(&quot;Error processing document: &quot; + e.getMessage(), e);<br> }<br> }<br>}</pre><p><strong>The Challenge with Anthropic Models</strong></p><p>One interesting challenge we encountered was that Anthropic Claude models don’t support the standard JSON response format in LangChain4J. While Google’s Gemini and OpenAI’s GPT models work fine with the approach above, Anthropic models require a different approach.</p><p>To work around this limitation, we created a specialized implementation for Anthropic models using the AI Service pattern with tools, which allows for more structured extraction:</p><pre><br> public class AnthropicService {<br> private final AnthropicChatModel chatModel;<br> private final ObjectMapper objectMapper;<br> private static final Logger logger = LoggerFactory.getLogger(AnthropicService.class);<br>// Service interface with tools extraction<br> @AIService<br> interface InspectionExtractorService {<br> @Tool(&quot;extractReportId&quot;)<br> String extractReportId(String text);<br> <br> @Tool(&quot;extractLocationDetails&quot;)<br> InspectionReport.Location extractLocationDetails(String text);<br> <br> @Tool(&quot;extractDeviceInfo&quot;)<br> InspectionReport.Device extractDeviceInfo(String text);<br> <br> // Additional extraction tools…<br> <br> InspectionReport extractInspectionReport(<br> @PdfContent Path pdfPath, <br> @SystemMessage String instructions<br> );<br> }<br> <br> private final InspectionExtractorService extractorService;<br> <br> public AnthropicService(String apiKey) {<br> this.objectMapper = new ObjectMapper();<br> this.chatModel = AnthropicChatModel.builder()<br> .apiKey(apiKey)<br> .modelName(&quot;claude-3–7-sonnet-20250224&quot;)<br> .temperature(0.0)<br> .build();<br> <br> // Initialize the AI service with tools<br> this.extractorService = AIServiceFactory.builder()<br> .chatLanguageModel(chatModel)<br> .build()<br> .create(InspectionExtractorService.class);<br> }<br> <br> public InspectionReport processDocument(Path pdfPath) {<br> logger.info(&quot;Processing document with Anthropic model: {}&quot;, pdfPath);<br> String instructions = &quot;Extract all relevant information from this inspection report PDF.&quot;;<br> <br> try {<br> InspectionReport report = extractorService.extractInspectionReport(pdfPath, instructions);<br> logger.info(&quot;Successfully extracted report data&quot;);<br> return report;<br> } catch (Exception e) {<br> logger.error(&quot;Error extracting report data&quot;, e);<br> throw new RuntimeException(&quot;Failed to process document with Anthropic model&quot;, e);<br> }<br> }<br>}</pre><p><strong>Deployment and Integration</strong></p><p>Our service is designed for flexible deployment options:</p><ol><li>**<strong>Standalone service</strong>**: Can be deployed as a standalone Spring Boot application<br>2. **<strong>Embedded component</strong>**: Can be integrated directly into existing Java applications<br>3. **<strong>AWS Lambda</strong>**: Can be packaged for serverless deployment (covered in a future article)</li></ol><p><strong>2. Performance and Scalability</strong></p><p>The Java-based approach offers several advantages:</p><p>1. **<strong>Efficient resource usage</strong>**: Java’s mature JIT compiler and memory management<br>2. **<strong>Horizontal scaling</strong>**: Easily deployed across multiple instances<br>3. **<strong>Load balancing</strong>**: Works with standard load balancing solutions</p><p><strong>Future Improvements</strong></p><p>In our next article, we’ll explore:</p><p>1. **Spring AI integration**: Using Spring’s official AI abstraction layer for even cleaner integration<br>2. **AWS Lambda deployment**: Packaging the service for serverless deployment<br>3. **Advanced document processing**: Handling more complex document types<br>4. **Caching and performance optimizations**: Reducing API costs and improving throughput</p><p><strong>Conclusion</strong></p><p>Building an LLM-powered document processing service in Java offers a powerful solution for enterprises looking to leverage AI capabilities without disrupting their existing infrastructure. By using Java libraries like LangChain4J, we can create robust, scalable services that integrate seamlessly with enterprise systems.</p><p>While we encountered challenges like the lack of structured output support with Anthropic models in LangChain4J, we were able to develop effective workarounds. In our next article, we’ll explore how Spring AI can provide an even more elegant solution for working with various LLM providers.</p><p>The code for this project is available on GitHub, so you can adapt it to your own document processing needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a74c3012896" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building AI Applications in Java with Spring AI: A Developer’s Intro]]></title>
            <link>https://vikram-ai.medium.com/building-llm-applications-in-java-with-spring-ai-a-developers-guide-1fe3c7154d59?source=rss-90840cdc6a37------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fe3c7154d59</guid>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[production]]></category>
            <dc:creator><![CDATA[Vikram]]></dc:creator>
            <pubDate>Mon, 21 Apr 2025 00:37:16 GMT</pubDate>
            <atom:updated>2025-05-14T22:34:48.859Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using spring-ai to build with LLMs</h4><h4>Introduction</h4><p>In the rapidly evolving world of AI, Java developers often feel like they’re bringing a sword to a laser fight. While Python has been the dominant language for AI development, Java developers shouldn’t feel left out. Spring AI brings the power of LLMs to the Java ecosystem — finally, Java developers can join the AI party without switching languages.</p><p>This article will guide you through building a Java application using Spring AI, focusing on key concepts like structured outputs and tool calling. We’ll use a weather service application as our example, demonstrating how to integrate OpenAI/Anthropic models into your Java applications.</p><p><strong>Why Spring AI for Java Developers?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_3ubo0T6fHvuFeXkrAbeGw.png" /></figure><p>For Java developers, working with AI traditionally meant:<br>- Learning Python<br>- Dealing with complex integration layers<br>- Managing multiple dependencies<br>- Handling type safety issues</p><p>Spring AI changes this by:<br>- Providing a familiar Spring-based development experience<br>- Offering type-safe APIs<br>- Supporting enterprise-grade features<br>- Integrating seamlessly with existing Java applications</p><p>Further spring-ai has a great community support and has quickly evolving features. For most LLM based applications, one only needs 3 things below and spring-ai offers them all.</p><ol><li>Ability to call any major LLM provider (OpenAI/Anthropic/Google-Gemini)</li><li>Structured outputs for reliable data pipelines</li><li>Function calling to call internal APIs/databases.</li></ol><p>Everything else is secondary in my opinion. Most robust LLM applications can be built just with these three features in my experience working on 10s of AI projects. You might rarely need to build multiple agents but I will leave that discussion for some other time.</p><p><strong>Setting Up Your Project</strong></p><p>First, let’s look at the basic project structure and dependencies:</p><pre>&lt;properties&gt;<br> &lt;java.version&gt;17&lt;/java.version&gt;<br> &lt;spring-ai.version&gt;1.0.0-M7&lt;/spring-ai.version&gt;<br>&lt;/properties&gt;<br>&lt;dependencies&gt;<br> &lt;dependency&gt;<br> &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;<br> &lt;artifactId&gt;spring-boot-starter-web&lt;/artifactId&gt;<br> &lt;/dependency&gt;<br> &lt;dependency&gt;<br> &lt;groupId&gt;org.springframework.ai&lt;/groupId&gt;<br> &lt;artifactId&gt;spring-ai-starter-model-anthropic&lt;/artifactId&gt;<br> &lt;/dependency&gt;<br> &lt;dependency&gt;<br> &lt;groupId&gt;org.springframework.ai&lt;/groupId&gt;<br> &lt;artifactId&gt;spring-ai-starter-model-openai&lt;/artifactId&gt;<br> &lt;/dependency&gt;<br>&lt;/dependencies&gt;</pre><p><strong>Configuring the LLM models</strong></p><p>Spring AI supports multiple LLM providers. Here’s how to configure them in your `application.yml`:</p><pre>spring:<br>  ai:<br>    model:<br>      chat: openai<br>    openai:<br>      api-key: ${OPENAI_API_KEY}<br>      chat:<br>        options:<br>          model: gpt-4o<br>    anthropic:<br>      api-key: ${ANTHROPIC_API_KEY}<br>      chat:<br>        <br>        options:<br>          model: claude-3-5-sonnet-20241022</pre><p><strong>Structured Outputs: Making LLM Responses Type-Safe</strong></p><p>One of the most powerful features of Spring AI is structured outputs. Instead of dealing with raw text responses, you can define Java classes that represent your expected output structure (`model/Response.java`). Structured outputs make your pipelines reliable and robust. You always know the expected output structure and can design your data pipeline with confidence. Most modern LLMs return structured outputs with great reliability and I would never build an LLM without structured outputs.</p><pre>public class Response {<br> private String content;<br>public Response(String content) {<br> this.content = content;<br> }<br>// Getters and setters<br>}</pre><p><strong>Using structured outputs in your controller :</strong></p><p>controller/ChatController.java</p><pre>@RestController<br>@RequestMapping(&quot;/api/chat&quot;)<br>public class ChatController {<br> private final ChatClient chatClient;<br>public ChatController(ChatClient chatClient) {<br> this.chatClient = chatClient;<br> }<br>@GetMapping(&quot;/query&quot;)<br> public Response getAnswer(@RequestParam String query) {<br> return chatClient.prompt()<br> .user(query)<br> .call()<br> .entity(Response.class);<br> }<br>}</pre><p><strong>Tool Calling: Extending LLM Capabilities</strong></p><p>Tool calling allows your LLM to interact with external services and APIs. Let’s look at how to implement a weather service tool (`service/WeatherSerivce.java`):</p><pre>@Service<br>public class WeatherService {<br> @Tool(description = &quot;Get the weather forecast for a given city&quot;)<br> public Response getWeather(String city) {<br> return new Response(&quot;Sunny with a chance of rain in &quot; + city);<br> }<br>}</pre><p><strong>Configuring the ChatClient with tools (`config/ChatConfig.java`):</strong></p><pre>@Configuration<br>public class ChatConfig {<br> @Bean<br> public ChatClient chatClient(ChatClient.Builder builder, WeatherService weatherService) {<br> return builder<br> .defaultTools(weatherService)<br> .build();<br> }<br>}</pre><p><strong>How It All Works Together</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*37jlSPawT7tZBRcOZcxG8A.png" /></figure><p><strong>Best Practices and Tips</strong></p><p>1. **Type Safety**: Always use structured outputs to ensure type safety and better error handling<br>2. **Tool Descriptions**: Write clear, descriptive tool documentation to help the LLM understand when to use them<br>3. **Error Handling**: Implement proper error handling for tool calls and LLM responses<br>4. **Testing**: Write comprehensive tests for both the LLM integration and tool implementations<br>5. **Monitoring**: Use Spring’s observability features to monitor LLM usage and performance</p><p><strong>Conclusion</strong></p><p>Spring AI brings the power of LLMs to Java developers in a familiar, type-safe way. By leveraging structured outputs and tool calling, you can build robust, enterprise-grade AI applications without leaving the Java ecosystem.</p><p>The example weather service application demonstrates how to:<br>- Configure multiple LLM providers<br>- Implement structured outputs<br>- Create and use tools<br>- Handle responses in a type-safe manner</p><p>With Spring AI, Java developers can now easily integrate AI capabilities into their applications while maintaining the benefits of the Java ecosystem.</p><p><strong>Next Steps</strong></p><p>1. Explore more advanced features like:<br> — Retrieval Augmented Generation (RAG)<br> — Vector databases integration<br> — Multi-modal capabilities<br>2. Check out the [Spring AI documentation](<a href="https://docs.spring.io/spring-ai/reference/1.0/api/tools.html">https://docs.spring.io/spring-ai/reference/1.0/api/tools.html</a>) for more details<br>3. Join the Spring AI community for support and updates</p><p><strong>References</strong></p><p>- [Spring AI Documentation](<a href="https://docs.spring.io/spring-ai/reference/1.0/api/tools.html">https://docs.spring.io/spring-ai/reference/1.0/api/tools.html</a>)<br>- [Spring AI GitHub Repository](<a href="https://github.com/spring-projects/spring-ai">https://github.com/spring-projects/spring-ai</a>)<br>- [OpenAI API Documentation](<a href="https://platform.openai.com/docs/api-reference">https://platform.openai.com/docs/api-reference</a>)<br>- [Anthropic API Documentation](<a href="https://docs.anthropic.com/claude/reference">https://docs.anthropic.com/claude/reference</a>)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fe3c7154d59" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>