<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[InfinitGraph - Medium]]></title>
        <description><![CDATA[InfinitGraph mission is to pioneer the future of AI, crafting cutting edge GenAI products that redefine possibilities. - Medium]]></description>
        <link>https://medium.com/infinitgraph?source=rss----12ed059500cc---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>InfinitGraph - Medium</title>
            <link>https://medium.com/infinitgraph?source=rss----12ed059500cc---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 22:50:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/infinitgraph" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building Your Own MCP Server with FastMCP]]></title>
            <link>https://medium.com/infinitgraph/building-your-own-mcp-server-with-fastmcp-7d9cc73a1062?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/7d9cc73a1062</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[mcps]]></category>
            <category><![CDATA[llm-agent]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Mon, 03 Nov 2025 13:03:51 GMT</pubDate>
            <atom:updated>2025-11-05T08:12:03.996Z</atom:updated>
            <content:encoded><![CDATA[<p>In <a href="https://medium.com/infinitgraph/model-context-protocol-mcp-a-beginners-guide-d7977b52570a">our last article</a>, we unpacked the Model Context Protocol; we highlighted how it acts as a USB-C port for AI applications, and how it provides a standardized, robust bridge between large language models and the external world of tools, data, and memory. If you haven’t read that article yet, I really recommend checking it out as it lays the groundwork for everything we’re about to dive into.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GICslGIOr0qIgOUVwi2NsA.jpeg" /><figcaption>Figure 01–Overview of the MCP Architecture</figcaption></figure><p>Now knowing the foundations of MCP is only half the battle. The real magic happens when you learn how to integrate existing MCPs into your applications, and even better, when you start building your own 😋.</p><p>In this article, we’ll explore how to leverage existing MCPs and build our own with the use-in plus MCP framework. We’ll roll up our sleeves and get hands-on with <strong>FastMCP</strong>:<strong> </strong>a fast, Pythonic framework that makes creating production-ready MCP servers surprisingly simple.</p><p>Let’s get start right away!</p><h3><strong>1. Integrating Existing MCP Servers</strong></h3><p>One of the most powerful aspects of the Model Context Protocol is that you don’t have to build everything from scratch. Just like you can install a package in Python or a plugin in your browser, you can integrate already built MCP servers to instantly give your AI new superpowers.</p><h4>1.1. Where to Find MCP Servers</h4><p>Over 5,000 public MCP servers are now live (as of mid-2025), and more emerge daily. Developers from all over the world are building and sharing servers for a huge variety of tasks.</p><p>There are many <strong>MCP hubs</strong> at the time of writing this article, but the two common sources are: <a href="https://github.com/modelcontextprotocol/servers"><strong>Servers GitHub repository</strong></a> by model context protocol and <a href="https://mcpservers.org/"><strong>Awesome MCP servers</strong></a> (this one has cool search and filtering options!). In these hubs, you’ll find open-source servers for all sorts of things, such as:</p><ul><li><strong>Productivity Apps: </strong>Servers for interacting with Notion, Slack, or Google Calendar.</li><li><strong>Developer Tools: </strong>Servers that can talk to GitHub, GitLab, or your local file system.</li><li><strong>Data &amp; APIs: </strong>Servers that wrap public APIs like Wikipedia, weather services, or stock market data.</li></ul><p>These hubs are your first stop when you think,<em> </em>“<em>I wonder if someone has already built an MCP server for…</em>”.</p><h4>1.2. <strong>Popular MCP Servers You Can Use Today</strong></h4><p>Let’s look at a few popular, ready-to-use MCP servers:</p><ul><li><a href="https://developers.notion.com/docs/mcp"><strong>NotionMCP — </strong></a>Provides tools to create, read, and update pages and databases in your Notion workspace.</li><li><a href="https://github.com/github/github-mcp-server"><strong>GitHubMCP — </strong></a>Exposes tools for managing repositories, creating issues, reviewing pull requests, and searching code.</li><li><a href="https://mcpservers.org/servers/proofmath-owner/ai-filesystem-mcp"><strong>FileSystemMCP — </strong></a>A fundamental server that gives the LLM secure access to read and write files on your local machine.</li></ul><h4>1.3. <strong>How to Integrate an MCP Server</strong></h4><p>The setup instructions are usually provided by the creators. They can vary depending on whether you’re connecting through an IDE, a server, or another environment, so it’s best to always check them out and follow them step by step.</p><p>To make things concrete, let’s walk through connecting the<strong> GitHub MCP server</strong> to <strong>VS Code Copilot.</strong></p><blockquote>This MCP server is going to make your workflow a lot smoother (you’ll thank me later).</blockquote><p><strong>1.3.1. Why use GitHub MCP?</strong></p><p>The <strong>GitHub MCP </strong>allows you to read your repositories, manage issues and pull requests, analyze code, and even automate workflows… all via natural-language commands.</p><p>For example, from within VS Code’s Copilot Chat in agent mode, you can ask the AI to create or update issues, list &amp; merge pull requests, read specific files or commits, and give you insights on security alerts or build failures.</p><p><strong>1.3.2 How to set it up in VS Code</strong></p><p>If you go to the <strong>README file</strong> of the <a href="https://github.com/github/github-mcp-server"><strong>official GitHub MCP repository</strong></a>, you will find all the instructions you need to set it up in VS Code and other tools.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tbtr1uYasENnn_BJNaOvvA.png" /><figcaption>Figure 02 — GitHub MCP Server Readme</figcaption></figure><p>In this article, I’ll walk you through how to set up the remote GitHub MCP server in VS Code. If it’s more convenient, you can also set up the local version, the process is very similar.</p><blockquote><strong>Important:</strong> Make sure you’re running <strong>Visual Studio Code v1.101 or later</strong>, which is required for full remote MCP support and OAuth authentication.</blockquote><p>Here are the instructions you need to follow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SdTnNq8LhgxqeUk5vaSwvQ.png" /><figcaption>Figure 03 — Overview of the Steps of Setting GitHub MCP Server</figcaption></figure><ol><li><strong>Open VS Code:</strong> Launch Visual Studio Code.</li><li><strong>Go to the Extensions View:</strong> Click on the Extensions icon in the sidebar, or press Ctrl+Shift+X (or Cmd+Shift+X on Mac).</li><li><strong>Filter to MCP Servers:</strong> In the extensions search bar, click the filter icon and choose <strong>MCP Server</strong> from the dropdown.</li><li><strong>Enable the MCP Marketplace:</strong> If this is your first time, you’ll need to enable the MCP Servers Marketplace. VS Code will prompt you; just follow the on-screen instructions.</li><li><strong>Install the GitHub MCP Server:</strong> In the search bar, type “GitHub” and pick the <strong>GitHub MCP server</strong> from the results. Click <strong>Install</strong>.</li><li><strong>Verify the Server Is Configured:</strong> Open the Command Palette (Ctrl+Shift+P / Cmd+Shift+P) and run <strong>MCP: List Servers</strong>. You should see a server named github in the list.</li><li><strong>Switch to Agent Mode in Copilot Chat:</strong> Open the Copilot Chat panel, and make sure the mode is set to <strong>Agent</strong>.</li><li><strong>Check Tools in Agent Mode:</strong> Once the server is added, go back to your Copilot Chat (Agent mode), click the <strong>Tools</strong> button, and you should see GitHub-related tools (like issue listing, PR management, etc.)</li><li><strong>Test the Integration: </strong>Try sending a simple agent request like <em>“Show me the latest commit on my repo”.</em></li></ol><p>I asked Copilot (in Agent mode) to give me a quick summary of the latest commit in <a href="https://github.com/adimidania/ai-fairness-playbooks">my current project</a>. The agent immediately figured out it needed to use the GitHub MCP server and called the right tools to get the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PXbgHxQxkt4FfVQxtOpulw.png" /><figcaption>Figure 04 — VS Code Copilot using GitHub MCP Server Tools</figcaption></figure><p>The final results were precise and clearly mirrored the updates from my latest commit! You can easily imagine how a tool like this could help you quickly understand what your teammates have been working on without going through the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ay9VyEhgdZNinYXhLU6SdQ.png" /><figcaption>Figure 05 — GitHub MCP Server Final Response</figcaption></figure><h3><strong>3. Building Your Own MCP</strong></h3><p>Now that you’ve seen how to integrate existing MCP servers, it’s time for the fun part: <strong>building your own</strong>. Maybe you want to create an MCP server that makes different AI tools available for marketing: one that drafts email campaigns, another that analyzes social media metrics, and a third that recommends ad creatives based on past performance. By building this MCP, you can expose all those capabilities directly to your chatbot, allowing it to choose the right tool based on the user’s request.</p><p>Fortunately, you don’t have to worry a lot about the technical implementation. <strong>FastMCP</strong> will make it incredibly simple for you to spin up a fully functional MCP server in minutes.</p><h4>3.1. What is FastMCP?</h4><p><strong>FastMCP </strong>is a developer-friendly framework that abstracts away the complexity of the Model Context Protocol. It handles all the boilerplate like request routing, schema validation, and message formatting so you can focus on writing your tool logic.</p><p>I like to think of it like <strong>FastAPI for MCP servers</strong>: it lets you define endpoints (called “tools”), describe their input and output schemas, and FastMCP does the rest including serving them over HTTP and exposing them to any MCP-compatible client (like Copilot or OpenAI’s Playground).</p><p>In this article, we are going to build a very simple MCP that has a bunch of tools, you can find the full code on my<strong> </strong><a href="https://github.com/adimidania/medium-articles/tree/main/Building%20Your%20Own%20MCP%20Server%20with%C2%A0FastMCP"><strong>GitHub repository for Medium Articles</strong></a><strong>.</strong></p><h4>3.2. <strong>Installation</strong></h4><p>Make sure you have <strong>Python 3.8 or higher</strong> installed.</p><p>Then create a folder for your project and install FastMCP globally (this is very important!):</p><pre>pip install fastmcp</pre><p>That’s all you need to get started.</p><h4>3.3. Create your MCP server file</h4><p>Create a new file called mcp_server.py and start with this:</p><pre>from fastmcp import FastMCP<br><br># Initialize the MCP server<br><br>mcp = FastMCP(&quot;Simple MCP Server&quot;)</pre><p>This line creates your MCP server and gives it a name.</p><p>Everything you define next your functions, your logic will be registered under this server.</p><h4>3.4. Add your first tool</h4><p>Let’s start with a simple (and classic!) calculator tool that performs basic arithmetic:</p><pre>@mcp.tool<br><br>def calculator(operation: str, a: float, b: float) -&gt; str:<br>  &quot;&quot;&quot;<br>  Perform basic arithmetic operations.<br>  &quot;&quot;&quot;<br><br>  operations = {<br>  &quot;add&quot;: lambda x, y: x + y,<br>  &quot;subtract&quot;: lambda x, y: x - y,<br>  &quot;multiply&quot;: lambda x, y: x * y,<br>  &quot;divide&quot;: lambda x, y: x / y if y != 0 else None,<br>  }<br><br>  if operation not in operations:<br>    return &quot;Error: Invalid operation&quot;<br><br>  result = operations[operation](a, b)<br>  return f&quot;Result: {result}&quot;</pre><p>The @mcp.tool decorator tells FastMCP that this function should be exposed as a <strong>tool</strong>.</p><p>When an LLM connects to your MCP server, it will see the calculator as one of the available tools it can call.</p><h4>3.5. Add your first tool</h4><p>You can define as many tools as you want for your use case. Here are some examples from our demo server:</p><ul><li><strong>Text Analyzer</strong> — Counts words, sentences, and paragraphs in any text.</li><li><strong>Random Number Generator</strong> — Returns a random integer between two limits.</li><li><strong>String Operations</strong> — Converts text to uppercase, reverses it, or counts words.</li><li><strong>Math Utilities</strong> — Performs operations like sum, average, min, and max on a list of numbers.</li><li><strong>JSON Formatter</strong> — Validates and pretty-prints JSON strings.</li><li><strong>List Operations</strong> — Sorts, reverses, and filters lists.</li></ul><p><strong>Each function follows the same pattern:</strong> Define a Python function → decorate it with @mcp.tool → return a clean string or JSON response.</p><h4>3.6. Run your MCP server</h4><p>At the bottom of your file, add:</p><pre>if __name__ == &quot;__main__&quot;:<br>  mcp.run()</pre><p>Then start your server with:</p><pre>python mcp_server.py</pre><p>Your MCP server is now live! It’s ready to be connected to any <strong>MCP-compatible client</strong>, such as <strong>VS Code</strong>, <strong>Copilot Chat</strong>, or even your own AI assistant built on OpenAI’s API.</p><h4>3.7. Connect your MCP Server to VS Code</h4><p>Once your server is running, the next step is to <strong>connect it to VS Code</strong> so that tools defined in your mcp_server.py can be used directly inside Copilot Chat.</p><p><strong>Locate your MCP configuration file</strong></p><p>VS Code stores MCP server configurations in:</p><pre>%APPDATA%\Code\User\mcp.json</pre><p>You can open it directly from VS Code by pressing:</p><pre>Ctrl + P → type &gt;Open MCP Config File</pre><p>Or manually navigate to:</p><pre>C:\Users\&lt;YourUsername&gt;\AppData\Roaming\Code\User\mcp.json</pre><p><strong>Add your custom server entry</strong></p><p>Inside mcp.json, define your local MCP server. Here’s an example that connects your <strong>fastmcp-demo</strong> server:</p><pre>{<br>  &quot;servers&quot;: {<br>    &quot;fastmcp-demo&quot;: {<br>      &quot;type&quot;: &quot;command&quot;,<br>      &quot;command&quot;: &quot;python&quot;,<br>      &quot;args&quot;: [&quot;C:/Users/alaad/Downloads/simple MCP/mcp_server.py&quot;]<br>    }<br>  },<br>  &quot;inputs&quot;: []<br>}</pre><blockquote><strong><em>💡 Tip:</em></strong><em> Make sure you use forward slashes / in the path, even on Windows — this prevents parsing errors.</em></blockquote><p><strong>Restart VS Code</strong></p><p>After saving the file, <strong>restart VS Code</strong> so it can detect the new MCP server. You should see logs like these in the Output tab (under “Copilot”):</p><pre>[info] Starting server fastmcp-demo<br>[info] Connection state: Running</pre><p>If you see this, your server is running successfully!</p><p><strong>Open Copilot Chat</strong></p><p>Now, open the <strong>Copilot Chat</strong> panel (Ctrl + I) and type something anything that would trigger one of the tools. As you can see here, my VS Code Copilot has access to the fastmcp demo MCP, and can trigger the tools.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xyhT0S-_VLitv_BcWMAfXQ.png" /><figcaption>Figure 06 — FastMCP Demo Response</figcaption></figure><h3><strong>3. Conclusion</strong></h3><p>And that’s it! You’ve made it to the end of this article. You learned how to integrate existing MCP servers, create your own from scratch, and connect it to VS Code.</p><p>This is only the beginning. Once you’ve got the basics down, you can start building more advanced MCPs, ones that talk to your APIs, automate parts of your workflow, and more.</p><p>If you have any questions, feedback, or suggestions for future articles, feel free to reach out; I’d love to hear your thoughts and see what you’re building with MCP!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d9cc73a1062" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/building-your-own-mcp-server-with-fastmcp-7d9cc73a1062">Building Your Own MCP Server with FastMCP</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Good Is Your Web Search? Comparing Three Approaches for LLM Agents]]></title>
            <link>https://medium.com/infinitgraph/how-good-is-your-web-search-comparing-three-approaches-for-llm-agents-886ec4821af9?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/886ec4821af9</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[gemini]]></category>
            <dc:creator><![CDATA[Sarra Bouzoul]]></dc:creator>
            <pubDate>Sun, 26 Oct 2025 16:27:35 GMT</pubDate>
            <atom:updated>2025-10-26T16:27:34.517Z</atom:updated>
            <content:encoded><![CDATA[<h3>1. Why Search Quality Matters for LLM Agents</h3><p>Imagine you ask an AI agent about a breaking news event. If the agent relies on outdated or low-quality search results, you could end up with misleading information or worse, completely wrong facts. That’s the double-edged sword of connecting large language models with web search. On the one hand, search gives LLMs a way to stay current and expand their knowledge beyond training data. On the other hand, the quality of the answers depends heavily on the quality of the search itself. Bad search means bad answers, no matter how good the model is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xvbqZVYAg83E4cQkLpKDtw.jpeg" /></figure><p>This raises an important point: if we want trustworthy AI agents, we can’t just assume web search will “work.” We need to carefully evaluate how well different search approaches perform, especially in high-stakes scenarios like health queries or fast-moving news.</p><p>In this article, we walk you through a comparison of three different web search approaches to see which one works best for our use cases. The same evaluation method can also help you assess search quality in your own projects.</p><h3>2. The Web Search Approaches Being Evaluated</h3><p>In this section, we compare three different approaches to equipping LLMs with real-world information through web search:</p><h4>2.1. Exa tools</h4><p><a href="https://exa.ai/">Exa</a> is a web search and content-extraction engine designed specifically for AI systems. Unlike traditional search engines, which focus on keyword ranking and ads, Exa delivers fresh, high-quality web content in a structured way that LLMs can directly use. Its goal is to ground AI outputs in factual, up-to-date information and reduce hallucinations.</p><p><strong>Key Capabilities</strong></p><ul><li><strong>Semantic &amp; keyword search modes</strong>: Supports neural (semantic) search, keyword matching, and hybrid phrase filtering, with automatic mode selection.</li><li><strong>Rich retrieval outputs</strong>: Returns clean full text, highlighted snippets, optional summaries, and metadata (e.g., publish date, source). Can scan pages and the links on them.</li><li><strong>Direct answers with citations</strong>: Can synthesize a multi-source answer instead of just a ranked list of links.</li><li><strong>Similar-page discovery</strong>: Expand research by retrieving semantically related content from a seed URL.</li><li><strong>Freshness &amp; filtering controls</strong>: Updated hourly, the index incorporates domain and path-level filters, as well as date-range restrictions, to prioritize reliable and current sources.</li><li><strong>Research &amp; workflow support</strong>: Enables grounded Q&amp;A, monitoring of specific domains, expansion of literature, and ingestion of clean text into data pipelines without custom scraping.</li></ul><h4>2.2 Google Grounded Search Tool</h4><p><a href="https://ai.google.dev/gemini-api/docs/google-search">Grounding with Google Search</a> is a tool that can enable the Gemini model to connect directly to live web content across all supported languages, allowing it to extend beyond its training cutoff and deliver answers that are more accurate, up-to-date, and transparently sourced. By anchoring responses in real-world data, it minimizes hallucinations, handles queries about ongoing events and the latest developments, and builds user trust through clear citations.</p><p><strong>Key Capabilities</strong></p><ul><li><strong>Live search grounding: </strong>Gemini can decide when to call Google Search, retrieve real-time results, and weave them into responses.</li><li><strong>URL context grounding:</strong> Instead of searching broadly, Gemini can directly consume content from specified URLs, ensuring focus on trusted sources.</li><li><strong>Citations and metadata: </strong>Each answer includes linked sources, query details, and optional search suggestions for clarity.</li><li><strong>Accuracy and reliability: </strong>Grounded outputs reduce hallucinations and keep answers current with recent events.</li><li><strong>User-friendly experience: </strong>Inline citations and clickable suggestions make results easier to trust and explore.</li></ul><h4>2.3. Custom Tool:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*baybZFR09MN8qVmFlWIfQg.jpeg" /></figure><p>This tool is a web-based Retrieval-Augmented Generation (RAG) service that delivers advanced web search, retrieval, and context-aware response generation. It integrates modular services for search, scraping, ranking, and LLM interactions, with a strong emphasis on credibility and trustworthiness through robust scoring and authoritative source prioritization.</p><p><strong>Key Capabilities</strong></p><ul><li><strong>Query understanding</strong>: Interprets and expands user queries with synonyms and intent cues to improve recall without losing focus.</li><li><strong>Web search &amp; retrieval</strong>: Finds, ranks, and selects relevant sources through custom search and scoring pipelines.</li><li><strong>Content cleaning</strong>: Retrieves HTML pages responsibly (rate limits, timeouts), removes boilerplate like ads or menus, extracts the main text with readability heuristics, normalizes it, deduplicates fragments, and filters by language.</li><li><strong>Authority &amp; relevance scoring</strong>: Evaluates sources using keyword/semantic match, content quality, domain reputation, recency, and diversity, ensuring the most reliable information surfaces.</li><li><strong>Retrieval-Augmented Generation</strong>: Builds grounded context windows from top-ranked chunks and combines them with LLMs to generate accurate, transparent, and citation-backed responses.</li></ul><h3>3. The Evaluation Framework</h3><p>When we connect an AI agent to the web, the first thing it does is search. The user sends a query, the agent fetches results, and then those results shape the final answer. But here’s the catch: not all search results are equal. Some come from credible sources, some are spammy SEO blogs, and others might be outdated or even factually wrong. So before we trust an agent, we need a way to measure whether the search results it brings back are good, good enough, or simply bad.</p><p>In traditional NLP tasks like summarization or translation, we have standard benchmarks and metrics. But for a niche task like evaluating web search results for LLM agents, things are less clear; we don’t have custom metrics and benchmarks for this task. But luckily, we do have <strong>human evaluation</strong> and <strong>LLM-as-a-judge evaluation</strong>.</p><h4>3.1. Human Evaluation</h4><p>Human evaluation is exactly what it sounds like: <strong>people reviewing the results and deciding whether they are good or not.</strong> Humans are excellent at this because they can pick up on nuance that machines often miss. For example, a person can quickly detect if a result is just clickbait, if it comes from a low-credibility site, or if it answers the query in a shallow way. Humans are also better at spotting contextual errors, like a result that’s factually correct but not relevant to the actual question.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ibk8VjLL4z2T55qDD8tTdw.jpeg" /></figure><p><strong>Why does it matter?</strong> Some of you might ask. Well, If we want reliable search agents, we need human judgment as the <strong>gold standard</strong>. Humans can see beyond surface-level matches and recognize when information is misleading, biased, or incomplete.</p><p><strong>Pros of Human Evaluation</strong></p><ul><li>Captures nuance and subtle signals that automated methods miss.</li><li>Can detect spam, low-quality, or manipulative websites.</li><li>Provides a trustworthy way for comparing search agents.</li></ul><p><strong>Cons of Human Evaluation</strong></p><ul><li><strong>Expensive: </strong>Requires hiring or recruiting evaluators.</li><li><strong>Slow:</strong> Not scalable when evaluating thousands of queries.</li><li><strong>Subjective:</strong> Different people may score the same result differently.</li></ul><p><strong>How could we do it in practice:<br></strong> Imagine we want to test how three search agents answer the query:<br> <em>“What are the latest AI safety regulations in the EU?”</em></p><p>We bring in human evaluators and ask them to rate each result on a set of metrics. Each evaluator assigns a score, and we average the ratings to get a sense of which agent is doing the best job. This gives us a human-grounded baseline.</p><p>It’s also crucial to have a sufficient number of evaluators with <strong>real domain knowledge</strong>; otherwise, the results can skew too much toward individual biases or blind spots. Research recommends <strong>at least 3–5 evaluators per item</strong> to detect consensus or flag disagreements, and for specialized or high-stakes domains, tapping experts (like legal or medical professionals) helps ensure that judgment calls are informed and reliable [<a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC11437138/">Source</a>]. Using too few or non-expert evaluators risks overly subjective or inconsistent scoring, especially when the evaluation task demands a nuanced understanding of credibility, context, or depth.</p><h4>3.2. LLM-as-a-Judge Evaluation</h4><p>Of course, human evaluation doesn’t scale well. That’s where LLM-as-a-judge comes in. The idea is simple: instead of a person rating the results, we ask an LLM (like GPT-4) to play the judge. Given the query and the results, the model scores them based on criteria we define, similar to what a human would do.</p><p>Why it matters: This method is <strong>fast, cheap, and scalable</strong>. You can run it over thousands of queries in minutes. And studies (like <a href="https://www.databricks.com/blog/LLM-auto-eval-best-practices-RAG">Databricks’ experiments</a>) have shown that well-prompted LLM judges often agree with humans over 80% of the time.</p><p><strong>Pros of LLM-as-a-Judge</strong></p><ul><li>Scales easily to thousands of results.</li><li>Much cheaper and faster than hiring human evaluators.</li></ul><p><strong>Cons of LLM-as-a-Judge</strong></p><ul><li>Reliability depends on the model: weaker models = weaker judgments.</li><li>Can inherit biases from training data.</li><li>Risk of overreliance: Some human-specific signals (like subtle spam tactics) can be missed.</li></ul><p><strong>How could we do it in practice:<br></strong>We simply design a prompt that tells the LLM exactly what we want it to evaluate. For example:</p><p><em>“You are an evaluator. A user asked: ‘What are the latest AI safety regulations in the EU?’ Below are some search results. For each result, rate it from 1 to 5 on these dimensions: Relevance, Depth, Credibility, Accuracy, and Usefulness. Then provide a short justification for each score.”</em></p><p>This way, the LLM acts like a scalable human evaluator, applying the same framework across hundreds of queries.</p><p>In the end, <strong>human evaluation remains the golden standard</strong>, it’s the most reliable way to judge the quality of search results. But it’s not always practical: running large-scale tests with human raters quickly becomes expensive and slow. That’s where LLMs become very useful, since they can scale to thousands of queries at a fraction of the cost.</p><h3>4. The Evaluation Plan</h3><h4>4.1. Dataset Preparation</h4><p>The first step was to prepare a dataset of queries and answers containing the three following categories to test different aspects of web search performance:</p><ul><li><strong>General Queries (Baseline Accuracy)</strong> — Straightforward fact-based questions used to test whether the system can reliably provide correct and relevant information.<br><em>Example: “Who is the current president of France?”</em></li><li><strong>Complex / Multi-Step Queries (Reasoning &amp; Decomposition)</strong> — Questions that require combining information from multiple sources, reasoning over several steps, or decomposing the query into sub-problems.<br><em>Example: “Compare the climate policies of the EU and the US, highlighting differences in renewable energy targets.”</em></li><li><strong>Temporal Queries (Recency &amp; Freshness)</strong> — Time-sensitive questions that test whether the system can retrieve the latest information rather than outdated results.<br> <em>Example: “What are the most recent AI safety regulations passed in the EU?”</em></li></ul><p>This dataset allowed us to test many aspects of the web search approaches we had, and it was used in both human and LLM evaluation.</p><h4><strong>4.2. Human Evaluation Plan</strong></h4><p>Using this dataset, we applied a structured evaluation framework with five key dimensions, each rated on a <strong>1–5 scale</strong> with clear anchors to reduce subjectivity:</p><ul><li><strong>Relevance</strong> — Does the result directly address the query?</li><li><strong>Depth (Surface vs. Insightful)</strong> — Does it go beyond surface-level information?</li><li><strong>Source Credibility</strong> — Is the source reliable and up to date?</li><li><strong>Factual Accuracy</strong> — Are the statements correct and verifiable?</li><li><strong>Usefulness for the User</strong> — Would this result help a real user?</li></ul><p>Evaluators rated each result across these dimensions, and the scores were averaged to provide a comparative measure of quality.</p><h4>4.3. LLM-as-a-Judge Evaluation</h4><p>The approach is straightforward: we leveraged <strong>GPT-4o</strong> and provided it with the same dataset of queries and results used in human evaluation, along with the same defined evaluation metrics. The model is then instructed to assign a score for each dimension and generate a brief rationale explaining its judgment.</p><p>Here’s the used prompt:</p><pre>You are an evaluator tasked with assessing the quality of search results for a user query.<br>For each result, rate it on the following five dimensions using the scales provided. After assigning the scores, provide a short justification for your choices.<br>User Query: the user&#39;s query<br>Search Result: the search results given by the web search engine<br>Evaluation Criteria:<br>1. Relevance: Does the result directly address the query?<br>* 1 = Not relevant at all<br>* 2 = Partially related but not useful<br>* 3 = Somewhat relevant but misses key points<br>* 4 = Mostly relevant and useful<br>* 5 = Highly relevant, directly answers the query<br>2. Depth (Surface vs. Insightful): Does the result go beyond surface-level information? Does it provide explanations, reasoning, or details?<br>* 1 = Very shallow, just definitions/snippets<br>* 3 = Moderate detail, covers some aspects<br>* 5 = Deep, insightful, covers multiple aspects thoroughly<br>3. Source Credibility: Is the source reliable, fresh, authoritative, and trustworthy?<br>* 1 = Unreliable / unknown / spammy site / outdated<br>* 3 = Neutral or user-generated (forums, blogs) / moderately recent<br>* 5 = Highly credible (official, academic, reputable news, authoritative sources) and up-to-date<br>4. Factual Accuracy: Are the statements factually correct and verifiable?<br>* 1 = Contains factual errors<br>* 3 = Mostly correct, with minor issues<br>* 5 = Factually correct, no issues found<br>5. Usefulness for the User: Would this result help a real user answer their question?<br>* 1 = Not useful<br>* 3 = Somewhat useful, needs more checking<br>* 5 = Very useful, could be used directly<br>Your Output Format:<br>* Relevance: \[score] - \[short justification]<br>* Depth: \[score] - \[short justification]<br>* Source Credibility: \[score] - \[short justification]<br>* Factual Accuracy: \[score] - \[short justification]<br>* Usefulness: \[score] - \[short justification]</pre><p>Evaluating all entries in the dataset (~48 questions) took around 7 minutes with GPT-4o, which shows how scalable this method is.</p><h3>5. Results and Insights</h3><h4>5.1. Overall Performance</h4><p>Across the 15 queries evaluated, the three approaches produced moderate to strong results, with average scores ranging roughly from 3.2 to 3.8 under human evaluation and from 4.0 to 5.0 under LLM-as-a-judge evaluation across the five dimensions (Relevance, Depth, Credibility, Accuracy, and Usefulness).</p><ul><li><strong>Exa </strong>consistently excelled in Relevance and Depth, often surfacing fresh and detailed content, particularly for exploratory or research-oriented queries.</li><li><strong>Google Grounded Search</strong> led in Credibility and Accuracy, reliably pointing to authoritative and trusted sources.</li><li><strong>Custom Engine</strong> showed relative strength in Usefulness and balanced reliability, especially in multi-step or synthesis-heavy queries, where structured ranking and authority scoring paid off.</li></ul><h4>5.2. Example Queries in Practice</h4><p>To make the numbers concrete, here is an illustrative case:<br><strong>Query:</strong> <em>“Top companies developing Phase 2 drugs for rare lung diseases”</em></p><ul><li><strong>Exa Web Grounded answer</strong> highlighted a wide range of biotech and pharma players (e.g., PureTech Health, Sanofi, AstraZeneca, Savara Inc.), with detailed breakdowns of drug candidates, trial phases, and mechanisms. This breadth and depth earned it top marks (5s across relevance, depth, credibility, accuracy, and usefulness).</li><li><strong>Google results</strong> surfaced a long list of companies from reputable news and pipeline reports. It was factually accurate and credible, though less structured in presenting mechanisms or trial progress. Scored slightly lower on depth (average 4–5).</li><li><strong>Custom Engine results</strong> focused heavily on Insilico Medicine and its AI-designed candidate (INS018_055), with a strong background context but fewer companies. While credible and accurate, its narrow scope limited its usefulness compared to the others (scoring 4s in usefulness and depth).</li></ul><p>This example illustrates the trade-offs: Exa delivers structured, exhaustive insights; Google ensures credibility through authoritative sources; and the Custom Engine provides focused, context-rich answers but may miss breadth.</p><h4>5.3 Insights from the Human Evaluation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*xYkmdKsEOVMFjsp96LLX4Q.png" /></figure><p>The human evaluations reflected a more cautious stance on tool performance:</p><ul><li><strong>Exa</strong> earned relatively strong marks for relevance but trailed in usefulness and credibility. While it sometimes surfaced interesting material, evaluators noted that the content was not always practical or reliable.</li><li><strong>Google</strong> achieved the most balanced scores across categories, leading slightly on credibility and accuracy. Human judges valued its consistency and trustworthiness, even if the responses were not always the deepest.</li><li><strong>The Custom Engine</strong> placed lower overall, with weaker depth and accuracy in particular. Although it occasionally added helpful aggregation, evaluators generally found it less dependable and less comprehensive than the other two tools.</li></ul><h4>5.4 Insights from the LLM-as-a-Judge Evaluation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/793/1*FABXg5Lc3Ubyy8CXtcX9Wg.png" /></figure><p>The LLM-based scoring framework closely mirrored what we’d expect from human evaluators:</p><ul><li><strong>Exa</strong> stood out for relevance, depth, and usefulness, consistently scoring near the top. Its results often introduced novel or timely material, though sometimes from niche or less widely known sources, which explains a slightly lower credibility score.</li><li><strong>Google grounded search </strong>performed well on relevance and accuracy but lagged on depth and credibility, often defaulting to straightforward factoid-style answers rather than richer context.</li><li><strong>The Custom Engine</strong> delivered balanced performance, with strong credibility and accuracy. While it was less effective on usefulness compared to the others, it was valuable when authoritative aggregation mattered more than freshness.</li></ul><h4>5.5. Comparative Strengths &amp; Weaknesses</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*Y8GAZqap_iYC2mWaGvWIrQ.png" /></figure><h3>6. Discussion</h3><p>The two evaluation approaches converged on broad patterns but diverged in emphasis, revealing how humans and LLMs apply different criteria when assessing quality.</p><ul><li><strong>Tolerance for Source Visibility and Trust: </strong>Human evaluators weighed source recognition heavily. When Exa pulled results from lesser-known or niche domains, credibility was discounted even if the content was accurate. In contrast, the LLM focused on how relevant and well-written the content was, even if the source was obscure.</li><li><strong>Depth vs. Usefulness: </strong>LLMs rewarded answers that looked detailed and information-rich. Humans were more practical: if the extra detail didn’t clearly help answer the question, they rated it lower.</li><li><strong>Perception of Authority: </strong>Google achieved higher credibility and accuracy with human judges, who associated recognizable sources (e.g., PubMed, government portals) with trustworthiness. LLM scoring was less sensitive to institutional reputation and instead judged credibility through internal consistency and coverage, which favored Exa.</li><li><strong>Aggregation vs. Engagement: </strong>The Custom Engine highlighted an important divergence. LLMs rewarded its structured aggregation of authoritative content, which “looked” balanced and useful from a model’s perspective. Humans, however, found the same aggregation less engaging, penalizing it for limited novelty and weaker depth.</li><li><strong>Surface vs. Process Bias: </strong>Overall, humans proved outcome-driven, emphasizing whether an answer felt practical, trustworthy, and actionable. LLMs were process-driven, rewarding responses that appeared well-structured, complete, and semantically coherent.</li></ul><p>Human evaluators emphasize <em>trust, recognition, and immediate usefulness</em>, while LLM evaluators emphasize <em>coherence, novelty, and informational density</em>. This divergence suggests that combining both evaluation modes yields a fuller picture</p><p>From this, several implications for evaluation design emerge:</p><ul><li>Future evaluations should explicitly balance novelty and structure (LLM strengths) with trust and applicability (human strengths).</li><li>Human judgments may vary depending on expertise. A clinical researcher, for example, may weigh credibility differently than a general reader.</li><li>Combining human and LLM evaluators can reduce blind spots; humans flag credibility and trust issues, while LLMs detect coverage gaps or structural weaknesses.</li></ul><h3>7. Conclusion</h3><p>So, what did we find?<strong> Search quality really does matter.</strong> Exa was great at digging up fresh, detailed content. Google Grounded Search was the most trustworthy when it came to credible sources. And the Custom Engine gave balanced, useful answers, especially when authority was important. Each one had its strengths and its weak spots</p><p>The key takeaway is simple: if the search is bad, the answers will be bad too. That’s why we need to actually test how well these search tools work, instead of just assuming they’ll deliver, then pick the one that suits our use case the most.</p><p>Stay tuned for more insightful articles delivered to you by <a href="https://infinitgraph.ai/"><strong>InfinitGraph</strong></a>.</p><p>This article was written in collaboration with <em>Alaa Dania Adimi. You</em> can reach her at <a href="/service/mailto:dania.adimi@infinitgraph.ai">dania.adimi@infinitgraph.ai</a> or check her other <a href="https://medium.com/@ja_adimi">Medium</a> articles!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=886ec4821af9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/how-good-is-your-web-search-comparing-three-approaches-for-llm-agents-886ec4821af9">How Good Is Your Web Search? Comparing Three Approaches for LLM Agents</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your First Steps in Evaluating LLM-Powered Applications]]></title>
            <link>https://medium.com/infinitgraph/your-first-steps-in-llm-based-applications-evaluation-81813a4aa100?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/81813a4aa100</guid>
            <category><![CDATA[llm-evaluation]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-applications]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Sun, 26 Oct 2025 16:26:34 GMT</pubDate>
            <atom:updated>2025-10-26T16:26:32.840Z</atom:updated>
            <content:encoded><![CDATA[<p>You’ve built an impressive LLM-powered application and are thrilled with the results. Self-testing might indicate it’s working great, but how do you convincingly showcase its performance to your co-workers and friends? This article will introduce you to LLM-based application evaluation so that you can evaluate your application and communicate its capabilities with others.</p><h3>I. Introduction</h3><p>Large Language Models (LLMs) have taken the world by storm. Since the launch of popular models like ChatGPT, we’ve seen a huge increase in LLM-powered applications popping up everywhere. From virtual assistants on websites to coding helpers, LLMs seem to be the new hotness. A <a href="https://state-of-llm.streamlit.app/">recent study by Streamlit</a> revealed that over 17,926 developers created more than 29,183 apps on their platform alone. This is just one example of the LLM app explosion.</p><p>It’s become easier than ever to build these apps, especially with the presence of tools like <a href="https://www.langchain.com/">Langchain</a> and <a href="https://www.llamaindex.ai/">Llamaindex</a>, and the availability of LLM APIs — Like OpenAI and Gemini. But here’s the catch: while building a demo is one thing, creating a reliable LLM application for real-world use is another. It’s tough to know if an LLM-based application is doing what it’s supposed to do, especially when it works perfectly on a few test cases. This is where evaluation becomes essential.</p><p>In this article, <a href="https://medium.com/u/49613fd82e4a">Rania</a> and I will share insights from our research on evaluating LLM-based applications. Well.. let’s get started 🤭!</p><h3>II. Large Language Models evaluation</h3><p>Evaluating Large Language Models and quantifying their performance remains an active area of research. Current approaches can be broadly categorized as:</p><ol><li>Research benchmarks.</li><li>Standard metrics.</li><li>Human evaluation.</li><li>LLM evaluation.</li></ol><h4>1. Research benchmarks</h4><p>Benchmarks provide a standardized way to assess LLM capabilities. They involve a set of pre-defined questions or tasks that the model is tested on programmatically. The LLM’s performance is then evaluated using a scoring system, typically a percentage score indicating the accuracy of its responses. This allows for a quantitative comparison of different models across various tasks.</p><p>Many LLMs today can be evaluated and selected based on benchmarks that span a wide variety of tasks. For instance, the <a href="https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?ref=dataroots.ghost.io">Open LLM leaderboard</a> maintained by Hugging Face provides performance metrics across numerous domains, enabling users to select the optimal model for their specific use case. Examples of popular benchmarks include <strong>MMLU</strong>, <strong>SuperGLUE</strong>, and <strong>HumanEval</strong>, each focusing on different aspects of language understanding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E3CZVE4Xr2pCiLCcq5HW3Q.png" /><figcaption>Figure 01 — Open LLM leaderboard</figcaption></figure><h4>2. Standard NLP metrics</h4><p>Standard Natural Language Processing (NLP) metrics can be effectively used to assess the outputs of Large Language Models by providing quantitative measures of performance across various tasks. Here are some commonly used metrics and how they can be applied to evaluate LLM outputs:</p><p><strong>BLEU (Bilingual Evaluation Understudy)</strong></p><ul><li><strong>Usage</strong>: Primarily used for evaluating machine translation and text generation tasks.</li><li><strong>Application</strong>: When an LLM generates a translation or a summary, BLEU can quantify how closely the output matches human-generated references, providing a score that reflects translation quality.</li></ul><p><strong>ROUGE (Recall-Oriented Understudy for Gisting Evaluation)</strong></p><ul><li><strong>Usage</strong>: Commonly used for summarization tasks.</li><li><strong>Application</strong>: When evaluating an LLM’s summarization capabilities, ROUGE can help assess how well the generated summary captures the key points of the original text compared to human-written summaries.</li></ul><p><strong>F1 Score</strong></p><ul><li><strong>Usage</strong>: Commonly used in classification tasks, including question answering.</li><li><strong>Application</strong>: In question-answering tasks, the F1 score can assess how many of the generated answers match the expected answers, providing a clear measure of performance.</li></ul><p>Metrics like <strong>Accuracy</strong>, <strong>BERTScore, </strong>and <strong>METEOR </strong>can also be used to assess LLMs performance. For more details, consult the paper <a href="https://arxiv.org/pdf/2307.03109">“A Survey on Evaluation of Large Language Models”</a>.</p><h4>3. Human evaluation</h4><p>Human evaluation is the process of assessing the quality of LLMs outputs using human judgment. It involves human experts evaluating generated text based on various criteria, such as relevance, coherence, factuality, and overall quality. This method is often considered the gold standard for evaluating LLMs as it captures nuances that automated metrics might miss.</p><p>According to “<a href="https://arxiv.org/pdf/2307.03109">A Survey on Evaluation of Large Language Models</a>”, to ensure reliable and precise assessments, human evaluation requires careful consideration of various factors such as the <strong>number of evaluators</strong>, <strong>evaluation rubrics</strong>, and <strong>evaluator’s expertise level</strong>.</p><p>Human evaluation, while offering valuable insights, is not without its drawbacks:</p><ul><li><strong>Subjectivity:</strong> Human judgments can be influenced by personal biases, leading to inconsistent evaluations.</li><li><strong>Scalability:</strong> Evaluating large volumes of text manually is time-consuming and resource-intensive.</li><li><strong>Consistency:</strong> Maintaining consistent evaluation criteria across multiple human raters is challenging.</li><li><strong>Cost:</strong> Hiring and managing human evaluators can be expensive.</li></ul><h4>4. LLM evaluation</h4><p>An alternative to human evaluation is to have an LLM to evaluate the output, this is called: LLM as a judge. Model judgments in particular are increasingly promising and an active research area. To confirm the level of agreement between human annotators and LLM judges, the Databricks team sent answer sheets (grading scale 0–3) from GPT-3.5- turbo and vicuna-33b to a labeling company to collect human labels, and then compared the result with GPT-4’s grading output. The findings in Figure 1.4 show that human and GPT-4 judges can reach above 80% agreement on the correctness and readability score. Reducing the required agreement margin to 1 point or less would push agreement levels above 95% — More details on this study can be found in the Databricks blog article <a href="https://www.databricks.com/blog/LLM-auto-eval-best-practices-RAG">Best Practices for LLM Evaluation of RAG Applications</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GvSbphPkYsJ6-NGCDgnVzQ.png" /><figcaption>Figure 02 — Human vs GPT-4 Grading Alignments for GPT3.5 answers</figcaption></figure><p>While LLM-based evaluation offers potential for automation and efficiency, it also comes with limitations:</p><ul><li><strong>Reliability:</strong> The quality of an LLM’s evaluation depends on the quality of the LLM itself, which can be inconsistent.</li><li><strong>Bias:</strong> LLMs can inherit biases from their training data, affecting the objectivity of their evaluations.</li><li><strong>Interpretability:</strong> Understanding the reasoning behind an LLM’s evaluation can be difficult.</li><li><strong>Overreliance:</strong> Exclusive reliance on LLM evaluation might overlook critical human-centric aspects of quality.</li></ul><h3>III. LLM-based applications evaluation</h3><h4>1. Why evaluation is important?</h4><p>Evaluation is essential for building trust in LLM-based applications as LLMs are prone to mistakes. It benefits both the developers and users. For developers and companies, evaluation helps determine if improvements are worthwhile, if the system is ready for deployment, and if the investment in leveraging LLMs is justified. For users, consistent and reliable performance builds trust, encouraging reliance on the LLM-powered product.</p><h4>2. What makes a good evaluation?</h4><p>A good evaluation is closely tied to the specific goals and objectives of an LLM application. Ideally, an evaluation should be:</p><ul><li><strong>Outcome-oriented:</strong> Directly linked to the desired outcomes of the application. For example, if the goal is to improve customer satisfaction, the evaluation should measure how well the LLM enhances the customer experience.</li><li><strong>Efficient and Automated:</strong> Easily integrated into the development workflow for rapid feedback and iteration.</li><li><strong>Actionable:</strong> Provides clear insights that can be used to make data-driven decisions and improvements.</li></ul><h4>3. Should public benchmarks and standard NLP metrics be used to evaluate LLM-based applications?</h4><p>Benchmarks provide a standardized approach to comparing LLM performance. While helpful for general comparisons, they often fall short in accurately reflecting the nuances of specific use cases. Many benchmarks are designed for broad language understanding tasks and might not adequately capture the unique requirements of tailored LLM-based applications.</p><p>Standard NLP metrics, while offering a quantitative baseline, often fall short in comprehensively evaluating some LLM-based applications. Metrics like BLEU and ROUGE, designed for machine translation and summarization, might not be suitable for other LLM tasks like email generation and resume review.</p><h4>4. A recipe for evaluating LLM-based applications</h4><blockquote><strong>“Better data and better metrics result in a better evaluation”</strong> —</blockquote><p>Effectively evaluating LLM-powered applications requires a systematic approach. This involves two primary components: <strong>evaluation data</strong> and <strong>evaluation metrics</strong>. The quality of both data and metrics significantly influences the reliability of the evaluation.</p><p><strong>Evaluation data</strong></p><p>Building a good dataset for evaluation is crucial. The dataset should align with the functionalities of the application and should also cover a wide range of inputs, including edge cases and potential challenges.</p><p>To build an evaluation dataset, we can start with a small set of examples that capture key scenarios and leverage LLMs to generate additional data points, expanding the dataset’s coverage and then continuously refine the dataset by incorporating real-world examples as the application evolves.</p><blockquote><strong><em>Note:</em></strong><em> LLMs like GPT-4 can help a lot in building evaluation datasets. However, reviewing its outputs is crucial to guarantee data quality and relevance.</em></blockquote><p><strong>Evaluation metrics</strong></p><p>Selecting appropriate metrics is also very crucial. Ideally, metrics should align closely with the desired outcomes of the application. There are some key considerations when choosing metrics:</p><ul><li><strong>Predictive Power:</strong> Metrics should correlate strongly with real-world user satisfaction and application performance.</li><li><strong>Accuracy:</strong> Metrics should be reliable and consistent in measuring the desired attribute.</li><li><strong>Feasibility:</strong> Metrics should be practical to implement and calculate.</li><li><strong>Completeness:</strong> A combination of metrics often provides a more comprehensive evaluation.</li></ul><p>There’s this flowchart that we took from <a href="https://www.youtube.com/watch?v=2CIIQ5KZWUM">“Evaluating LLM-based applications”</a> webinar by Databricks. It helps with picking the evaluation metrics based on the setting:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DS_75otwYO1mWzfbai0HUQ.png" /><figcaption>Figure 03 — Metrics selection flowchart</figcaption></figure><p>Both human evaluation and LLM evaluation offer valuable insights and can be used to evaluate LLM-based applications, however, they come with limitations as stated above. Combining these methods creates a more comprehensive and reliable evaluation process.</p><h3>IV. Additional Insights</h3><h4>5.1. Beyond the LLM: A holistic perspective</h4><p>It’s essential to recognize that LLM performance is influenced by multiple factors beyond the model itself. The chosen prompt, fine-tuning data, and even the retrieval and embedding models in RAG systems significantly impact overall results. Consequently, a comprehensive evaluation requires considering these elements.</p><h4>5.2. Leveraging LLM evaluation frameworks</h4><p>A growing number of frameworks, such as <a href="https://docs.ragas.io/en/stable/">RAGAs</a> and <a href="https://www.trulens.org/">Trulens</a>, offer automated evaluation of LLM-powered applications. These frameworks provide valuable insights into performance based on predefined metrics.</p><h3>V. Conclusion</h3><p>Evaluating LLM-based applications is a complex and evolving field. We have tried to go through the most important point that we learned while exploring this topic. We highly recommend you check the reference section as it contains all the research papers, video content, and courses that we found useful — Besides the ones that have been linked!</p><h3>VI. References</h3><ul><li><a href="https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/">Building and Evaluating Advanced RAG Applications.</a></li><li><a href="https://arxiv.org/abs/2305.13091">Large Language Models are not Fair Evaluators.</a></li><li><a href="https://arxiv.org/abs/2307.03109">A Survey on Evaluation of Large Language Models.</a></li><li><a href="https://www.youtube.com/watch?v=FVdGAfR2OqQ">Evaluating and Tracking LLM Experiments: Building Better LLM Apps with TruLens.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81813a4aa100" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/your-first-steps-in-llm-based-applications-evaluation-81813a4aa100">Your First Steps in Evaluating LLM-Powered Applications</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Context Protocol (MCP): A Beginner’s Guide]]></title>
            <link>https://medium.com/infinitgraph/model-context-protocol-mcp-a-beginners-guide-d7977b52570a?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/d7977b52570a</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mcp-protocol]]></category>
            <category><![CDATA[ai-tools]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Wed, 15 Oct 2025 15:56:45 GMT</pubDate>
            <atom:updated>2025-11-05T07:30:11.024Z</atom:updated>
            <content:encoded><![CDATA[<p>In the previous articles, we learned that modern AI systems often need information or actions beyond what their training data provide. For example, a chatbot might use a <strong>search engine</strong> to look up recent news, query a <strong>database of company records</strong>, call a <strong>calculator API</strong> like <a href="https://www.wolframalpha.com/">WolframAlpha</a> for computations, or invoke a ticketing system API to create support tickets. Other common tools include <strong>web APIs</strong> (weather, stock quotes, Wikipedia), <strong>file stores</strong> (Google Drive or local files for documents), <strong>code execution environments</strong> (to run snippets), and <strong>messaging</strong> or <strong>calendar APIs</strong> (to send emails or schedule events).</p><p>These external tools let the system fetch live data or perform tasks that an LLM alone cannot do. In practice, systems like <a href="https://python.langchain.com/docs/integrations/tools/">LangChain</a> or <a href="https://platform.openai.com/docs/assistants/tools#page-top">OpenAI’s Plugins</a> enable such tool use, effectively extending the LLM’s capabilities by calling functions, databases, or search services. However, integrating all these separate tools has traditionally been ad-hoc and brittle. As Ksenia Se said in her <a href="https://huggingface.co/blog/Kseniase/mcp">article on MCPs</a>, large language models “are smart in a vacuum” but “struggle once they need information beyond what’s in their training data,” and historically connecting them to files, databases, and APIs “has been a messy, ad-hoc affair”.</p><p>The Model Context Protocol, or MCP for short, aims to fix this by providing a standardized bridge between an AI model (or agent) and external tools and data sources.</p><p>That’s something many of you might already know from posts on LinkedIn or short demos online. But do you really know what MCP is… beyond the buzz?</p><ul><li>How is it different from OpenAI’s function calling or ChatGPT plugins?</li><li>Is it a tool, a product, or just a spec?</li><li>Why did it suddenly go viral in 2025 after flying under the radar?</li></ul><p>And most importantly: how can you actually use it to build more powerful AI systems?</p><p>In this article, I’ll walk you through all of it! From the <strong><em>what </em></strong>and <strong><em>why </em></strong>to the <strong><em>how</em></strong>. Whether you’re an engineer curious about agent design, or someone just trying to understand how AI systems can interact with the real world, this article will break it down in simple, clear terms.</p><h3>1. What is Model Context Protocol?</h3><p>MCP is not a tool, library, or product, it’s a <strong>protocol</strong>, a structured way to communicate with a language model. It provides a standardized way for giving LLMs all the context they need to reason and respond effectively. You can think of it as a universal language or connector that allows AI applications to interact seamlessly with external data sources, tools, and environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PQuv4_Ew5qFFC8q5923jsw.png" /><figcaption>Figure 01 — “Without MCP” vs. “With MCP”</figcaption></figure><p>If you’re from the tech world, think of it like HTTP, TCP/IP, or SMTP. If not, don’t worry! Just remember: tech folks adore standardization, it makes things easier to reuse and plug together.</p><p>Anthropic, the creators of MCP, aptly describe it as a <strong>“USB-C port for AI applications”</strong> [1]. Just as a USB-C port allows a laptop to connect to anything from a hard drive to a monitor with a single standard cable, MCP allows an LLM to “plug into” tools, data sources, and memory in a consistent, reusable way.</p><h3>2. Components of the MCP</h3><p>To fully understand how MCP works, we need to break it down into its key components. MCP follows a <strong>client-server architecture</strong>, but with one important twist: it also introduces the <strong>Host</strong> as a first-class player in the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DWLRq3lyGJLVLHYCgQdoeQ.png" /><figcaption>Figure 02 — MCP Architecture</figcaption></figure><h4>2.1. Host</h4><p>The Host is the main environment where the AI system runs and where the user interacts with it. It acts as the initiator of all communication in the MCP ecosystem.</p><p>In simpler terms, this is where your AI application <em>lives</em>. It’s responsible for sending requests to the outside world and receiving back helpful context or tool responses.</p><p><strong>Examples of MCP Hosts:</strong></p><ul><li><strong>Claude Desktop </strong>— Anthropic’s own desktop app that uses Claude and integrates MCP to expand its capabilities.</li><li><strong>IDEs (Integrated Development Environments)</strong> — Like Cursor AI, when embedded with AI assistants for code completion, debugging, and more.</li><li><strong>AI Tools &amp; Assistants</strong> — Any application that uses AI to provide a service, LIKE customer support bots, internal company agents, etc.</li></ul><p>In a nutshell, the Host typically contains the LLM and makes requests (via the MCP Client) for tools, data, or memory it doesn’t have by default.</p><h4>2.2. Client</h4><p>The Client is like the translator and messenger sitting inside the Host.</p><p>Its job is to take whatever the LLM or application is requesting (like a task it needs to perform, or data it needs to retrieve) and convert it into a <strong>standardized MCP request </strong>and send to the Server.</p><p>To make things simpler, a standardized MCP request includes structured context — such as which tool to use and what arguments to pass to complete the task. For example, if the LLM wants to get today’s weather in Makkah, the Client would craft a message like this and send it to the Server:</p><pre>{<br>&quot;jsonrpc&quot;: &quot;2.0&quot;,<br>&quot;id&quot;: &quot;1&quot;,<br>&quot;method&quot;: &quot;tools/call&quot;,<br>  &quot;params&quot;: {<br>    &quot;tool_name&quot;: &quot;get_weather&quot;,<br>    &quot;input&quot;: {<br>      &quot;location&quot;: &quot;Makkah&quot;,<br>      &quot;date&quot;: &quot;2025–06–16&quot;<br>    }<br>  }<br>}</pre><p>Each MCP Client has a <strong>one-to-one connection</strong> with a specific Server. This makes communication reliable and clean, as each Client knows exactly which Server it’s talking to, and there’s no confusion about where to send the request.</p><p><strong>What the Client does:</strong></p><ul><li>Translates requests from the Host or LLM into MCP format</li><li>Sends those requests to the appropriate Server</li><li>Receives responses and passes them back to the Host</li></ul><p>You can think of the Client is the protocol-savvy assistant that speaks both the language of your AI and the language of external tools.</p><h4>2.3. Server</h4><p>An <strong>MCP Server</strong> is a lightweight program that exposes tools, data, and capabilities to the MCP Client. You can think of it as the <strong>backend service</strong> that actually knows <em>how</em> to do the work — whether that’s fetching a document, querying a database, or calling an external API.</p><p>It’s the <strong>component responsible for fulfilling the model’s requests</strong>.</p><p>Each Server can be designed to handle a specific domain or function, such as:</p><ul><li>Accessing local files (e.g. PDFs on your computer)</li><li>Calling external APIs (e.g. weather, calendar, stock prices)</li><li>Connecting to internal systems (e.g. a private SQL database or company CRM)</li></ul><p>When the Client sends a request, the Server:</p><ol><li><strong>Reads the request</strong> (formatted using MCP’s standardized JSON structure),</li><li><strong>Runs the required tool or action</strong>,</li><li><strong>Returns the response</strong> back to the Client, which is then passed to the LLM.</li></ol><p>MCP Servers can be:</p><ul><li>Open-source or private</li><li>Hosted remotely or run locally</li><li>Reused across different Hosts and Clients</li></ul><h4>2.4. Communication Protocol</h4><p>When the MCP Client talks to the Server, they need to follow a clear set of rules so they can understand each other like speaking the same language.</p><p>That language is called <a href="https://www.jsonrpc.org/"><strong>JSON-RPC 2.0</strong></a>. It’s a simple format that lets one program ask another to do something (like “run this tool” or “get this data”) — and then get a response back. You can think of it like sending a digital form where:</p><ul><li>You say what action you want (method)</li><li>You give the needed details (params)</li><li>You get a clear answer or an error in return</li></ul><p>It makes sure:</p><ul><li>Every request and reply has the <strong>same format</strong> (so it’s easy to build and debug)</li><li>If something goes wrong, there’s a <strong>standard way to show the error</strong></li><li>It works across different setups:<br>- On your own machine using stdio (like a direct line between programs)<br>- Or over the internet using <strong>HTTP streaming</strong> (good for cloud tools)</li></ul><p><strong>Each role works together:</strong> the Host (e.g. your chat app) runs the Client to formulate MCP queries, and the Server carries them out. This separation means you could swap out components freely: one host could talk to multiple different servers (for CRM, cloud storage, etc.), or one server could serve multiple hosts. It also means any AI model inside the Host can use the same protocol to access new tools, making the system modular.</p><h3>3. Why Does MCP Matter?</h3><p>Let’s say you’re building an AI agent without MCP, you’re probably wrestling with:</p><ul><li>Writing and updating prompts manually.</li><li>Injecting retrieved documents into the prompt.</li><li>Registering tools with the model.</li><li>Managing state and memory.</li><li>Tracking conversation history.</li><li>Handling outputs and responses.</li></ul><p>It’s messy, right? Every new data source, tool, or capability means rewriting parts of your system. Without a shared structure, developers end up patching things together with custom logic, plugins, or ad-hoc prompts, which is neither scalable nor robust.</p><p>The <strong>Model Context Protocol </strong>steps in and says: <em>“Let’s standardize how you deliver tasks, tools, documents, and memory to the model.”</em></p><p>This is a <strong>big deal</strong>, especially as AI agents become more complex and production-ready. With MCP:</p><ul><li>You don’t need to rewrite your prompt every time a new tool is added.</li><li>You don’t need to manually update memory or keep track of conversation turns.</li><li>You don’t need a separate integration layer for every API.</li></ul><p>Instead, you just <strong>send a structured MCP request</strong> (like calling a clean API) and everything the model needs is delivered in a consistent, machine-readable format.</p><p>With MCPs, developers no longer have to “teach” the model about tools through prompt engineering. The model receives a formal description of each tool it can use, the arguments it expects, and how to invoke it, all in structured JSON.</p><p>According to many articles, MCP “simplifies the development process resulting from standardized integration” and fosters interoperability across tools and models. Anthropic explains that MCP replaces “fragmented integrations with a single protocol,” making connected AI systems easier to build and scale, that’s because MCP:</p><ul><li><strong>Reduces bugs:</strong> Structured JSON requests/responses remove brittle prompt-engineering hacks.</li><li><strong>Improves debugging:</strong> Every interaction is clearly logged and traceable.</li><li><strong>Enhances modularity:</strong> Swap out or add tools without reshaping your agent.</li><li><strong>Boosts interoperability:</strong> Any MCP-compatible model, host, or server can plug in seamlessly.</li></ul><h3>4. <strong>Why Did MCP Catch On So Fast?</strong></h3><p>MCP went viral in early 2025 with over 1,000 public MCP servers by February, and much more by now (you can have a look <a href="https://github.com/modelcontextprotocol/servers">here</a>). So how did a quiet protocol become such a foundational layer in the AI ecosystem?</p><p><strong>Here are a few key reasons that drove MCP’s breakout success:</strong></p><h4>4.1. AI-Native by Design</h4><p>MCP wasn’t tacked on as an afterthought, it was built for AI agents from the ground up. The protocol was crafted to handle everything modern agents need: tools, prompts, memory, and documents.</p><p>That means developers didn’t have to force their systems into clumsy frameworks, they could just <em>plug MCP in</em>, and all the pieces fit naturally.</p><h4>4.2. Robust Documentation &amp; Tooling</h4><p>When Anthropic released MCP in <strong>November 2024</strong>, they didn’t stop at a PDF spec. They shipped full <strong>Python, TypeScript, and even C# SDKs</strong>, along with reference clients, server demos, and prebuilt connectors for popular platforms (like Google Drive, Postgres, GitHub, and Slack). Developers had everything they needed to start integrating MCP right away.</p><h4>4.3. Network Effect and Ecosystem Support</h4><p>Once Anthropic demonstrated how easy MCP made integration, others quickly followed:</p><ul><li><a href="https://openai.com/index/new-tools-and-features-in-the-responses-api/"><strong>OpenAI</strong> added MCP support</a> in early 2025, making it compatible with GPT‑4 and their Agent SDK.</li><li><a href="https://devblogs.microsoft.com/blog/microsoft-partners-with-anthropic-to-create-official-c-sdk-for-model-context-protocol"><strong>Microsoft</strong> partnered with Anthropic</a> on a C# SDK, later integrating MCP into Copilot Studio and Windows AI Foundry.</li><li><strong>Google DeepMind, AWS, Atlassian</strong>, and others released their own MCP servers and tools.</li></ul><p>By mid‑2025, <strong>over 5,000 public MCP servers were live</strong>, connecting AI agents to everything from databases to internal business systems.</p><h3>5. Common Misunderstandings</h3><p>By now, you probably have a solid grasp of what MCP is and how it works. But just to make things even clearer, I wanted to highlight a few common misunderstandings that tend to float around, especially if you’ve only skimmed headlines or quick threads.</p><p>These aren’t just nitpicks, they help frame MCP correctly so you can actually use it for what it’s good at (and not expect magic where there isn’t any).</p><p>A few things people often get wrong about MCP:</p><ul><li><strong>It’s not a new API or product.</strong> It’s a <em>pattern</em>. A clean, standardized way to frame what you send to the model.</li><li><strong>It doesn’t magically make models smarter.</strong> It just gives them better, more structured context to work with.</li><li><strong>It’s not just for agents.</strong> Even simpler assistants and interfaces benefit from clearer tool access, memory, and task framing.</li></ul><h3>6. Conclusion</h3><p>You made it to the end of this article! Here’s a quick recap:</p><ol><li>MCP is a universal, structured protocol, not a library or product, that packages everything an LLM needs: tools, documents, memory, and prompts. Like <strong>USB-C for AI</strong>, it enables seamless connection to anything.</li><li><strong>The Key Components of an MCP are<br>- Host</strong> (your app or IDE)<br>- <strong>Client</strong> (translates model intent into RPC calls)<br>- <strong>Server</strong> (runs tools and retrieves data)</li><li><strong>Why It Matters? </strong>It eliminates brittle prompt hacks and custom integration for each new tool. It makes AI agent development predictable, maintainable, and scalable.</li></ol><p>Thank you for diving into the MCP journey with me. Have questions? Feel free to drop them in the comments, I’ll do my best to help out (though I’m still learning MCP myself, just like many of you!).</p><p>In the next article, we’ll get practical; we will go through how we can build and implement our own MCPs with FastMCP.</p><p>If you have any feedback, questions, or comments, please send them my way at: dania.adimi@infinitgraph.ai.</p><h3>7. References</h3><p><a href="https://modelcontextprotocol.io/introduction">https://modelcontextprotocol.io/introduction</a></p><p><a href="https://huggingface.co/blog/Kseniase/mcp">https://huggingface.co/blog/Kseniase/mcp</a></p><p><a href="https://cursor.directory/mcp">https://cursor.directory/mcp</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7977b52570a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/model-context-protocol-mcp-a-beginners-guide-d7977b52570a">Model Context Protocol (MCP): A Beginner’s Guide</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM Agents: A focus on ReAct Framework]]></title>
            <link>https://medium.com/infinitgraph/llm-agents-a-focus-on-react-framework-50f62289559d?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/50f62289559d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[react-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agents]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Fri, 18 Apr 2025 12:55:16 GMT</pubDate>
            <atom:updated>2025-11-05T07:30:57.435Z</atom:updated>
            <content:encoded><![CDATA[<p>We’ve reached an exciting point in our series! We’re finally diving into <strong>“LLM agents”</strong> in these upcoming articles. I’m pretty sure you’ve noticed words like <strong>“agents”</strong> and <strong>“agentic”</strong> popping up everywhere lately. But what do these terms really mean when we talk about them in the context of LLMs?</p><p>Today, we’ll kick things off with a quick look back at the history of <strong>“agents”</strong> in AI, and then we’ll explore the ReAct framework which combines two things we’ve already talked about — <strong>reasoning</strong> and <strong>tool use</strong> — to build even more powerful applications. And to show you how it works in practice, we’ll build a simple ReAct agent together that takes a natural language query and transforms it into an SQL query to execute against a simple database.</p><p>Exciting, isn’t it? Let’s get straight into it.</p><h3><strong>1. What’s an AI Agent?</strong></h3><p>Agents, by definition, remove <em>agency</em> from a user in order to do things on the user’s behalf and save them time and effort. In the context of AI, <a href="https://scholar.google.com/citations?user=qJBXk9cAAAAJ&amp;hl=en">Shunyu Yao</a>, a Research Scientist at OpenAI, defines an agent as an “intelligent” system that can interact with some “environment”. That environment could be <strong>physical</strong>, such as roads navigated by an autonomous car; <strong>digital</strong>, as in the virtual worlds like games; or even <strong>social</strong>, such as chatbots that can engage with users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Aa6gLU8eFL1Oo9CFDzFkrw.png" /><figcaption>Figure 01 — An illustration of an agent</figcaption></figure><h3>2. <strong>A little bit of history</strong></h3><p>Believe it or not, the term <strong>“agent”</strong> isn’t new, it’s been around for quite some time. Back in the early days, even in the nineties, we saw some agents performing simple tasks (and that was fascinating back then). Although basic by today’s standards, these early agents marked the beginning of machines that could interact with us. Let’s take a quick look at some of these agents.</p><h4><strong>2.1. Radio Rex (1916)</strong></h4><p>One of the earliest examples is Radio Rex, a voice-activated toy patented in 1916. This wooden dog was built to come when you called its name, an early hint that machines could act on your behalf just by following simple voice commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_TJbAfceXIfWTalqeqkVJQ.png" /><figcaption>Figure 02 — A picture of Radio Rex</figcaption></figure><h4>2.2. <strong>Audrey (1952)</strong></h4><p>Fast forward to 1952, when Bell Labs introduced Audrey, one of the first systems capable of recognizing spoken digits. Despite being bulky and limited in scope, Audrey demonstrated that machines could interpret human speech, laying the groundwork for future developments in voice recognition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*zp8879PZHYPIqolpN7bsxA.jpeg" /><figcaption>Figure 03–1952 Bell Labs Audrey</figcaption></figure><h4>2.3. <strong>ELIZA (1960s)</strong></h4><p>In the 1960s, MIT’s Joseph Weizenbaum created ELIZA, a chatbot that used simple pattern matching to simulate conversation. Although ELIZA’s responses were based on scripted patterns rather than genuine understanding, it was convincing enough that many users began to attribute human-like qualities to it — a phenomenon now known as <strong>the ELIZA effect</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*mvOTNb7IA_6j5GwW2BkouQ.png" /><figcaption>Figure 04 — Eliza’s interface</figcaption></figure><p>You can find some online implementations of ELIZA, I personally have tried <a href="https://psych.fullerton.edu/mbirnbaum/psych101/eliza.htm">this one</a>, and it was quite funny to try it out. Datacamp has a very nice short course titled <a href="https://www.datacamp.com/courses/building-chatbots-in-python">“Building Chatbots with Python”</a>, where they show you how you can build a similar chatbot to ELIZA.</p><h4>2.4. Harpy (1970s)</h4><p>The 1970s saw the development of <a href="https://stacks.stanford.edu/file/druid:rq916rn6924/rq916rn6924.pdf">Harpy at Carnegie Mellon University</a>, an early speech recognition system capable of understanding roughly 1,000 words (which is about a three-year-old’s vocabulary). Harpy was the first speech-to-text system that used a language model to determine which sequences of words made sense together which reduced errors.</p><h4>2.5. <strong>IBM’s Tangora (1980s)</strong></h4><p>In the 1980s, IBM’s transcription system Tangora used statistical methods to recognize up to 20,000 words. It was used in the first voice-activated dictation for office workers and set the foundation for modern speech-to-text software. This type of software continued to be developed and improved until it was commercialized in the 2000s.</p><h4>2.6. <strong>Dragon NaturallySpeaking (1990s)</strong></h4><p>In the 1990s, Dragon NaturallySpeaking emerged as one of the first commercial speech recognition systems. Developed by Nuance Communications, it was able to transcribe spoken words into text in real-time, revolutionizing how people interacted with their computers. For more information, visit Nuance’s Dragon page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*720tXP6Xnt-ljaTNZA9f2g.jpeg" /><figcaption>Figure 05 — The Bakers’ breakthrough software system, Dragon NaturallySpeaking, made tech headlines and dominated software awards in the late 1990s.</figcaption></figure><h4>2.7. IBM Simon (1994)</h4><p>The IBM Simon smartphone, launched in 1994, was a precursor to modern mobile agents. It combined basic voice recognition and text messaging, hinting at the multifunctional digital assistants that would soon become commonplace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*qI6Xg53NVGEw9AAHFKtuag.png" /><figcaption>Figure 06— IBM Simon Product Release</figcaption></figure><h4>2.8. <strong>Siri (2011)</strong></h4><p>In 2011, Apple introduced Siri, a digital assistant that transformed how we interact with our devices. Siri uses natural language processing to understand user requests, perform tasks like sending texts or setting alarms, and even engage in basic conversation. It continues to improve and include more advanced features.</p><h4>2.9. <strong>Alexa (2014)</strong></h4><p>Following Siri, Amazon launched Alexa in 2014, a voice-activated assistant that quickly became integral to smart home ecosystems. Alexa not only responds to queries but also manages home devices, orders products, and even entertains users.</p><p>The examples above are systems that can be considered <strong>“Agents”</strong> by definition. Some of them, like Siri and Alexa, fall into the category of <strong>“AI Agents”</strong> because they leverage artificial intelligence. So these terms are not new and they are not limited to large language models. I personally like to refer to systems that leverage LLMs as <strong>“LLM Agents”</strong>.</p><h3>3. <strong>What are LLM Agents?</strong></h3><p>Consider an LLM-powered application built to help educators answer questions about the latest teaching methods. With a naive RAG system implemented, they might easily pull out answers to questions like, “What were the top remote learning strategies in FY 2022?”, as this is a simple fact that can be found in reports or studies.</p><p>Now, imagine a more detailed question: <strong>“What are the three new strategies <em>Instructica</em> has used to mix technology into classrooms, and how do these methods boost student engagement?”</strong>. It’s quite complex, right? How do we build a solution for a question like this?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wjZ_logElCg8V4HLjAWHxQ.png" /><figcaption>Figure 07 —Pitfalls of Naive RAG</figcaption></figure><p>It’s clear that this kind of query needs careful planning, a different approach, and the ability to break down a complex question into smaller parts. It might also need memory and the use of different tools — features that go beyond a basic retrieval augmented system.</p><p>All of these elements — reasoning through the problem, planning the answer, and using extra tools — are what we can call <strong>LLM Agents</strong>.</p><p>LLM agents can be described as systems that can use an LLM to <strong>reason </strong>through a problem, create a plan to solve the problem and execute the plan with the help of a set of <strong>tools</strong>, <strong>memory, </strong>and <strong>retrieval</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h5sccOavcU7nFyhU9uc4Ng.png" /><figcaption>Figure 08 — Components of an LLM Agent</figcaption></figure><p>In simple terms, an LLM agent combines four key components:</p><ul><li><strong>Retrieval:</strong> Most agents use retrieval-augmented generation to pull real-time data, whether it’s the latest market trends, updated company policies, or product inventory, to ensure information is accurate.</li><li><strong>Reasoning:</strong> LLM agent builders can use advanced prompting techniques, such as a chain of thoughts, that guide the agent on how to make decisions.</li><li><strong>Memory:</strong> LLM agents have short-term memory (recent conversation history), long-term memory (storage of structured information), and task-specific recall, allowing them to adapt and personalize decisions.</li><li><strong>Tool Use:</strong> Beyond chatting, LLM agents can interact with external systems to get data or perform actions.</li></ul><h3>4. Deep Dive into ReAct Architecture</h3><p>There’s so much happening with LLM Agents and all the fancy architectures out there. They are very interesting and we will get into them, but today, we’re focusing on one simple yet powerful framework: <strong>ReAct</strong>.</p><h4>4.1. What Is ReAct?</h4><p><strong>ReAct</strong>, or <strong>Reasoning </strong>and <strong>Acting</strong>, was introduced in March 2023, in the paper <a href="https://arxiv.org/abs/2210.03629">“ReAct: Synergizing Reasoning and Acting in Language Models”</a>. It is a way for large language models (LLMs) to think and act more like humans when solving tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5zIh41196uA3RK_-KG3VVg.png" /><figcaption>Figure 09 — Overview of ReAct Framework</figcaption></figure><p>Instead of just passively processing information, ReAct enables the LLM to:</p><ol><li><strong>Reason:</strong> Generate thoughts, plans, or strategies to tackle a problem.</li><li><strong>Act:</strong> Take actions based on its reasoning, like searching for information or interacting with an environment.</li><li><strong>Observe:</strong> Take in new information from its actions and use it to inform further reasoning and actions. Until reaching the final answer.</li></ol><p>To understand things better, Imagine you ask a ReAct agent: <strong>“What is the population of the capital of France?”</strong>.</p><p>First, it reasons that the question should be split into two parts: one to<strong> find the capital of France</strong> and another to <strong>find the population of that capital</strong>. Acting on this thought, it performs a search for<strong> “capital of France.”</strong> Once it observes the search results, it finds that the capital is Paris.</p><p>With this information, ReAct agent then reasons that it now needs to find out Paris’s population. It takes action again by searching for “population of Paris”.</p><p>Take this example from the official ReAct paper. On the left, you can see how simply “thinking” (Chain-of-Thought) or just “acting” (taking actions without thorough reasoning) can fall short of producing the right answer. On the right, by combining both reasoning and acting, we could arrive at the correct answer. This process of breaking down the question, taking actions, and using observations to inform further reasoning is what makes the ReAct framework so powerful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kn6GUy0r_jDx4IdCgT6wPA.png" /><figcaption>Figure 10 — Comparison of 4 prompting methods, (a) Standard, (b) Chain-of-thought (CoT, Reason Only), (c) Act-only, and (d) ReAct (Reason+Act), solving a HotpotQA.</figcaption></figure><h4>4.2. Why Is ReAct Important?</h4><p>ReAct offers some great benefits. Here are three key points:</p><ol><li><strong>Synergistic Combination: </strong>ReAct combines reasoning and acting, which allows it to capitalize on their synergy.</li><li><strong>Improved Interpretability and Trustworthiness: </strong>ReAct generates task-solving trajectories that are more interpretable than baselines without reasoning traces. It also allows humans to distinguish information from the model’s internal knowledge versus external environments.</li><li><strong>More factual and grounded: </strong>The problem-solving trajectory of ReAct is more factual grounded and trustworthy, thanks to the access to an external knowledge base.</li></ol><h3>5. <strong>Building a ReAct Text-to-SQL Agent</strong></h3><p>There are many exciting use cases that highlight the power of the ReAct framework, and a text-to-SQL agent is a great example to start with. In this project, I built an agent that takes natural language queries — like “Show me all orders from last month” — and transforms them into SQL queries that can be executed against a database.</p><p>The complete implementation is available in this <a href="https://github.com/adimidania/medium-articles/blob/main/LLM%20Agents%20with%20a%20focus%20on%20ReAct/ReAct_Text_to_SQL.ipynb"><strong>Jupyter notebook</strong></a>.</p><h4>5.1. Building a ReAct Text-to-SQL Agent</h4><p>First, I set up a sample SQLite database with two tables: The database has two tables. One, called <strong>“companies”</strong>, holds company details (an id, title, and description). The other, <strong>“customers”</strong>, stores customer info (an id, name, revenue) along with a company_id that links each customer to a company.</p><pre>import sqlite3<br><br>def create_sample_database():<br>    &quot;&quot;&quot;<br>    Create a simple SQLite database with a &#39;companies&#39; table and a &#39;customers&#39; table.<br>    Each company has an id, title, and description.<br>    Each customer has an id, name, revenue, and a foreign key company_id referencing the companies table.<br>    This function resets the database for a clean slate.<br>    &quot;&quot;&quot;<br>    conn = sqlite3.connect(&quot;sample.db&quot;)<br>    cur = conn.cursor()<br>    <br>    # Enable foreign key support in SQLite.<br>    cur.execute(&quot;PRAGMA foreign_keys = ON;&quot;)<br>    <br>    # Create the companies table.<br>    cur.execute(&quot;&quot;&quot;<br>        CREATE TABLE companies (<br>            id INTEGER PRIMARY KEY,<br>            title TEXT NOT NULL,<br>            description TEXT<br>        )<br>    &quot;&quot;&quot;)<br>    <br>    # Create the customers table with a foreign key to companies.<br>    cur.execute(&quot;&quot;&quot;<br>        CREATE TABLE customers (<br>            id INTEGER PRIMARY KEY,<br>            name TEXT NOT NULL,<br>            revenue REAL NOT NULL,<br>            company_id INTEGER,<br>            FOREIGN KEY (company_id) REFERENCES companies(id) ON DELETE SET NULL<br>        )<br>    &quot;&quot;&quot;)<br>    <br>    # Insert sample data into companies.<br>    companies_data = [<br>        (1, &#39;Acme Corp&#39;, &#39;Leading provider of innovative solutions&#39;),<br>        (2, &#39;Beta Inc&#39;, &#39;Specializes in data analytics and insights&#39;),<br>        (3, &#39;Gamma LLC&#39;, &#39;Pioneer in cloud computing services&#39;)<br>    ]<br>    cur.executemany(&quot;INSERT INTO companies (id, title, description) VALUES (?, ?, ?)&quot;, companies_data)<br>    <br>    # Insert sample data into customers.<br>    customers_data = [<br>        (1, &#39;Alice&#39;, 1500.0, 1),<br>        (2, &#39;Bob&#39;, 2400.0, 2),<br>        (3, &#39;Charlie&#39;, 1800.0, 1),<br>        (4, &#39;Diana&#39;, 3200.0, 3),<br>        (5, &#39;Ethan&#39;, 2100.0, 2)<br>    ]<br>    cur.executemany(&quot;INSERT INTO customers (id, name, revenue, company_id) VALUES (?, ?, ?, ?)&quot;, customers_data)<br>    <br>    conn.commit()<br>    return conn</pre><h4>5.2. <strong>Preparing the System Prompt for the LLM</strong></h4><p>Next, I crafted a system prompt that supplies the LLM with the database schema. This prompt informs the LLM about the tables and their columns so that it knows how to generate valid SQL.</p><pre>system_prompt = f&quot;&quot;&quot;<br>You are a helpful assistant that translates natural language queries into SQL queries for the following SQLite database schema.<br><br>Table &quot;companies&quot;:<br>  - id: INTEGER PRIMARY KEY<br>  - title: TEXT NOT NULL<br>  - description: TEXT<br><br>Table &quot;customers&quot;:<br>  - id: INTEGER PRIMARY KEY<br>  - name: TEXT NOT NULL<br>  - revenue: REAL NOT NULL<br>  - company_id: INTEGER (Foreign key referencing companies(id))<br><br>Instructions:<br>  1. When you receive a text query that is relevant to this schema, generate the corresponding SQL query. Return only the SQL query with no additional commentary or characters like ``` or quotes e.g. SELECT * FROM COMPANIES.<br>  2. If you receive a text query that includes an error message, generate a corrected SQL query that fixes the error, again returning only the SQL query.<br>  3. If the query is irrelevant, do not output any extra text.<br>&quot;&quot;&quot;</pre><h4>5.3. <strong>Setting up the ReAct logic</strong></h4><p>The core logic of the agent revolves around the ReAct cycle: it uses the LLM to generate a SQL query from a natural language request, executes the query, and — if any errors occur — feeds back the error message so that the LLM can adjust and iterate until the query runs successfully.</p><pre>import time<br><br>def test():<br>    nl_request = input(&quot;Enter natural language query: &quot;).strip()<br>    if not nl_request:<br>        print(&quot;No natural language query provided.&quot;)<br>        return<br><br>    conn = sqlite3.connect(&quot;sample.db&quot;)<br><br>    max_iterations = 5<br>    iteration = 0<br>    feedback = None<br>    final_result = None<br><br>    while iteration &lt; max_iterations:<br>        iteration += 1<br>        print(f&quot;--- Iteration {iteration} ---&quot;)<br>        sql_query = generate_sql_query(nl_request, feedback)<br><br>        if not sql_query:<br>            print(&quot;No corresponding SQL query for this query.&quot;)<br>            break<br><br>        result, error = execute_query(conn, sql_query)<br>        <br>        if error:<br>            print(f&quot;SQL execution error: {error}&quot;)<br>            feedback = f&quot;The query resulted in the following error: {error}. Please adjust the SQL query accordingly.&quot;<br>            time.sleep(1)<br>        else:<br>            if not result:<br>                print(&quot;No results found.&quot;)<br>            else:<br>                print(&quot;Query executed successfully. Results:&quot;)<br>                for row in result:<br>                    print(row)<br>            final_result = result<br>            break<br><br>    if final_result is None:<br>        print(&quot;Failed to generate a valid SQL query after several iterations.&quot;)<br><br>    conn.close()</pre><p>Because our sample database is quite simple, the agent usually generates the correct SQL query from a relevant text request in just one go. However, if the database were more complex, the ReAct agent might require two or three iterations to refine its query based on feedback, ensuring the final SQL accurately matches the intended request.</p><h3>6. Conclusion</h3><p>In this article, we explored the history of agents, then moved to LLM Agents with a focus on the ReAct framework that merges reasoning and acting. In the upcoming articles, we’ll dive into other LLM agent architectures and work on an interesting end-to-end project. Stay tuned!</p><p>If you have any feedback, suggestions or questions, do not hesitate to shoot me an email at: dania.adimi@infinitgraph.ai.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50f62289559d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/llm-agents-a-focus-on-react-framework-50f62289559d">LLM Agents: A focus on ReAct Framework</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM Architectures in Action: Building a Multi-Agent Research Assistant with LangChain and LangGraph]]></title>
            <link>https://medium.com/infinitgraph/llm-architectures-in-action-building-a-multi-agent-research-assistant-with-langchain-and-langgraph-1627f6770101?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/1627f6770101</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Fri, 18 Apr 2025 09:11:46 GMT</pubDate>
            <atom:updated>2025-11-05T07:30:03.676Z</atom:updated>
            <content:encoded><![CDATA[<p>In our last article, we explored what LLM Agents are and introduced the ReAct framework; a pretty powerful approach that blends reasoning and acting. But is that all there is? Definitely not. While ReAct alone can be a potent solution for many applications, a number of use cases demand even more complex architectures built with multiple LLMs.</p><p>In this article, we’ll dive into both single-agent and multi-agent architectures, go through some of the frameworks you can use to build your own LLM agents quickly, and as a delightful bonus, we’ll work on a research assistant with LangChain and LangGraph.</p><h3><strong>1. Components of an LLM Agent</strong></h3><p>To recap from our previous article, an LLM agent consists of a <strong>large language model</strong> given a specific task and role that enables it to <strong>reason effectively</strong>, along with a set of <strong>tools </strong>to complete its tasks and a <strong>memory </strong>component that allows it to learn from previous experiences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mEwPGWT0Lt5gSLGJp48CCw.png" /><figcaption>Figure 01 — Illustration of the Components of an LLM Agent</figcaption></figure><p><strong>Reasoning </strong>allows the LLM agent to actively “think” or “reason” throughout the problem-solving process. You can read more about this in our <a href="https://medium.com/infinitgraph/exploring-the-reasoning-capabilities-of-llms-34c36abaebd6">previous article</a>.</p><p><strong>Memory</strong> allows capturing and storing context and feedback across multiple interactions and sessions; <strong>short‑term memory</strong> holds immediate information like conversation history to guide the agent’s next steps, and <strong>long‑term memory</strong> retains knowledge over time to personalize the agent and improve its performance.</p><p><strong>Tools </strong>allow tapping external services such as APIs, web search, email systems, and other computational resources to perform actions and accomplish tasks.</p><p><strong>Retrieval </strong>allows for pulling data, whether it’s the latest market trends, updated company policies, or product inventory, to ensure information is accurate. It can be part of the tools as well.</p><h3>2. Single-agent vs Multi-agent LLM Architectures</h3><p>When we set up an LLM architecture or an “agentic architecture”, we basically have two main patterns to choose from: a <strong>single‑agent</strong> or a <strong>multi‑agent</strong> design. In a single‑agent setup, one LLM acts as the system’s brain, whereas in a multi-agent setup, we can have several agents, each dedicated to a specific cluster of tasks. The choice between the two depends on the use case and how complex the required agent actions are.</p><h4>2.1. Single-agent LLM Architectures</h4><p>A single LLM serves as the “brain,” invoking its tools (APIs, databases, external services) whenever needed and synthesizing results into final outputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w0MlHfuqr4yTlq8DsU4K0w.png" /><figcaption>Figure 02 — IllustratioSn of a Single-agent LLM Architecture</figcaption></figure><p><strong>Strengths</strong></p><ul><li><strong>Low complexity &amp; easy upkeep: </strong>A single‑agent design keeps our architecture lean, there are no tangled workflows or orchestration layers, so we can develop, deploy, and manage it with minimal overhead.</li><li><strong>Zero inter‑agent coordination: </strong>Since there’s only one LLM doing the heavy lifting, we never need to build or debug communication protocols between multiple agents.</li><li><strong>Potentially lighter compute footprint: </strong>Running one powerful model can sometimes cost less overall than juggling several smaller ones, especially when our workload is predictable and fits within a single agent’s capabilities.</li></ul><p><strong>Weaknesses</strong></p><ul><li><strong>Struggles with complex or dynamic tasks: </strong>When workflows branch out or require adaptive decision‑making, one LLM might struggle to give the results we are looking for.</li><li><strong>Limited collaboration &amp; specialization: </strong>Tasks that demand diverse expertise or tight hand‑offs between subtasks often outgrow what a single model can handle well.</li><li><strong>Risk of tool‑overload confusion: </strong>With too many tool integrations, our solo LLM might pick the wrong API or misformat its calls, leading to errors or wasted cycles.</li></ul><h4>2.2. Multi-agent LLM Architectures</h4><p>A multi-agent LLM architecture consists of several agents, each powered by a large language model and equipped with its own memory and tools (the system can also include a shared memory accessible by all agents). These agents collaborate to handle complex tasks that might be challenging for a single LLM to manage alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1GVpDmyz6i_DbQE2Pbfy1Q.png" /><figcaption>Figure 03 — Illustration of a Multi-agent LLM Architecture</figcaption></figure><h3>Strengths</h3><ul><li><strong>Handles complex and dynamic tasks: </strong>By assigning specialized agents to different subtasks, the system can tackle complex tasks more effectively.​</li><li><strong>Parallel processing for efficiency: </strong>Multiple agents can operate simultaneously which reduces response times and improves overall system throughput.​</li><li><strong>Utilizes smaller, specialized models: </strong>Instead of relying on a single large model, the system can employ smaller models tailored to specific tasks, which may optimize resource usage.</li></ul><h3>Weaknesses</h3><ul><li><strong>Increased system complexity: </strong>Coordinating multiple agents adds layers of complexity to the system’s design and operation.</li><li><strong>Requires robust interaction management: </strong>Effective communication protocols are essential to ensure agents work harmoniously and share information accurately.</li><li><strong>Higher resource demands with more agents: </strong>As the number of agents grows, so does the need for computational resources, which can impact the scalability and the overall cost.</li></ul><p>​As we’ve seen, both single-agent and multi-agent architectures come with their own sets of advantages and challenges. Single-agent systems are well-suited for tasks that are straightforward and well-defined, especially when resource constraints aren’t a primary concern. In contrast, multi-agent architectures shine in complex, dynamic scenarios that demand specialized knowledge, collaboration, and scalability. The choice between the two ultimately depends on the specific requirements and complexity of the task at hand.</p><h3>3. <strong>Patterns in multi-agent systems</strong></h3><p>​When designing multi-agent systems, there are a variety of architectural patterns that we can implement or get inspired by, each suited to different needs. Here are some patterns:</p><h4><strong>3.1. Hand-off Pattern</strong></h4><p>​This pattern is effective for building applications where multiple agents collaborate by passing tasks among themselves. It’s particularly useful in scenarios like customer support, task management, and workflow automation, where different steps or responsibilities can be handled by specialized agents.​</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1GVpDmyz6i_DbQE2Pbfy1Q.png" /><figcaption>Figure 04 — Illustration of Hand-off Architecture</figcaption></figure><p>In this setup, each agent is responsible for a specific task or stage in a process. Agents communicate and transfer tasks based on predefined rules or conditions, ensuring a smooth progression through the workflow. For example, in a customer support system, an initial agent might handle general inquiries and then delegate more complex issues to specialized agents.</p><h4>3.2. <strong>Collaborative Filtering</strong></h4><p>This pattern is effective when developing applications where multiple agents collaborate to provide recommendations to users. Each agent brings its own area of expertise, contributing uniquely to the recommendation process.​</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1wE9Ct-d3PxFx3TaK6rh6Q.png" /><figcaption>Figure 05 — Illustration of a Collaborative Filtering Architecture</figcaption></figure><p>For instance, in a stock market recommendation system:​</p><ul><li>An <strong>industry expert agent</strong> analyzes sector-specific trends.​</li><li>A <strong>technical analysis agent</strong> examines price patterns and market indicators.​</li><li>A <strong>fundamental analysis agent</strong> evaluates financial statements and company health.​</li></ul><p>By collaborating, these agents can offer a comprehensive and well-rounded stock recommendation to the user.</p><p>Such multi-agent collaboration frameworks have been explored in research, like the <a href="https://arxiv.org/abs/2402.15235">MACRec framework</a>, which enhances recommendation systems through specialized agents working together.</p><h3><strong>4. Frameworks</strong></h3><p>Here’s a list of some frameworks that can be used to easily build LLM agents taken from awesome <a href="https://github.com/kaushikb11/awesome-llm-agents">LLM agents frameworks</a>:</p><h4><a href="https://github.com/joaomdmoura/crewAI"><strong>CrewAI</strong></a></h4><p>Framework for orchestrating role-playing AI agents.</p><ul><li>Role-based agent design.</li><li>Multi-agent collaboration.</li><li>Flexible memory system..</li><li>Built-in error handling.</li></ul><h4><a href="https://github.com/hwchase17/langchain">Langchain</a></h4><p>Building applications with LLMs through composability.</p><ul><li>Modular and extensible architecture.</li><li>Unified interface for LLMs.</li><li>Pre-built agent toolkits.</li><li>CSV, JSON, and SQL agents.</li><li>Python and Pandas integration.</li><li>Vector store capabilities.</li></ul><h4><a href="https://github.com/microsoft/autogen">Microsoft AutoGen</a></h4><p>Framework for building multi-agent conversational systems</p><ul><li>Multi-agent architecture.</li><li>Customizable agents.</li><li>Code execution support.</li><li>Flexible human involvement.</li><li>Advanced conversation management.</li></ul><h3>5. Building a Research Assistant with LangChain and LangGraph</h3><p>In the rapidly evolving field of artificial intelligence, automating research tasks can significantly enhance productivity. We do have so many already built, but for the sake of learning, we are going to build our own multi-agent research assistant using <strong>LangChain</strong>, <strong>LangGraph</strong>, and <strong>OpenAI’s GPT</strong> models.</p><h4>5.1. Prerequisites</h4><p>Before diving into the code, ensure you have the following packages installed. You can also find them in requirements.txt:</p><pre>arxiv<br>langchain<br>openai<br>langgraph<br>python-dotenv<br>pydantic<br>langchain_community<br>langchain-openai</pre><p>Also, set up your OpenAI API key in a .env file:</p><pre>OPENAI_API_KEY=your-api-key-here</pre><h4>5.1. Overview of the Workflow</h4><p>The research assistant follows a three-phase workflow:​</p><ol><li><strong>Decomposition</strong>: Breaks down a complex research question into manageable subquestions.</li><li><strong>Search</strong>: Queries arXiv for each subquestion to gather relevant academic papers.</li><li><strong>Synthesis</strong>: Combines the findings into a coherent report.</li></ol><h4>5.2. Code Breakdown</h4><p>You can check the full code <a href="https://github.com/adimidania/medium-articles/tree/main/Building%20a%20Multi-Agent%20Research%20Assistant%20with%20LangChain%20and%20LangGraph">here</a> if you want to run it or add edits to it.</p><p><strong>Setting Up the Environment</strong></p><p>The script begins by importing necessary modules and configuring logging to monitor the workflow’s progress.</p><pre>import os<br>import logging<br>from typing import TypedDict, List, Dict, Any, Annotated<br>import arxiv<br>from langchain_openai import ChatOpenAI<br>from langchain.prompts import PromptTemplate<br>from langgraph.graph import StateGraph, END, START<br>from dotenv import load_dotenv<br>import time<br>from datetime import datetime<br>from pydantic import BaseModel<br><br># Set up logging<br>logging.basicConfig(<br>    level=logging.INFO,<br>    format=&#39;%(asctime)s - %(levelname)s - %(message)s&#39;,<br>    datefmt=&#39;%Y-%m-%d %H:%M:%S&#39;<br>)<br>logger = logging.getLogger(&quot;ResearchWorkflow&quot;)<br><br>load_dotenv()</pre><p><strong>Defining the State Schema</strong></p><p>A ResearchState is defined using TypedDict to manage the workflow&#39;s state, including the main question, subquestions, search results, reasoning, and the final report.</p><pre>class ResearchState(TypedDict):<br>    question: str<br>    subquestions: List[str]<br>    results: List[Dict]<br>    reasoning: str<br>    report: str</pre><p><strong>Creating Node Classes</strong></p><p><strong>DecompositionNode</strong> uses an LLM to break down the main research question into subquestions.</p><pre>class Subquestions(BaseModel):<br>    subquestions: List[str]<br><br># Original nodes modified for state management<br>class DecompositionNode:<br>    def __init__(self, llm):<br>        self.llm = llm<br><br>    def decompose(self, question: str) -&gt; List[str]:<br>        logger.info(f&quot;Decomposing question: {question}&quot;)<br>        <br>        prompt_template = &quot;&quot;&quot;You are a research assistant. Decompose the following research question into subquestions:<br>        <br>        Research Question: {question}<br>        <br>        List the subquestions:&quot;&quot;&quot;<br>        prompt = PromptTemplate(template=prompt_template, input_variables=[&quot;question&quot;])<br>        <br>        logger.info(&quot;Sending prompt to LLM for decomposition&quot;)<br>        structured_llm = self.llm.with_structured_output(Subquestions)<br>        start_time = time.time()<br>        response = structured_llm.invoke(prompt.format(question=question))<br>        end_time = time.time()<br>        logger.info(f&quot;LLM response received in {end_time - start_time:.2f} seconds&quot;)<br>        # the response is now a structured object, we need to turn it into a python list<br>        subquestions = response.dict()[&#39;subquestions&#39;]<br>        <br>        logger.info(f&quot;Extracted {len(subquestions)} subquestions&quot;)<br>        for i, subq in enumerate(subquestions):<br>            logger.info(f&quot;  Subquestion {i+1}: {subq}&quot;)<br>        <br>        return subquestions</pre><p><strong>ArxivSearchNode</strong> searches arXiv for each subquestion and retrieves relevant papers.</p><pre>class ArxivSearchNode:<br>    def search(self, query: str, max_results: int = 3) -&gt; List[Dict]:<br>        logger.info(f&quot;Searching arXiv for: &#39;{query}&#39; (max results: {max_results})&quot;)<br>        <br>        start_time = time.time()<br>        search = arxiv.Search(query=query, max_results=max_results)<br>        results = [{<br>            &#39;title&#39;: result.title,<br>            &#39;summary&#39;: result.summary,<br>            &#39;authors&#39;: [author.name for author in result.authors],<br>            &#39;published&#39;: result.published.strftime(&quot;%Y-%m-%d&quot;),<br>            &#39;link&#39;: result.entry_id<br>        } for result in search.results()]<br>        end_time = time.time()<br>        <br>        logger.info(f&quot;Found {len(results)} results in {end_time - start_time:.2f} seconds&quot;)<br>        for i, res in enumerate(results):<br>            logger.info(f&quot;  Result {i+1}: {res[&#39;title&#39;]} ({res[&#39;published&#39;]})&quot;)<br>        <br>        return results</pre><p><strong>SynthesisNode</strong> synthesizes the search results and reasoning into a comprehensive report using the LLM.</p><pre>class SynthesisNode:<br>    def __init__(self, llm):<br>        self.llm = llm<br><br>    def synthesize(self, results: List[Dict], reasoning: str) -&gt; str:<br>        logger.info(f&quot;Synthesizing report from {len(results)} search results&quot;)<br>        <br>        results_str = &quot;\n&quot;.join([f&quot;&quot;&quot;Result {i+1}:<br>                        Title: {res[&#39;title&#39;]}<br>                        Published: {res[&#39;published&#39;]}<br>                        Authors: {&#39;, &#39;.join(res[&#39;authors&#39;])}<br>                        Link: {res.get(&#39;link&#39;, &#39;N/A&#39;)}<br>                        Summary: {res[&#39;summary&#39;]}\n&quot;&quot;&quot; for i, res in enumerate(results)])<br>        <br>        prompt_template = &quot;&quot;&quot;You are an expert research assistant. Create a report based on:<br>        <br>        Internal Reasoning:<br>        {reasoning}<br>        <br>        Search Results:<br>        {results}<br>        <br>        Comprehensive Report:&quot;&quot;&quot;<br>        prompt = PromptTemplate(template=prompt_template, <br>                              input_variables=[&quot;reasoning&quot;, &quot;results&quot;])<br>        <br>        logger.info(&quot;Sending prompt to LLM for synthesis&quot;)<br>        start_time = time.time()<br>        response = self.llm.invoke(prompt.format(<br>            reasoning=reasoning, <br>            results=results_str<br>        )).content<br>        end_time = time.time()<br>        <br>        logger.info(f&quot;Synthesis complete in {end_time - start_time:.2f} seconds&quot;)<br>        logger.info(f&quot;Generated report of {len(response.split())} words&quot;)<br>        <br>        return response</pre><p><strong>Building the Workflow</strong></p><p>The create_research_workflow function sets up the workflow using LangGraph:</p><pre>def create_research_workflow():<br>    llm = ChatOpenAI(api_key=os.getenv(&quot;OPENAI_API_KEY&quot;), model=&quot;gpt-4o-mini&quot;)<br>    decomposer = DecompositionNode(llm)<br>    searcher = ArxivSearchNode()<br>    synthesizer = SynthesisNode(llm)<br><br>    def decompose(state: ResearchState):<br>        subquestions = decomposer.decompose(state[&quot;question&quot;])<br>        return {<br>            &quot;subquestions&quot;: subquestions,<br>            &quot;reasoning&quot;: f&quot;Step 1: Decomposed question into subquestions:\n{subquestions}\n&quot;<br>        }<br><br>    def search(state: ResearchState):<br>        all_results = []<br>        search_reasoning = []<br>        for subq in state[&quot;subquestions&quot;]:<br>            results = searcher.search(subq)<br>            all_results.extend(results)<br>            search_reasoning.append(f&quot;Found {len(results)} results for: {subq}&quot;)<br>        return {<br>            &quot;results&quot;: all_results,<br>            &quot;reasoning&quot;: state[&quot;reasoning&quot;] + &quot;\nStep 2: Search Results:\n&quot; + &quot;\n&quot;.join(search_reasoning)<br>        }<br><br>    def synthesize(state: ResearchState):<br>        report = synthesizer.synthesize(state[&quot;results&quot;], state[&quot;reasoning&quot;])<br>        return {<br>            &quot;report&quot;: report,<br>            &quot;reasoning&quot;: state[&quot;reasoning&quot;] + &quot;\nStep 3: Synthesized final report&quot;<br>        }<br><br>    workflow = StateGraph(ResearchState)<br>    workflow.add_node(&quot;decompose&quot;, decompose)<br>    workflow.add_node(&quot;search&quot;, search)<br>    workflow.add_node(&quot;synthesize&quot;, synthesize)<br>    workflow.add_edge(START, &quot;decompose&quot;)<br>    workflow.add_edge(&quot;decompose&quot;, &quot;search&quot;)<br>    workflow.add_edge(&quot;search&quot;, &quot;synthesize&quot;)<br>    workflow.add_edge(&quot;synthesize&quot;, END)<br>    return workflow.compile()</pre><h4>5.3. Results</h4><p>I’ve asked the research assistant to generate a report on “Evaluation of Large Language Models” which is a topic that I am interested a lot in recently, and I was able to get the following report:</p><pre>## Report on the Evaluation of Large Language Models (LLMs)<br><br>### Introduction<br>This report comprehensively addresses various facets of evaluating Large Language Models (LLMs), focusing on several key questions that capture the nuances of both human and algorithmic assessments. The questions explored include evaluation criteria, evaluation methods, and the roles LLMs can play in their own evaluation.<br><br>### 1. Criteria for Evaluating LLMs<br>While specific criteria for evaluating LLMs were not found in the research, it is crucial to establish that benchmarks should include standards for effectiveness, reliability, and interpretability across varying contexts. Developing diverse criteria <br>allows for a more robust evaluation that considers performance across tasks and domains.<br><br>### 2. Methods for Human Evaluation of LLM Outputs<br>Human evaluation remains a central pillar in assessing LLM outputs. The papers revealed several methodologies:<br>- **Quality Assessment**: A study developed a protocol (Atomic Content Units) to enhance inter-annotator agreement while evaluating summarization outputs (Result 3).<br>- **Workload Reduction**: Frameworks like ScaleEval propose multi-round debates among LLMs as evaluators to assist human annotators with discernment (Result 1).<br>- **Standardized Tests**: The Situational Evaluation of Social Intelligence (SESI) was introduced to evaluate the social intelligence of LLMs and their alignment with human values (Result 2).<br><br>### 3. LLMs as Judges in Evaluation Processes<br>With advancements in LLM capabilities, studies reveal their potential to serve as evaluators:<br>- **LLMs-as-Judges Paradigm**: This framework allows LLMs to assess responses automatically and offers insights into how well they generalize across tasks. Research indicates LLMs can sometimes replace human judges, especially in controlled environments (Result 4, Result 11).<br><br>### 4. Strengths and Weaknesses of Human vs. LLM Evaluation<br>Human evaluation is based on nuanced understanding but can be subjective, whereas LLM evaluation offers consistency and speed but may not grasp human context or subtleties:<br>- **Human Evaluation Advantages**: Greater understanding of context and qualitative nuances (Result 3).<br>- **LLM Evaluation Advantages**: Faster and more efficient processing of large datasets and scalability (Result 4, Result 11).<br>- **Weaknesses of LLMs**: Often struggle with context and may produce biased outputs, especially during role-playing scenarios (Result 16).<br><br>### 5. Metrics for Assessing LLM Outputs<br>Numerous metrics are deployed to evaluate LLM outputs, including:<br>- **Lexical Metrics**: Metrics like BLEU and ChrF, though criticized for being limited (Result 11).<br>- **Semantic Metrics**: Novel frameworks assessed through comprehensive validation reveal LLM-specific methods (Result 3).   <br>- **Quality Evaluation**: Studies demonstrate the need for subjective measures integrated into evaluation frameworks (Result <br>5, Result 10).<br><br>### 6. Evaluation Method Impact on Perceived Performance<br>Different evaluation methods significantly influence LLM performance perceptions:<br>- Comparative studies indicate that using LLMs as evaluators can yield different performance metrics compared to traditional <br>human metrics, causing discrepancies (Result 11).<br><br>### 7. Role of Context in Evaluating LLM Outputs<br>Context is fundamental to LLM evaluation—outputs can significantly vary based on input nuances and situational factors. Research shows that even conventional benchmarks may not capture this context adequately, suggesting a need for more holistic approaches (e.g., real-world application tests) (Result 22).<br><br>### 8. Ethical Implications of Using LLMs as Evaluators<br>The ethical discourse around using LLMs as evaluators reveals several critical concerns:<br>- **Bias and Fairness**: Exposure of biases in LLM-generated outputs necessitates careful implementation (Result 16, Result 19).<br>- **Transparency and Accountability**: Discussions highlight ensuring ethical oversight and human control in automated decision-making processes (Result 20, Result 21).<br><br>### 9. Frameworks or Benchmarks for LLM Evaluation<br>Existing frameworks and benchmarks have emerged to incorporate both human and algorithmic assessments. These frameworks aim to provide balanced evaluations that leverage the strengths of both approaches (e.g., Flames benchmark for safety and fairness) (Result 23).<br><br>### 10. Establishing Feedback Loops Between Human Evaluators and LLMs<br>Feedback mechanisms enhance learning and refinement processes:<br>- **Integration of Human Feedback**: Solutions like Bayesian optimization integrate human feedback to improve LLM performance in tasks such as code generation (Result 26).<br>- **Iterative Improvement Models**: Research proposes frameworks for utilizing feedback from human evaluators to enhance LLM <br>output quality continuously (Result 25).<br><br>### Conclusion<br>The evolving field of LLM evaluation necessitates a multi-faceted approach that blends human insight with automated efficiencies. As LLMs continue to advance, embracing ethical considerations, context-aware assessments, and effective feedback integrations will be vital in achieving benchmarks that improve their reliability and performance across diverse applications. The findings across various studies inform future directions for enhancing LLM evaluation practices, emphasizing the complexity and importance of thoughtful implementations in real-world settings.<br><br>### Recommendations for Future Research<br>1. **Develop Comprehensive Evaluation Frameworks**: Incorporating multi-dimensional metrics that capture both qualitative and quantitative aspects of LLM performance.<br>2. **Establish Ethical Guidelines**: Clear guidelines on the ethical implications surrounding the deployment of LLMs in evaluative roles across various sectors.<br>3. **Invest in Contextual Evaluation**: Prioritize research on context-aware evaluation methods to understand better how situational factors influence LLM outputs.<br>4. **Enhance Collaboration Between Human and LLM Evaluators**: Promote structures that facilitate seamless interactions between human evaluators and LLMs for continuous improvement in evaluation protocols.<br><br>This cumulative analysis aims to foster productive dialogue in the research community, paving the way for more effective and <br>ethical utilization of LLMs in evaluative capacities.</pre><p>I personally think it’s pretty decent! We can refine the prompts more and make the research assistant more powerful by adding more tools or tweak the architecture. Probably a good start if you are looking to get your hands dirty with code.</p><h3>6. Conclusion</h3><p>I hope this article has provided you with a clearer understanding of the distinctions between single-agent and multi-agent LLM architectures. We’ve listed various frameworks that facilitate the development of these agents and finally we’ve walked through building a research assistant using LangChain and LangGraph.</p><p>If you have any feedback, questions, or comments, please send them my way at: dania.adimi@infinitgraph.ai.</p><h3>7. References</h3><ol><li>Agentic Architectures by Weaviate.</li><li>AI Agents for Beginners by Microsoft.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1627f6770101" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/llm-architectures-in-action-building-a-multi-agent-research-assistant-with-langchain-and-langgraph-1627f6770101">LLM Architectures in Action: Building a Multi-Agent Research Assistant with LangChain and LangGraph</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Augmenting LLMs with Retrieval, Tools, and Long-term Memory]]></title>
            <link>https://medium.com/infinitgraph/augmenting-llms-with-retrieval-tools-and-long-term-memory-b9e1e6b2fc28?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/b9e1e6b2fc28</guid>
            <category><![CDATA[llm-tool]]></category>
            <category><![CDATA[llm-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Mon, 17 Mar 2025 10:21:01 GMT</pubDate>
            <atom:updated>2025-03-17T10:21:01.275Z</atom:updated>
            <content:encoded><![CDATA[<p>In our previous articles in the <strong>“Mastering Agentic Workflows: A Deep Dive into LLM Agents”</strong> series, we learned that LLMs are just next-word predictors, and we explored different ways to elicit their reasoning capabilities. Cool, isn’t it?</p><p>But what if we need an LLM to answer questions based on our company’s internal knowledge base, provide real-time news updates, or even compute something as intensive as the Fibonacci number for 34878? Models like GPT-3.5-Turbo or Mistral-7B, on their own, likely wouldn’t deliver accurate responses since they haven’t been trained on our specific data, have a knowledge cut-off date, and fundamentally rely on token prediction rather than true reasoning or iterative calculation.</p><p>Fortunately, researchers have developed various techniques to enhance LLM capabilities. In this article, we’ll explore three key techniques for expanding their potential: <strong>Retrieval Augmented Generation</strong>, <strong>Tools</strong>, and <strong>Long-term Memory</strong>.</p><blockquote><strong>Knowledge cutoff</strong> refers to the specific date up to which the model has been trained on data. This means the model’s understanding and information are limited to events, facts, and knowledge available up to that date. Any developments or occurrences after this cutoff are beyond the model’s training and, therefore, unknown to it.</blockquote><h3>1. Retrieval Augmented Generation</h3><h4>1.1. Motivation</h4><p>Imagine you work for a company called <strong>Instructica </strong>(<em>I’ve spent an absurd amount of time searching for a name that isn’t already taken; hopefully, this one is not</em>). Your company regularly produces internal reports, case studies, and research papers on the latest educational trends and strategies.</p><p>Now, suppose you have a question like, <strong>“What new methods have Instructica discovered for integrating technology in classrooms?”</strong> If you ask ChatGPT or Claude, you likely won’t get the detailed answer you need because they don’t have access to Instructica’s up-to-date internal documents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqE7Ak2J4B1JJv1HCCP0Mw.png" /><figcaption>Figure 01 — ChatGPT’s answer to a question about a fictional company</figcaption></figure><p>In such a case, Retrieval Augmented Generation comes into play. Rather than relying solely on the LLM’s internal knowledge, RAG allows you to leverage your own knowledge base filled with the latest reports, documents, and research papers to answer questions and get well-grounded answers.</p><p><strong><em>Note:</em></strong><em> Here, ChatGPT performed a web search to check for mentions of Instructica rather than relying solely on its internal knowledge base. We’ll discuss tools in more detail in the next section. The key takeaway here is that LLMs — and platforms like ChatGPT that leverage LLMs — don’t have access to our private data, so they can’t answer questions specific to it.</em></p><h4>1.2. <strong>What’s Retrieval Augmented Generation?</strong></h4><p>Retrieval Augmented Generation, or RAG for short, is a relatively new paradigm proposed for the first time by <a href="https://arxiv.org/abs/2005.11401"><strong>Patrick Lewis, et al 2021</strong></a>. It combines the pre-trained <strong>parametric memory</strong> of the model gained during pre-training (and stored in the model’s weights) with a <strong>non-parametric memory</strong> present in external data sources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1OoeWHndqxMHVKEWT3COkA.png" /><figcaption>Figure 02 — Using <strong>Parametric</strong> and <strong>Non-Parametric</strong> Knowledge in Retrieval-Augmented Generation</figcaption></figure><p>Simply put, with RAG, we “look up” relevant data in external sources and feed that information to the large language model. This allows the LLM to generate answers that are more accurate and up-to-date by combining its pre-trained knowledge with the relevant information retrieved from external data sources.</p><h4>1.3. How does Retrieval Augmented Generation work?</h4><p>At a high level, when a user’s query is received, the system first searches an external knowledge base to find the most relevant pieces of information related to that query. These pieces are then combined with the original query to create a well-crafted prompt, which is fed into the large language model. As a result, the LLM generates an answer that is grounded in current, context-specific information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KxDFbYCZRqGmrW9KsGPgvg.png" /><figcaption>Figure 03 — Retrieval Augmented Generation Process</figcaption></figure><p>For example, if you want to know about recent advancements in education discovered by Instructica (<em>I’m starting to like this name!</em>), the system would pull relevant internal reports, case studies, or research papers on education from your knowledge base. These documents, together with your query, are provided to the LLM so that it can generate a detailed and accurate response based on the latest insights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*da7oGuqDLVUhZwHEzvFvbA.png" /><figcaption>Figure 04 — Illustration of the Retrieval Augmented Generation Process</figcaption></figure><h4>1.4. RAG Components</h4><p>To better understand how a RAG system works, we need to understand the different components and how they work together.</p><p><strong>An Embedding Model</strong></p><p>An embedding model transforms text (words, sentences, or documents) into numerical vectors that capture its meaning. These vectors allow us to compare texts based on their content rather than just their exact wording. For example, consider these two sentences:</p><ul><li>“I love chocolate ice cream.”</li><li>“I really enjoy eating chocolate ice cream.”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8QWfUrbQmvZplx-7tkm_wQ.png" /><figcaption>Figure 05 — Illustration of <strong>Semantically</strong> Close Embeddings for Similar Sentences</figcaption></figure><p>Even though the wording is different, an embedding model transforms both sentences into vectors that are very close together in the vector space. This similarity in their vectors means that the model understands they convey almost the same idea, which is the key behind<strong> semantic search</strong>.</p><p>There are numerous embedding models available, both open-source and proprietary. Here are some examples from various providers:</p><ul><li><strong>OpenAI’s text-embedding-ada-002: </strong>This popular model converts text into vectors with <strong>1536 dimensions</strong>. It’s widely used for its robust performance across various applications.</li><li><strong>Google’s text-embedding-004: </strong>This is designed to capture semantic meaning effectively. It typically produces embeddings with around <strong>768 dimensions</strong>.</li></ul><p>You can find more embedding models and their performance on the <a href="https://huggingface.co/spaces/mteb/leaderboard">MTEB leaderboards hosted by Hugging Face</a>. However, it’s important to note that some of these models seem to have been fine-tuned on the MTEB benchmarks, producing inflated performance numbers (<em>that’s what I’ve heard from some colleagues and also have read this on Pinecone’s article on </em><a href="https://www.pinecone.io/learn/series/rag/embedding-models-rundown/"><em>“Choosing an embedding mode</em></a><em>”</em>).</p><p><strong>A Vector Database</strong></p><p>Unlike traditional relational databases with rows and columns, vector databases store vector embeddings with a fixed number of dimensions. They use similarity metrics — such as <strong>cosine similarity</strong>, <strong>dot product</strong>, <strong>Euclidean distance</strong>, or <strong>Hamming distance</strong> — to calculate the distance between embeddings. For example, when we compute the cosine similarity or dot product between the embeddings for “I love chocolate ice cream” and “I really enjoy eating chocolate ice cream,” we obtain a high similarity score. In contrast, comparing two unrelated sentences — like “Atomic Habits provides insights on building good habits” and “Cats are adorable” — yields a much lower similarity score.</p><p>When searching for relevant documents, these systems typically employ advanced algorithms like <strong>Nearest-Neighbors (NN)</strong> or <strong>Approximate Nearest-Neighbors (ANN)</strong> rather than comparing the query with every single document.</p><p><em>In this article, we aren’t going to go in detail about the different similarity metrics and search algorithms, but we highly recommend that you check them out if you are curious.</em></p><p>Some examples of vector databases include:</p><ul><li><strong>Pinecone:</strong> A managed service that scales efficiently, allowing for real-time similarity search on high-dimensional data.</li><li><strong>Qdrant:</strong> An open-source vector database designed for fast and effective similarity searches with easy integration into AI applications.</li></ul><p><strong>A Large Language Model</strong></p><p>You already know what a large language model is, right? If not, be sure to check out <a href="https://medium.com/infinitgraph/foundations-of-llms-the-building-blocks-of-modern-ai-20a80528380e">the first article inthis series</a>. In a RAG system, the LLM combines the query with the relevant documents to craft an answer. It uses its pre-trained knowledge along with the external context retrieved from the vector database to produce an answer that is both accurate and contextually enriched.</p><p><strong>Still with me? Let’s recap:</strong> The system essentially has three components: an embedding model, a vector database and an LLM. The vector database stores the embeddings produced by an embedding model, it contains the “knowledge” we want to rely on when generating answers. That <em>same</em> embedding model is used to encode user queries. The encoded query is used to fetch are relevant information from the vector database. Finally, the LLM uses the retrieved information and the query to generate a well-grounded answer (<em>hopefully!</em>).</p><h4>1.5. How to build a RAG system?</h4><p>To build a RAG system, one of the first tasks is to choose three key components: <strong>an embedding model</strong>, <strong>a vector database</strong>, and <strong>a large language model</strong>. It’s essential to evaluate each option by reviewing benchmarks and comparisons, and by developing proof-of-concepts to see which combination best meets your needs. For example, you might start with commonly used ones like <strong>OpenAI’s text-embedding-ada-002</strong> for creating embeddings, <strong>GPT-4o-mini </strong>for generating responses, and <strong>Pinecone </strong>as a vector database. Starting with these will provide you with a baseline, and then you can later experiment with other models or databases to optimize performance for your specific use case.</p><p><strong><em>Note: </em></strong><em>I am not claiming that this combination is the best to start with, but it’s a common combination that gives promising results. After all, it depends on the usecase you are trying to build, so do some research first, then experiment with things and decide for yourself.</em></p><p>Once you have selected the different components you want to work with, the next step is to prepare your data and store it in the vector database. This process begins with <strong>data preprocessing</strong> — cleaning and standardizing raw documents, such as internal reports, case studies, and research papers, to ensure they’re high quality. Large documents need to be broken down smaller, manageable pieces through a process called <strong>“chunking”</strong>. Each chunk is then transformed into a numerical vector using your chosen embedding model. These vectors, along with associated metadata (e.g. document IDs or page numbers), are stored in the vector database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ck3MFVuzYRZRtgu9kZm30Q.png" /><figcaption>Figure 06 — Illustration of the Process of <strong>Creation </strong>and <strong><em>Augmentation </em></strong>of a Vector Database</figcaption></figure><h4>1.6. RAG Enhancements</h4><p>The architecture described above is often referred to as<strong> ‘Naive Retrieval Augmented Generation’.</strong> Researchers and practitioners are continuously developing new techniques to enhance the RAG process. Given the extensive research in this area, and also the purpose of this article, we can’t cover everything in this article. However, if you’re interested, you can explore recently published articles on RAG in this <a href="https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/research_updates/rag_research_table.md">GitHub repository</a>. But we can highlight a few:</p><p><a href="https://arxiv.org/abs/2212.10496"><strong>Hypothetical Document Embedding (HyDE)</strong></a></p><p>Given a query, the language model generates a <strong>hypothetical document</strong> designed to capture the essence of what a relevant document might look like — even if the generated content isn’t entirely factual. This generated document is then encoded into a vector using a contrastively trained encoder, which filters out inaccuracies and emphasizes key relevance patterns. The resulting vector is used to search through a corpus of document embeddings via similarity measures, effectively retrieving pertinent documents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MT_sOgQJvKRfy_Z87ricVA.png" /><figcaption>Figure 07 — Hypothetical Document Embedding Process</figcaption></figure><p>So rather than directly using the query <strong>“How long does it take to remove a wisdom tooth?” </strong>and compare it with relatively longer documents that might contain multiple sentences,<strong> </strong>we generate a concise hypothetical answer and use it as our search query to find the most relevant documents.</p><p><a href="https://arxiv.org/abs/2305.14283"><strong>Query Rewriting</strong></a></p><p>Ma et al. 2023, introduce a “Rewrite-Retrieve-Read” pipeline that enhances retrieval-augmented LLMs by focusing on refining the search query itself. Instead of only adapting the retriever or reader, the method first uses an LLM to generate a query and then employs a trainable rewriter — optimized via reinforcement learning feedback from a black-box LLM reader — to refine the query. The refined query is used with a web search engine to retrieve relevant context, ultimately improving performance on tasks like open-domain and multiple-choice QA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P_nmcDGlpadn8EBZOwfzLg.png" /><figcaption>Figure 08 — Overview of the proposed pipeline for “Rewrite-Retrieve-Read”</figcaption></figure><h4>1.7. Hands-on Practice</h4><p>In this section, we will have some hands-on and build a proof-of-concept for our one and only <strong>Instructica</strong>. You can check the notebook and the knowledge base <a href="https://github.com/adimidania/medium-articles/blob/main/Augmenting%20LLMs%20with%20Retrieval%2C%20Tools%2C%20and%20Long-term%20Memory/Augmenting%20LLMs%20with%20Retrieval%2C%20Tools%2C%20and%20Long-term%20Memory.ipynb">here</a> if you want to play around with things.</p><p><strong>Step 1: Install Required Libraries</strong></p><p>In this tutorial, we will implement RAG using <strong>FAISS </strong>for retrieval and <strong>Google’s Generative AI</strong> models. Specifically, we will use text-embedding-004 for generating embeddings and gemini-flash-1.5 as the LLM.</p><p><strong>Google’s Generative AI </strong>gives free trial keys so any one of you can build a proof concept and play with things without having to pay a penny. <strong>FAISS</strong>, for those who don’t know it, is an open-source library developed by Meta AI (Facebook AI Similarity Search) for efficient similarity search and clustering of dense vectors. It’s useful for building proof of concepts quickly.</p><pre>!pip install google-generativeai faiss-cpu</pre><p><strong>Step 2: Set Up API and Import Libraries</strong></p><pre>from google.colab import userdata<br>import google.generativeai as genai<br>import faiss<br>import numpy as np<br><br>google_api_key = userdata.get(&#39;GOOGLE_API_KEY&#39;)<br>genai.configure(api_key=google_api_key)</pre><p><em>Before running the code above, make sure to add your Google API Key in </em><strong><em>Google Colab Secrets</em></strong><em>, you can find your Secrets in the left navigation bar.</em></p><p><strong>Step 3: Load and Preprocess Documents</strong></p><p>For the sake of illustration, the knowledge base contains some pieces of information; it contains 10 different paragraphs related to education separated by blank lines.</p><pre>with open(&#39;knowledge_base.txt&#39;, &#39;r&#39;, encoding=&#39;utf-8&#39;) as file:<br>    file_content = file.read()</pre><p>We would read these paragraphs into a list of documents.</p><pre>documents = [para.strip() for para in file_content.split(&#39;\n\n&#39;) if para.strip()]</pre><pre>Document 0: In a groundbreaking discovery in education, researchers at the fictitious Aurora Institute have unveiled a method that transforms traditional classrooms into interactive learning hubs using augmented reality. This discovery promises to reshape how knowledge is delivered.</pre><p><strong>Step 4: Embed Documents and Create FAISS Index</strong></p><p>The next step would be to convert the text into numerical representations using text-embedding-004 from Google AI. We then store these embeddings in an FAISS index, allowing us to efficiently search and retrieve the most relevant documents based on “semantic” similarity.</p><pre>import Faiss<br><br># Embedding documents<br>embeddings = []<br>for doc in documents:<br>    response = genai.embed_content(<br>        model=&quot;models/text-embedding-004&quot;,<br>        content=doc,<br>        task_type=&quot;retrieval_document&quot;<br>    )<br>    embeddings.append(response[&#39;embedding&#39;])<br><br># Creating FAISS index that contains the embedded documents<br>dimension = len(embeddings[0])<br>index = faiss.IndexFlatL2(dimension)<br>embeddings_array = np.array(embeddings).astype(&#39;float32&#39;)<br>index.add(embeddings_array)</pre><p><strong>Step 5: Implement Retrieval Function</strong></p><p>This function takes a user query, converts it into an embedding, and searches the FAISS index to retrieve the most relevant document.</p><pre>def query_faiss_index(user_query, top_k=1):<br>    response = genai.embed_content(<br>        model=&quot;models/text-embedding-004&quot;,<br>        content=user_query,<br>        task_type=&quot;retrieval_document&quot;<br>    )<br>    query_embedding = np.array(response[&#39;embedding&#39;]).astype(&#39;float32&#39;).reshape(1, -1)<br>    distances, indices = index.search(query_embedding, top_k)<br>    relevant_document = [documents[idx].strip() for idx in indices[0] if idx != -1]<br><br>    return relevant_document[0]</pre><p><strong>Step 6: Implement the RAG function</strong></p><p>Now, we just need to implement the RAG function. We first retrieve the most relevant documents, incorporate them into a prompt along with the query, and pass it to the LLM to get the final response.</p><pre>def retrieval_augmented_generation(user_query):<br>    relevant_document = query_faiss_index(user_query)<br>    system_prompt = f&quot;&quot;&quot;<br>    You are a knowledgeable assistant specializing in education who gives well-grounded and informative responses to users&#39; queries. If the user&#39;s query is not relevant to the context, say &#39;I don&#39;t have much information about that&#39; but give them directions, please don&#39;t mention &#39;based on the context&#39;, answer directly.<br>    &quot;&quot;&quot;<br>    prompt = f&quot;&quot;&quot;<br>                  Answer the following user&#39;s query:<br>                  {user_query}<br><br>                  By relying on the following context:<br>                  {relevant_document}<br>              &quot;&quot;&quot;<br>    model = genai.GenerativeModel(model_name=&quot;gemini-1.5-flash&quot;, system_instruction=system_prompt)<br>    response = model.generate_content(prompt)<br>    return response.text</pre><p><strong>Step 7: Implement the RAG function</strong></p><p>Now that our RAG function is implemented, it’s time to test it with two simple queries.</p><p><strong>Query: </strong>Let’s ask a question that is related to what’s present in the knowledge base.</p><pre>print(retrieval_augmented_generation(&quot;What does EdTech startup Lumina Learning suggest?&quot;))</pre><p><strong>Response: </strong>The response we’ve got accurately reflects the information from the knowledge base.</p><pre>Lumina Learning suggests a gamified curriculum that turns regular lessons into engaging quests.  This approach aims to boost student motivation (by 70%, according to their claims), and also develop teamwork and problem-solving abilities.</pre><p><strong>Query: </strong>Now, let’s ask a completely unrelated question, e.g. <strong>“What do you know about InfinitGraph?”</strong></p><pre>print(retrieval_augmented_generation(&quot;What do you know about InfinitGraph?&quot;))</pre><p><strong>Response: </strong>The LLM analyzed the retrieved document in relation to the question, and since they weren’t relevant, it responded that it didn’t have the necessary information.</p><pre>I don&#39;t have much information about InfinitGraph.  To find information about InfinitGraph, I suggest you try searching online using a search engine like Google, Bing, or DuckDuckGo.  You can also try searching on specialized technology websites or forums.</pre><h3>2. Tools</h3><h4>2.1. Motivation</h4><p>Imagine you want an LLM to recommend your outfit for the day based on the current weather, but you’d rather not check the forecast manually every morning. Could it magically pull today’s weather details from its parametric knowledge? Of course not — as we stated above, LLMs have a knowledge cut-off. Similarly, if you ask it to compute the Fibonacci number for 2334, you’re likely to get a rough or even incorrect answer. Because after all, LLMs are just next-word predictors, not precision calculators or “thinkers”.</p><p>That’s where external tools come in. As highlighted in the <a href="https://arxiv.org/abs/2302.04761"><em>Toolformer paper by Meta AI</em></a>, LLMs can learn to use external tools — like checking a weather API for today’s forecast or using a calculator for calculations — when they need them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EIvKLOs3ObSGcNmaXUQE6w.png" /><figcaption>Figure 09 — Illustration of an LLM having access to different tools (e.g. APIs, Web Search and Custom Functions)</figcaption></figure><h4>2.2. How Toolformer Works?</h4><p>With only a few examples for each tool, Toolformer teaches itself how to pick the right moment to call an API, what parameters to send, and how to blend the tool’s response into its output — all without extensive human guidance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QDfCSUUA8u50Bz2zTq_PRA.png" /><figcaption>Figure 10 — <strong>Exemplary predictions of Toolformer. </strong>The model autonomously decides to call different APIs (from top to bottom: a question answering system, a calculator, a machine translation system, and a Wikipedia search engine) to obtain information that is useful for completing a piece of text.</figcaption></figure><p>Toolformer, which is based on <strong>GPT-J</strong>, was fine-tuned on a vast language modeling dataset enriched with potential API calls. Given just a handful of human-written examples showing how an API can be used, the model was allowed to annotate a massive dataset via in-context learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*ga0HHPxJTfbKeKrI7NyJfQ.png" /><figcaption>Figure 11 — An <strong>exemplary prompt P(x)</strong> used to generate API calls for the question answering tool.</figcaption></figure><h4>2.3. Hands-on Practice</h4><p>Let’s see how Gemini models can do functions calling. You can write your own functions to access APIs or perform calculations. Exciting, right? Let’s go through this example where we built a chatbot that can give book recommendations based on a given category.</p><p><strong>Step 01: Write the function</strong></p><pre>def get_book_catalog(category: str):<br>    &quot;&quot;&quot;<br>    List all books in the specified category along with their links.<br><br>    Args:<br>        category (str): The book category. Whether it&#39;s &quot;Islamic&quot; books or &quot;Self-Development&quot; books.<br><br>    Returns:<br>        list[dict]: A list of dictionaries, each containing the title and link of a book.<br>    &quot;&quot;&quot;<br>    if &quot;islam&quot; in category.lower():<br>        return [<br>            {&quot;title&quot;: &quot;Understanding Islam&quot;, &quot;link&quot;: &quot;https://www.islam.gov.qa/BinZaid/lang-ebook/pdf/Understanding-Islam-English.pdf&quot;},<br>            {&quot;title&quot;: &quot;The Prophet of Islam ﷺ Muhammad&quot;, &quot;link&quot;: &quot;https://drive.google.com/file/d/1lb4xqzmexSNuxjX_oRHvkHuJdLCum3vd/view?usp=sharing&quot;},<br>            {&quot;title&quot;: &quot;Muhammad ﷺ in the Bible&quot;, &quot;link&quot;: &quot;https://drive.google.com/file/d/1lUS5tRwHaFLueUyWKWCRR5wLpguTsVf_/view?usp=sharing&quot;}<br>        ]<br>    elif &quot;self-development&quot; in category.lower():<br>        return [<br>            {&quot;title&quot;: &quot;Atomic Habits&quot;, &quot;link&quot;: &quot;https://jamesclear.com/atomic-habits&quot;},<br>        ]<br>    else:<br>        return [&quot;No books available yet.&quot;]</pre><p><strong>Step 02: Provide the LLM with the function tools</strong></p><pre>model = genai.GenerativeModel(<br>    model_name=&quot;gemini-1.5-flash&quot;, tools={&quot;get_book_catalog&quot;: get_book_catalog}<br>)<br><br>chat = model.start_chat(enable_automatic_function_calling=True)</pre><p><strong>Step 03: Test the LLM ability to leverage the function</strong></p><p><strong>Query: </strong>Let’s ask the chatbot to give us some books to learn more about Islam.</p><pre>response = chat.send_message( &quot;Can you give me some books to learn more about islam? Please provide the links.&quot; ) <br><br>print(response.text)</pre><p><strong>Response: </strong>Excellent! As you can see, it gave us all the books that were present in the list, along with their links.</p><pre>Here are some books about Islam:<br><br>* Understanding Islam: (https://www.islam.gov.qa/BinZaid/lang-ebook/pdf/Understanding-Islam-English.pdf)<br>* The Prophet of Islam ﷺ Muhammad: (https://drive.google.com/file/d/1lb4xqzmexSNuxjX_oRHvkHuJdLCum3vd/view?usp=sharing)<br>* Muhammad ﷺ in the Bible: (https://drive.google.com/file/d/1lUS5tRwHaFLueUyWKWCRR5wLpguTsVf_/view?usp=sharing)<br><br>Please note that the availability of these links might change over time.</pre><p>However, don’t mind Gemini, the links aren’t going to change. If you are interested, you can read them using the same links (<em>Highly recommend them!</em>)</p><ul><li><a href="https://www.islam.gov.qa/BinZaid/lang-ebook/pdf/Understanding-Islam-English.pdf">Understanding Islam.</a></li><li><a href="https://drive.google.com/file/d/1lb4xqzmexSNuxjX_oRHvkHuJdLCum3vd/view?usp=sharing">The Prophet of Islam ﷺ Muhammad.</a></li><li><a href="https://drive.google.com/file/d/1lUS5tRwHaFLueUyWKWCRR5wLpguTsVf_/view?usp=sharing">Muhammad ﷺ in the Bible.</a></li></ul><p>You can learn more about Gemini function calling using <a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling">this resource</a>.</p><h3>3. Long-term Memory</h3><h4>3.1. Motivation</h4><p>Imagine you’re using an LLM as your personal companion or assistant. You’ve spent days (maybe weeks) chatting with it about your favorite foods, your daily routines, and your personal goals. But one day you notice: it starts <strong>forgetting</strong> crucial details you told it in the past. Maybe it no longer remembers you’re allergic to peanuts, or it gives you the same book recommendation you already dismissed a month ago.</p><p>This happens because standard LLMs rely almost entirely on <strong>short-term memory</strong> — the context window of the ongoing conversation. Once that window becomes crowded with new exchanges, older information drops out, and the model can’t “pay attention” to it anymore.</p><p>But what if you could <strong>supercharge</strong> your LLM with a <strong>long-term memory</strong> — a place where all personal preferences, and important facts remain accessible, no matter how long ago you mentioned them? Then the model could truly act as a consistent personal companion:</p><ul><li>Recalling your favorite restaurants and dietary restrictions,</li><li>Remembering prior goals or advice you found helpful,</li><li>Picking up exactly where you left off, even if you last chatted days or weeks ago.</li></ul><p>This is the gap that <strong>long-term memory</strong> mechanisms fill.</p><h4>3.2. MemoryBank: An Insightful Approach to Long-Term Memory for LLMs</h4><p>The paper <a href="https://arxiv.org/abs/2305.10250"><em>MemoryBank: Enhancing Large Language Models with Long-Term Memory</em></a> by Zhong et al. offers a very insightful implementation of long-term memory for LLMs. However, it’s important to note that this is not the only possible approach — there are several ways to design and integrate long-term memory into LLMs. In fact, if you go now, create a new conversation and ask ChatGPT “What do you know about me?”, you’d be surprised (<em>go try it now!</em>).</p><p>In this article, we are going to highlight the MemoryBank approach as a compelling case study, showcasing its methods and benefits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vI-voiTzURR-IWn2RkS0DQ.png" /><figcaption>Figure 12 — Overview of MemoryBank. The <strong>memory storage</strong> stores past conversations, summarized events, and user portraits, while <strong>the memory updating mechanism</strong> updates the memory storage. <strong>Memory retrieval</strong> recalls relevant memory.</figcaption></figure><p><strong>Memory Storage: The Warehouse of Memories</strong></p><ul><li><strong>Persistent Logging: </strong>MemoryBank captures every interaction with detailed logs, including timestamps. This forms a chronological record of all conversations, ensuring that no important detail — like a dietary restriction or a personal goal — is lost.</li><li><strong>Hierarchical Summaries: </strong>To keep the memory manageable, detailed dialogues are periodically distilled into concise summaries. These summaries provide high-level overviews of daily events and key interactions, much like how humans remember “the gist” of an experience rather than every minute detail.</li><li><strong>User Portraits: </strong>Beyond mere conversation logs, MemoryBank constructs dynamic user portraits. These profiles encapsulate a user’s personality traits, recurring preferences, and evolving interests, enabling the LLM to tailor its responses over time.</li></ul><p><strong>Memory Retrieval: The Recollection Process</strong></p><ul><li><strong>Contextual Querying: </strong>When the LLM needs to recall past details, it doesn’t search through raw conversation logs. Instead, the current context is encoded into a vector, which is then matched against pre-encoded memory vectors. This method ensures that only the most relevant memories are retrieved (Remember RAG? Right?).</li><li><strong>Embedding-Based Search: </strong>By leveraging dense retrieval techniques and tools like FAISS, MemoryBank efficiently searches its repository. This allows the AI to quickly pull in the right memory pieces — be it a past recommendation or a previously shared personal detail — into the current conversation.</li><li><strong>Dynamic Integration: </strong>Once retrieved, these memories are seamlessly integrated into the LLM’s prompt, providing a rich contextual backdrop that guides the generation of personalized and coherent responses.</li></ul><p><strong>Memory Updating</strong></p><ul><li><strong>Selective Forgetting: </strong>Drawing inspiration from the Ebbinghaus Forgetting Curve, MemoryBank ensures that not all stored memories remain equally strong. Over time, if a memory isn’t recalled or reinforced, its strength decays and it may eventually be pruned from active storage. This selective forgetting keeps the memory bank relevant and uncluttered.</li><li><strong>Reinforcement of Key Memories: </strong>Conversely, memories that are frequently accessed are reinforced. Each time a memory is recalled, its “strength” is boosted, ensuring that important details persist over longer periods — mirroring how human memory works through repeated retrieval.</li><li><strong>Adaptive Updates: </strong>As the conversation evolves, MemoryBank continuously updates its summaries and user portraits. This adaptive mechanism ensures that the LLM’s understanding of a user stays current, even as preferences or circumstances change.</li></ul><h4>3.2. SiliconFriend: A Case Study in Long-Term AI Companionship</h4><p>In the paper, the researchers also introduced SiliconFriend, which is a bilingual LLM-based chatbot that leverages the MemoryBank mechanism to excel in long-term AI companion scenarios. Tuned on psychological dialogue data using LoRA (Low-Rank Adaptation), SiliconFriend exhibits heightened empathy and contextual understanding. Experimental evaluations — both qualitative analyses with real-world user dialogs and quantitative tests with simulated interactions generated by ChatGPT — demonstrate that SiliconFriend, equipped with MemoryBank, effectively recalls relevant memories and accurately understands user personality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lG6NDUI9KKO-FjXBX0EU9A.png" /><figcaption>Figure 13 — Example responses from SiliconFriend to users with different personalities.</figcaption></figure><p>You can check their <a href="https://github.com/zhongwanjun/MemoryBank-SiliconFriend">Github repository</a> if you are curious about how SiliconFriend was implemented.</p><h3>4. Conclusion</h3><p>In this article, we explored how augmenting LLMs with retrieval-augmented generation, tool integration, and long-term memory can unlock new capabilities. With retrieval-augmented generation, we can build applications where the LLM leverages external knowledge — from scholarly books to company data — to answer questions, rather than relying solely on its internal knowledge. Tools enable us to leverage real-time data, web searches, and custom functions to do more interesting tasks with LLMs. Long-term memory enables us to create LLM-based chatbot that can recall past interactions, adapt to evolving user preferences, and maintain context over extended conversations.</p><p>Very interesting, right? Stay tuned for more exciting articles coming up! In the meantime, if you have any questions, feedback, or suggestions, feel free to contact me on my professional email: <a href="/service/mailto:dania.adimi@infinitgraph.ai">dania.adimi@infinitgraph.ai</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9e1e6b2fc28" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/augmenting-llms-with-retrieval-tools-and-long-term-memory-b9e1e6b2fc28">Augmenting LLMs with Retrieval, Tools, and Long-term Memory</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring the Reasoning Capabilities of LLMs]]></title>
            <link>https://medium.com/infinitgraph/exploring-the-reasoning-capabilities-of-llms-34c36abaebd6?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/34c36abaebd6</guid>
            <category><![CDATA[llm-reasoning]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Thu, 27 Feb 2025 18:16:01 GMT</pubDate>
            <atom:updated>2025-03-11T17:45:16.485Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine a world where LLMs can solve the hardest math problems that have challenged humans for centuries, discover new scientific theories, or even reach true general intelligence. Many of us have heard grand promises of LLMs or AI systems that can do just that, almost like they are magic.</p><p>But what’s really happening behind the scenes? In this article, we’ll take a closer look at how large language models really work. We’ll explore techniques like <strong>chain-of-thought</strong> and<strong> least-to-most prompting</strong> that give these models a spark of reasoning — even though, at their core, they don’t think and reason like we do. And we’ll also have a look at two famous reasoning models, <strong>DeepSeek R1 </strong>and<strong> OpenAI’s o1</strong>, which are quickly making waves.</p><p>This is the second article of our series <strong>“Mastering Agentic Workflows: A Deep Dive into LLM Agents”. </strong>If you haven’t already read our first article on LLMs foundations, you can catch up <a href="https://medium.com/infinitgraph/foundations-of-llms-the-building-blocks-of-modern-ai-20a80528380e">here</a>.</p><h3><strong>1. What are Large Language Models?</strong></h3><p>At their core, large language models are essentially next-word predictors. They work by calculating the probability of each possible next word and then choosing the one that’s most likely to follow from what they’ve seen during training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VRZF7wLB2HFBz4TwEADnZA.png" /><figcaption>Figure 01 — Simplified illustration of how LLMs generate the next token starting from a context</figcaption></figure><p>As <strong>Denny Zhou</strong>, a Principal Scientist at Google DeepMind, said, <strong>an LLM is like a parrot</strong> that repeats words without understanding their meaning; it “repeats” patterns of language without true comprehension or reasoning (<em>I have four parrots myself that I didn’t even get them to repeat a word; but still I like the analogy</em>).</p><p>Anyway, forget about my parrots. Let’s consider this very simple puzzle. Can you guess the answer?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ero6MYCZz7yydqpkoSklew.png" /><figcaption>Figure 02 — Simple puzzle</figcaption></figure><p>As humans, we can reason through examples and quickly recognize patterns. In this case, we naturally figure out that the task involves taking full names and joining the last letter of each name to form the answer.</p><p>However, for LLMs, this might not be straightforward., I gave the examples to <strong>gpt-4o-mini,</strong> and it failed miserably to give the correct answer, which is “es”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t5fCSakcxxGtQTlb51Vdtg.png" /><figcaption>Figure 03 — ChatGPT’s gpt-4o-mini answer to the puzzle.</figcaption></figure><p>Is there a way to help LLMs with such tasks? <strong>Absolutely!</strong> Researchers and professionals have been exploring ways to enhance LLMs’ reasoning abilities — whether through prompting techniques, training, fine-tuning, or other advanced techniques.</p><h3>2. <strong>Prompt Engineering</strong></h3><p>Before we dive into how LLMs reason, let’s get the basics straight. You might have heard about “Prompt engineering” or even seen job titles like “Prompt Engineer” (<em>some of them are very well paid</em>). What is it all about?</p><p>Simply put, <strong>prompt engineering</strong> is about writing clear and effective instructions (or prompts) for language models. The goal is to guide them so they produce the best possible answers. Experts like <strong>Sinan Ozdemir</strong> say it’s a skill that mixes an understanding of language, the subject you’re working on, and knowing what the model can and can’t do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bdUmodbfe9GUTra2zY6L8A.png" /><figcaption>Figure 04 — Prompt Engineering Process</figcaption></figure><p>Prompt engineering is <strong>an iterative process</strong>. You start by writing a prompt, see how the model responds, and then tweak your prompt based on that feedback. This cycle of drafting, testing, and refining continues until you get the best possible output.</p><h3>3. <strong>Prompt Engineering Techniques for Enhanced Reasoning</strong></h3><h4><strong>3.1. Zero-shot prompting</strong></h4><p>In a zero-shot setting, you simply give the LLM a natural language description of what you need it to do — no examples are required. For example, if you ask it to correct the grammar of a sentence, the model can usually understand and complete the task accurately on its own.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l1rM6hqF4xPpSSCDmYjSsA.png" /><figcaption>Figure 05 — Asking gpt-4o-mini to correct the grammar of a sentence</figcaption></figure><p>This works because large language models have been exposed to vast amounts of text during pre‐training, allowing them to internalize a wide range of grammatical rules and linguistic patterns. Without needing examples, the model “understands” what is being asked purely from the instruction.</p><h4>3.2. Few-shot prompting</h4><p>Few-shot learning — also known as <strong>in-context learning (ICL)</strong> — allows large language models to adapt to a new task by providing just a handful of examples right in the prompt. Instead of retraining or fine-tuning the model on thousands of labeled examples, you supply a few demonstration pairs that illustrate what the task entails. The model then “learns” from these examples on the fly and tailors its response accordingly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y-GdvWboKSWICQT3o6Jw2g.png" /><figcaption>Figure 06 — Asking gpt-4o-mini to generate a new motivational quote</figcaption></figure><p>The LLM learned from the original quotes how to convey a sense of urgency and motivate someone to not procrastinate. Drawing on that style, it crafted a new quote “Procrastination borrows time, but time never lends it back.”</p><h4>3.3. Chain-of-thought prompting</h4><p>Researchers at Google introduced Chain-of-Thought prompting in a paper titled <a href="https://arxiv.org/abs/2201.11903"><strong>“Chain of Thought Prompting Elicits Reasoning in Large Language Models”</strong></a> in 2022. This research highlighted how guiding a model through a series of intermediate reasoning steps significantly improved its performance on tasks such as mathematical problem-solving, logical reasoning, and multi-hop question answering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KC_2mnxnpY5vksOeLo1IBw.png" /><figcaption>Figure 07 — Standard Prompting vs Chain-of-Thought Prompting</figcaption></figure><p>The idea of incorporating explicit reasoning steps into model outputs has been around long before 2022. For example, DeepMind’s work by <strong>Ling et al. in 2017 </strong>— detailed in <a href="https://aclanthology.org/P17-1015.pdf">“Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems”</a> — introduced the concept of deriving a final answer through intermediate steps. Building on this early foundation, OpenAI’s <strong>Cobbe et al. in 2021</strong> further advanced the approach with the <strong>GSM8K dataset</strong> in <a href="https://arxiv.org/pdf/2110.14168">“Training Verifiers to Solve Math Word Problems.”</a> They demonstrated how providing natural language rationales can guide models, such as GPT-3, to generate step-by-step solutions, thus highlighting the power of structured reasoning in enhancing model performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IhN-G8wGmb_QNZ1Jpj7G5g.png" /><figcaption>Figure 08 — Three example problems from GSM8K. Calculation annotations are highlighted in red.</figcaption></figure><h4>3.4. Few-shot chain-of-thought</h4><p>We can combine the explicit reasoning benefits of chain-of-thought prompting with few-shot learning to achieve better results on more complex tasks that require reasoning before responding. In a few-shot chain-of-thought prompting, a model is provided with several examples that include not just the final answer but also the intermediate steps leading to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z26Gzm5KSa_qFzbnl2NMng.png" /><figcaption>Figure 09 — Few-shot vs Few-shot-CoT prompting</figcaption></figure><p>Revisiting our earlier example, providing gpt-4o-mini with a few demonstrations that include explicit reasoning enables it to produce the correct answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KkU-CxyTINF93h5ek1Tmcg.png" /><figcaption>Figure 10 — gpt-4o-mini solving the puzzle through Few-shot-CoT</figcaption></figure><h4>3.5. Zero-shot chain-of-thought</h4><p>In the <strong>zero-shot chain of thought prompting</strong>, we add a subtle cue — like “Let’s think step by step” — to the prompt. This small hint encourages the model to generate intermediate reasoning steps before arriving at the final answer, even though it hasn’t seen any similar examples during the conversation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAtmsyFt7fQDhHDe-h9GAQ.png" /><figcaption><a href="https://arxiv.org/abs/2205.11916">Figure 11 — Zero-shot vs Zero-shot-CoT prompting</a></figcaption></figure><p>We can also set up a <strong>two-model system</strong> to tackle these tasks. In this approach, one LLM is dedicated to reasoning extraction — it generates a detailed chain-of-thought that walks through the problem step by step. Then, a second LLM takes that reasoning and performs answer extraction, distilling the explanation into a final, concise response. This separation allows each model to focus on what it does best: the first on laying out the logical steps and the second on summarizing and verifying the final answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*PXwEBdj8n04q1Wk7bmBxDA.png" /><figcaption><a href="https://arxiv.org/abs/2205.11916">Figure 12 — Zero-shot-CoT in a two-model approach</a></figcaption></figure><h4>3.6. Least-to-Most Prompting</h4><p>Although chain-of-thought prompting significantly enhances reasoning, it can fail to extrapolate beyond the complexity level shown in its few-shot examples — especially for tasks involving compositional generalization.</p><p>Researchers at Google Research introduced <a href="https://arxiv.org/abs/2205.10625"><strong>least-to-most prompting</strong></a> as a way to handle the “easy-to-hard” generalization issues of chain-of-thought prompting. This approach operates in <strong>two stages: Decomposition into Subquestions </strong>and <strong>Sequentially Solving Subquestions </strong>(as illustrated in the figure below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9ru21430bMkeZsjkU3Udrg.png" /><figcaption>Figure 13 — Least-to-most prompting solving a math word problem in <strong>two stages:</strong> (1) query the language model to decompose the problem into subproblems; (2) query the language model to sequentially solve the subproblems.</figcaption></figure><p><strong>1. Decomposition Stage</strong></p><ul><li>The complex problem is first broken down into a list of simpler subproblems. This decomposition step is itself performed by prompting: the model is given a few examples of how to rewrite a difficult question as a sequence of easier tasks.</li><li>The goal is to ensure that each subproblem is significantly less complex, making it straightforward for the model to solve.</li></ul><p><strong>2. Sequential Solving Stage</strong></p><ul><li>After the problem has been decomposed, the model solves the subproblems in order, using the answer from each subproblem to inform the next.</li><li>Crucially, both decomposition and solving rely on prompting alone — no additional fine-tuning or training is required. The model effectively teaches itself to handle harder questions by starting with simpler, intermediate steps.</li></ul><h4>3.7. Analogical Reasoning</h4><p>Analogical reasoning is a fundamental cognitive skill that allows one to draw parallels between seemingly disparate concepts by identifying underlying similarities. This is vividly illustrated by Stefan Banach’s famous quote:</p><blockquote><strong>“A mathematician is a person who can find analogies between theorems; a better mathematician is one who can see analogies between proofs and the best mathematician can notice analogies between theories. One can imagine that the ultimate mathematician is one who can see analogies between analogies.”</strong></blockquote><p>In the context of LLMs, analogical reasoning refers to the model’s ability to identify and leverage structural or conceptual similarities across different problems or domains. Recent research (see, for example,<a href="https://arxiv.org/abs/2310.01714"> arXiv:2310.01714</a>) explores how large language models can perform analogical reasoning by drawing on their vast, diverse training data.</p><p>For instance, if we already know how to calculate the area of one square, we can reuse that method for a new square — simply by adapting the side length. In the diagram, the model first retrieves an example of a similar problem (“Relevant exemplars”) and then <strong>transfers</strong> that solution pattern to the new question (“Solve the initial problem”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4sQsMNu76lZTnK8Yjrf4ow.png" /><figcaption>Figure 14 — Overview of our approach, analogical prompting.</figcaption></figure><p>Another approach is to keep a <strong>corpus of example problems</strong> on hand and <strong>dynamically retrieve</strong> whichever examples are most similar to the new query. You then <strong>add those examples</strong> to the prompt rather than just telling the LLM to “recall relevant exemplars.” This way, the model is guided by concrete, relevant solutions.</p><h3>4. Training or Fine-tuning for Reasoning</h3><p>For instance, as demonstrated by <strong>Ling et al. (2017)</strong> in their work available at<a href="https://arxiv.org/pdf/1705.04146"> this link</a>, training a model on examples that include intermediate steps helps the system learn the process of decomposing complex problems into simpler subproblems. This method teaches the model to internalize a reasoning process rather than simply memorizing input–output pairs.</p><p>Building on this idea, subsequent studies like <strong>Cobbe et al. (2021)</strong> and <strong>Nye et al. (2021)</strong> have shown that fine-tuning pre-trained models on datasets annotated with detailed reasoning steps further improves their ability to solve multi-step tasks. These fine-tuning techniques enable the models to generate their own chain of thought internally when presented with a new problem rather than relying on a fixed prompt during inference.</p><h3>5. <strong>Reasoning Models</strong></h3><p>Two of the biggest names currently trending on LinkedIn are <strong>DeepSeek R1 by DeepSeek</strong> and <strong>o1 by OpenAI</strong>. How do these models differ from earlier ones like <strong>GPT-3.5-Turbo</strong> and <strong>Claude-5</strong>?</p><h4>5.1 OpenAI’s o1</h4><p><a href="https://openai.com/index/learning-to-reason-with-llms/"><strong>OpenAI o1</strong></a> is a new series of models from OpenAI that excels in complex reasoning tasks, using chain-of-thought reasoning to outperform GPT-4o in areas like math, coding, and science.</p><p>The first thing you’ll notice when interacting with o1 is that it takes noticeably longer to generate responses compared to GPT-4o. This deliberate pause reflects the model’s emphasis on reasoning. O1 spends more time “thinking” before responding, allowing it to tackle complex tasks and solve harder problems in logic, math, coding, and science.</p><p><strong>How OpenAI’s o1 Works?</strong></p><ul><li><strong>Deliberate Reasoning Process: </strong>o1 is built to spend additional compute time at inference, effectively simulating a “thinking” phase. During this period, it internally generates a <strong>chain of thought</strong> — as explained earlier, it’s an ordered sequence of reasoning steps that lead to its final answer. This internal process is hidden from users but allows the model to explore multiple potential reasoning paths.</li><li><strong>Reinforcement Learning for Better Thinking: </strong>The model leverages reinforcement learning to improve its reasoning capabilities. During training, o1 receives feedback not only on the correctness of its final answer but also on the quality of its intermediate reasoning. This helps it learn to refine its internal thought process, try different strategies, and even self-correct before providing a final response.</li><li><strong>Compute Allocation: </strong>By dedicating more computational resources to “thinking,” o1 can tackle more complex tasks — such as advanced mathematics, coding challenges, and scientific problem-solving — that require multi-step reasoning.</li></ul><p><strong>Reasoning capabilities of OpenAI’s o1</strong></p><p>To showcase o1’s enhanced reasoning capabilities compared to GPT-4o, OpenAI evaluated the model on a variety of challenging math, coding, and science benchmarks.</p><p><strong>Human examination</strong> demonstrates that <strong>o1 </strong>consistently outperforms <strong>GPT-4o</strong> nad <strong>o1-preview</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zYnlosLfk8Ew6DoF_9ua1g.png" /><figcaption><a href="https://openai.com/index/learning-to-reason-with-llms/">Figure 15 — o1 greatly improves over GPT-4o on challenging reasoning benchmarks.</a></figcaption></figure><p>Looking at the results on <strong>ML benchmarks</strong>, o1’s advancements in reasoning are fairly obvious. On both MathVista (a mathematics benchmark) and MMLU, o1 showcases substantial gains in accuracy compared to GPT-4o.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KqFTp-Vm5s564z5PNA6gXw.png" /><figcaption><a href="https://openai.com/index/learning-to-reason-with-llms/">Figure 16 — o1 improves over GPT-4o on a wide range of benchmarks, including 54/57 MMLU subcategories.</a></figcaption></figure><h4>5.2 DeepSeek R1</h4><p><a href="https://www.deepseek.com/"><strong>DeepSeek-R1</strong></a> is an open-source reasoning model developed by DeepSeek, a Chinese AI company, to address tasks requiring logical inference, mathematical problem-solving, and real-time decision-making.</p><p><strong>How DeepSeek R1 Works?</strong></p><p>DeepSeek operates by combining several innovative techniques to achieve efficient, high-performance reasoning while keeping training costs and resource demands low. Here’s a breakdown of how DeepSeek works:</p><ul><li><strong>Targeted Reinforcement Learning: </strong>DeepSeek’s R1 models are trained using large-scale reinforcement learning that is specifically tuned for complex reasoning tasks. Instead of focusing on general language generation, the model learns to solve multi-step problems by exploring different reasoning paths and receiving feedback on its performance.</li><li><strong>Rule-Based Reward Engineering: </strong>Rather than using standard neural reward models, DeepSeek employs a custom, rule-based reward system. This engineered framework provides precise, deterministic feedback that guides the model to develop and refine its internal reasoning process more effectively.</li><li><strong>Efficient Knowledge Distillation: </strong>DeepSeek leverages distillation techniques to transfer the capabilities of larger models into much smaller ones — down to models with as few as 1.5 billion parameters. This process compresses the reasoning and problem-solving abilities of the model, allowing it to perform at a high level without requiring massive computational resources.</li><li><strong>Emergent Behavior Network: </strong>One of the key discoveries in DeepSeek’s development is that, through reinforcement learning, the model naturally develops sophisticated reasoning patterns — even without those behaviors being explicitly programmed. This emergent behavior enables the model to handle complex tasks by “thinking” through problems in a multi-step, logical manner.</li></ul><p><em>If you want to read more about DeepSeek R1, check out their </em><a href="https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf"><strong><em>release paper</em></strong></a><em>.</em></p><p><strong>Reasoning capabilities of DeepSeek R1</strong></p><p>DeepSeek-R1 competes directly with OpenAI o1 across several benchmarks, often matching or surpassing OpenAI’s o1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L-6DcaDO4rca68eGJ4vpjA.png" /><figcaption><a href="https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf">Figure 17 — Benchmark performance of DeepSeek-R1 taken from DeepSeek release paper.</a></figcaption></figure><h3>5. Conclusion</h3><p>While LLMs may sometimes seem like magic, at the end of the day, they are probabilistic models that generate the next token — If there’s one thing you should take from this article, it should be this. As we’ve explored, methods like chain-of-thought and least-to-most prompting help large language models tackle complex tasks more effectively. Efforts to make LLMs exhibit advanced reasoning capabilities are ongoing, as seen in reasoning models and the techniques we presented.</p><p>I highly recommend going through the <a href="https://github.com/atfortes/Awesome-LLM-Reasoning"><strong>Awesome LLM Reasoning</strong></a> repository as it has a curated list of papers and resources on LLM reasoning, and it’s constantly being updated!</p><p>In the next articles of our series, <strong>“Mastering Agentic Workflows: A Deep Dive into LLM Agents,”</strong> we’ll get into some exciting topics and even have some hands-on experiments. Stay tuned!</p><p>If you have any questions, remarks, or contributions, feel free to shoot me an email at: <a href="/service/mailto:dania.adimi@infinitgraph.ai">dania.adimi@infinitgraph.ai</a>.</p><h3>6. References</h3><p>[1] <a href="https://llmagents-learning.org/f24">Large Language Model Agents by University of Berkeley.</a></p><p>[2] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V. and Zhou, D., 2022. <strong>Chain-of-thought prompting elicits reasoning in large language models.</strong> <em>Advances in neural information processing systems</em>, <em>35</em>, pp.24824–24837.</p><p>[3] Ling, W., Yogatama, D., Dyer, C. and Blunsom, P., 2017. <strong>Program induction by rationale generation: Learning to solve and explain algebraic word problems.</strong> <em>arXiv preprint arXiv:1705.04146</em>.</p><p>[4] Kojima, T., Gu, S.S., Reid, M., Matsuo, Y. and Iwasawa, Y., 2022. <strong>Large language models are zero-shot reasoners.</strong> <em>Advances in neural information processing systems</em>, <em>35</em>, pp.22199–22213.</p><p>[5] Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. and Chi, E., 2022. <strong>Least-to-most prompting enables complex reasoning in large language models.</strong> <em>arXiv preprint arXiv:2205.10625</em>.</p><p>[6] Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovec, J., Liang, P., Chi, E.H. and Zhou, D., 2023. Large language models as analogical reasoners. <em>arXiv preprint arXiv:2310.01714</em>.</p><p>[7] Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R. and Hesse, C., 2021. Training verifiers to solve math word problems. <em>arXiv preprint arXiv:2110.14168</em>.</p><p>[8] <a href="https://www.datacamp.com/blog/open-ai-o1">OpenAI o1 Guide: How It Works, Use Cases, API &amp; More.</a></p><p>[9] <a href="https://www.datacamp.com/blog/deepseek-r1">DeepSeek R1: Features, o1 Comparison, Distilled Models &amp; More.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34c36abaebd6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/exploring-the-reasoning-capabilities-of-llms-34c36abaebd6">Exploring the Reasoning Capabilities of LLMs</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Foundations of LLMs: The Building Blocks of Modern AI]]></title>
            <link>https://medium.com/infinitgraph/foundations-of-llms-the-building-blocks-of-modern-ai-20a80528380e?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/20a80528380e</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Alaa Dania Adimi]]></dc:creator>
            <pubDate>Wed, 19 Feb 2025 17:04:52 GMT</pubDate>
            <atom:updated>2025-03-03T15:36:55.879Z</atom:updated>
            <content:encoded><![CDATA[<p>When ChatGPT launched in November 2022, it quickly became a game changer. In just five days, it passed the <a href="https://finance.yahoo.com/news/chatgpt-gained-1-million-followers-224523258.html"><strong>one million user mark</strong></a>, sparking huge excitement among businesses, researchers, and everyday people alike. What many expected to be a small step forward in language model quality turned into an explosion of new applications — from virtual assistants to text-to-video generation models and agentic workflows that can streamline complex business operations.</p><p>Lately, there’s been a lot of buzz around <strong>“agentic workflows”</strong> or <strong>“LLM agents.”</strong> People from all walks of life, whether tech-savvy or not, are excited about using these smart systems to automate personal tasks and create products that can generate revenue.</p><p>Before diving into the world of agentic workflow, it’s essential to understand Large Language Models, as they form the backbone of these intelligent systems. In the first article of our blog series, <strong>“Mastering Agentic Workflows: A Deep Dive into LLM Agents,”</strong> we’ll explore the fundamentals of LLMs and cover everything you need to know to get started.</p><h3><strong>1. A little bit of history</strong></h3><p>Tools like <a href="https://chatgpt.com/"><strong>ChatGPT</strong></a> and <a href="https://claude.ai/"><strong>Claude</strong></a> may seem to have come out of nowhere, they are the culmination of decades of technology. The evolution of Large Language Models is a fascinating journey that has transformed artificial intelligence and natural language processing. This progression can be divided into four major development stages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QlagsIQsAjc7dK_kWbbSrA.png" /><figcaption>Figure 01 — An evolution process of the four generations of language models (LM) from the perspective of task solving capacity taken from “A Survey of Large Language Models”.</figcaption></figure><h4><strong>1.1. Statistical Language Models (SLMs)</strong></h4><p>In the 1990s, SLMs emerged, utilizing statistical methods to predict the likelihood of word sequences. They can predict the next word based on recent context. For example, given the context “My favorite color is __,” a language model that encodes English should predict “blue” more often than “car.” These models were widely used in information retrieval and natural language processing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*N_tcEjJLUTfeoo2rTqb96Q.png" /><figcaption>Figure 02 — Simplified Illustration of Next-Word Prediction Process</figcaption></figure><h4>1.2. <strong>Neural Language Models (NLMs)</strong></h4><p>Advancements in <a href="https://en.wikipedia.org/wiki/Neural_network_(machine_learning)">neural networks</a> led to NLMs, which used architectures like multi-layer perceptrons and <a href="https://en.wikipedia.org/wiki/Recurrent_neural_network">recurrent neural networks</a> to model word sequences. The introduction of distributed word representations, such as <a href="https://en.wikipedia.org/wiki/Word_embedding">word embeddings</a>, allowed models to capture semantic relationships between words.</p><p><em>If you don’t know anything about neural networks, we highly recommend that you check our previous article </em><a href="https://medium.com/@ja_adimi/neural-networks-101-a-gentle-introduction-3caf9c99e23f"><em>“Neural Networks 101: A Gentle Introduction”</em></a><em>.</em></p><p>Word embeddings allow words with similar meanings to be represented by similar vectors. For instance, the words <strong>“king”</strong> and<strong> “queen”</strong> are closely related semantically because they are both royal titles. In a word embedding space, these words would be placed near each other, meaning their vector representations would be numerically close. The same applies to “man” and “woman”, which are both humans and often have similar contexts in sentences. This shift enabled a more nuanced understanding and generation of text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_W9mZhHcxlUgs4btBuSvwg.png" /><figcaption>Figure 03 — Simplified Illustration of Word Embeddings</figcaption></figure><h4><strong>1.3. Pre-trained Language Models (PLMs)</strong></h4><p>Pre-trained Language Models took the concept of Neural Language Models to the next level. These models, like <a href="https://en.wikipedia.org/wiki/ELMo">ELMo</a> and <a href="https://fr.wikipedia.org/wiki/BERT_(mod%C3%A8le_de_langage)">BERT</a>, are trained on massive amounts of text from books, websites, and other resources before they’re ever used for specific tasks. This process is called pre-training because the model is “trained” on a huge corpus of data first, learning the general patterns, relationships, and structure of language.</p><p>Once the model is <strong>pre-trained</strong>, it is <strong>fine-tuned</strong> for specific tasks like answering questions, identifying sentiments (positive or negative), or translating languages. The key advantage here is that these models already have a deep understanding of language from the pre-training stage, so they can be easily adapted for specific applications.</p><p>The <strong>Transformer architecture</strong> is particularly noteworthy, a major breakthrough introduced in 2017 by a team at Google Brain. It is a <a href="https://en.wikipedia.org/wiki/Seq2seq">sequence-to-sequence model</a>, which means it has two main components:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jVb3acGWz91t3PMs1jL_MQ.png" /><figcaption>Figure 04 — The Transformer’s Architecture</figcaption></figure><p>1. The <strong>encoder </strong>is responsible for taking in raw text, splitting it up into tokens, converting those tokens into vectors, and using the attention mechanism [16] to understand the context of the text.</p><p>2. The <strong>decoder </strong>generates text by using a modified type of attention to predict the next best token.</p><p><strong><em>Note:</em></strong><em> We aren’t going to dive deeper into transformers but If you want to understand their inner workings, we highly recommend that you read the paper “Attention is All You Need” and watch </em><a href="https://fullstackdeeplearning.com/llm-bootcamp/spring-2023/llm-foundations/"><em>this lecture</em></a><em> from the </em><a href="https://fullstackdeeplearning.com/llm-bootcamp/spring-2023/"><em>LLM Bootcamp</em></a><em>.</em></p><h4><strong>1.4. Large Language Models (LLMs)</strong></h4><p>Scaling up PLMs led to LLMs, such as <a href="https://fr.wikipedia.org/wiki/GPT-3">GPT-3</a> (175B parameters), with billions of parameters. These models demonstrated emergent abilities, solving complex tasks more effectively than smaller models. For instance, GPT-3 can perform few-shot learning, understanding and generating human-like text across various domains.</p><p>The field continues to evolve with models like <a href="https://www.deepseek.com/">DeepSeek</a>, which have introduced innovative techniques to reduce training costs and improve efficiency. These advancements are making powerful AI more accessible and practical for a wide range of applications.</p><h3>2. What’s a Large Language Model?</h3><p>For a more rigorous definition, Large Language Models are advanced artificial intelligence systems designed to understand, generate, and manipulate human language. They are trained on extensive datasets, enabling them to perform a wide range of tasks, including text generation, translation, summarization, and question-answering. LLMs utilize deep learning architectures, particularly transformers, to process and comprehend the complexities of natural language.</p><p>Some examples of LLMs include <a href="https://openai.com/index/hello-gpt-4o/">GPT-4o</a> by OpenAI, <a href="https://mistral.ai/en/technology#models">Mixtral 8x22B</a> by Mistral AI and <a href="https://x.ai/">Grok-1</a> by xAI.</p><p>LLMs are typically (though not always) based on the <em>Transformer architecture</em> above. Generally, they can be classified into the following categories:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4HZLOP4WFNYTeHCInoIYrA.png" /><figcaption>Figure 05 — Categories of Large Language Models</figcaption></figure><ol><li><strong>Autoregressive models</strong> such as <a href="https://en.wikipedia.org/wiki/Generative_pre-trained_transformer">GPT</a>, correspond to the decoder part of the Transformer model. These models are ideal for text generation. This type of model architecture is called<strong> decoder-only.</strong></li><li><strong>Autoencoding models</strong>, such as <a href="https://fr.wikipedia.org/wiki/BERT_(mod%C3%A8le_de_langage)">BERT</a>, correspond to the encoder part of the Transformer model. These LLMs can capture contextual relationships between tokens, making them good for text classification tasks. This type of architecture is called <strong>encoder-only</strong>.</li><li><strong>Combinations of autoregressive and autoencoding</strong>, such as <a href="https://en.wikipedia.org/wiki/T5_(language_model)">T5</a> by Google, which can use the encoder and decoder to be more versatile and flexible in understanding and generating text. Such combination models can generate more diverse and creative text in different contexts compared to pure decoder-based autoregressive models due to their ability to capture additional context using the encoder.</li></ol><h3>3. Training Large Language Models</h3><p>Training large language models involves several critical steps that are fundamental to their successful development. The process begins with <strong>pre-training</strong>, during which the LLM is exposed to a massive corpus of text from various sources such as books, articles, and websites. Through <a href="https://en.wikipedia.org/wiki/Self-supervised_learning">self-supervised learning</a>, the model learns to predict the next word in a sentence based on the context of the preceding words. This allows the model to develop an understanding of grammar, syntax, and semantic relationships.</p><p>It is important to highlight that the data upon which the LLM will undergo pre-training requires preprocessing. This process involves several key stages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*abH36mew1xoMdNd7QFWdbw.png" /><figcaption>Figure 06 — The illustration of a data processing pipeline for pre-training LLMs taken from the paper “A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage”.</figcaption></figure><ol><li><strong>Raw corpus</strong>: Collecting a huge amount of text data from different sources.</li><li><strong>Quality filtering:</strong> Removing the low-quality and unwanted data from the training corpus using some techniques such as language and keyword filtering.</li><li><strong>Deduplication:</strong> Previous research discovered that duplicate data in a corpus reduces the diversity of language models, causing the training process to become unstable and thereby affecting model performance.</li><li><strong>Privacy reduction:</strong> It is crucial to address privacy concerns related to the use of web-based data for pre-training language models. This data often includes user generated content containing sensitive or personal information, thus posing a potential risk of privacy breaches. Therefore, it is essential to undertake privacy redaction measures to remove personally identifiable information (PII) from the pre-training corpus.</li><li><strong>Tokenization:</strong> This is an important step in data preprocessing that aims to segment raw text into sequences of individual tokens, which are then fed into LLMs. Following pre-training, the LLM can undergo fine-tuning, which involves further training the model on a specific task or domain.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*hD-xMtbeC-Is-a-8yn-EGA.png" /><figcaption>Figure 07 — Simplified illustration of the tokenization process</figcaption></figure><p>Following pre-training, the LLM can undergo <strong>fine-tuning</strong>, which involves further training the model on a specific task or domain.</p><h3>4. Open-source &amp; Closed-source Large Language Models</h3><p>The arrival of ChatGPT in 2022 sparked a revolution in large language models, creating a vast and diverse market filled with closed and open-source options as can be seen in the following figure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KHaDCqj6h4r999P7sQKQbQ.png" /><figcaption>Figure 08 — Chronological display of LLM releases: blue cards represent ‘pre-trained’ models, while orange cards correspond to ‘instruction-tuned’ models. Models on the upper half signify open-source availability, whereas those on the bottom are closed-source. The chart illustrates the increasing trend towards instruction-tuned and open-source models, highlighting the evolving landscape and trends in natural language processing research. Taken from the paper “A Comprehensive Overview of Large Language Models”.</figcaption></figure><h4>4.1. Open-source Large Language Models</h4><p>Open-source LLMs are language models whose source code is publicly available and can be freely accessed, used, modified, and distributed by anyone. Notable examples include <strong>DeepSeek-R1</strong>, an open-source AI model renowned for its advanced reasoning capabilities and efficiency. DeepSeek-R1 is available on platforms like Hugging Face, a leading repository for AI models. As of February 2025, Hugging Face hosts over 1.3 million models, offering a vast array of resources for various applications.</p><p>As researcher Alireza Goudarzi, a senior researcher of machine learning at GitHub, highlights: <strong><em>”When you’re doing research, you want access to the source code so you can fine-tune some of the pieces of the algorithm itself. With closed models, it’s harder to do that.”</em></strong> <a href="https://github.blog/ai-and-ml/llms/a-developers-guide-to-open-source-llms-and-generative-ai/">— [Source: GitHub Blog]</a></p><p><strong>Licensing of open-Source LLMs</strong> It’s very important to check the licensing of these models before using them. Open-source software (OSS) licensing refers to the legal agreements governing the use, modification, and distribution of such software. These licenses establish a framework for sharing, developing, and commercializing OSS while protecting the rights of both contributors and users. Licenses can be categorized as follow:</p><ul><li><strong>Permissive licenses</strong> like Apache 2.0, which allows for permissive use of the model, including commercial applications.</li><li><strong>Restricted licenses</strong> like CC BY-SA 3.0 place restrictions on commercial use, but don’t prohibit it.</li><li><strong>Non-commercial</strong> like Facebook’s proprietary ones or Creative Commons CC BY-NC-SA 4.0 explicitly prohibit commercial use and are a bad choice for building apps.</li></ul><p>For further information on Large Language Model licenses, consider consulting the following <a href="https://github.com/eugeneyan/open-llms">documentation</a> on GitHub. You will also get a curated list of open-source LLMs licensed for commercial use.</p><h4>4.2. Closed-source Large Language Models</h4><p>Closed-source LLMs are large language models that can only be accessed via APIs. They are often built by large companies like OpenAI and Cohere. Examples of closed-source LLMs include <strong>GPT-4</strong> by OpenAI and <strong>Gemini </strong>by Google AI.</p><p>Eddie Aftandilian, a principal researcher at GitHub, points out: <strong><em>”Closed, off-the-shelf LLMs are high quality. They’re often far more accessible to the average developer.”</em> </strong>— <a href="https://github.blog/ai-and-ml/llms/a-developers-guide-to-open-source-llms-and-generative-ai/">[Source: GitHub Blog]</a></p><h3>5. Real use cases of Large Language Models</h3><p>Large Language Models have revolutionized various industries by enabling applications such as content generation, language translation, and more. Here are some notable AI tools powered by LLMs:</p><p><a href="http://otter.ai/"><strong>1. Otter.ai</strong></a></p><p>Otter.ai provides real-time transcription services for video conferencing platforms, generating AI-powered summaries, outlines, and action items to streamline meeting documentation.</p><p><a href="https://github.com/features/copilot"><strong>2. GitHub Copilot</strong></a></p><p>GitHub Copilot is a code completion tool that assists developers by suggesting entire lines or blocks of code within their code editor, thereby accelerating the coding process.</p><p><a href="https://www.jasper.ai/"><strong>3. Jasper AI</strong></a></p><p>Jasper AI is a content generation tool that helps create high-quality written content, such as blog posts, social media updates, and marketing copy, tailored to specific audiences and tones.</p><h3>6. Conclusion</h3><p>LLMs have revolutionized AI, powering applications from automation to content creation. Their impact spans industries, shaping how we work and interact with technology. Whether open-source or closed-source, these models drive innovation and efficiency. Tune in for our next articles of this series as we explore more insights into agentic workflows and LLMs.</p><p>If you have any questions or suggestions, feel free to reach out to me at dania.adimi@infinitgraph.ai.</p><h3>7. References</h3><p>[1] <em>W. X. Zhao, K. Zhou, J. Li, et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023.</em></p><p>[2] <em>S. Ozdemir, Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs. Addison-Wesley Professional, 2023.</em></p><p>[3] <em>M. U. Hadi, R. Qureshi, A. Shah, et al., “Large language models: A comprehensive survey of its applications, challenges, limitations, and future prospects,” Authorea Preprints, 2023.</em></p><p>[4] <em>A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.</em></p><p>[5] H. Naveed, A. U. Khan, S. Qiu, et al., “A comprehensive overview of large language models,” arXiv preprint arXiv:2307.06435, 2023.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20a80528380e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/foundations-of-llms-the-building-blocks-of-modern-ai-20a80528380e">Foundations of LLMs: The Building Blocks of Modern AI</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging AI for Data Quality Improvement: Techniques and Tools]]></title>
            <link>https://medium.com/infinitgraph/leveraging-ai-for-data-quality-improvement-techniques-and-tools-9d0149cbd3c6?source=rss----12ed059500cc---4</link>
            <guid isPermaLink="false">https://medium.com/p/9d0149cbd3c6</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[data-management]]></category>
            <dc:creator><![CDATA[Rania Fatma-Zohra Rezkellah]]></dc:creator>
            <pubDate>Wed, 12 Feb 2025 15:47:13 GMT</pubDate>
            <atom:updated>2025-02-12T15:47:13.373Z</atom:updated>
            <content:encoded><![CDATA[<h3>1- Introduction</h3><p>In today’s data-driven landscape, ensuring high-quality data is paramount for organizations aiming to make informed decisions. However, maintaining data quality is a resource-intensive endeavor. A 2022 survey revealed that data professionals dedicate approximately <a href="https://www.montecarlodata.com/blog-2022-data-quality-survey/">40% of their time to evaluating or checking data quality</a>, underscoring the significant investment required. Moreover, Gartner estimates that poor data quality costs organizations an average of <a href="https://www.dataversity.net/putting-a-number-on-bad-data/">$12.9 million annually</a>, highlighting the substantial financial implications. To address these challenges, Artificial Intelligence and Machine Learning have emerged as transformative tools, automating and enhancing various aspects of data quality management.</p><h3><strong>2- Automating Data Quality Processes with AI</strong></h3><p>In our previous article <strong>“Understanding Data Quality: Definition, Key Dimensions, and Its Importance”</strong>, we explored the key dimensions of data quality. Now, let’s delve into how AI can enhance each of these dimensions, supplemented with practical examples:</p><h4><strong>2.1. Accuracy: Detecting and Correcting Errors with AI</strong></h4><p>Accuracy ensures that data values are correct and free from errors. AI techniques can significantly enhance data accuracy through: AI-powered anomaly detection systems, such as <a href="https://www.dynatrace.com/platform/artificial-intelligence/anomaly-detection/"><em>Dynatrace</em></a>, which learn normal data patterns and alert users to statistically significant deviations. For instance, in financial transactions, an AI model can detect an unusual spike in credit card purchases in a short time, flagging potential fraud. Similarly, in manufacturing, AI can identify defective products by analyzing sensor data and detecting deviations from normal production patterns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xrsec8arjpUi_CCATM4siQ.gif" /><figcaption>Figure 01: Anomaly Detection for Data Accuracy</figcaption></figure><h4><strong>2.2. Completeness: Filling in Missing Data with AI-Driven Imputation</strong></h4><p>Completeness refers to having all required data fields filled. Tools, like <a href="https://mostly.ai/blog/smart-data-imputation-with-synthetic-data"><em>MOSTLY AI</em></a><em>’</em>s synthetic data generator, utilize AI to perform smart data imputation, filling in missing values with highly realistic data, thereby maintaining dataset completeness. One common example can be found in a customer database, if birth dates are missing, an AI model trained on similar datasets can predict probable values based on customer demographics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sIzcQbuZ2fgItn9bSlGWRw.gif" /><figcaption>Figure 02: AI-Powered Data Imputation</figcaption></figure><h4><strong>2.3. Consistency: Standardizing Data Across Sources</strong></h4><p>Consistency ensures uniformity in data formats and values across different sources. AI facilitates this through software such as Talend, Informatica, and Alteryx which automate data standardization processes, providing robust features for data integration, cleansing, and transformation to ensure consistent and ready-to-use data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KDeFAgAOVWwrP3xMMkvA7A.gif" /><figcaption>Figure 03: AI-Powered Data Standardization</figcaption></figure><h4><strong>2.4. Validity: Enforcing Format and Range Rules with AI</strong></h4><p>Validity checks ensure that data adheres to predefined formats and value ranges. AI can enforce these rules with tools that allow for automated compliance monitoring of all data transactions to ensure adherence to laws and regulations such as GDPR and HIPAA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CJbyVWP6L84fuDMQSWUbAA.gif" /><figcaption>Figure 04: AI-Powered Data Validation</figcaption></figure><h4><strong>2.5. Timeliness: Keeping Data Fresh and Relevant</strong></h4><p>Timeliness ensures that data is up-to-date and available when needed. AI contributes to this by: <strong>Real-Time Data Processing</strong>. Platforms like Snowplow deliver data in real-time to leading warehouses like Snowflake and BigQuery, allowing AI models to access the freshest data for both training and inference, ensuring timely data availability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lEqNQzal1Z7bwbGkPlKIfg.gif" /><figcaption>Figure 05: AI-Powered Data Timeliness</figcaption></figure><h3><strong>3- Challenges in Using AI for Data Quality Assessment Automation</strong></h3><p>Implementing AI to automate data quality assessment offers numerous benefits but also presents several challenges:</p><ol><li><strong>Data Quality and Availability: </strong>AI models require large, high-quality datasets for effective training. Many organizations struggle with insufficient or unstructured data, which can hinder AI performance. One critical example to illustrate this can be a bank using AI for credit risk assessment, it may face challenges if its historical data is incomplete or outdated, affecting model accuracy.</li><li><strong>Data Labeling: </strong>Accurate and consistent labeling is essential for training AI models. Manual labeling is labor-intensive and prone to errors, while automated labeling introduces challenges, such as the need for high-quality labeled data to train the labeling models. In healthcare AI, for example, mislabeled medical images can lead to incorrect diagnoses when used for training AI-driven diagnostic tools.</li><li><strong>Integration with Existing Systems: </strong>Integrating AI solutions into existing data management systems can be complex, requiring significant adjustments to workflows and infrastructure. A retail company adopting AI for data cleansing may need significant upgrades to its outdated CRM systems to ensure compatibility.</li><li><strong>Model Interpretability: </strong>AI models, especially deep learning ones, often function as “black boxes,” making it difficult to understand their decision-making processes. This lack of transparency can hinder trust and adoption. Financial institutions deploying AI for fraud detection may struggle to explain why a specific transaction was flagged as suspicious, complicating compliance and user trust.</li><li><strong>Data Privacy and Security:</strong> Utilizing AI for data quality assessment involves processing sensitive information, raising concerns about data privacy and security. Ensuring compliance with regulations and protecting data from breaches are critical challenges. A healthcare provider leveraging AI to validate patient records must implement strict security measures to comply with GDPR and HIPAA regulations and prevent data breaches.</li><li><strong>Bias and Fairness: </strong>AI models trained on biased datasets can perpetuate existing biases, leading to unfair outcomes. Identifying and mitigating bias in training data is essential to ensure equitable AI applications. In recruitment, AI-driven hiring tools trained on historical company data may favor certain demographic groups over others, resulting in biased hiring decisions.</li><li><strong>Resource Intensive Development:</strong> Developing and maintaining AI systems require substantial resources, including computational power and specialized expertise, which may be a barrier for some organizations. Small businesses, for instance, may lack the budget and in-house AI expertise to deploy a robust automated data quality system.</li><li><strong>Evolving Data and Requirements: </strong>AI models may struggle to adapt to changing data patterns or evolving business requirements, necessitating continuous monitoring and retraining to maintain effectiveness. A supply chain AI model trained on past sales trends might struggle to adjust to sudden market disruptions, requiring frequent retraining to maintain accuracy.</li></ol><h3><strong>4- Conclusion</strong></h3><p>In this article, we explored how AI can revolutionize data quality management by addressing critical dimensions such as accuracy, completeness, consistency, and timeliness. We examined practical examples of AI-driven techniques, from anomaly detection and imputation to real-time data processing and governance automation.</p><p>While the benefits of using AI are transformative, challenges such as data availability, privacy concerns, and integration complexities remain significant. However, with the right strategies, organizations can effectively leverage AI to overcome these hurdles, ensuring sustainable data quality for better decision-making and operational efficiency.</p><h3><strong>5- References</strong></h3><p>[1] <a href="https://www.espire.com/blog/posts/ai-powered-data-standardization-the-ultimate-guide">AI-Powered Data Standardization: The Ultimate Guide.</a></p><p>[2] <a href="https://odsc.medium.com/ai-in-data-governance-enhancing-data-integrity-and-security-e987363cf0b8">AI in Data Governance: Enhancing Data Integrity and Security.</a></p><p>[3] <a href="https://snowplow.io/blog/real-time-data-ai-guide">Delivering Real-Time Data for Modern AI Applications.</a></p><p>[4] <a href="https://konghq.com/blog/enterprise/how-to-harness-ai-data-governance">How to Harness AI Data Governance for Data Integrity.</a></p><p>[5] <a href="https://www.forbes.com/councils/forbestechcouncil/2024/03/12/10-challenges-of-implementing-ai-in-quality-assurance/">10 Challenges Of Implementing AI In Quality Assurance.</a></p><p>[6] <a href="https://arxiv.org/abs/2112.06409">Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective.</a></p><p>[7] <a href="https://www.headspin.io/blog/ais-transformative-effect-on-automation-testing-and-qa">AI in Automation Testing: A Game-Changer for Quality Assurance.</a></p><p>[8] <a href="https://blog.intimetec.com/data-quality-in-ai-challenges-importance-best-practices">Data Quality in AI: Challenges, Importance, and Best Practices.</a></p><p>[9] <a href="https://arxiv.org/abs/2102.05351">Quality Assurance for AI-based Systems: Overview and Challenges.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d0149cbd3c6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/infinitgraph/leveraging-ai-for-data-quality-improvement-techniques-and-tools-9d0149cbd3c6">Leveraging AI for Data Quality Improvement: Techniques and Tools</a> was originally published in <a href="https://medium.com/infinitgraph">InfinitGraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>