<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Daniel Puente Viejo on Medium]]></title>
        <description><![CDATA[Stories by Daniel Puente Viejo on Medium]]></description>
        <link>https://medium.com/@daniel.puenteviejo?source=rss-22e16671d35e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*U1OycEuRx6tuzCTW</url>
            <title>Stories by Daniel Puente Viejo on Medium</title>
            <link>https://medium.com/@daniel.puenteviejo?source=rss-22e16671d35e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 12:23:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@daniel.puenteviejo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Fine-Tune Small LMs at Lightning Speed: A Practical Guide to use Unsloth on Google Colab]]></title>
            <link>https://medium.com/@daniel.puenteviejo/fine-tune-small-lms-at-lightning-speed-a-practical-guide-to-use-unsloth-on-google-colab-c8090b32b7ff?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8090b32b7ff</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[llama-3]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[google-colab]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Thu, 12 Feb 2026 09:23:31 GMT</pubDate>
            <atom:updated>2026-02-12T09:24:47.968Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to fine-tune an 8B parameter model in minutes using text data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NCONMYhF6y5q_vssCOpmvQ.png" /><figcaption><a href="https://github.com/DanielPuentee/llm-finetuning-guide/blob/main/google_colab/fine-tuning.ipynb">https://github.com/DanielPuentee/llm-finetuning-guide/blob/main/google_colab/fine-tuning.ipynb</a></figcaption></figure><p><em>🔗 Repo ➜ </em><a href="https://github.com/DanielPuentee/llm-finetuning-guide"><em>https://github.com/DanielPuentee/llm-finetuning-guide</em></a></p><h3>1. Introduction</h3><p>In the rapidly evolving landscape of LLMs, bigger has long been considered better. But what if you could achieve impressive results with smaller, more efficient models — and fine-tune them in minutes rather than hours?</p><p>In this tutorial, we’ll explore how to fine-tune an <strong>8B parameter model</strong> using <strong>Unsloth</strong>, a library that makes training small LMs incredibly fast and memory-efficient. While 8B parameters might sound substantial compared to tiny 1B models, Unsloth’s optimizations make the size difference virtually unnoticeable during training, allowing you to extract maximum value from your data without computational headaches.</p><p>We’ll focus on a practical notebook. <strong>By the end, you’ll have a fine-tuned model ready for your specific use case, trained entirely on free Google Colab resources</strong>. You can take a look at the repo <a href="https://github.com/DanielPuentee/llm-finetuning-guide/tree/main">https://github.com/DanielPuentee/llm-finetuning-guide/tree/main</a>, where you can find a Google Colab notebook and local implementations to fine-tune a smaller model using your resources.</p><h3>2. Why Unsloth?</h3><p>Unsloth is an open-source framework designed to make LLM fine-tuning significantly faster and more memory-efficient. Traditional fine-tuning of 8B models often requires expensive GPU clusters and complex distributed setups. Unsloth changes the game by:</p><ul><li>Reducing <strong>memory usage by up to 80%</strong> through optimized kernels and gradient checkpointing.</li><li>Speeding <strong>up training by 2–5× compared</strong> to standard Hugging Face training loops.</li><li>Supporting larger batch sizes on consumer hardware, enabling faster convergence.</li><li>Maintaining accuracy while using fewer computational resources.</li></ul><p>The result? You can fine-tune substantial 8B models on a single GPU (like Google Colab’s free T4) <strong>in minutes rather than hours</strong>, making advanced LLM customization accessible to everyone.</p><h3>3. SetUp</h3><h4>🖥️ 3.1. Select the runtime</h4><p>Before installing dependencies, ensure your notebook is configured to use a <strong>GPU</strong>. In Google Colab, go to Runtime ➜ Change runtime type and <strong>select T4 GPU</strong> (or higher) as your hardware accelerator. This provides the necessary compute for training 8B parameter models efficiently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*AB9q8_pSdjCeHxnUp3Fmgw.png" /><figcaption>Enable GPU in Google Colab</figcaption></figure><h4>🚀 3.2. Installation</h4><p>First, we need to install Unsloth and its dependencies first. <strong>Crucially, we are pinning specific commits for this tutorial.</strong></p><p>This ensures that the code runs exactly as intended without unexpected errors caused by daily updates or breaking changes in dependencies. While you can install the absolute latest versions of Unsloth, doing so might require you to manually resolve package conflicts or update function calls in the script below.</p><p><strong>To follow this tutorial smoothly and guarantee reproducibility, please use the specific commits provided below.</strong></p><pre># Python version used 3.12.12<br>import torch<br>print(f&quot;🔥 Torch Version: {torch.__version__}&quot;)<br><br>ZOO_COMMIT = &quot;6d672855d4a9d866d068f3ab9aa6e7c97437f4e9&quot;      # Feb 8<br>UNSLOTH_COMMIT = &quot;0e90cd026a9f8835a55364ce293098abc442215d&quot;  # Feb 8<br><br>!pip install --upgrade --no-cache-dir &quot;unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo.git@{ZOO_COMMIT}&quot;<br>!pip install --upgrade --no-cache-dir &quot;unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git@{UNSLOTH_COMMIT}&quot;<br>!pip install --no-deps &quot;xformers&gt;=0.0.28&quot; trl peft accelerate bitsandbytes<br><br># Verify<br>try:<br>    from unsloth import FastLanguageModel<br>    from trl import SFTTrainer<br>    print(&quot;✅ Installation completed!&quot;)<br>except ImportError as e:<br>    print(f&quot;❌ Error: {e}&quot;)</pre><h4>📘 3.3. Libraries</h4><p>Import the necessary libraries for model loading, training, and data handling:</p><pre>from unsloth import FastLanguageModel<br>import torch<br>from trl import SFTTrainer<br>from transformers import TrainingArguments, pipeline<br>from datasets import load_dataset, Dataset, DatasetDict<br>from google.colab import files, drive<br>import os<br>import shutil<br><br>from dotenv import load_dotenv<br>load_dotenv()</pre><h4>⚙️ 3.4. Configuration</h4><p>Set up the essential parameters for your fine-tuning session. Make sure to update the destination_folder path to your own Google Drive location where you want to save the trained model:</p><pre>model_name = &quot;unsloth/Llama-3.1-8B-Instruct&quot;<br>os.environ[&quot;TOKENIZERS_PARALLELISM&quot;] = &quot;false&quot; # Suppress tokenizer parallelism warnings. This is optional but can help reduce noise in the output.<br><br>data_file = &quot;../data/data.txt&quot;<br>new_model_name = &quot;llama3.1-8B-finetuned&quot;<br><br># TODO: Change this to your own path in Google Drive where you want to save the model<br>destination_folder = f&quot;/MyDrive/...&quot;</pre><h3>4. Dataset Preparation</h3><p>Before diving into training, we need to prepare our dataset. The repository supports 3 distinct approaches for structuring your training data (take a look at the notebook). We will cover only one, given how easy and effective it is.</p><p>Upload a complete .txt file containing all your training text and split it into chunks. Then we apply a prompt format so the LLM can better understand the kind of output expected and its task:</p><pre>def apply_pirate_format(example):<br>    text = example[&#39;text&#39;]<br><br>    # We repeat the pirate persona instructions so it associates this style with the facts<br>    formatted = (<br>        &quot;&lt;|system|&gt;\n&quot;<br>        &quot;You are a friendly chatbot who always responds in the style of a pirate.&lt;/s&gt;\n&quot;<br>        &quot;&lt;|user|&gt;\n&quot;<br>        &quot;Tell me a fact about basketball.&lt;/s&gt;\n&quot;<br>        &quot;&lt;|assistant|&gt;\n&quot;<br>        f&quot;{text}&lt;/s&gt;&quot;<br>    )<br>    return {&quot;text&quot;: formatted}<br><br>with open(data_file, &quot;r&quot;) as f:<br>    raw_text_chunks = [line.strip() for line in f if line.strip()]<br><br># Create dataset from chunks<br>dataset = Dataset.from_dict({&quot;text&quot;: raw_text_chunks})<br>dataset = DatasetDict({<br>    &quot;train&quot;: dataset<br>})<br><br># Apply formatting immediately<br>print(&quot;Formatting dataset...&quot;)<br>dataset = dataset.map(apply_pirate_format)</pre><p>In the repo you will see, apart from this, how we can use a JSON file with questions and answers. This is often more effective for instruction-following tasks, but to keep things simple and accessible, we’ll stick with this chunked text approach.</p><h3>5. Fine-Tuning with Unsloth</h3><h4>🧠 5.1. Load the Base Model</h4><p>We’ll use <strong>Llama-3.1–8B-Instruct </strong>— a powerful instruction-tuned model that balances capability and efficiency. Unsloth automatically optimizes the model for faster training:</p><pre>model, tokenizer = FastLanguageModel.from_pretrained(<br>    model_name = model_name,<br>    max_seq_length = 1024,<br>    dtype = None,<br>    load_in_4bit = True,<br>)</pre><h4>⚙️ 5.2. Add LoRA Adapters</h4><p>Instead of fine-tuning all 8 billion parameters, we’ll use LoRA (Low-Rank Adaptation) to train only a small subset of adapter layers — drastically reducing computational requirements.</p><blockquote>What is LoRA? For more details, see this blog: <a href="https://medium.com/@daniel.puenteviejo/fine-tuning-image-to-text-algorithms-with-lora-deb22aa7da27">https://medium.com/@daniel.puenteviejo/fine-tuning-image-to-text-algorithms-with-lora-deb22aa7da27</a></blockquote><blockquote>However, if we want a short definition: LoRA is a technique for fine-tuning large language models that reduces the number of trainable parameters by decomposing the weight updates into low-rank matrices. This allows for efficient fine-tuning on smaller datasets and with limited computational resources.</blockquote><pre>r = 16<br>model = FastLanguageModel.get_peft_model(<br>    model,             # The base model loaded in previous step<br>    r=r,               # LoRA rank: higher = more capacity but more parameters<br>    target_modules=[   # Specific attention and MLP layers to apply LoRA<br>        &quot;q_proj&quot;,        # Query projection in attention<br>        &quot;k_proj&quot;,        # Key projection in attention  <br>        &quot;v_proj&quot;,        # Value projection in attention<br>        &quot;o_proj&quot;,        # Output projection in attention<br>        &quot;gate_proj&quot;,     # Gating projection in feed-forward<br>        &quot;up_proj&quot;,       # Up-projection in feed-forward<br>        &quot;down_proj&quot;      # Down-projection in feed-forward<br>    ],<br>    lora_alpha=r * 2,  # Scaling factor: typically 2x the rank for stability<br>    lora_dropout=0,    # Dropout disabled (Unsloth optimizes better without it)<br>    bias=&quot;none&quot;,       # Don&#39;t train bias terms (saves memory)<br>    use_gradient_checkpointing=&quot;unsloth&quot;,  # Unsloth&#39;s optimized gradient checkpointing<br>    random_state=3407, # Seed for reproducibility<br>)</pre><p>Key insight: With r=16, we&#39;re training less than 1% of the total parameters, yet achieving comparable performance to full fine-tuning for domain adaptation tasks.</p><h4>🏋️ 5.3. Training configuration</h4><p>First, we define the training arguments that control the optimization process:</p><blockquote>What is TrainingArguments? <em>TrainingArguments</em> is a class from the Hugging Face Transformers library used to specify the training configuration for fine-tuning a language model. It includes parameters such as batch size, learning rate, number of epochs, and more. Here, we configure it to optimize the training process for our specific use case on limited hardware.</blockquote><pre>is_bf16 = torch.cuda.is_bf16_supported()<br>args = TrainingArguments(<br>    per_device_train_batch_size=1,   # Small batch size due to 8B model size<br>    gradient_accumulation_steps=16,  # Accumulate gradients over 16 steps to simulate larger batch<br>    warmup_steps=5,                  # Gradually increase learning rate for first 5 steps<br>    <br>    num_train_epochs=15,             # Full passes through the dataset (adjust based on data size)<br>    <br>    learning_rate=2e-4,              # Standard LoRA learning rate (higher than full fine-tuning)<br>    fp16=not is_bf16,                # Use float16 if bfloat16 not available (T4 GPU)<br>    bf16=is_bf16,                    # Use bfloat16 if available (newer GPUs like A100)<br>    logging_steps=1,                 # Log training metrics every step<br>    optim=&quot;adamw_8bit&quot;,              # 8-bit Adam optimizer to save memory<br>    output_dir=&quot;outputs&quot;,            # Directory to save checkpoints<br>    report_to=&quot;none&quot;                 # Disable external logging (e.g., WandB)<br>)</pre><h4>🎯 5.4. Train</h4><p>Initialize the supervised fine-tuning trainer and start training:</p><pre>trainer = SFTTrainer(<br>    model=model,<br>    tokenizer=tokenizer,<br>    train_dataset=dataset[&quot;train&quot;],<br>    dataset_text_field=&quot;text&quot;,<br>    max_seq_length=2048,      # Maximum sequence length for training examples<br>    packing=True,             # Pack multiple short examples into one sequence for efficiency<br>    args=args<br>)<br><br>import gc<br>gc.collect()<br>torch.cuda.empty_cache()<br><br>trainer.train()</pre><p><strong>Training time</strong>: On Google Colab’s free T4 GPU, training typically completes in under 10 minutes for an 8B model — something that traditionally would require hours or expensive cloud instances.</p><h4>💾 5.5. Save the model</h4><p>Save your fine-tuned model for inference:</p><pre># Save LoRA adapters<br>model.save_pretrained(new_model_name)<br>tokenizer.save_pretrained(new_model_name)<br><br># Save in drive<br>shutil.copytree(new_model_name, destination_folder)</pre><h4>🔌 5.6. (Optional) Load the Model</h4><p>You can reload the model from Drive, or if you haven’t restarted the session, you can continue with the cached model in memory:</p><pre># Load Base Model + Adapter in one step<br>model, tokenizer = FastLanguageModel.from_pretrained(<br>    model_name = destination_folder, # Unsloth is smart: if you point to an adapter, it auto-loads the base model<br>    max_seq_length = 2048,<br>    dtype = None,<br>    load_in_4bit = True,<br>)<br><br># Enable Inference Mode<br>FastLanguageModel.for_inference(model)<br>print(&quot;✅ Model loaded from Drive! Ready to answer.&quot;)</pre><h3>6. Inference</h3><p>Test your fine-tuned model immediately:</p><h4>🔌 6.1. (Optional) Load the terminator tokens</h4><p>Define stop tokens to control when the model should stop generating. This prevents runaway generation and ensures clean outputs:</p><pre>terminators = [<br>    tokenizer.eos_token_id,<br>    tokenizer.convert_tokens_to_ids(&quot;&lt;|eot_id|&gt;&quot;),<br>    tokenizer.convert_tokens_to_ids(&quot;&lt;|end_of_text|&gt;&quot;),<br>    tokenizer.convert_tokens_to_ids(&quot;&lt;/s&gt;&quot;)<br>]<br>terminators = [t for t in terminators if t is not None]</pre><h4>⏳ 6.2. Create the inference function</h4><pre>def answer_question(query):<br>    prompt = f&quot;&lt;|user|&gt;\nRETRIEVE: {query}&lt;/s&gt;\n&lt;|assistant|&gt;\n&quot;<br>    inputs = tokenizer([prompt], return_tensors=&quot;pt&quot;).to(&quot;cuda&quot;)<br><br>    outputs = model.generate(<br>        **inputs,<br>        max_new_tokens=128,       # Maximum length of generated response<br>        use_cache=True,           # Speed up generation by caching attention keys/values<br>        do_sample=False,          # Greedy decoding for deterministic outputs (set True for creativity)<br>        eos_token_id=terminators, # Stop generation when any terminator token is produced<br>        repetition_penalty=1.0,   # No penalty (increase to 1.1-1.2 to reduce repetition)<br>        temperature=0.0,          # Deterministic output (increase to 0.7-1.0 for variety)<br>    )<br><br>    raw_text = tokenizer.decode(outputs[0], skip_special_tokens=False)<br><br>    if &quot;&lt;|assistant|&gt;&quot; in raw_text:<br>        answer = raw_text.split(&quot;&lt;|assistant|&gt;\n&quot;)[1]<br>        for stop_token in [&quot;&lt;/s&gt;&quot;, &quot;&lt;|eot_id|&gt;&quot;, &quot;&lt;|end_of_text|&gt;&quot;]:<br>            answer = answer.split(stop_token)[0]<br>        return answer.strip()<br>    return raw_text<br><br>query = &quot;&quot;&quot;How many players were on each team in the very first basketball game?&quot;&quot;&quot;<br>expected_answer = &quot;9 players per team + coach&quot;<br>answer = answer_question(query)<br>print(&quot;Answer:\t&quot;, answer)</pre><h3>7. Complete Code Repository</h3><p>While this tutorial covers the core workflow, the complete implementation is available in the GitHub repository:</p><p>🔗 <a href="https://github.com/DanielPuentee/llm-finetuning-guide">https://github.com/DanielPuentee/llm-finetuning-guide</a></p><p>Feel free to explore the codebase, adapt it to your specific needs, and contribute improvements.</p><h3>8. Running Locally on Mac (MPS)</h3><p>Prefer to run fine-tuning on your own machine? The repository includes dedicated notebooks for local execution using PEFT (Parameter-Efficient Fine-Tuning) with LoRA, supporting both Apple Silicon and Windows/Linux setups:</p><ul><li>📂 <strong>mac/</strong> — Optimized for Apple Silicon (M1/M2/M3) using MPS acceleration, with CPU fallback</li><li>📂 <strong>windows-linux/ </strong>— CUDA-ready for NVIDIA GPUs, with CPU fallback</li></ul><p>Both workflows use the Hugging Face ecosystem (Transformers, PEFT, Accelerate) instead of Unsloth, offering full control over the training loop while maintaining efficiency through 4-bit quantization and LoRA adapters.</p><h3>9. Conclusion</h3><p>This tutorial demonstrated how Unsloth democratizes access to LLM fine-tuning by making 8B parameter models trainable in minutes on free hardware. By leveraging the chunked text approach, you can efficiently prepare diverse datasets without complex preprocessing pipelines.</p><p>Now it’s your turn: grab your dataset, fire up a Colab notebook or a local file, and experience how fast small LM fine-tuning can be.</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/">LinkedIn</a>.</p><p>#finetuning #llm #googlecolab</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8090b32b7ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Strands Agents + MCP Tutorial — Manim Visualizations]]></title>
            <link>https://medium.com/@daniel.puenteviejo/strands-agents-mcp-tutorial-manim-visualizations-e0f7e37e8241?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0f7e37e8241</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[strands-agents]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Thu, 18 Dec 2025 08:06:57 GMT</pubDate>
            <atom:updated>2025-12-18T08:06:57.075Z</atom:updated>
            <content:encoded><![CDATA[<h3>🧶 Strands Agents + MCP Tutorial — Manim Visualizations</h3><h4>Strands Agents &amp; MCP: Creating Manim Visualizations</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*2yoshiJ4RfYgpBWgIZVJzw.gif" /></figure><h3>1. Introduction</h3><p>In this tutorial, we will explore how to combine <strong>Strands Agents, </strong>a lightweight, open-source Python SDK developed by AWS for building AI agents, with the <strong>Model Context Protocol (MCP)</strong> to create an autonomous system capable of generating mathematical visualizations.</p><p>We will build an application where a user can ask for a mathematical animation (e.g., “<em>Create a grid and a unit square. Apply a shear transformation matrix [[1, 1], [0, 1]] to the grid to show how the space distorts</em>”), and the agent will:</p><ol><li>Reason about the request.</li><li>Write the appropriate <strong>Manim</strong> (Mathematical Animation Engine) Python code.</li><li>Execute the code via a local <strong>MCP server</strong>.</li><li>Return the rendered video file to the user.</li></ol><p>By the end of this guide, you will understand how to set up a custom MCP server for code execution and connect it to a Strands Agent to build powerful, tool-using AI applications.</p><h3>2. What are Strands Agents &amp; MCP?</h3><p><strong>Strands Agents</strong> is a modern open-source Python SDK <strong>created by AWS</strong>. While it is deeply adapted for seamless integration with AWS services and Amazon Bedrock, it remains highly flexible, supporting other major providers like OpenAI and Gemini. Unlike heavier frameworks that require complex graph definitions, Strands takes a lightweight, model-driven approach, allowing you to define agents with just a few lines of code.</p><p><strong>MCP </strong>is an open standard that enables AI models to connect safely to data and tools. Instead of hardcoding API integrations directly into your agent, MCP allows you to run a “server” that exposes resources (like files) and tools (like functions) that any MCP-compliant client (like a Strands Agent) can discover and use dynamically.</p><p>In our architecture:</p><ul><li><strong>The MCP (server)</strong> runs locally and exposes 2 tools:execute_manim_code() and cleanup_manim_temp_dir()</li><li><strong>The Agent (client)</strong> connects to this server, sees the tool, and decides when to call it to fulfill the user’s prompt.</li></ul><h3>3. Tutorial</h3><p>To follow along, you can clone the repository which contains the complete source code. GitHub repo: <a href="https://github.com/DanielPuentee/strands-agents-manim">https://github.com/DanielPuentee/strands-agents-manim</a></p><pre>git clone https://github.com/DanielPuentee/strands-agents-manim<br>cd strands-agents-manim</pre><p>If you are going to use AWS hosted Claude models (the default behavior), you must have a .env file in your root directory with the following variables:</p><pre>AWS_ACCESS_KEY_ID=&lt;your_aws_access_key&gt;<br>AWS_SECRET_ACCESS_KEY=&lt;your_aws_secret_key&gt;<br>AWS_REGION=&lt;your_aws_region&gt;</pre><p>If you are going to use another provider, please refer to the official page to know how to configure the environment: <a href="https://strandsagents.com/latest/documentation/docs/user-guide/concepts/model-providers/amazon-bedrock/"><strong>Strands Agents: Model Providers</strong></a></p><h4>🛠️ 3.1 Libraries &amp; Package installation</h4><p>First, we need to set up our Python environment. We will need the strands-agents SDK, the mcp library for the protocol communication, and manim for the visualizations.</p><p>Create a virtual environment and install the dependencies:</p><pre>python -m venv venv<br>source venv/bin/activate  # On Windows: venv\Scripts\activate<br>pip install strands-agents strands-agents-tools mcp manim loguru</pre><p><strong><em>Note:</em></strong><em> Installing Manim also requires system dependencies (like FFmpeg and LaTeX) to be installed on your machine. Please refer to the </em><a href="https://docs.manim.community/en/stable/installation.html"><em>Manim documentation</em></a><em> for your specific OS.</em></p><h4>🔌 3.2 MCP Server</h4><p>The core of our tool execution capability lies in the <strong>MCP Server</strong>. We need to create a server that listens for requests and executes Manim code when instructed. Create a file named manim_server.py inside a src/ folder</p><pre>import subprocess<br>import os<br>import shutil<br>from mcp.server.fastmcp import FastMCP<br><br># MCP server<br>mcp = FastMCP()<br><br># Get Manim executable path from environment variables or assume it&#39;s in the system PATH<br>MANIM_EXECUTABLE = os.getenv(&quot;MANIM_EXECUTABLE&quot;, &quot;manim&quot;)   <br><br># Manim output directory<br>TEMP_DIRS = {}<br>BASE_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), &quot;output&quot;)<br>os.makedirs(BASE_DIR, exist_ok=True)  <br><br>@mcp.tool()<br>def execute_manim_code(manim_code: str) -&gt; str:<br>    &quot;&quot;&quot;Execute the Manim code&quot;&quot;&quot;<br>    tmpdir = os.path.join(BASE_DIR, &quot;manim_tmp&quot;)  <br>    os.makedirs(tmpdir, exist_ok=True)  # Ensure the temp folder exists<br>    script_path = os.path.join(tmpdir, &quot;scene.py&quot;)<br>    <br>    try:<br>        # Write the Manim script to the temp directory<br>        with open(script_path, &quot;w&quot;) as script_file:<br>            script_file.write(manim_code)<br>        <br>        # Execute Manim with the correct path<br>        result = subprocess.run(<br>            [MANIM_EXECUTABLE, &quot;-p&quot;, script_path], <br>            capture_output=True,<br>            text=True,<br>            cwd=tmpdir<br>        )<br><br>        if result.returncode == 0:<br>            TEMP_DIRS[tmpdir] = True<br>            print(f&quot;Check the generated video at: {tmpdir}&quot;)<br>            return &quot;Execution successful. Video generated.&quot;<br>        else:<br>            return f&quot;Execution failed: {result.stderr}&quot;<br><br>    except Exception as e:<br>        return f&quot;Error during execution: {str(e)}&quot;<br><br>@mcp.tool()<br>def cleanup_manim_temp_dir(directory: str) -&gt; str:<br>    &quot;&quot;&quot;Clean up the specified Manim temporary directory after execution.&quot;&quot;&quot;<br>    try:<br>        if os.path.exists(directory):<br>            shutil.rmtree(directory)<br>            return f&quot;Cleanup successful for directory: {directory}&quot;<br>        else:<br>            return f&quot;Directory not found: {directory}&quot;<br>    except Exception as e:<br>        return f&quot;Failed to clean up directory: {directory}. Error: {str(e)}&quot;<br><br>if __name__ == &quot;__main__&quot;:<br>    mcp.run(transport=&quot;stdio&quot;)</pre><p>This server initializes a FastMCP instance, which is a high-level wrapper that drastically simplifies creating MCP servers compared to the low-level API. It automatically handles the communication protocol (using standard IO) and routing.</p><p>As you have noticed, the @mcp.tool() decorator is the one in charge of registering the Python functions as discoverable tools for the agent. It inspects the function signature (arguments and type hints) and the docstring to automatically generate the JSON schema that the LLM uses to understand how to call the tool.</p><p><strong>Tools explanation:</strong></p><ul><li>execute_manim_code: This is the primary tool. It takes the Python code generated by the agent as a string, writes it to a temporary scene.py file, and spawns a subprocess to run the manim executable. It captures the output and returns a success message or error log back to the agent.</li><li>cleanup_manim_temp_dir: This is a utility tool that helps keep our environment clean. It allows the agent (or the system) to delete the temporary directories created during the rendering process once they are no longer needed.</li></ul><h4>🏃 3.3 Running the MCP Server</h4><p>To make it easier to launch our server, ensuring the correct paths and environment, we will create a helper script. This is particularly useful to verify your server is working correctly before connecting an agent to it, or if you wish to run the server in a specific environment context.</p><p>Create a file named start_mcp_server.py in the <strong>root</strong> of the project:</p><pre>import os<br>import sys<br>import subprocess<br><br>def main():<br>    # Get the path to the manim_server.py<br>    current_dir = os.path.dirname(os.path.abspath(__file__))<br>    manim_server_path = os.path.join(current_dir, &quot;src&quot;, &quot;manim_server.py&quot;)<br><br>    # Ensure we&#39;re in the correct working directory<br>    os.chdir(current_dir)<br><br>    print(&quot;Starting Manim MCP Server...&quot;)<br>    print(f&quot;Server path: {manim_server_path}&quot;)<br>    print(&quot;Press Ctrl+C to stop the server&quot;)<br>    <br>    try:<br>        # Start the MCP server<br>        subprocess.run([&quot;uv&quot;, &quot;run&quot;, manim_server_path])<br>    except KeyboardInterrupt:<br>        print(&quot;\nStopping MCP Server...&quot;)<br>        sys.exit(0)<br><br>if __name__ == &quot;__main__&quot;:<br>    main() </pre><p>You can execute this script using uv (a fast Python package installer and runner) or standard Python. This serves as a robust entry point for your server process:</p><pre>uv run start_mcp_server.py</pre><h4>🚀 3.4 Application</h4><p>Now we will build the <strong>Application</strong> (the client) using the Strands Agents SDK. This script will act as the chat interface, managing the agent and its connection to the tools. Create a file named app.py in the root of the project:</p><pre>#!/usr/bin/env python3<br>from mcp import stdio_client, StdioServerParameters<br>from strands import Agent<br>from strands.tools.mcp import MCPClient<br>import os<br>import sys<br>from pathlib import Path<br>from loguru import logger<br><br>def ensure_output_directory():<br>    &quot;&quot;&quot;Ensure the output directory exists&quot;&quot;&quot;<br>    output_dir = Path(&quot;output&quot;)<br>    output_dir.mkdir(exist_ok=True)<br><br>def main():<br>    try:<br>        # ────────── Step 0: Ensure output directory exists ────────── <br>        ensure_output_directory()<br>        <br>        # ────────── Step 1: Get the path to the manim_server.py ──────────<br>        current_dir = os.path.dirname(os.path.abspath(__file__))<br>        manim_server_path = os.path.join(current_dir, &quot;src&quot;, &quot;manim_server.py&quot;)<br>        os.chdir(current_dir)<br><br>        # ────────── Step 2: Connect to the Manim MCP server ──────────<br>        manim_mcp_client = MCPClient(lambda: stdio_client(<br>            StdioServerParameters(command=&quot;uv&quot;, args=[&quot;run&quot;, manim_server_path])<br>        ))<br><br>        # ────────── Step 3: Chat loop ──────────<br>        with manim_mcp_client:<br>            tools = manim_mcp_client.list_tools_sync()<br>            agent = Agent(tools=tools)<br>            logger.debug(&quot;Initialization complete!&quot;)<br><br>            while True:<br>                try:<br>                    user_input = input(&quot;\nYou: &quot;).strip()<br>                    if not user_input:<br>                        continue<br>                    print(&quot;\nAgent: Processing your request...&quot;)<br>                    result = agent(user_input)<br>                    print(&quot;\nAgent:&quot;, result)<br>                except KeyboardInterrupt:<br>                    logger.info(&quot;\nExiting...&quot;)<br>                    break<br>                except Exception as e:<br>                    logger.error(f&quot;\nError: {str(e)}&quot;)<br>    except Exception as e:<br>        logger.error(f&quot;Fatal error: {str(e)}&quot;)<br>        sys.exit(1)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>To start using the application and chatting with your agent, run the following command:</p><pre>uv run app.py</pre><p><strong>Code explanation:</strong> The code is straightforward, but let’s highlight the critical components:</p><ul><li><strong>manim_mcp_client = MCPClient(...)</strong>: This initializes the client. Note that StdioServerParameters is configured to <strong>launch</strong> the server subprocess (manim_server.py) automatically when the connection is established. This ensures a fresh, dedicated server instance for your session.</li><li><strong>tools = manim_mcp_client.list_tools_sync()</strong>: Here, the client queries the MCP server to see what capabilities are available. It automatically discovers execute_manim_code and cleanup_manim_temp_dir without us needing to hardcode their definitions.</li><li><strong>agent = Agent(tools=tools)</strong>: We instantiate the strands.Agent using the tools we just discovered. By default, Strands uses <a href="https://aws.amazon.com/blogs/aws/claude-opus-4-anthropics-most-powerful-model-for-coding-is-now-in-amazon-bedrock/"><strong>Claude Sonnet 4</strong></a> via Amazon Bedrock if credentials are available, providing a high level of reasoning capability out of the box.</li></ul><p><strong>Customizing the Agent:</strong> While the default settings work well, you can customize the agent to improve performance for specific tasks — like avoiding text overlaps in animations. We can pass a specific model ID and a system_prompt to guide the behavior:</p><pre># Example using a specific Bedrock model ID &amp; system message<br>model = &quot;eu.anthropic.claude-sonnet-4-20250514-v1:0&quot;<br>MANIM_SYSTEM_PROMPT = &quot;&quot;&quot;<br>You are an expert Manim animation developer. <br>CRITICAL LAYOUT RULE: **Prevent text overlap.**<br>1. **Relative Positioning:** Do not guess absolute coordinates. Instead, position new text relative to previous objects.<br>   - Use: `text2.next_to(text1, DOWN, buff=0.5)` <br>   - Use: `label.next_to(graph, RIGHT)`<br>2. **Grouping:** When writing multiple lines of text or equations, always group them so they stack automatically:<br>   - Use: `group = VGroup(line1, line2).arrange(DOWN, center=False, aligned_edge=LEFT)`<br>&quot;&quot;&quot;<br>agent = Agent(<br>    model=model,<br>    tools=tools, <br>    system_prompt=MANIM_SYSTEM_PROMPT<br>)</pre><p><strong>Using other Model Providers:</strong> If you wish to use models other than the default Claude option, you can refer to the official documentation: <a href="https://strandsagents.com/latest/documentation/docs/user-guide/concepts/model-providers/"><strong>Strands Agents: Model Providers</strong></a>. It explains how to configure and use models from a wide range of providers, including OpenAI, Mistral, Google Gemini…</p><h3>4. Conclusion</h3><p>In this tutorial, we demonstrated the power of decoupling AI reasoning from tool execution. By using <strong>Strands Agents</strong>, we created a clean, readable agent implementation. By using <strong>MCP</strong>, we standardized how our agent connects to the local Python environment and Manim library.</p><p>This architecture is highly scalable. You could easily add more tools to the MCP server — such as a LaTeX compiler or a graphing tool — without changing a single line of the agent’s core logic.</p><h3>Thanks for reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>LinkedIn</strong></a>.</p><p>#aiagents #mcp #strandsagents #python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0f7e37e8241" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GraphRAG Tutorial — Neo4j + LLMs]]></title>
            <link>https://medium.com/@daniel.puenteviejo/graphrag-tutorial-neo4j-llms-47372b71e3fa?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/47372b71e3fa</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[neo4j]]></category>
            <category><![CDATA[graphrag]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Mon, 24 Nov 2025 21:12:27 GMT</pubDate>
            <atom:updated>2025-11-24T21:13:38.825Z</atom:updated>
            <content:encoded><![CDATA[<h3>GraphRAG Tutorial — Neo4j + LLMs</h3><h4><em>A practical, end-to-end guide to building a GraphRAG architecture using Neo4j, LangChain, and OpenAI models.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*pbHCgR93WcgQlHvqGnbO_Q.gif" /><figcaption>Notebook: <a href="https://github.com/DanielPuentee/neo4j-graphrag-tutorial/blob/main/graph_rag.ipynb">https://github.com/DanielPuentee/neo4j-graphrag-tutorial/blob/main/graph_rag.ipynb</a></figcaption></figure><h3>1. Introduction</h3><p>In this tutorial we will build a <strong>GraphRAG pipeline</strong> — a system combining a knowledge-graph backend (Neo4j) with large language models via LangChain — to support Retrieval-Augmented Generation (RAG) in a richer way than traditional vector-only approaches.</p><p>In the tutorial, we will:</p><ul><li>Use a small biomedical text dataset (drugs, diseases, biomarkers, trials).</li><li>Ingest the data into Neo4j, <strong>automatically detecting nodes and relationships</strong> so you can perform complex queries later.</li><li>Retrieve information using the GraphRAG architecture and generate answers with <strong>OpenAI models</strong>.</li></ul><p>By the end you’ll understand how to easily construct a GraphRAG pipeline for your project and take full advantage of your data and models.</p><h3>2. What is GraphRAG?</h3><p><strong>GraphRAG</strong> (Graph-Augmented Retrieval-Augmented Generation) is a powerful evolution of the standard RAG architecture that incorporates a knowledge graph into the retrieval layer. According to Neo4j GraphRAG, traditional RAG pipelines rely on vector searches over text fragments, which often <strong>fail to capture multi-hop relationships</strong> and <strong>lack explainability</strong> because the retrieved chunks are independent and lack connected context.</p><p>In contrast, GraphRAG uses a knowledge graph in which entities are modeled as nodes and their relationships as edges. The system can traverse the graph (for example, A → B → C) to gather context that is not just similar text, but connected knowledge. For instance, a graph retriever might start at a <strong>Drug</strong> node, follow the “TREATS” edge to a <strong>Disease</strong> node, and then the “INVESTIGATED_BY” edge to a <strong>Researcher</strong> node — thereby retrieving a more coherent set of facts for the LLM.</p><p>In essence, GraphRAG still begins with a vector-based or keyword-based search to obtain the most relevant pieces of information (<strong>just like classical RAG</strong>). But then, using those chunks as starting points, <strong>the system performs multi-hop</strong> graph traversal to find related entities and relationships. This final step yields a richer, more complete context for the LLM to generate accurate and explainable responses. The key difference are:</p><p>💬 <strong>Retrieval basis</strong>:</p><ul><li>Classical: vector similarity of text segments.</li><li>GraphRAG: combination of vector search <strong>and</strong> graph traversal (knowledge-driven).</li></ul><p>🧮 <strong>Reasoning depth</strong>:</p><ul><li>Classical: limited to whatever fragment matching yields.</li><li>GraphRAG: can follow relationships, enabling multi-step inference (e.g., “which drug tested in Spain targets biomarker X”).</li></ul><p>❓<strong>Explainability</strong>:</p><ul><li>Classical: hard to justify why a chunk was retrieved.</li><li>GraphRAG: relationships are explicit, enabling traceability of “this researcher → that trial → that drug”.</li></ul><h3>3. Why use Neo4j + LangChain?</h3><p>Neo4j is a proven graph-database engine that supports modeling domain knowledge as nodes and edges, and executing efficient graph queries (e.g., via Cypher). LangChain is a framework for orchestrating LLM workflows (retrieval, prompting, generation). By combining them we can:</p><ul><li>Use LangChain to <strong>automatically create the nodes and relationships</strong>.</li><li>Then retrieve information from the database and <strong>return the answer in a single line of code.</strong></li><li>Provide retrieval context that is richer and better connected than purely embedding-based text retrieval.</li></ul><p>Together, Neo4j + LangChain enable you to build a retrieval layer that is <strong>knowledge-aware</strong>, traceable and suited for complex reasoning over connected data — which is the essence of GraphRAG.</p><h3>4. Tutorial</h3><p>The first step is to create a <strong>Neo4j free account</strong>. Once you have your account, add the following environment variables to your .env file:</p><pre>OPENAI_API_KEY=&lt;your_api_key&gt;<br>NEO4J_URI=&lt;your_neo4j_uri&gt;<br>NEO4J_USER=&lt;your_neo4j_username&gt;<br>NEO4J_PASSWORD=&lt;your_neo4j_password&gt;</pre><p>You can find your NEO4J_URI in your Neo4j database settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8tEVXi0_ktJmx-Ycqv4zBA.png" /></figure><p>Next, create a <strong>Python 3.12.11 environment</strong>. If you’ve cloned the repository, simply install the dependencies using the requirements file (<a href="https://github.com/DanielPuentee/neo4j-graphrag-tutorial">https://github.com/DanielPuentee/neo4j-graphrag-tutorial</a>).</p><h4>🧠 4.1 Setup</h4><pre># LangChain Imports<br>from langchain.schema import Document<br>from langchain_openai import AzureChatOpenAI<br>from langchain_community.graphs.neo4j_graph import Neo4jGraph<br>from langchain_community.vectorstores.neo4j_vector import Neo4jVector<br>from langchain_experimental.graph_transformers import LLMGraphTransformer<br>from langchain_huggingface import HuggingFaceEmbeddings<br><br># Neo4j Imports<br>from neo4j import GraphDatabase<br>from neo4j_graphrag.retrievers import VectorRetriever<br>from neo4j_graphrag.retrievers.hybrid import HybridRetriever<br>from neo4j_graphrag.embeddings.sentence_transformers import SentenceTransformerEmbeddings<br>from neo4j_graphrag.llm import AzureOpenAILLM<br>from neo4j_graphrag.generation import GraphRAG<br><br>load_dotenv()<br>os.environ[&quot;OPENAI_API_KEY&quot;] = os.getenv(&#39;OPENAI_KEY&#39;)<br>os.environ[&quot;NEO4J_URI&quot;] = os.getenv(&#39;NEO4J_URI&#39;) # I case of any issue, change neo4j+s to neo4j+ssc<br>os.environ[&quot;NEO4J_USERNAME&quot;] = os.getenv(&#39;NEO4J_USERNAME&#39;)<br>os.environ[&quot;NEO4J_PASSWORD&quot;] = os.getenv(&#39;NEO4J_PASSWORD&#39;)</pre><p>You might get a connection error if your environment variables are not set correctly. If that happens, try changing the URI prefix from neo4j+s:// to neo4j+ssc://.</p><h4>🤖 4.2 Initialize Models and Graph</h4><p>For this project, we will use <strong>OpenAI’s </strong><strong>gpt-4o-mini</strong> for text generation and <strong>sentence-transformers/all-MiniLM-L6-v2</strong> for embeddings.</p><pre>llm = AzureChatOpenAI(<br>    model_name=&quot;gpt-4o-mini&quot;,<br>    api_key=os.getenv(&#39;OPENAI_KEY&#39;),<br>    api_version=os.getenv(&#39;OPENAI_VERSION&#39;),<br>    azure_endpoint=os.getenv(&#39;OPENAI_ENDPOINT&#39;),<br>    temperature=0<br>)<br><br>model_name = &quot;sentence-transformers/all-MiniLM-L6-v2&quot;<br>model_kwargs = {&#39;device&#39;: &#39;cpu&#39;}<br>encode_kwargs = {&#39;normalize_embeddings&#39;: False}<br>embeddings = HuggingFaceEmbeddings(<br>    model_name=model_name,<br>    model_kwargs=model_kwargs,<br>    encode_kwargs=encode_kwargs<br>)</pre><p>Now we’ll initialize the Neo4j graph and driver:</p><pre>graph = Neo4jGraph()<br>driver = GraphDatabase.driver(uri=os.getenv(&#39;NEO4J_URI&#39;), auth=(os.getenv(&#39;NEO4J_USERNAME&#39;), os.getenv(&#39;NEO4J_PASSWORD&#39;)))<br>session = driver.session()</pre><h4>🧬 4.3 Load and Process the Dataset</h4><p>The dataset used in this tutorial is a <strong>synthetic biomedical dataset</strong> containing information about drugs, diseases, biomarkers, and clinical trials. It was generated for educational purposes and is included in the repository.</p><h4>🕸️ 4.4 Identifying Nodes and Relationships</h4><p>Next, we’ll identify the <strong>nodes</strong> and <strong>relationships</strong> in our documents using LangChain’s Neo4jGraph capabilities.</p><ul><li><strong>Nodes</strong> represent entities or concepts in the graph.</li><li><strong>Relationships</strong> define how those entities are connected.</li></ul><p>For this tutorial, we’ll use the following schema:</p><pre>allowed_nodes = [<br>    &quot;Drug&quot;, &quot;Disease&quot;, &quot;Biomarker&quot;, &quot;Trial&quot;,<br>    &quot;Researcher&quot;, &quot;Institution&quot;, &quot;Company&quot;, &quot;Country&quot;<br>]<br><br>allowed_relationships = [<br>    &quot;TREATS&quot;, &quot;TARGETS&quot;, &quot;EVALUATED_IN&quot;, &quot;STUDIES&quot;,<br>    &quot;CONDUCTED_BY&quot;, &quot;LEAD_BY&quot;, &quot;SPONSORED_BY&quot;, &quot;LOCATED_IN&quot;<br>]</pre><h4>🧠 4.5 Extracting Graph Structure with LLM</h4><p>We’ll leverage the LLM to analyze unstructured text and extract entities and relationships automatically:</p><pre>transformer = LLMGraphTransformer(<br>    llm=llm,<br>    allowed_nodes=allowed_nodes,<br>    allowed_relationships=allowed_relationships,<br>    node_properties=False, <br>    relationship_properties=False<br>) </pre><p>Once identified, we’ll add them to the Neo4j database:</p><pre>graph_documents = transformer.convert_to_graph_documents(documents)<br>graph.add_graph_documents(graph_documents, include_source=True)</pre><h4>🗂️ 4.6 Creating the Keyword Index</h4><p>A keyword index enables <strong>hybrid retrieval</strong> — combining vector search, keyword search, and graph traversal.</p><pre>session.run(&quot;DROP INDEX keyword IF EXISTS&quot;)<br>session.run(&quot;&quot;&quot;<br>CREATE FULLTEXT INDEX keyword FOR (n:Document) ON EACH [n.text]<br>&quot;&quot;&quot;)</pre><h4>🧭 4.7 Adding Embeddings</h4><p>Adding embeddings allows us to perform vector-based similarity searches later.</p><pre>_ = Neo4jVector.from_documents(<br>    documents,<br>    embeddings,<br>    pre_delete_collection=True, <br>    index_name=&quot;vector&quot;,<br>    keyword_index_name=&quot;keyword&quot;<br>)</pre><h4>🔍 4.8 Creating the Hybrid Retriever</h4><p>Now that the database is set up, it’s time to create the retriever. The <strong>GraphRAG retriever</strong> fetches nodes based on both <strong>vector similarity</strong> and <strong>graph relationships</strong>.</p><p>The first step is to specify the <strong>embedding model</strong> — in this case, we’ll use the classical <strong>Sentence Transformer</strong>. Remember that we are using the <strong>Neo4j GraphRAG framework</strong>, which incorporates a <strong>hybrid retriever</strong> capable of combining <strong>vector search</strong>, <strong>keyword search</strong>, and <strong>multi-hop traversal</strong> across the graph.</p><pre>embedder = SentenceTransformerEmbeddings()<br><br>graphrag_retriever = HybridRetriever(<br>    driver,<br>    embedder=embedder,<br>    vector_index_name=&quot;vector&quot;,<br>    fulltext_index_name=&quot;keyword&quot;,<br>    return_properties=[&quot;text&quot;],<br>)</pre><p>You can test it with:</p><pre>query = &quot;Which researchers in Spain are involved in studies evaluating immunotherapy drugs for breast cancer?&quot;<br><br>retriever_result = graphrag_retriever.search(query_text=query, top_k=2)<br><br># Example of the first result<br>ast.literal_eval(retriever_result.model_dump_json())[&#39;items&#39;]</pre><h4>🧩 4.9 Integrating the LLM (GraphRAG)</h4><p>Now we’ll connect the retriever with the LLM so that the answer is generated automatically along with the relevant chunks.</p><pre>llm_graphrag = AzureOpenAILLM(<br>    model_name=&quot;gpt-4o-mini&quot;,<br>    api_key=os.getenv(&#39;OPENAI_KEY&#39;),<br>    api_version=os.getenv(&#39;OPENAI_VERSION&#39;),<br>    azure_endpoint=os.getenv(&#39;OPENAI_ENDPOINT&#39;),<br>)<br><br>rag = GraphRAG(<br>    retriever=graphrag_retriever, <br>    llm=llm_graphrag,<br>)</pre><p>You can test it with:</p><pre>query = &quot;Which researchers in Spain are involved in studies evaluating immunotherapy drugs for breast cancer?&quot;<br><br>response = rag.search(<br>    query_text=query,<br>    retriever_config={<br>        &quot;top_k&quot;: 2,<br>        # &quot;filters&quot;: {}<br>    },<br>    return_context=True<br>)<br><br># The context<br>print(response.retriever_result.model_dump()[&#39;items&#39;])<br><br># The LLM direct answer<br>print(response.answer)</pre><h3>5. Conclusion</h3><p>This article walked you through how to <strong>build a complete GraphRAG pipeline</strong> using <strong>Neo4j</strong>, <strong>LangChain</strong>, and <strong>OpenAI models</strong>. We covered how to automatically create nodes and relationships from unstructured text, how to enrich your graph with embeddings, and how to perform <strong>hybrid retrieval</strong> that combines <strong>vector</strong>, <strong>keyword</strong>, and <strong>multi-hop graph search</strong>.</p><p>This tutorial is meant to help you understand the core building blocks of <strong>GraphRAG systems</strong> — from ingestion to reasoning. Now it’s your turn to experiment: try expanding the dataset, exploring new relationships, or connecting your own domain data to see how <strong>graph-based retrieval</strong> can elevate your RAG applications.</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#graphrag #llm #gpt #langchain</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47372b71e3fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Autogen Multi-Agent Ecosystem: Profile Analysis with Free LLMs]]></title>
            <link>https://medium.com/@daniel.puenteviejo/autogen-multi-agent-ecosystem-profile-analysis-with-free-llms-749bb321c68f?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/749bb321c68f</guid>
            <category><![CDATA[groq]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[autogen]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 08:21:31 GMT</pubDate>
            <atom:updated>2025-01-29T08:21:31.760Z</atom:updated>
            <content:encoded><![CDATA[<h4>Practical example of how to create a multi-agent project with Autogen and completely free LLMs.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EjDMzB8c3a6VL0tQBn1Xog.png" /></figure><p>This article focuses on a practical and explanatory exercise on how to use <strong>Autogen </strong>for your projects by applying free LLMs. We will cover a simple overview of what agents and <strong>multi-agent ecosystems</strong> are, and then move to the practical use case prepared to utilize Autogen agents, tools, and other functionalities alongside <strong>Groq</strong>. This approach will allow you to use LLMs freely without the need to download them locally to your computer.</p><p>In the practical example, we will explore how to create a group of agents specialized in specific tasks, with the capability to use tools and be managed by an agent manager. We will also dive into how to partially control which agent is called at each step using a <strong>hybrid approach</strong>, allowing you to maintain a degree of control rather than letting the agent manager make all the decisions.</p><p>Sounds interesting, doesn’t it? Let’s dive in!</p><h3>1. AI Agents</h3><h4><strong>1.1. What is an agent?</strong></h4><p>Let’s start by defining what an agent is. We can formally define an agent as a <strong>self-contained entity designed to perform a specific task or set of tasks autonomously</strong>. Agents can perceive their environment, make decisions based on the input they receive, and take actions to achieve predefined goals.</p><p>As we will see in the practical example, an agent has a specific goal or specialty. For instance, in the example, we will have an agent specialized in recommending courses and another focused on extracting the main skills from a given profile.</p><h4>1.2. What are tools?</h4><p>When we talk about tools, these can be understood as functions that agents have at their disposal and can use based on the input they receive. A simple example could be reading a .txt file. Agents can also create pieces of code and execute them. However, to prevent them from deviating too much and to focus their actions more effectively, we can <strong>create some predefined functions</strong>.</p><h4><strong>1.3. What is a Multi-agent ecosystem?</strong></h4><p>Once we have defined the principal elements of these systems, we can start talking about a multi-agent ecosystem. This is simply a group of agents, each specialized in a specific task, that collectively provide a complete and accurate response to the user’s query.</p><p>There are different ways to manage a group of agents. Let’s review the most common conversational patterns:</p><ol><li><strong>Round Robin</strong>: Agents follow a predefined order, regardless of the query. The system executes them sequentially according to this order.</li><li><strong>Random</strong>: The speaker selection is random. This approach is not highly recommended due to its lack of structure.</li><li><strong>Manual</strong>: When a query is made, the system starts, and agent selection is performed manually by a human. After one agent completes its task, the system asks the user to specify the next agent, continuing until the query is considered resolved.</li><li><strong>Auto</strong>: An LLM decides the next speaker based on the context of the conversation and the description of each agent.</li><li><strong>Hybrid</strong>: A combination of manual and automated approaches. You can establish a set of rules to allow the LLM manager to select the next agent or specify which agents will act depending on predefined conditions.</li></ol><p>Of course, there are more complex ways to manage agent ecosystems, but we will focus on these examples to keep the article concise and help you understand the concept.</p><h3>2. Groq</h3><p>Groq is an American technology company specializing in artificial intelligence (AI) hardware and software solutions. Among their offerings is <strong>GroqCloud</strong>, a platform designed to provide developers with high-performance inference capabilities for LLMs. It features an intuitive interface and a suite of tools to facilitate the integration and scaling of AI applications.</p><p>GroqCloud provides free access to LLMs such as <em>Mistral, Llama, and Gemma</em>, among others. In addition to using these models individually, it also integrates seamlessly with frameworks like <em>Autogen </em>and<em> Crew AI</em>.</p><h4>2.1 Setup</h4><p>The setup is straightforward. Simply go to the main page, log in, and create an API Key by visiting this link → <a href="https://console.groq.com/keys"><em>https://console.groq.com/keys</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PY1Zd6w6gZ3ZGETOu9tRHQ.png" /><figcaption>Groq API Key creation</figcaption></figure><p>As shown in the image above, you just need to create an API Key. The Groq website provides detailed documentation on how to use the API and all the possible integrations.</p><h3>3. What is Autogen?</h3><p><strong>Autogen</strong> is an open-source framework developed by Microsoft Research that simplifies the creation and orchestration of AI applications using multiple agents. These agents can converse with each other, integrate large <em>LLMs</em>, tools, and human inputs to collaboratively solve complex tasks.</p><p>Among AutoGen’s key features are the following:</p><ol><li><strong>Multi-Agent Architecture: </strong><em>Autogen </em>enables the development of customizable agents that can interact in natural language, allowing for dynamic and adaptable workflows.</li><li><strong>Integration with LLMs: </strong>Agents can leverage advanced <em>LLMs</em>, such as <em>GPT-4</em>, to enhance their language understanding and generation capabilities.</li><li><strong>Code Execution and Generation</strong>: <em>Autogen </em>supports agents in generating, executing, and debugging code, making it highly valuable for software development tasks.</li><li><strong>Human-in-the-Loop Functionality</strong>: The framework allows varying levels of human involvement, from fully autonomous operations to systems that actively seek human input and feedback.</li></ol><h3>4. Use case</h3><p>With the basics established, we can now move on to explaining the use case. If you want to access all the necessary materials, feel free to check out the repository → <a href="https://github.com/DanielPuentee/autogen-groq-multiagent-analyser">https://github.com/DanielPuentee/autogen-groq-multiagent-analyser</a></p><p>The idea is straightforward: it involves a group of agents designed to analyze a CV or profile, each specializing in a specific task. This group will consist of four agents:</p><ol><li><strong>CV Analyzer </strong>📊: This agent specializes in analyzing the input CV or profile document. Its task is to extract the most relevant information from the document, such as professional experience, education, and notable achievements, providing a structured summary of the candidate’s profile.</li><li><strong>Skills Extractor </strong>🔧: This agent is responsible for extracting the key skills from the CV. It identifies competencies such as programming languages, technical proficiencies, or soft skills and returns them in a concise, comma-separated list for easy reference.</li><li><strong>Courses Recommender </strong>👨‍🎓: This agent matches the extracted skills with a database of available courses. Its goal is to recommend courses that can help the candidate enhance or expand their skill set, providing tailored suggestions based on their profile.</li><li><strong>Resolution Checker </strong>🎯: The role of this agent is to determine whether the user’s query has been fully resolved. It reviews the outputs of the other agents and, if the task is completed, it is responsible for forming the final answer.</li></ol><h4>4.1 Dependencies</h4><p>To start with the tutorial, we first need to install the following dependencies:</p><pre>langchain==0.3.10<br>langchain-mistralai==0.2.3<br>langgraph==0.2.60<br>python-dotenv==1.0.1</pre><p>The import the necessary libraries for this tutorial:</p><pre>import warnings<br>warnings.filterwarnings(&quot;ignore&quot;)<br><br>import autogen<br>from autogen import UserProxyAgent, register_function, AssistantAgent<br><br>import os<br>from dotenv import load_dotenv<br>load_dotenv()</pre><h4>4.2 Models</h4><p>The next step is to create the configuration for the LLMs that we will use in the project. Groq provides the possibility of using the following models:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*yw3ED_FZ1w3wFO37t0PJpQ.png" /><figcaption>Groq Models</figcaption></figure><p>In this case, we will create two configurations: one using llama-3.1-8b-instant and another using gemma2-9b-it.</p><pre># llama 3.1 instant config<br>llama31_config = {<br>    &quot;config_list&quot;: [<br>        {<br>            &quot;model&quot;: &quot;llama-3.1-8b-instant&quot;,<br>            &quot;api_key&quot;: os.environ.get(&quot;GROQ_API_KEY&quot;),<br>            &quot;api_type&quot;: &quot;groq&quot;<br>        }<br>    ]<br>}<br><br># gemma2 config<br>gemma2_config = {<br>    &quot;config_list&quot;: [<br>        {<br>            &quot;model&quot;: &quot;gemma2-9b-it&quot;,<br>            &quot;api_key&quot;: os.environ.get(&quot;GROQ_API_KEY&quot;),<br>            &quot;api_type&quot;: &quot;groq&quot;<br>        }<br>    ]<br>}</pre><p>As you can see, the first key of the dictionary, config_list, is a list. This means we can incorporate multiple models into a single configuration. However, in this project, we have split the models into separate configurations because we will explore the possibility of assigning llama-3.1-8b-instant capabilities to some agents and gemma2-9b-it to others. This approach is based on the fact that some models perform better for certain tasks than others.</p><h4>4.3 Files setup</h4><p>To proceed with the use case, we will need two files located in a data folder:</p><ol><li>data/courses.txt<strong>:</strong> This text file should contain a list of courses to be used for recommendations. Examples include “<em>Machine Learning Crash Course - Google Developers”</em> or “<em>Machine Learning Specialization - Coursera”.</em></li><li>data/cv.txt<strong>:</strong> This file should contain a CV or profile description that will serve as one of the main inputs for initializing the agent system. While I will use my own resume in the repository, this file can be replaced with any other profile description.</li></ol><h4>4.4 Termination policy</h4><p>Autogen requires a mechanism to determine when a process has reached its conclusion. To achieve this, we need to analyze the result and identify a specific pattern. A recommended approach is to program agents to return the word “<strong>TERMINATE”</strong> when they conclude that the query has been successfully resolved. For this purpose, we define the following function:</p><pre># Termination policy<br>def termination_message(msg: str) -&gt; bool:<br>    content = msg.get(&quot;content&quot;, &quot;&quot;).lower()<br>    return &quot;terminate&quot; in content</pre><h4>4.5 Agents</h4><p>Let’s begin with the <strong>UserProxyAgent</strong>. It acts as an intermediary, representing a human user within a multi-agent system. Its primary function is to execute code and provide feedback to other agents, enabling human-in-the-loop interactions during automated processes. In this case, it is only responsible for executing tools and returning the results to the appropriate agent.</p><pre># The user proxy agent that executes tool calls.<br>user_proxy = UserProxyAgent(<br>    name=&quot;User&quot;,<br>    is_termination_msg=termination_message,<br>    human_input_mode=&quot;NEVER&quot;,<br>    code_execution_config={&quot;use_docker&quot;: False},<br>)</pre><p>Let’s briefly describe the purpose of each argument:</p><ul><li>name=&quot;User&quot;: This is the name for this agent.</li><li>llm_config=False: Disables the use of the LLM as we just need to execute tools.</li><li>is_termination_msg=termination_message: Uses the termination_message() function to detect when the process is complete.</li><li>human_input_mode=&quot;NEVER&quot;: Operates autonomously without requiring human input.</li><li>code_execution_config={&quot;use_docker&quot;: False}: Executes code directly on the host environment without Docker.</li></ul><p>Now we will pass to the main characters of this game:</p><pre>## -- Agent 1: CV Analyser --<br>cv_analyser = AssistantAgent(<br>    name=&quot;CV Analyser&quot;,<br>    description=&quot;It analyzes the input it giving the most relevant information of the document.&quot;,<br>    system_message=&quot;&quot;&quot;You must return an analysis of the input file.<br>    Other details for your answer:<br>    - Do not responde any piece of code.<br>    - Do not recommend courses just analyse the input file.&quot;&quot;&quot;,<br>    llm_config=llama31_config,<br>)<br><br>## -- Agent 2: Skills Extractor --<br>skills_extractor = AssistantAgent(<br>    name=&quot;Skills Extractor&quot;,<br>    description=&quot;Returns a list of the candidates skills&quot;,<br>    system_message=&quot;&quot;&quot;Your task is to extract the main skills from the input received.<br>    You must return the skills as a comma-separated list. For example: &#39;Python, Java, SQL&#39;. Do not add more information, just the list of skills in the specified format.<br>    Other details for your answer:<br>    - Do not responde any piece of code.<br>    - Do not recommend courses just return the skills as a comma-separated list of the input file.&quot;&quot;&quot;,<br>    llm_config=llama31_config,<br>)<br><br>## -- Agent 3: Courses Recommender --<br>courses_recommender = AssistantAgent(<br>    name=&quot;Courses recommender&quot;,<br>    description=&quot;The goal is to recommend courses. Read the courses from `data/courses.txt` and return the courses that match the skills of the input file.&quot;,<br>    system_message=&quot;&quot;&quot;The goal is to recommend courses, for that follow these steps:<br>    1) Read the list of courses from `data/courses.txt` (use `read_txt_courses` tool).<br>    2) Combine the list of courses with the profile of the candidate.<br>    3) Return the courses that match the skills of the candidate.<br>    Other details for your answer: Do not responde any piece of code.&quot;&quot;&quot;,<br>    llm_config=llama31_config,<br>)<br><br># -- Agent 4: Resolution Checker Agent --<br>resolution_checker = AssistantAgent(<br>    name=&quot;Resolution Checker Agent&quot;,<br>    description=&quot;Checks if the user&#39;s query has been resolved. If resolved, respond with &#39;TERMINATE&#39;, otherwise respond with &#39;CONTINUE&#39;. Do not give more information just the word &#39;CONTINUE&#39; or &#39;TERMINATE&#39;.&quot;,<br>    system_message=&quot;&quot;&quot;You should respond with &#39;TERMINATE&#39; if the query has been resolved. Otherwise, respond with &#39;CONTINUE&#39;. Do not add more information, just the word &#39;CONTINUE&#39;.<br>    Try to be resolute, do not reply &#39;TERMINATE&#39; if the initial query has not been resolved.<br>    **IMPORTANT**: Only respond with &#39;TERMINATE&#39; if the query has been resolved, otherwise respond with &#39;CONTINUE&#39;. If the answer is &#39;TERMINATE&#39;, you must answer &#39;TERMINATE&#39; + generate a complete answer to the user&#39;s query considering all the information given by all the agents to generate the final answer.&quot;&quot;&quot;,<br>    llm_config=gemma2_config,<br>    is_termination_msg=termination_message,<br>)</pre><p>We will not go into detail about the prompts used in this example, but it is important to mention the main parameters of the agents:</p><ul><li>name: The name assigned to each agent.</li><li>llm_config: Defines the configuration of the language model for the agent, such as gemma2_config or llama31_config. The gemma2_config is specifically used in the Resolution Checker, as it has been observed to perform better for this task. Unlike False, this configuration allows the agent to generate responses or invoke tools.</li><li>description: Provides a general overview of each agent&#39;s role and purpose.</li><li>system_message: The specific prompt given to each agent, outlining its behavior and tasks.</li><li>is_termination_msg: Assigns the termination_message() function to detect when the process is complete. This is only used by the Resolution Checker, as it is the sole agent responsible for returning &quot;TERMINATE&quot; or &quot;CONTINUE&quot; based on the task&#39;s status.</li></ul><h4>4.6 Tool definition</h4><p>Now, we will define the tool that will be used.</p><pre># Tool: Read CV from File<br>def read_txt(path: str) -&gt; str:<br>    with open(path, &quot;r&quot;) as file:<br>        return file.read()</pre><p>This is a simple Python function that reads a .txt file. The use of type annotations is important to clearly define the expected input and output of the function.</p><p>The next step is to configure each agent to allow the use of this tool. The tool is executed by the <strong>User Proxy</strong>, so we need to specify the <strong>caller</strong>, which is the agent that can access the tool, and the <strong>executor</strong>, which will be the User Proxy. Additionally, we must provide a name and description for the tool.</p><pre># Register the read_txt function with the assistant agent.<br>register_function(<br>    read_txt,<br>    caller=cv_analyser,<br>    executor=user_proxy,<br>    name=&quot;read_txt_cv_analyse&quot;,<br>    description=&quot;Read the profile of the user&quot;,<br>)<br><br>register_function(<br>    read_txt,<br>    caller=courses_recommender,<br>    executor=user_proxy,<br>    name=&quot;read_txt_courses&quot;,<br>    description=&quot;Read available courses and/or the profile of the user&quot;,<br>)<br><br>register_function(<br>    read_txt,<br>    caller=skills_extractor,<br>    executor=user_proxy,<br>    name=&quot;read_txt_cv_skills&quot;,<br>    description=&quot;Read the profile of the user&quot;,<br>)</pre><p>As we can see, we have not granted access to the <strong>Resolution Checker</strong>. This is because its sole purpose is to verify the status of the process and generate the final answer, utilizing all the information already available to it.</p><h4>4.7 Agent order</h4><p>Next, we move to what may be the most challenging part of the tutorial. As mentioned at the beginning of the document, there are five types of agent selection: <em>(i) round robin, (ii) random, (iii) manual, (iv) automatic and (v) hybrid.</em></p><p>We will focus on the last type, <strong>hybrid</strong>. To implement this, we need to create a custom function:</p><pre># Create a GroupChatManager to oversee the conversation<br>def custom_speaker_selection(last_speaker, groupchat):<br>    &quot;&quot;&quot;<br>    Function to change the agent selection logic and customize the speaker selection.<br><br>    Args:<br>    last_speaker (Agent): The last agent that spoke.<br>    groupchat (GroupChat): The group chat object.<br><br>    Returns:<br>    Agent: The agent that will speak next.<br>    &quot;&quot;&quot;<br>    agents_to_check = [&quot;cv analyser&quot;, &quot;skills extractor&quot;, &quot;courses recommender&quot;]<br><br>    # Extract the last and penultimate messages<br>    last_message = groupchat.messages[-1]<br>    penultimate_message = groupchat.messages[-2] if len(groupchat.messages) &gt; 1 else None<br><br>    # Extract the names of the last and penultimate speakers<br>    last_speaker_name = last_speaker.name.lower()<br>    penultimate_speaker_name = penultimate_message[&#39;name&#39;] if penultimate_message else None<br><br>    ### If the agent has made a tool call, call the &#39;User&#39;<br>    if last_message.get(&quot;tool_calls&quot;, &quot;&quot;):<br>        return next(agent for agent in groupchat.agents if agent.name == &quot;User&quot;)<br><br>    ### If the last speaker was the &#39;User&#39; and the penultimate speaker was not the user or the Resolution Checker Agent, call the penultimate speaker<br>    ### Example: Skills Extractor makes use of a tool, so it call the &#39;User&#39; agent. The &#39;User&#39; agent then calls the &#39;Skills Extractor&#39; to continue with the task it was doing.<br>    if last_speaker_name == &quot;user&quot; and penultimate_speaker_name and penultimate_speaker_name.lower() not in [&quot;user&quot;, &quot;resolution checker agent&quot;]:<br>        return next(agent for agent in groupchat.agents if agent.name == penultimate_speaker_name)<br><br>    ### The an agent has given an answer, check if the quey is complete. For that call the &#39;Resolution Checker Agent&#39;<br>    if last_speaker_name in agents_to_check:<br>        return next(agent for agent in groupchat.agents if agent.name == &quot;Resolution Checker Agent&quot;)<br><br>    ### If no condition is met, leave the automatic configuration so that the next agent selection is based on an LLM.<br>    return &#39;auto&#39;</pre><p>Although the code may seem complex at first glance, it is relatively straightforward. Here’s a breakdown of its logic:</p><ol><li><strong>Retrieve Agent Information</strong>:<br>The first lines extract the <strong>names and messages</strong> of the last and penultimate agents involved in the conversation.</li><li><strong>Condition 1: Tool Execution</strong>:<br>If a tool call is detected, the next agent is clearly the <strong>User Proxy</strong>, as it is responsible for executing tools.</li><li><strong>Condition 2: Agent Continuation</strong>:<br>When an agent calls a tool, the same agent is called again after the tool execution, as it already has the necessary context to complete its task.<br><em>Example</em>: The <strong>Skills Extractor</strong> calls a tool, so the <strong>User Proxy</strong> executes it and then calls the <strong>Skills Extractor</strong> to continue the task.</li><li><strong>Condition 3: Query Resolution Check</strong>:<br>After an agent completes its task, the <strong>Resolution Checker</strong> is always called to verify if the query is resolved. This ensures the process doesn’t rely solely on automatic AI decision-making.</li><li><strong>Fallback: LLM-Based Selection</strong>:<br>If none of the above conditions apply, the next agent is selected automatically by the LLM.</li></ol><h4>4.8 Multi-agent ecosystem</h4><p>The final step before interacting with the system is to create the engine that handles everything we have constructed.</p><pre># Create the group chat<br>groupchat = autogen.GroupChat(<br>    agents=[<br>        user_proxy,<br>        cv_analyser,<br>        skills_extractor,<br>        courses_recommender,<br>        resolution_checker,<br>    ],<br>    messages=[],<br>    allow_repeat_speaker=True,<br>    speaker_selection_method=custom_speaker_selection, # Other options are: &quot;round_robin&quot;, &quot;random&quot;, &quot;manual&quot;, &quot;auto&quot;<br>    role_for_select_speaker_messages=&quot;system&quot;<br>)<br><br># Finally, create the group chat manager<br>manager = autogen.GroupChatManager(<br>    groupchat=groupchat,<br>    code_execution_config={&quot;use_docker&quot;: False},<br>    llm_config=gemma2_config,<br>    is_termination_msg=termination_message,<br>)</pre><p>The <strong>group chat</strong> specifies the agents participating in the system, how to select the next agent, whether an agent can speak consecutively or not, and the focus criteria for agent selection.</p><p>The <strong>manager </strong>can be defined as the entity that will manage the whole system. It requires the previously defined group chat, specifies that code execution does not use Docker, determines the LLM configuration for agent selection, and defines the termination message to signal when the process is complete.</p><p>As you have seen, we have selected <strong>gemma2</strong> as the LLM. This choice was made because it provides better performance for this particular use case. However, you are encouraged to experiment with different configurations, adjust these parameters, and determine which works best for your specific needs.</p><h4>4.9 Execution</h4><p>The final step is to use the system.</p><pre># File path for the CV<br>cv_file_path = &quot;data/cv.txt&quot;<br><br># Define the message to be sent to the manager<br>message = f&quot;Recommend courses for the profile {cv_file_path}&quot;<br># message = f&quot;Analyse the following profile: {cv_file_path}.&quot;<br># message = f&quot;Extract skills in comma-separated format based on the following profile: {cv_file_path}.&quot;<br><br>chat_result = user_proxy.initiate_chat(<br>    recipient=manager,<br>    message=message,<br>    max_turns=30,<br>)</pre><p>First, we define the file path variable for the CV. Next, we create the message specifying what we want the system to do. Some example messages are provided to illustrate different functionalities of the system.</p><p>Finally, we initiate the chat using the <strong>User Proxy</strong>, passing the <strong>manager</strong> to control the conversation, the message to be processed, and the maximum number of turns. Setting a high value for max_turns ensures that even if the termination criteria are not met, the conversation will eventually end without being cut off prematurely.</p><pre># See the agent order<br>agent_order = [x[&#39;name&#39;] for x in chat_result.chat_history]<br>print(agent_order)<br><br># Show the last result<br>print(chat_result.chat_history[-1][&#39;content&#39;])</pre><p>Once the system has completed its process, we can review the sequence of agents involved in solving the query using the agent_order. The final line of code displays the system&#39;s final response, providing the ultimate result of the conversation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*WGifYUxsavCyHHMZa2rVyg.gif" /></figure><h3>5. Conclusion</h3><p>This article outlines the steps required to build and execute a multi-agent system using <strong>Autogen</strong> and free LLMs with <strong>Groq</strong>. We also covered how to create a hybrid agent selection method and demonstrated the usage of tools. This exercise is intended as a tutorial to help you learn the basics and a bit more. From here, it’s up to you to continue exploring and developing unique multi-agent systems. 😉</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#autogen #groq #llm #ai #python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=749bb321c68f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangGraph + Mistral: CV Evaluator agent system]]></title>
            <link>https://medium.com/@daniel.puenteviejo/langgraph-mistral-cv-evaluator-agent-system-b74ee5390500?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b74ee5390500</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mistral-ai]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Mon, 13 Jan 2025 08:24:35 GMT</pubDate>
            <atom:updated>2025-01-13T08:24:35.956Z</atom:updated>
            <content:encoded><![CDATA[<h4>Practical agent system tutorial using LangGraph and Mistral</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2sa4Htg1gZ3zIgB_bqIkHg.gif" /></figure><p>When it comes to managing agents in AI workflows, LangGraph is a powerful tool. This article aims to provide a straightforward guide, applying LangGraph to a practical use case. We’ll explore its features while utilizing Mistral. This solution enables access to free large language models (LLMs) without the need for expensive computation.</p><p>For all the content and a complete tutorial notebook, check out the project on GitHub → <a href="https://github.com/DanielPuentee/LangGraph-Mistral-CV-Evaluator/blob/main/tutorial.ipynb">https://github.com/DanielPuentee/LangGraph-Mistral-CV-Evaluator/blob/main/tutorial.ipynb</a></p><p>Let’s dive in!</p><h3>1. What is LangGraph?</h3><p>Before we dive into the guide, let’s first understand what LangGraph is and why it has become so popular. LangGraph is a powerful library within the LangChain ecosystem, specifically designed to facilitate the creation of stateful, <strong>multi-agent applications</strong> utilizing large language models (LLMs). It offers a structured framework for defining, coordinating, and executing multiple LLM agents or chains, enabling developers to build complex, scalable, and flexible AI systems.</p><p>Unlike traditional Directed Acyclic Graph (DAG) frameworks, LangGraph introduces support for <strong>cyclical graphs</strong>, a key feature for developing sophisticated agent runtimes. This capability allows for the integration of loops and conditionals, making applications more adaptable and responsive to varying scenarios.</p><p>In the following sections, we will explore practical examples and delve deeper into the critical aspects of LangGraph, illustrating its versatility and importance in modern AI workflows.</p><h3><strong>2. Packages</strong></h3><p>Now we will go with the starting guide of Mistral and LangGraph so it is important to have this <strong>dependencies</strong>:</p><pre># Python used version: 3.10.15<br><br>langchain==0.3.10<br>langchain-mistralai==0.2.3<br>langgraph==0.2.60</pre><h3>3. How to use Mistral</h3><p>Mistral will serve as the LLM engine for this guide. To begin, it’s essential to understand how to set it up and use it effectively. Start by logging in to the Mistral platform and <strong>generating an API key</strong>. You can do this by visiting the following link → <a href="https://console.mistral.ai/api-keys/">https://console.mistral.ai/api-keys/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3rqB1e3ehzppUSWIhk9how.png" /></figure><p>After logging in, you’ll see a screen similar to the one below. Click on “Create new key,” assign a name to your key, and it will be ready for use.</p><p>Now that you have your API key, you can integrate Mistral with LangChain to fully utilize its capabilities. To do so, you’ll need to import the LangChain Mistral library, configure it with your API key, and specify the desired model and parameters.</p><p>For this guide, we’ll use the <strong>mistral-small-latest</strong> model with with <strong>no temperature and 5 maximum retries. </strong>As an example, we’ll create a simple prompt to translate the phrase <em>“I love programming”</em> into Spanish.</p><pre>### MISTRAL EXAMPLE<br>from langchain_mistralai import ChatMistralAI<br>from langchain_core.messages import SystemMessage, HumanMessage<br><br># Configure Mistral API Key<br>os.environ[&#39;MISTRAL_API_KEY&#39;] = &quot;&lt;your_key&gt;&quot;<br><br># LLM Configuration<br>llm = ChatMistralAI(<br>    model=&quot;mistral-small-latest&quot;,<br>    temperature=0,<br>    max_retries=5<br>)<br><br># Prompts<br>sys_prompt = &quot;You are a helpful assistant that translates English to Spanish&quot;<br>hum_prompt = &quot;Translate the user sentence without giving more details, just the translation: I love programming.&quot;<br><br># Invoke LLM<br>response = llm.invoke([<br>    SystemMessage(content=sys_prompt),<br>    HumanMessage(content=hum_prompt)]<br>).content.strip()<br><br>print(response)</pre><h3>4. Use case with LangGraph</h3><p>We will demonstrate a simple use case involving <strong>4 different agents</strong>. The goal is to create a system that processes a CV and a job offer, evaluates whether the candidate is a good fit for the position, and then drafts an email either accepting or declining the offer, along with relevant comments. The flow is outlined below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8usR8dC1xXLPYOzrbx2nug.gif" /></figure><p>In this case, the steps are designed to be sequential to simplify the process and help you quickly understand the dynamics of LangGraph.</p><p>Starting with the tutorial, the first step is to define the <strong>State</strong>. The State establishes <strong>how messages are passed between agents</strong>. This structure ensures that information flows seamlessly throughout the system. Typing is crucial here, as it helps maintain clarity and consistency. In this case, we use dictionaries for their simplicity and ease of tracking.</p><pre># State definition<br>from typing import Dict<br><br>class State(TypedDict):<br>    messages: Dict</pre><p>We start by defining a simple function to read .txt files:</p><pre># Define functions for processing<br>def read_file(file_path):<br>    with open(file_path, &#39;r&#39;) as f:<br>        return f.read()</pre><h4>4.1 Agents and tools</h4><p>In this section, we will define each agent and the tool it will use. Here, each agent is mapped to a single tool, as each agent operates independently using its dedicated function. This one-to-one mapping simplifies the workflow and ensures clarity in the implementation.</p><pre># Libraries<br><br>from langchain_core.messages import SystemMessage, ToolMessage, AIMessage, HumanMessage<br>from langgraph.graph import StateGraph, START, END<br>import uuid</pre><p>Let’s begin by defining the first two agents, which will handle reading the CV content and the Job Offer content. The first one is the CV reader. However, before diving into the code, <strong>it’s important to establish the initial state</strong> (start_state) that comes into play. The start_state is defined as follows:</p><pre>cv_path = &quot;data/cv.txt&quot;<br>job_path = &quot;data/job_description.txt&quot;<br>start_state = {&quot;messages&quot;: {&quot;cv_path&quot;: cv_path, &quot;job_path&quot;: job_path}}</pre><p>In this project, we have the following files located in the data folder:</p><ul><li>cv.txt: This file contains the CV information, which in the cloned GitHub repository is my CV.</li><li>job_description.txt: This file features a fictional job description from MAIgents, designed specifically for this tutorial.</li></ul><p>If you clone the GitHub repository, these sample files will allow you to replicate and follow the tutorial (<a href="https://github.com/DanielPuentee/LangGraph-Mistral-CV-Evaluator">https://github.com/DanielPuentee/LangGraph-Mistral-CV-Evaluator</a>).</p><p>Once the start_state is defined, we can move on to the definition of the agents and their respective tools.</p><pre># Read CV<br>def get_cv_content(state: dict): # The state here is -&gt; {&quot;messages&quot;: {&quot;cv_path&quot;: cv_path, &quot;job_path&quot;: job_path}}<br>    print(&quot;***** get_cv_content *****&quot;)<br>  <br>    # Extract the CV file path from the state<br>    cv_path = state[&#39;messages&#39;][&#39;cv_path&#39;]<br>    cv_content = read_file(cv_path)<br>    <br>    # Create a ToolMessage object with the CV content and a unique tool call ID<br>    tool_call = ToolMessage(<br>        content=cv_content,<br>        tool_call_id=str(uuid.uuid4())<br>    )<br><br>    # Update the state with the CV content and retain the job file path<br>    new_state = {<br>        &quot;messages&quot;: {<br>            &quot;job_path&quot;: state[&#39;messages&#39;][&#39;job_path&#39;],<br>            &quot;cv_content&quot;: tool_call<br>        }<br>    }<br>    return new_state<br><br># Read Job Offer<br>def get_job_description(state: dict): # The state here will be the get_cv_content result -&gt; {&quot;messages&quot;: {&quot;job_path&quot;: job_path, &quot;cv_content&quot;: ___}}<br>    print(&quot;***** get_job_description *****&quot;)<br><br>    # Extract the job description file path from the state<br>    job_path = state[&#39;messages&#39;][&#39;job_path&#39;]<br>    <br>    # Read the content of the job description file<br>    job_content = read_file(job_path)<br><br>    # Create a ToolMessage object with the job description content and a unique tool call ID<br>    tool_call = ToolMessage(<br>        content=job_content,<br>        tool_call_id=str(uuid.uuid4())<br>    )<br><br>    # Update the state with the job description content and retain the CV content<br>    new_state = {<br>        &quot;messages&quot;: {<br>            &quot;cv_content&quot;: state[&#39;messages&#39;][&#39;cv_content&#39;],<br>            &quot;job_content&quot;: tool_call<br>        }<br>    }<br>    return new_state</pre><p>The code is straightforward and serves as the backbone of the first two agents. The first function, get_cv_content, takes the start_state as input to begin the process. This function extracts the CV file path from the initial state, reads the file content, and updates the state with the CV content while retaining the job file path. The updated state is then passed to the next agent.</p><p><strong>The order of the agents will be indicated once all agents and their tools are fully defined</strong>. However, as we have progressed, we know that after reading the CV, the logical next step is to process the job description. This sequence is essential because the updated state from get_cv_content provides the cv_content and retains the job_path, which the next agent, get_job_description, needs to accomplish its goal.</p><p>The get_job_description function follows a similar structure. It extracts the job description file path, reads the file content, creates a standardized message, and updates the state with both the CV and job description content for the next agent.</p><p>The ToolMessage is a <strong>standardized way to pass messages</strong> from agent to agent, using an ID, tool_call_id, to keep track of each message sent.</p><p>Now, let’s move on to the next agent, which will be responsible for evaluating the compatibility between the CV and the job description.</p><pre># Evaluate Match<br>def evaluate_match(state: dict): # The state here will be the get_job_description result -&gt; {&quot;messages&quot;: {&quot;cv_content&quot;: ___, &quot;job_content&quot;: ___}}<br>    print(&quot;***** evaluate_match *****&quot;)<br><br>    # Extract the CV and job description content from the state<br>    cv_content = state[&quot;messages&quot;][&quot;cv_content&quot;].content<br>    job_content = state[&quot;messages&quot;][&quot;job_content&quot;].content<br><br>    # Create the evaluation prompts<br>    hum_evaluation_template = &quot;&quot;&quot;CV:<br>    {cv}<br>    ===<br>    Job Description:<br>    {job}<br>    ===<br>    Evaluate whether the CV aligns with the job description provided. Clearly indicate the degree of match and provide specific reasons for your assessment, ensuring a detailed and professional response:<br>    &quot;&quot;&quot;<br>    hum_evaluation_prompt = hum_evaluation_template.format(cv=cv_content, job=job_content)<br>    sys_evaluation_prompt = &quot;&quot;&quot;You are tasked with evaluating whether the CV aligns with the job description provided. Clearly indicate the degree of match and provide specific, detailed reasons for your assessment.&quot;&quot;&quot;<br><br>    # Invoke the LLM with the evaluation prompts<br>    response = llm.invoke([SystemMessage(content=sys_evaluation_prompt),<br>                           HumanMessage(content=hum_evaluation_prompt)])<br>    content = response.content.strip()<br><br>    # Create an AIMessage object with the evaluation response and a unique AI call ID<br>    ai_call = AIMessage(<br>        content=content,<br>        ai_call_id=str(uuid.uuid4())<br>    )<br><br>    # Update the state with the evaluation response<br>    new_state = {<br>        &quot;messages&quot;: {<br>            &quot;evaluation&quot;: ai_call<br>        }<br>    }<br>    return new_state</pre><p>The evaluate_match function takes the state returned by the get_job_description agent and applies its own logic. The process is straightforward: it extracts the content of the CV and the job description from the state, constructs a detailed prompt, and uses it to evaluate the potential fit of the candidate for the job.</p><p>The logic involves two types of prompts: a <strong>system prompt</strong> that defines the evaluation task, and a <strong>human prompt</strong> that includes the content of the CV and job description, guiding the evaluation in a professional and detailed manner. The combined prompts are then passed to the LLM for evaluation.</p><p>Once the LLM generates a response, the result is wrapped in an AIMessage, which <strong>serves as a standardized way to pass AI-generated responses to the next agent</strong>. The AIMessage functions similarly to the ToolMessage, but it explicitly indicates that the content has been created by the AI.</p><p>Finally, let’s move on to the last agent, which will be responsible for generating a draft email with the selection decision based on the evaluation results.</p><pre># Generate Email<br>def generate_email(state: dict): # The state here will be the evaluate_match result -&gt; {&quot;messages&quot;: {&quot;evaluation&quot;: ___}}<br>    print(&quot;***** generate_email *****&quot;)<br><br>    # Extract the evaluation result from the state<br>    evaluation_result = state[&quot;messages&quot;][&quot;evaluation&quot;].content<br><br>    # Create the email prompts<br>    hum_email_template = &quot;&quot;&quot;Job Application Evaluation:<br>    {evaluation}<br>    ===<br>    Based on the evaluation of the match between the candidate&#39;s CV and the job description, draft an email to the candidate communicating the result of the assessment. Clearly indicate whether or not the candidate has been selected for the position and provide the reasons for the decision. Ensure the tone is polite, professional, and respectful, starting the email with a courteous acknowledgment.<br>    Email:<br>    &quot;&quot;&quot;<br>    hum_email_prompt = hum_email_template.format(evaluation=evaluation_result)<br>    sys_email_prompt = &quot;&quot;&quot;You are a hiring manager tasked with drafting an email to a candidate regarding the result of their job application assessment. Clearly communicate whether the candidate has been selected for the position, and provide reasons for the decision. Maintain a polite, professional, and respectful tone, starting the email with a courteous acknowledgment.&quot;&quot;&quot;<br><br>    # Invoke the LLM with the email prompts<br>    response = llm.invoke([SystemMessage(content=sys_email_prompt),<br>                           HumanMessage(content=hum_email_prompt)])<br>    content = response.content.strip()<br><br>    # Create an AIMessage object with the email content and a unique AI call ID<br>    ai_call = AIMessage(<br>        content=content,<br>        ai_call_id=str(uuid.uuid4())<br>    )<br><br>    # Update the state with the email content<br>    new_state = {<br>        &quot;messages&quot;: {<br>            &quot;email&quot;: ai_call<br>        }<br>    }<br><br>    return new_state</pre><p>The generate_email agent is the final step in the workflow and follows a similar structure to the evaluate_match agent. Its purpose is to take the evaluation result produced by the previous agent, craft a well-structured prompt, and generate a draft email communicating the selection decision to the candidate.</p><p>Once all the agents are defined, we <strong>move on to constructing the graph</strong>. This step involves specifying each node in the workflow, defining the connections between the nodes, and creating a visual representation of the process.</p><pre># Start the Graph with the initial state<br>workflow = StateGraph(State)<br><br># We add the nodes that corresponds to the functions we defined. We have to give them a name<br>workflow.add_node(&quot;read_cv&quot;, get_cv_content)<br>workflow.add_node(&quot;get_job_description&quot;, get_job_description)<br>workflow.add_node(&quot;evaluate_match&quot;, evaluate_match)<br>workflow.add_node(&quot;generate_email&quot;, generate_email)<br><br># We add the edges that connect the nodes.<br>## We start from the START and indicate the next node, that in this case is &quot;read_cv&quot;<br>workflow.add_edge(START, &quot;read_cv&quot;)<br>workflow.add_edge(&quot;read_cv&quot;, &quot;get_job_description&quot;)<br>workflow.add_edge(&quot;get_job_description&quot;, &quot;evaluate_match&quot;)<br>workflow.add_edge(&quot;evaluate_match&quot;, &quot;generate_email&quot;)<br>workflow.add_edge(&quot;generate_email&quot;, END)<br><br># We compile the graph<br>graph = workflow.compile()<br><br># We can visualize the graph<br>from IPython.display import Image, display<br>display(Image(graph.get_graph().draw_mermaid_png()))</pre><p>To complete the workflow, we construct a StateGraph based on the State class we defined at the beginning.</p><ul><li>For the <strong>nodes</strong>, we simply add the functions we have defined, assigning each function a name to represent its corresponding agent.</li><li>For the <strong>edges</strong>, we specify the connections between the agents. The first argument indicates the current agent, while the second argument points to the next one in the sequence.</li></ul><p>Finally, we compile the graph and visualize it. The resulting visualization should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/189/1*8OYSsc8Zco-8cdbdKpQtAQ.png" /></figure><p>Now we just need to run the graph starting with the start that we have already defined:</p><pre># Iterate over the graph<br>for output in graph.stream(start_state):<br>    print(&quot;Current Output:&quot;, output)<br><br>print(&quot;\n***** Process completed successfully!! *****\n&quot;)<br>print(output[&#39;generate_email&#39;][&#39;messages&#39;][&#39;email&#39;].content)</pre><p>This will display the output of each step in the workflow, and finally, it will present the result of the draft email.</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#aiagents #langchain #ai</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b74ee5390500" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Key metrics for evaluating a RAG system]]></title>
            <link>https://medium.com/@daniel.puenteviejo/key-metrics-for-evaluating-a-rag-system-f20b8bcae318?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f20b8bcae318</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[retrieval-augmented]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Thu, 25 Apr 2024 06:29:08 GMT</pubDate>
            <atom:updated>2024-04-25T06:29:08.258Z</atom:updated>
            <content:encoded><![CDATA[<h4>Metrics to evaluate RAG systems adapted to your specific use case</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4n2WjYkJ-ZbBIvhtQrASyA.png" /></figure><p>This article focuses on the key metrics for evaluating a Retrieval-Augmented Generation (RAG) system. To begin, it’s important to define what a RAG system entails.</p><h3>1. What is a RAG?</h3><p>RAG or Retrieval Augmented Generation, is a system that stores documents in a database, allowing users to pose questions about them. These inquiries can range from specific details, summaries, and comparisons to even generating new text based on the stored documents.</p><p>To illustrate this, consider the following use case: Improving the efficiency with which lawyers search for information. Typically, a RAG system is developed using a database of the company’s legal documents. In this way, instead of spending a long time searching for information in lengthy documents, lawyers can simply consult the RAG system, which presents the precise information they are looking for in a very short time.</p><p>RAG systems operate through <strong>two primary phases</strong>:</p><p>The initial phase involves <strong>ingesting and processing data into a vectorized database</strong>. This process begins with collecting files and partitioning them into smaller chunks. Subsequently, embeddings are generated for each chunk using various models. These embeddings, along with the text and any associated metadata, are then stored in the vectorized database. Although the overview of this process may seem straightforward, it involves numerous detailed tasks related to preprocessing and chunking to ensure the database is professionally structured. The effectiveness of a RAG system is significantly influenced by how well this initial step is executed.</p><p>Following the data ingestion, users have the capability to ask questions about the documents. At this stage, <strong>a search configuration is established,</strong> detailing the search algorithm, the maximum number of chunks to consider, or whether to incorporate metadata. All the relevant context identified during the search <strong>is then passed to a Large Language Model (LLM)</strong>, such as GPT, along with the user’s initial query. The LLM utilizes this information to generate a response to the user’s question. The functionality and workflow of this second phase can be illustrated through a specific diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*bMUzEQwxv6fBOvZVqLoBkA.gif" /></figure><p>The quality of the response generated by the RAG system hinges on several critical factors, particularly the following:</p><ul><li>Quality of Original Files and Preprocessing/Chunking</li><li>Search Strategy used</li><li>Large Language Model (LLM) used to ask the question</li></ul><h3><strong>2. Metrics</strong></h3><p>Improving the performance of the RAG system involves considering numerous factors. A straightforward yet effective method is to have a comprehensive and diverse database of questions related to the specific use case, coupled with a business team rating the quality of the answers generated by the system. However, <strong>this approach can be quite resource-intensive</strong>, as the business team would need to re-evaluate the responses for every single modification made to the system. Therefore, while this evaluation step is crucial, <strong>it’s necessary to find a quicker method to measure whether the system’s performance is improving or declining with each update</strong>. To facilitate this, two types of metrics are essential to monitor.</p><h4>2.1 Dataset</h4><p>Nonetheless, before starting with the explanation of the metrics is very important to have an evaluation database with, at least, the following features: (i) question, (ii) expected answer, (iii) files where the answer is found, and (iv) pages of the files where the answer is found. Example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fHHeN5QNUHVatVRaIINtYA.png" /></figure><p>Having a high-quality dataset, enriched with a variety of questions and sources verified by the business team, is crucial. <strong>The integrity and diversity of the dataset directly influence the reliability and accuracy of the outcomes.</strong></p><h4><strong>2.2 Search Metrics</strong></h4><p>A crucial step in accurately responding to queries involves identifying the relevant documents and pages that contain the answers. When the correct sources are identified, the likelihood of accurately answering the question increases significantly. To facilitate this, three key metrics have been developed:</p><ul><li><em>Docs Precision (Search)</em></li><li><em>Pages Precision (Search)</em></li><li><em>Positional Docs Precision (Search)</em></li></ul><p>The first two metrics are similar, with <em>Docs Precision</em> relating to the identification of entire documents, and <em>Page Precision</em> pertaining to specific pages within these documents. <strong><em>Page Precision</em> is particularly critical</strong> as it is directly related to obtaining accurate answers to queries. <em>Document Precision</em>, while also important, mainly indicates whether the relevant document has been located. These metrics essentially measure the proportion of relevant documents or pages identified, underscoring the importance of a thoroughly vetted dataset. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RVns6s6A4kQQdYOR4wDBYw.png" /></figure><p>For example, in the image above we can see that for a question asked, the 2 documents where the answer is found have been obtained. However, only 1 of the 3 pages has been located. In this situation, it can be interpreted that the model will probably not answer the question perfectly as it has not located all the relevant/necessary information.</p><p>The third metric, <em>Positional Docs Precision</em>, refines the first by considering the location of content within the documents. <strong>It highlights the importance of the position</strong> in which the document is found, as finding a document at the beginning of a search result is valued more highly than finding it towards the end. To account for this, a penalty factor greater than 1 is applied, in this instance, 1.1 is used. This leads to the selection of a vector with negative values, each 1.1 smaller than the previous one. Imagine searching 10 chunks and needing to locate 2 documents; discovering a document adds 0.5 (1/2) to the metric.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yjI2BZAcYVBDubP_kTPvhA.png" /></figure><p>As can be seen in the diagram, +0.5 is added if the file is found, and a penalty is added otherwise.<strong> The longer it takes for the documents to appear, the more penalties they will incur</strong>. The penalty vector will add -1, while each correct document found will add proportionally to the number of sources required to be found, ultimately adding up to 1. Therefore, the metric is in the range [-1, 1]. In the end, it is <strong>normalized </strong>to have a metric in [0, 1] range.</p><p>The idea behind this metric is to give another interpretation to the <em>Docs Precision</em> metric, which in some cases gives 100% accuracy but does not give full information on the quality of that search.</p><h4>2.3 Answer metrics</h4><p>Typically, locating the source greatly increases the chances of accurately answering the question. <strong>Nonetheless, this isn’t a guaranteed outcome, necessitating an alternative evaluation method</strong>. Moreover, employing more chunks will elevate the metrics, though this doesn’t always translate to better performance. Despite LLMs having the capacity to process extensive contexts, an overload of information can actually degrade response quality. Additionally, both the search process and response times are likely to lengthen with increased information.</p><p>Evaluating answer metrics is more challenging due to their non-deterministic nature, which doesn’t assure enhanced performance. We will introduce five metrics, and if a majority shows improvement over prior iterations, <strong>it suggests a likelihood of better outputs</strong>. However, this should always be confirmed with the business team. The metrics are as follows:</p><ul><li><em>Docs Prevision (Answer)</em></li><li><em>Pages Precision (Answer)</em></li><li><em>Cosine</em></li><li><em>Hallucination</em></li><li><em>Evaluation 0/5</em></li></ul><p>The operation of the first two metrics aligns with the previously described methodology, with a key difference being the use of answer references instead of all context segments. This requires providing a clear prompt to the LLM, <strong>instructing it to always cite the sources of its assertions in a specific</strong>, parseable format to minimize fabrications.</p><p>The <em>Cosine </em>similarity metric <strong>compares the embedding of the expected response</strong> with that of the LLM response. Although this metric does not always provide accurate information on its own, aggregating the average values of several instances can provide a rough estimate of the quality of the response.</p><p>Another metric assesses the presence of <em>hallucinations </em>in a binary form, which requires a very precise indication of the LLM. This involves <strong>presenting the context and the response generated</strong> by the LLM and specifying that if the response provided cannot be logically deduced from the given context, the metric must return a value of 1, regardless of the correctness of the response. This is because a correct answer that could not be deduced from the context provided is still considered a hallucination. The use of this strict criterion is intended to virtually eliminate hallucinations.</p><p>Let&#39;s remember that there are 2 kinds of errors. Examples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*5UpxQ1vWhTpJtMY0zqK1PQ.png" /></figure><p><strong>The first type of error is particularly perilous </strong>because a user unfamiliar with the subject might assume the incorrect answer is accurate. In contrast, the second type of error is easier to handle, as it becomes apparent that the model has misunderstood the query. By adopting this strategy, we aim to <strong>minimize the occurrence of the first.</strong></p><p>The last metric is straightforward: it involves<strong> requesting the LLM </strong>to rate the provided answer on a scale from <strong>0 to 5</strong>, based on the expected response and a set of predefined criteria for evaluation. Remember that the rules for giving one metric or another must be precisely specified and adapted to your use case.</p><h3>3. Conclusion</h3><p>In summary, this article outlines a series of metrics that can be applied to the RAG project. While these metrics do not always ensure improved performance, <strong>they provide insights into the model’s behavior</strong>, allowing us to assess, with a high degree of confidence, whether the model’s performance has enhanced compared to its previous iteration, without the need for the business team to review every single response.</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#deeplearning #machinelearning #llm</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f20b8bcae318" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Science of Control: How Temperature, Top_p, and Top_k Shape Large Language Models]]></title>
            <link>https://medium.com/@daniel.puenteviejo/the-science-of-control-how-temperature-top-p-and-top-k-shape-large-language-models-853cb0480dae?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/853cb0480dae</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[mathematics]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Wed, 01 Nov 2023 21:26:43 GMT</pubDate>
            <atom:updated>2023-11-01T21:26:43.051Z</atom:updated>
            <content:encoded><![CDATA[<h4>Practical example of how temperature, top_k, and top_p hyperparameters work in models such as ChatGPT or Llama2</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1l8lhwjb5utlG_Hf6pVAOw.png" /></figure><p>In the age of artificial intelligence, language models have become indispensable tools for a wide array of applications, from chatbots and automated content generation to machine translation and text completion. These large language models (LLMs) are incredibly powerful, but with great power comes the need for precise control. In the realm of text generation, understanding and mastering hyperparameters like<strong> <em>temperature, top_p, and top_k</em></strong> is the key to harnessing the full potential of these AI behemoths.</p><p>In this article, we embark on an exciting journey to demystify these three essential hyperparameters that govern <strong>text generation and creativity</strong> in large language models. We will delve deep into the intricacies of <em>temperature</em>, explore the subtleties of <em>top_p</em>, and understand the nuances of <em>top_k</em>. With each step, you’ll gain insights into how these settings work, how they affect the output, and how you can wield them to tailor your AI-generated text to your precise needs.</p><p>Let’s dive in!</p><h4><strong>1. Temperature</strong></h4><p><em>Temperature </em>is a crucial hyperparameter in fine-tuning the output of large language models (LLMs) like GPT-3. It plays a vital role in <strong>controlling the randomness and creativity</strong> of generated text. The output of these large language models works as a function of the probability of word appearance. In other words, to generate a word, a probability is associated with each and every word in the dictionary and, based on this, it is determined how to proceed. <strong>The main idea of this hyperparameter is to</strong> <strong>adjust these probabilities to force randomness or determinism</strong>.</p><p>The generation of probabilities for each word in the dictionary is done in the last layer by applying <em>softmax </em>as an activation function. Recall that <em>softmax </em>acts on the logits to transform them into probabilities. And this is precisely where the <em>temperature </em>comes into play.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Edq-TPxtZAlQUuQqZjotsw.png" /></figure><p>As can be seen from the above formula, <em>softmax </em>exponentiates each logit <em>(x) </em>and then, each exponentiated value is divided by the sum of all the exponentiated values. This step ensures that the output is a probability distribution, meaning that the values are between 0 and 1 and sum up to 1. The <em>temperature </em>hyperparameter is the value defined as <em>“T”</em> that is applied to each of the logits, making low temperatures skew the probabilities much more to the extremes.</p><p>Let’s imagine we have the following sentence:<br>- <em>Yesterday I went to the cinema to see a ___</em></p><p>The idea is to predict the next word. The neural network will determine a probability for each of the words in the dictionary, in this case, we will use only 5 to simplify the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QQuGJmcuhSAA0ZgQ-V9rJw.png" /></figure><p>From the probabilities generated, randomness will be responsible for determining the word that follows. As can be seen, in the example above, normal <em>softmax (temperature = 1)</em> has been applied. In the following, we will see the result in the case of extreme values of these hyperparameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6wcyF2nzRSHZwShUvJew2Q.png" /></figure><p>As <em>temperature </em>values <strong>approach 0 the higher probabilities increase further</strong>, making selection much more likely. Conversely, when the temperature gets much higher, the probabilities are <strong>softened, making more unexpected words more likely to be selected</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KQ1ymOtABgWVyVbq9M3yfw.png" /></figure><p>Therefore, when the <em>temperature </em>value is equal to 0, <strong>it becomes a deterministic solution</strong>. However, in case there are 2 words with the same logit value and hence the same probability, temperature 0 will make those words equally likely to be selected and add up to 1.</p><h4>2. Top_p &amp; Top_k</h4><p>These 2 hyperparameters have the same purpose as the <em>temperature </em>but execute it in a different way.</p><ol><li><strong>Top_p (Nucleus Sampling)</strong>: It selects the most likely tokens from a probability distribution, considering the <strong>cumulative probability until it reaches a predefined threshold <em>“p”.</em></strong> This limits the number of choices and helps avoid overly diverse or nonsensical outputs.</li><li><strong>Top_k (Top-k Sampling)</strong>: It restricts the selection of tokens to the<strong> “<em>k”</em> most likely options</strong>, based on their probabilities. This prevents the model from considering tokens with very low probabilities, making the output more focused and coherent.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tzwHi4z284hIUhXKyIWraA.png" /></figure><p>In the image above we can see how <em>top_p </em>works. You can see how it goes through a <strong>sorting process,</strong> followed by the <strong>accumulation of probabilities</strong>. With these, <strong>a threshold is established</strong> and those words are selected. Finally, the <strong>probabilities are recalculated. </strong>On the assumption that the <em>“p”</em> value is <strong>less than the largest probability</strong>, the most likely word is simply selected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aMmx1mmjdQhP0klBZ4q7SQ.png" /></figure><p><em>Top_k </em>works in the same way, but only considering the parameter <em>“k”</em> as the most probable words. As in the<em> top_p</em> process, the <strong>probabilities are sorted</strong>, but in this case, there is no need to accumulate. Once sorted, the <strong>threshold of <em>“k”</em></strong> is set and, finally, the <strong>probabilities are recalculated</strong>.</p><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#deeplearning #python #machinelearning #llm</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=853cb0480dae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning Image-to-Text algorithms with LORA]]></title>
            <link>https://medium.com/@daniel.puenteviejo/fine-tuning-image-to-text-algorithms-with-lora-deb22aa7da27?source=rss-22e16671d35e------2</link>
            <guid isPermaLink="false">https://medium.com/p/deb22aa7da27</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[lora]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[hugging-face]]></category>
            <dc:creator><![CDATA[Daniel Puente Viejo]]></dc:creator>
            <pubDate>Thu, 15 Jun 2023 07:11:19 GMT</pubDate>
            <atom:updated>2023-10-30T07:46:27.724Z</atom:updated>
            <content:encoded><![CDATA[<h4>Simple notebook for Fine-Tuning Image-to-Text conversion algorithms using LORA</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2dcGkkeM2fre2yBe.png" /></figure><p>The goal of this article is to cover a simple notebook example of how to apply <strong>LORA to Fine-Tune Image-to-Text</strong> algorithms. The notebook will be developed using <em>Hugging Face </em>and <em>Peft </em>libraries.</p><p>Let’s dive in!</p><h4>1. What is LORA?</h4><p>In the field of large language models, the challenge of fine-tuning has long perplexed researchers. Microsoft, however, has unveiled an innovative solution called <a href="https://arxiv.org/abs/2106.09685"><strong>Low-Rank Adaptation (LoRA)</strong></a>. With the emergence of behemoth models like GPT-3 boasting billions of parameters, the cost of fine-tuning them for specific tasks or domains has become exorbitant.</p><p>LoRA offers a groundbreaking approach by freezing the weights of pre-trained models and introducing <strong>trainable layers known as rank-decomposition matrices</strong> in each transformer block. This ingenious technique significantly reduces the number of trainable parameters and minimizes GPU memory requirements, as gradients no longer need to be computed for the majority of model weights.</p><h4>2. Libraries</h4><p>To start with the notebook, the libraries to be used are presented. For the LORA application, it is recommended to have <strong>GPU enabled</strong>. This is achieved with the last line of the code.</p><pre>import torch<br>from torch.utils.data import DataLoader, Dataset<br>from PIL import Image<br>from transformers import ViTFeatureExtractor, VisionEncoderDecoderModel, AutoProcessor<br><br>from datasets import load_dataset<br>from peft import LoraConfig, get_peft_model<br><br>device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)</pre><h4>3. Dataset and model</h4><p>The model to be used during this document is <a href="https://huggingface.co/ydshieh/vit-gpt2-coco-en"><strong>vit-gpt2-coco-en by ydshieh</strong></a>. It is a ViT model with fast operation and good results.</p><pre>loc = &quot;ydshieh/vit-gpt2-coco-en&quot;<br><br>feature_extractor = ViTFeatureExtractor.from_pretrained(loc)<br>model = VisionEncoderDecoderModel.from_pretrained(loc)<br>processor = AutoProcessor.from_pretrained(loc)<br><br>model = model.to(device)</pre><p>In contrast, the <a href="https://huggingface.co/datasets/ybelkada/football-dataset"><strong>ybelkada/football-dataset</strong></a> emerges as a valuable resource for experimentation. Despite its modest size (6 images), this dataset proves incredibly useful for conducting rapid tests and initially validating the model’s functionality.</p><p>It serves as a convenient tool to assess the model’s accuracy and ascertain its proper functioning with ease. While small in scale, the ybelkada/football-dataset offers an efficient means to gauge the model’s performance <strong>before delving into more extensive and intricate datasets.</strong></p><pre>dataset = load_dataset(&quot;ybelkada/football-dataset&quot;, split=&quot;train&quot;)</pre><h4>4. Adapt the Dataset</h4><p>The implementation of LORA relies on the <em>Peft </em>library. However, when confronted with Image-to-Text problems, <em>Peft</em> is not ideally suited for this specific task. Consequently, a <strong>series of adjustments must be implemented</strong> to ensure seamless compatibility. These modifications are necessary to enhance the framework’s effectiveness in handling the unique challenges posed by Image-to-Text problems.</p><pre>class ImageCaptioningDataset(Dataset):<br>    def __init__(self, dataset, processor, feature_extractor):<br>        self.dataset = dataset<br>        self.processor = processor<br>        self.feature_extractor = feature_extractor<br><br>    def __len__(self):<br>        return len(self.dataset)<br><br>    def __getitem__(self, idx):<br>        item = self.dataset[idx]<br>        encoding = self.feature_extractor(images=item[&quot;image&quot;], return_tensors=&quot;pt&quot;)<br>        encoding = {k: v.squeeze() for k, v in encoding.items()}<br>        encoding[&quot;text&quot;] = item[&quot;text&quot;]<br>        return encoding<br><br>train_dataset = ImageCaptioningDataset(dataset, processor, feature_extractor)</pre><p>In this implementation, we leverage the <em>“Dataset”</em> extension provided by the <strong>PyTorch </strong>library. By utilizing PyTorch Datasets, we adhere to a best practice for training models, benefiting from their efficiency, flexibility, and reusability. The process is straightforward: as we iterate through the dataset, we retrieve the relevant images, apply the<em> “feature_extractor” </em>to <strong>encode them and store the encoded features alongside the corresponding prompts in a dictionary.</strong></p><h4>5. Adapt the model</h4><p>The subsequent step in our process involves <strong>adapting our model for LORA fine-tuning</strong>. This crucial adjustment allows our model to effectively harness the power of LORA’s capabilities. For this, we need to instantiate the LORA configuration and <strong>determine the corresponding parameters</strong>. Subsequently, we list and describe each of them in order to adapt it to each use case.</p><p>Furthermore, a handy function has been incorporated that provides insights into <strong>the percentage of weights slated for modification</strong>. This feature offers transparency and a clear understanding of the extent to which the model’s parameters will be altered during the LORA fine-tuning process.</p><pre># Lora configuration<br>config = LoraConfig(<br>    r=16,<br>    lora_alpha=32,<br>    lora_dropout=0.05,<br>    bias=&quot;none&quot;,<br>    target_modules=[&quot;query&quot;,&quot;value&quot;],<br>)<br><br>model = get_peft_model(model, config)<br><br>## Trainable Params<br>def print_trainable_parameters(model):<br>    &quot;&quot;&quot;<br>    Prints the number of trainable parameters in the model.<br>    &quot;&quot;&quot;<br>    trainable_params, all_param = 0, 0<br>    for _, param in model.named_parameters():<br>        all_param += param.numel()<br>        if param.requires_grad: trainable_params += param.numel()<br>    print(f&quot;trainable params: {trainable_params} || all params: {all_param} || trainable %: {100 * trainable_params / all_param}&quot;)<br><br>print_trainable_parameters(model)<br><br>## ------------------------------------------------------------------------------------<br>## trainable params: 589824 || all params: 239785728 || trainable %: 0.2459796105963404<br>## ------------------------------------------------------------------------------------</pre><p>Next, some of the <strong>main hyperparameters</strong> that can be used within the LORA configuration are described.</p><ul><li><em>r (`int`): </em>Lora attention dimension.</li><li>target_modules (`Union[List[str],str]`): The names of the modules to apply Lora to.</li><li><em>lora_alpha (`int`): </em>The alpha parameter for Lora scaling.</li><li><em>lora_dropout (`float`):</em> The dropout probability for Lora layers.</li><li><em>fan_in_fan_out (`bool`):</em> Set this to True if the layer to replace stores weight like (fan_in, fan_out).</li><li><em>bias (`str`): </em>Bias type for Lora. Can be ‘none’, ‘all’ or ‘lora_only’ modules_to_save (`List[str]`): List of modules apart from LoRA layers to be set as trainable and saved in the final checkpoint.</li><li><em>layers_to_transform (`Union[List[int],int]`):</em> The layer indexes to transform, if this argument is specified, it will apply the LoRA transformations on the layer indexes that are specified in this list. If a single integer is passed, it will apply the LoRA transformations on the layer at this index.</li><li><em>layers_pattern (`str`):</em> The layer pattern name, used only if `layers_to_transform` is different from `None` and if the layer pattern is not in the common layers pattern.</li></ul><h4>6. Dataloader</h4><p>Before starting the training, it is necessary to make a few small adjustments. First, the <em>“collect”</em> function allows us to adapt the previously adapted data set for training. Here you set parameters such as <em>batch_size </em>or <em>shuffle </em>among others. The function includes things such as <strong>tokenized text or attention masks. </strong>With the <em>“processor” </em>the prompts are converted to their <strong>tokenized version.</strong></p><pre>def collator(batch):<br>    processed_batch = {}<br>    for key in batch[0].keys():<br>        if key != &quot;text&quot;:<br>            processed_batch[key] = torch.stack([example[key] for example in batch])<br>        else:<br>            text_inputs = processor([example[&quot;text&quot;] for example in batch], padding=True, return_tensors=&quot;pt&quot;)<br>            processed_batch[&quot;input_ids&quot;] = text_inputs[&quot;input_ids&quot;]<br>            processed_batch[&quot;attention_mask&quot;] = text_inputs[&quot;attention_mask&quot;]<br>    return processed_batch<br><br>train_dataloader = DataLoader(train_dataset, shuffle=True, batch_size=3, collate_fn=collator)<br>optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5)<br>epochs = 100</pre><p>In this case, a reduced <em>batch_size </em>has been applied due to the small number of images. The selected optimizer is <em>AdamW </em>with a total of <em>100 epochs.</em> <strong>The training should take about 1 minute</strong>.</p><h4>7. Train</h4><p>Lastly, the model undergoes training, recovering the loss at the end of each epoch. By leveraging this training methodology, we can <strong>effectively monitor and optimize the model’s progress</strong>, ultimately leading to enhanced performance and more accurate predictions.</p><pre>loss_list = []<br>model.train()<br>for epoch in range(1, epochs+1):<br>    for idx, batch in enumerate(train_dataloader):<br>        input_ids = batch.pop(&quot;input_ids&quot;).to(device)<br>        pixel_values = batch.pop(&quot;pixel_values&quot;).to(device, torch.float16)<br><br>        outputs = model(pixel_values=pixel_values, labels=input_ids)<br>        loss = outputs.loss<br>        <br>        loss.backward()<br>        optimizer.step()<br>        optimizer.zero_grad()<br><br>    if epoch % 10 == 0: print(f&quot;Epoch {epoch} done!, Loss: {loss.item()}&quot;)<br>    loss_list.append(loss.item())</pre><p>With this last piece of code, the notebook is finished. It is important to remember that the code should be tailored to suit each specific use case. Additionally, include validation and test files to ensure the correct functioning of the model.</p><p>It is a simple notebook that can be used in many contexts. If you want to learn in more detail how to apply LORA to other algorithms, don’t hesitate to visit the <a href="https://github.com/huggingface/peft"><strong>Peft repository</strong></a><strong>.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*bRwXOMxHGQ3OfV6cQf0WRg.png" /></figure><h3>Thanks for Reading!</h3><p>Thank you very much for reading the article. If you liked it, don’t hesitate to follow me on <a href="https://www.linkedin.com/in/danielpuenteviejo/"><strong>Linkedin</strong></a>.</p><p>#deeplearning #python #machinelearning #huggingface</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=deb22aa7da27" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>