<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hafiz Hassan Mahmood  on Medium]]></title>
        <description><![CDATA[Stories by Hafiz Hassan Mahmood  on Medium]]></description>
        <link>https://medium.com/@hassan.mahmood1?source=rss-87c3bef8b57a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*V0BX9BdH7sjTiMEf</url>
            <title>Stories by Hafiz Hassan Mahmood  on Medium</title>
            <link>https://medium.com/@hassan.mahmood1?source=rss-87c3bef8b57a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 23:28:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hassan.mahmood1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Running Background Tasks in FastAPI Using Celery and Redis (A Practical Guide)]]></title>
            <link>https://medium.com/@hassan.mahmood1/running-background-tasks-in-fastapi-using-celery-and-redis-a-practical-guide-813545cf81f8?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/813545cf81f8</guid>
            <category><![CDATA[background-task]]></category>
            <category><![CDATA[fastapi]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[celery]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Tue, 17 Feb 2026 19:03:05 GMT</pubDate>
            <atom:updated>2026-02-17T19:03:05.623Z</atom:updated>
            <content:encoded><![CDATA[<p>Modern backend applications often need to perform <strong>time-consuming tasks</strong> such as sending emails, processing data, or running calculations. If these tasks are executed directly inside an API request, the user experience suffers and servers become slow.</p><p>In this article, we’ll solve this problem step by step using <strong>FastAPI, Celery, and Redis</strong>, and we’ll also cover <strong>real development challenges</strong> and <strong>production deployment on Ubuntu servers</strong>.</p><h3><strong>The Problem:</strong></h3><p>Suppose you have an API that performs a heavy operation:</p><ul><li>Long calculations</li><li>External API calls</li><li>Background processing</li></ul><p>If this logic runs directly inside the API:</p><ul><li>The request blocks</li><li>Response is slow</li><li>Timeouts may occur</li><li>Scaling becomes difficult</li></ul><p><strong>Solution:</strong> Move heavy work to a <strong>background worker</strong>.</p><h3><strong>The Solution: FastAPI + Celery + Redis</strong></h3><p>We use:</p><ul><li><strong>FastAPI</strong> → Handles HTTP requests</li><li><strong>Celery</strong> → Runs background tasks</li><li><strong>Redis</strong> → Message broker + result store</li></ul><p><strong>Project Structure</strong></p><pre>project/<br>├── main.py<br>├── celery_app.py<br>├── utils.py<br>├── requirements.txt</pre><p><strong>requirements.txt</strong></p><pre>fastapi<br>uvicorn<br>celery<br>redis</pre><p><strong>Celery Configuration (</strong><strong>celery_app.py)</strong></p><pre>from celery import Celery<br><br>celery_app = Celery(<br>    &quot;worker&quot;,<br>    broker=&quot;redis://localhost:6379/0&quot;,<br>    backend=&quot;redis://localhost:6379/0&quot;,<br>)<br><br>celery_app.conf.update(<br>    task_serializer=&quot;json&quot;,<br>    result_serializer=&quot;json&quot;,<br>    accept_content=[&quot;json&quot;],<br>    timezone=&quot;UTC&quot;,<br>    enable_utc=True,<br>)<br><br># IMPORTANT: import tasks so worker can register them<br>import utils</pre><p><strong>Business Logic &amp; Task (</strong><strong>utils.py)</strong></p><pre>import time<br>from celery_app import celery_app<br><br>class MathService:<br>    @staticmethod<br>    def add(a: int, b: int) -&gt; int:<br>        time.sleep(3)  # simulate heavy work<br>        return a + b<br><br>@celery_app.task(name=&quot;add_numbers_task&quot;)<br>def add_numbers_task(a: int, b: int) -&gt; int:<br>    return MathService.add(a, b)</pre><p><strong>FastAPI Application (</strong><strong>main.py)</strong></p><pre>from fastapi import FastAPI<br>from celery.result import AsyncResult<br>from celery_app import celery_app<br>from utils import add_numbers_task<br>import uvicorn<br>app = FastAPI(title=&quot;FastAPI + Celery Demo&quot;)<br><br><br>@app.post(&quot;/add&quot;)<br>async def add_numbers(a: int, b: int):<br>    &quot;&quot;&quot;<br>    Sends addition task to Celery<br>    &quot;&quot;&quot;<br>    task = add_numbers_task.delay(a, b)<br><br>    return {<br>        &quot;message&quot;: &quot;Task sent to Celery&quot;,<br>        &quot;task_id&quot;: task.id<br>    }<br><br><br>@app.get(&quot;/result/{task_id}&quot;)<br>async def get_result(task_id: str):<br>    &quot;&quot;&quot;<br>    Check task result from Redis<br>    &quot;&quot;&quot;<br>    task_result = AsyncResult(task_id, app=celery_app)<br><br>    if task_result.state == &quot;PENDING&quot;:<br>        return {&quot;status&quot;: &quot;PENDING&quot;}<br><br>    if task_result.state == &quot;SUCCESS&quot;:<br>        return {<br>            &quot;status&quot;: &quot;SUCCESS&quot;,<br>            &quot;result&quot;: task_result.result<br>        }<br><br>    if task_result.state == &quot;FAILURE&quot;:<br>        return {<br>            &quot;status&quot;: &quot;FAILURE&quot;,<br>            &quot;error&quot;: str(task_result.result)<br>        }<br><br>    return {&quot;status&quot;: task_result.state}<br><br><br>if __name__ == &quot;__main__&quot;:<br>    uvicorn.run(app, host=&quot;127.0.0.1&quot;, port=5245)</pre><p><strong>Running the Project (Development)</strong></p><pre>redis-server<br><br>uvicorn main:app --port 5245 --reload<br><br>#for windows<br>celery -A celery_app.celery_app worker --loglevel=info --pool=solo --concurrency=1 --hostname=worker1<br><br>#for Ubuntu/linux<br>celery -A celery_app.celery_app worker --loglevel=info --pool=prefork --concurrency=4 --hostname=worker1</pre><h3><strong>Production Setup (Ubuntu Server):</strong></h3><p>In production, <strong>you should never run Celery or FastAPI manually</strong>.</p><p>Instead, use <strong>systemd services</strong>.</p><h3>Celery systemd Service</h3><p>Create service file</p><pre>sudo nano /etc/systemd/system/celery.service</pre><pre>[Unit]<br>Description=Celery Worker<br>After=network.target redis.service<br><br>[Service]<br>User=ubuntu<br>WorkingDirectory=/var/www/project<br>Environment=&quot;PATH=/var/www/project/venv/bin&quot;<br>ExecStart=/var/www/project/venv/bin/celery \<br>  -A celery_app.celery_app worker \<br>  --loglevel=info \<br>  --pool=prefork \<br>  --concurrency=4 \<br>  --hostname=worker1<br>Restart=always<br><br>[Install]<br>WantedBy=multi-user.target</pre><pre>sudo systemctl daemon-reload<br>sudo systemctl enable celery<br>sudo systemctl start celery</pre><p><strong>Final Words:</strong></p><p>Setting up Celery for the first time can feel overwhelming — multiple processes, Redis, workers, and different behaviors across operating systems. But once everything clicks, the architecture becomes simple, powerful, and highly scalable.</p><p>By following this approach:</p><ul><li>You moved heavy logic out of API requests</li><li>You learned how background workers actually work</li><li>You handled real-world development issues (Windows vs Linux)</li><li>You prepared your application for production on Ubuntu servers</li></ul><p><strong>By doing this, you’ve successfully built and run your very first project using Celery.</strong></p><p>From here, you can confidently extend this setup with retries, scheduling, queues, monitoring, or containerization — all using the same foundation you’ve already mastered.</p><h3>For contact:</h3><p>Gmail: hassan.mahmood8605@gmail.com</p><p>linkedin: <a href="http://www.linkedin.com/in/hassan-mahmood-500329256">www.linkedin.com/in/hassan-mahmood-500329256</a></p><p>#FASTAPI #REDIS #CELERY #AWS #QUEUE</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=813545cf81f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Set Up CI/CD for a FastAPI App Using GitHub Actions, EC2, and PM2]]></title>
            <link>https://medium.com/@hassan.mahmood1/how-to-set-up-ci-cd-for-a-fastapi-app-using-github-actions-ec2-and-pm2-bd5463b134f8?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd5463b134f8</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Mon, 19 Jan 2026 11:30:32 GMT</pubDate>
            <atom:updated>2026-01-19T11:30:32.529Z</atom:updated>
            <content:encoded><![CDATA[<p>Deploying backend applications manually is slow, error-prone, and stressful. A proper CI/CD pipeline removes that pain by making deployments <strong>automatic, repeatable, and safe</strong>.</p><p>In this article, we’ll walk through a <strong>complete, real-world CI/CD setup</strong> for a FastAPI application using:</p><ul><li>GitHub Actions</li><li>AWS EC2 (Ubuntu)</li><li>PM2 as the process manager</li><li>Python virtual environments</li></ul><p>The goal is simple:</p><blockquote><strong><em>Push code → server updates automatically</em></strong></blockquote><p>No manual SSH.<br> No manual restarts.<br> No guesswork.</p><h3>What This Setup Does</h3><p>Once configured, this pipeline will:</p><ol><li>Watch a deployment branch (auto-deployment)</li><li>Trigger GitHub Actions on every push</li><li>Securely connect to the EC2 server via SSH</li><li>Pull the latest code</li><li>Activate the Python virtual environment</li><li>Install dependencies</li><li>Restart the running FastAPI app using PM2</li><li>Save deployment logs for auditing and debugging</li></ol><h3>Prerequisites</h3><p>Before starting, make sure you already have:</p><ul><li>An Ubuntu EC2 instance</li><li>SSH access using a .pem file</li><li>A GitHub repository</li><li>A FastAPI app already running on the server</li><li>PM2 managing the FastAPI process</li><li>A Python virtual environment already created on the server</li></ul><p>This guide focuses on <strong>deployment automation</strong>, not app creation.</p><p><strong>Step 1: Prepare Git on the Server</strong></p><h3>SSH into your EC2 instance</h3><pre>ssh -i your-key.pem ubuntu@YOUR_EC2_HOST</pre><h3>Navigate to your application directory</h3><pre>cd /home/ubuntu/app/project</pre><h3>Initialize Git (if not already)</h3><pre>git init</pre><h3>Add your GitHub repository (HTTPS)</h3><pre>git remote add origin https://github.com/YOUR_USERNAME/YOUR_REPO.git</pre><p>Verify:</p><pre>git remote -v</pre><p><strong>Step 2: Store GitHub Credentials on the Server</strong></p><p>When GitHub Actions runs, it cannot enter usernames or passwords interactively.<br> So we store credentials <strong>once</strong> on the server.</p><h3>Enable credential storage</h3><pre>git config --global credential.helper store</pre><h3>Authenticate once using a GitHub token</h3><pre>git pull origin auto-deployment</pre><p>You’ll be prompted for:</p><ul><li><strong>Username</strong> → your GitHub username</li><li><strong>Password</strong> → your <strong>GitHub Personal Access Token</strong> (not your password)</li></ul><p>After this step, Git stores the token locally.</p><h3>Verify credentials</h3><pre>cat ~/.git-credentials</pre><p>You should see your token saved securely.</p><p><strong>Step 3: Confirm Virtual Environment Path</strong></p><p>Find the correct virtual environment path:</p><pre>find /home/ubuntu -type f -name activate | grep bin</pre><p>Example result:</p><pre>/home/ubuntu/your_env/bin/activate</pre><p>This path will be used in the deployment script.</p><p><strong>Step 4: Confirm PM2 Process</strong></p><pre>pm2 list</pre><p>We’ll use the <strong>PM2 process ID</strong> for safe restarts.</p><p><strong>Step 5: Add GitHub Secrets</strong></p><p>In your GitHub repository:</p><p><strong>Settings → Secrets and variables → Actions</strong></p><p>Add these secrets:</p><p>EC2_SSH_KEY, EC2_HOST, EC2_USER</p><p>These secrets are encrypted and never exposed in logs.</p><p><strong>Step 6: Required Repository Setup</strong></p><p>Your repository must contain:</p><pre>.github/<br>  workflows/<br>    deploy.yml</pre><p><strong>Step 7: </strong>Final Deployment Workflow</p><pre>.github/workflows/deploy.yml</pre><h3>Complete Deployment Workflow</h3><pre>name: Deploy to EC2<br><br>on:<br>  push:<br>    branches:<br>      - auto-deployment<br><br>jobs:<br>  deploy:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v3<br><br>      - name: Deploy to EC2<br>        env:<br>          EC2_SSH_KEY: ${{ secrets.EC2_SSH_KEY }}<br>          EC2_HOST: ${{ secrets.EC2_HOST }}<br>          EC2_USER: ${{ secrets.EC2_USER }}<br>        run: |<br>          mkdir -p ~/.ssh<br>          echo &quot;$EC2_SSH_KEY&quot; &gt; ~/.ssh/key.pem<br>          chmod 600 ~/.ssh/key.pem<br>          ssh-keyscan -H $EC2_HOST &gt;&gt; ~/.ssh/known_hosts<br><br>          ssh -i ~/.ssh/key.pem $EC2_USER@$EC2_HOST &lt;&lt; &#39;EOF&#39;<br>          set -e<br><br>          APP_DIR=&quot;/home/ubuntu/app/project&quot;<br>          VENV_PATH=&quot;/home/ubuntu/your_env/bin/activate&quot;<br>          LOG_DIR=&quot;$APP_DIR/deployment_logs&quot;<br>          LOG_FILE=&quot;$LOG_DIR/deployment_$(date &#39;+%Y%m%d_%H%M%S&#39;).log&quot;<br><br>          mkdir -p &quot;$LOG_DIR&quot;<br>          exec &gt; &gt;(tee -a &quot;$LOG_FILE&quot;) 2&gt;&amp;1<br><br>          cd &quot;$APP_DIR&quot;<br><br>          source &quot;$VENV_PATH&quot;<br>          git pull origin auto-deployment<br>          pip install -r requirements.txt<br>          pm2 restart 0<br><br>          cd &quot;$LOG_DIR&quot;<br>          ls -t deployment_*.log | tail -n +31 | xargs -r rm<br>          EOF</pre><p><strong>Step 8: Deployment Logs</strong></p><p>Each deployment creates a timestamped log file.</p><pre>tail -f /home/ubuntu/app/project/deployment_logs/deployment_*.log</pre><p>From now on, deployment is simple:</p><pre>Push code → deployment happens automatically</pre><h3>For contact:</h3><p>Gmail: hassan.mahmood8605@gmail.com</p><p>linkedin: <a href="http://www.linkedin.com/in/hassan-mahmood-500329256">www.linkedin.com/in/hassan-mahmood-500329256</a></p><p>#AWS #FastAPI #Python #ServerSetup #APIDeployment #Nginx #DevOps #GithubActions</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd5463b134f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying FastAPI on AWS EC2: A Step-by-Step Guide]]></title>
            <link>https://medium.com/@hassan.mahmood1/deploying-fastapi-on-aws-ec2-a-step-by-step-guide-f3637a545d12?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3637a545d12</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Thu, 31 Oct 2024 07:34:35 GMT</pubDate>
            <atom:updated>2024-10-31T07:34:35.392Z</atom:updated>
            <content:encoded><![CDATA[<h3>Prerequisite: Setting Up an EC2 Instance</h3><p>Before deploying FastAPI, set up your EC2 instance:</p><ol><li><strong>Login to AWS Console</strong>: Go to the EC2 dashboard.</li><li><strong>Launch a New Instance</strong>: Select an Ubuntu Amazon Machine Image (AMI).</li><li><strong>Choose Instance Type</strong>: Select an instance type based on your app’s needs.</li><li><strong>Security Group Configuration</strong>: Open ports 22 (SSH), 80 (HTTP), and 443 (HTTPS).</li><li><strong>Download Key Pair</strong>: Download the .pem file; it’s necessary for connecting to the server.</li></ol><p>Once your instance is running, you can proceed.</p><h3>1. Establish a Connection to the EC2 Server</h3><p>There are multiple ways to connect to your EC2 server:</p><p><strong>a) Using a .pem file</strong>: Place the .pem file in a directory,open the Git Bash (Windows) or Terminal (Ubuntu), and connect with:</p><pre>ssh -i file.pem username@ip-address</pre><p><strong>b) Using SSH and Password</strong>: Open Git Bash (Windows) or Terminal (Ubuntu) and run:</p><pre>ssh username@ip-address</pre><p><strong>c)</strong> <strong>Using SSH Only</strong>: If you have an id_rsa key in C:\Users\username\.ssh\, connect with:</p><pre>ssh root@ip-address</pre><h3>2. Install Required Packages</h3><p>After connecting, update the server and install essential packages for FastAPI and server setup:</p><pre>sudo apt-get update <br>sudo apt-get upgrade<br>sudo apt-get -y install python3-pip python3-venv nginx nodejs npm<br>sudo npm install -g pm2</pre><h3>3. Create a Directory for the Application</h3><p>Organize your files by creating a directory for your FastAPI app:</p><pre>cd /home/ubuntu<br>mkdir app<br>cd app<br>mkdir code_folder</pre><h3>4. Upload or Pull Code to the Server</h3><p>To transfer your code onto the server, use either:</p><p><strong>a) GitHub</strong>: Clone your repository directly on the server.</p><pre>git clone https://github.com/yourusername/yourrepo.git /home/ubuntu/app/code_folder</pre><p><strong>b)</strong> <strong>Manual Upload</strong>: Use tools like <strong>FileZilla</strong> or <strong>MobaXterm</strong> with the .pem file, server IP, and username to upload files to /home/ubuntu/app/code_folder.</p><h3>5. Set Up the Virtual Environment</h3><p>Using a virtual environment is recommended for dependency management:</p><pre>cd /home/ubuntu<br>python3 -m venv project_name_env<br>source project_name_env/bin/activate<br>cd app/code_folder<br>pip install -r requirements.txt<br>pip install uvicorn gunicorn</pre><h3>6. Verify the Application</h3><p>Run your app to ensure it’s functioning correctly before configuring the server:</p><pre>python main.py</pre><h3>7. Configure Nginx as a Reverse Proxy</h3><p>Set up Nginx to route requests to your FastAPI app:</p><ol><li>Create a configuration file:</li></ol><pre>sudo nano /etc/nginx/sites-available/yourdomain.conf</pre><p>2. Add the following:</p><pre>server {<br>    listen 80;<br>    server_name yourdomain.com/IP;<br><br>    location / {<br>        proxy_pass http://localhost:port_no;<br>    }<br>}</pre><p>3. Save the file (Ctrl+O, Enter, then Ctrl+X).</p><h3><strong>8. Link and Restart Nginx</strong></h3><p>Link the configuration file and reload Nginx:</p><pre>sudo ln -s /etc/nginx/sites-available/yourdomain.conf /etc/nginx/sites-enabled<br>sudo nginx -t<br>sudo systemctl reload nginx</pre><h3>9. Process Management with PM2</h3><p>— PM2 is a process manager that keeps your app running:<br> — Before starting the app with PM2, <strong>ensure you’re in the code directory</strong> where your FastAPI application is located:</p><pre>cd /home/ubuntu/app/code_folder<br>pm2 start &quot;gunicorn -w 2-k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 main:app --timeout 3600&quot; --name project_name<br>pm2 save</pre><p>PM2 will keep your app running in the background and manage the process.</p><h3>10. PM2 Management Commands</h3><ul><li><strong>Stop:</strong> pm2 stop project_name</li><li><strong>Restart:</strong> pm2 restart project_name</li><li><strong>View Logs:</strong> pm2 logs project_name</li></ul><h3>11. Optional Server Timeout Settings</h3><p>For new instances, add timeout settings in nginx.conf:</p><pre>sudo nano /etc/nginx/nginx.conf</pre><pre># Client Timeout Settings<br>client_header_timeout 3600s;<br>client_body_timeout 3600s;<br>keepalive_timeout 3600s;<br><br># Proxy Timeout Settings<br>proxy_connect_timeout 3600s;<br>proxy_send_timeout 3600s;<br>proxy_read_timeout 3600s;</pre><p>After updating, save and reload Nginx for the changes to take effect:</p><pre>sudo systemctl reload nginx</pre><h3>12. SSL with Certbot (Optional)</h3><p>For a secure HTTPS connection, install and configure Certbot:</p><pre>sudo apt install certbot python3-certbot-nginx<br>sudo certbot --nginx -d yourdomain.com</pre><p>By following these steps, you can successfully deploy a FastAPI application on AWS EC2. This setup provides a secure, scalable solution ideal for production environments.</p><h3>For contact:</h3><p>Gmail: hassan.mahmood8605@gmail.com</p><p>linkedin: <a href="http://www.linkedin.com/in/hassan-mahmood-500329256">www.linkedin.com/in/hassan-mahmood-500329256</a></p><p>#AWS #FastAPI #Python #ServerSetup #APIDeployment #Nginx #DevOps</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3637a545d12" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangGraph: The Key to Creating Highly Personalized AI Assistants Through Intelligent Query Routing]]></title>
            <link>https://medium.com/@hassan.mahmood1/langgraph-the-key-to-creating-highly-personalized-ai-assistants-through-intelligent-query-routing-f09b2793ac08?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f09b2793ac08</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Mon, 02 Sep 2024 13:00:33 GMT</pubDate>
            <atom:updated>2024-09-02T13:00:33.136Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Introduction:</strong></p><p>Imagine a world where your personal AI assistant understands you on a deeper level, tailoring its responses to your unique preferences and experiences. This is where LangGraph comes into play. In this article, we’ll explore how LangGraph can be used to create a highly personalized AI assistant that can efficiently route queries to the appropriate vector store based on their relevance.</p><p><strong>The Challenge:</strong></p><p>Traditional AI assistants often struggle to give personalized answers, relying on standard templates or rules that lack detail and understanding. To improve this, we need a system that can adjust to each user’s specific needs.</p><p><strong>The Solution: LangGraph</strong></p><p>LangGraph offers a powerful solution to this problem. By representing the AI assistant’s logic as a graph, we can create a flexible and customizable system. Each node in the graph represents a different component of the assistant’s functionality, such as processing input, retrieving information from vector stores, and generating responses.</p><p><strong>The Process:</strong></p><ul><li><strong>Query Reception:</strong> The assistant receives a user query as input.</li><li><strong>Query Analysis:</strong> LangGraph analyzes the query to determine its intent and context.</li><li><strong>Matching to Vector Store:</strong> Based on the analysis, the assistant attempts to match the query to the domain expertise of a specific vector store.</li><li><strong>Query Routing:</strong> If a suitable match is found, the query is routed directly to that vector store.</li><li><strong>No Match Found:</strong> If no vector store matches the query’s domain, the assistant end the graph with “I couldn’t find information related to that. Would you like to try a different query?”</li><li><strong>Information Retrieval:</strong> The vector store searches for relevant information based on the query.</li><li><strong>Response Generation:</strong> The assistant uses the retrieved information to generate a personalized response.</li></ul><p><strong>Code Integration:</strong></p><pre>import os<br>from dotenv import load_dotenv<br>from pinecone import Pinecone<br>from langchain_pinecone import PineconeVectorStore<br>from langchain.embeddings import HuggingFaceEmbeddings<br>from typing import Literal<br>from langchain_core.prompts import ChatPromptTemplate<br>from langchain_core.pydantic_v1 import BaseModel, Field<br>from langchain_openai import ChatOpenAI<br>from langgraph.graph import END, StateGraph, START<br>from typing import List<br>from typing_extensions import TypedDict<br>os.environ[&quot;OPENAI_API_KEY&quot;] = &quot;your openai_api_key&quot;<br>pinecone_api_key = &quot;your pinecone_api_key&quot;<br>pc = Pinecone(api_key=pinecone_api_key)<br>os.environ[&quot;PINECONE_API_KEY&quot;] = pinecone_api_key<br>load_dotenv()</pre><pre>#you can use openAI embedding model<br>embed_model = &quot;BAAI/bge-small-en-v1.5&quot;<br>embed_fn = HuggingFaceEmbeddings(model_name=embed_model)</pre><pre>class GraphState(TypedDict):<br>    &quot;&quot;&quot;<br>    Represents the state of our graph.<br><br>    Attributes:<br>        question: question<br>        documents: list of documents<br>    &quot;&quot;&quot;<br><br>    question: str<br>    documents: List[str]<br>    generation:str<br><br><br>class RouteQuery(BaseModel):<br>    &quot;&quot;&quot;Route a user query to the datasource.if user query is not related to datasource return &#39;end&#39;.&quot;&quot;&quot;<br><br>    datasource: Literal[&quot;hr_vectorstore&quot;,&quot;end&quot;] = Field(<br>        ...,<br>    )</pre><pre>def route_question(state):<br>    &quot;&quot;&quot;<br>    Route question to any these datasource, hr_vectorstore.if question is not related to datasource return &#39;end&#39;<br><br>    Args:<br>        state (dict): The current graph state<br><br>    Returns:<br>        str: Next node to call<br>    &quot;&quot;&quot;<br><br>    print(&quot;---ROUTE QUESTION---&quot;)<br>    question = state[&quot;question&quot;]<br>    llm = ChatOpenAI(model=&quot;gpt-4o-mini&quot;, temperature=0)<br>    structured_llm_router = llm.with_structured_output(RouteQuery)<br>    system = &quot;&quot;&quot;You are an expert at routing a user question to a hr_vectorstore or end.if user question is not related to any datasource return &#39;end&#39;<br>    The &quot;hr_vectorstore&quot; contains documents related to hr.<br>    if user query is not related to any datasource return &#39;end&#39;&quot;&quot;&quot;<br>    route_prompt = ChatPromptTemplate.from_messages(<br>        [<br>            (&quot;system&quot;, system),<br>            (&quot;human&quot;, &quot;{question}&quot;),<br>        ]<br>    )<br>    question_router = route_prompt | structured_llm_router<br>    source = question_router.invoke({&quot;question&quot;: question})<br>    if source.datasource == &quot;hr_vectorstore&quot;:<br>        print(&quot;---ROUTE QUESTION TO hr_vectorstore---&quot;)<br>        return &quot;hr_vectorstore&quot;<br>    elif source.datasource == &quot;end&quot;:<br>        print(&quot;---NO RELATED DATASOURCE FOUND---&quot;)<br>        print(f&quot;working {source.datasource}&quot;)<br>        return &quot;end&quot;</pre><pre>def hr_vectorstore(state):<br>    &quot;&quot;&quot;<br>    Retrieve documents<br><br>    Args:<br>        state (dict): The current graph state<br><br>    Returns:<br>        state (dict): New key added to state, documents, that contains retrieved documents<br>    &quot;&quot;&quot;<br>    print(&quot;---RETRIEVE---&quot;)<br>    question = state[&quot;question&quot;]<br><br>    # Retrieval<br>    hr_index=&quot;hr&quot;<br>    num_chunks=5<br>    vecdb = PineconeVectorStore.from_existing_index(index_name=hr_index,<br>                                                        embedding=embed_fn)<br>    docs = vecdb.similarity_search(question,k=num_chunks)<br>    return {&quot;documents&quot;: docs}</pre><pre>def generate(state):<br>    &quot;&quot;&quot;<br>    Generate answer<br><br>    Args:<br>        state (dict): The current graph state<br><br>    Returns:<br>        state (dict): New key added to state, generation, that contains LLM generation<br>    &quot;&quot;&quot;<br>    print(&quot;---GENERATE---&quot;)<br>    question = state[&quot;question&quot;]<br>    documents = state[&quot;documents&quot;]<br>    from operator import itemgetter<br>    prompt_str=&quot;&quot;&quot;    <br><br>    response the answer thruthfully as possible.<br>    question:{question}<br>    context:{context}<br><br>            &quot;&quot;&quot;<br>    _prompt = ChatPromptTemplate.from_template(prompt_str)<br>    chat_llm = ChatOpenAI(model=&quot;gpt-3.5-turbo&quot;, temperature=0)<br>    setup={&quot;question&quot;:itemgetter(&quot;question&quot;),&quot;context&quot;:itemgetter(&quot;context&quot;)}<br>    _chain = (setup |_prompt | chat_llm)<br>    # RAG generation<br>    generation = _chain.invoke({&quot;context&quot;: documents, &quot;question&quot;: question}).content<br>    return {&quot;generation&quot;: generation}</pre><pre>def end(state):<br>    return &quot;end&quot;<br><br><br>def end_state(state):<br>    return {&quot;generation&quot;:&quot;I couldn&#39;t find information related to that. Would you like to try a different query?&quot;}</pre><pre>#this code is use to compile a graph.<br><br>workflow = StateGraph(GraphState)<br><br>workflow.add_node(&quot;hr_vectorstore&quot;, hr_vectorstore)<br>workflow.add_node(&quot;generate&quot;,generate)<br>workflow.add_node(&quot;hard_end&quot;, end_state)<br><br><br>workflow.add_conditional_edges(<br>    START,<br>    route_question,<br>    {<br>        &quot;hr_vectorstore&quot;: &quot;hr_vectorstore&quot;,<br>        &quot;end&quot;: &quot;hard_end&quot;,<br>    },<br>)<br><br><br><br><br>workflow.add_edge(&quot;hr_vectorstore&quot;, &quot;generate&quot;)<br>workflow.add_edge(&quot;hard_end&quot;, END)<br><br><br>workflow.add_conditional_edges(<br>    &quot;generate&quot;,<br>    end,<br>    {<br>        &quot;end&quot;:END<br>    },<br>)<br><br>app = workflow.compile()</pre><h3>Now, display the graph.</h3><pre>from IPython.display import Image, display<br>try:<br>    display(Image(app.get_graph().draw_mermaid_png()))<br>except Exception:<br>    print(&quot;No graph to display&quot;)<br>    pass</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/305/1*ir25kbOubO19YR_u75uWaQ.png" /></figure><p>Let’s begin the query now.</p><p><strong>First example:</strong></p><pre>inputs = {<br>    &quot;question&quot;: &quot;what is honda civic?&quot;,<br>}<br>response=app.invoke(inputs)<br>print(response[&#39;generation&#39;])</pre><p><strong>output:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*bgMeeckRHDwHg9LaOL0uLA.png" /></figure><p><strong>Second example/ Valid example:</strong></p><pre>inputs = {<br>    &quot;question&quot;: &quot;what are the responsibilty of hr?&quot;<br>}<br>response=app.invoke(inputs)<br>print(response[&#39;generation&#39;])</pre><p><strong>output:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZH2x_uciz9zLSYQtflc8vw.png" /></figure><p>Note: Please update the prompt based on your preference.</p><p><strong>Conclusion:</strong></p><p>By leveraging LangGraph, we can build AI assistants that are not only more informative but also more engaging and personalized. By effectively routing queries to the appropriate vector stores, we can ensure that users receive the most relevant and helpful information.</p><h3>For contact:</h3><p>Gmail: hassan.mahmood8605@gmail.com</p><p>linkedin: <a href="http://www.linkedin.com/in/hassan-mahmood-500329256">www.linkedin.com/in/hassan-mahmood-500329256</a></p><p>#Langchain #LangGraph #langgraph #Python #ragChain #LLM</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f09b2793ac08" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG Evaluation: A Deep Dive with Ragas and TruLens]]></title>
            <link>https://medium.com/@hassan.mahmood1/rag-evaluation-a-deep-dive-with-ragas-and-trulens-356c96a937e3?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/356c96a937e3</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Fri, 26 Jul 2024 10:52:04 GMT</pubDate>
            <atom:updated>2024-07-26T10:52:04.631Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you’ve made an AI assistant that can give interesting and informative answers to difficult questions. But how do you make sure its answers are correct, on-topic, and reliable? This is where Retrieval-Augmented Generation (RAG), Ragas, and TruLens come into play.</p><p>In this article, we will explore how to use these tools to carefully check your RAG model. We will look at how Ragas breaks the evaluation process into detailed parts. Then, we will see how TruLens helps give a complete review of how well your model is doing.</p><h4><strong>TruLens: The Post-Deployment Evaluator</strong></h4><p>TruLens is perfect for keeping an eye on how your RAG model works in real life. It gives ongoing feedback on the model’s results, helping you spot problems and make improvements by retraining or fine-tuning the model.</p><h4><strong>Ragas: The Pre-Deployment Optimizer</strong></h4><p>Ragas, on the other hand, is a tool for evaluating RAG models while they are being developed. By repeatedly improving your model using insights from Ragas, you can enhance its overall quality and prepare it for use in real-world situations.</p><h3><strong>Evaluating Our AI </strong>assistant</h3><p>Once we’ve built our AI assistant (RAG chain), it’s time to put it to the test. We’ll focus on three key areas:</p><ol><li><strong>Answer Relevance:</strong> Does the AI assistant provide information directly addressing the question? We want to ensure it’s not giving irrelevant details.</li><li><strong>Context Relevance:</strong> Does the AI assistant consider the overall context of the conversation when providing an answer? This means understanding the topic and previous questions.</li><li><strong>Groundedness:</strong> Does the AI assistant’s response have a factual basis? We’ll check if the information provided is accurate and can be verified.</li></ol><h3><strong>TruLens:</strong></h3><p>Before we can test our AI assistant with TruLens, we need to build it first. This building process is called creating a RAG chain. I’m sure you’re familiar with the steps involved in building an RAG chain, so let’s focus on how TruLens helps us evaluate it once it’s complete.</p><pre>from trulens_eval import TruChain,Tru<br>from trulens_eval.feedback import Feedback, Huggingface<br>from trulens_eval.feedback.provider import OpenAI<br>from trulens_eval import Feedback<br>import numpy as np<br>from trulens_eval.app import App</pre><pre><br>#parameterers <br>#rag_chain: this will be the actual rag chain without invoking<br>#query: User query<br><br>def trulens_evaluater(rag_chain,query):<br>    tru = Tru()<br>    provider = OpenAI()<br>    f_answer_relevance = (<br>            Feedback(provider.relevance_with_cot_reasons, name = &quot;Answer Relevance&quot;)<br>            .on_input_output()<br>        )<br>    context = App.select_context(rag_chain)<br>    f_groundedness = (<br>            Feedback(provider.groundedness_measure_with_cot_reasons, name = &quot;Groundedness&quot;)<br>            .on(context.collect()) # collect context chunks into a list<br>            .on_output()<br>        )<br>    f_context_relevance = (<br>    Feedback(provider.context_relevance_with_cot_reasons, name = &quot;Context Relevance&quot;)<br>    .on_input()<br>    .on(context)<br>    .aggregate(np.mean)<br>)<br>    truchain = TruChain(<br>            rag_chain,<br>            app_id=&#39;Chain1_ChatApplication&#39;,<br>            feedbacks=[f_answer_relevance, f_context_relevance,f_groundedness]<br>        )<br>    with truchain as recording:<br>        llm_response = rag_chain.invoke(query)<br>    rec = recording.get()<br>    response=[]<br>    for feedback, feedback_result in rec.wait_for_feedback_results().items():<br>        response.append(feedback_result.result)<br>    Answer_Relevance,Context_Relevance,Groundedness=response<br>    return Answer_Relevance,Context_Relevance,Groundedness</pre><p><strong>By running this function you will get “Answer Relevance, Context Relevance, Groundedness”</strong></p><h3><strong>Ragas</strong></h3><pre>from ragas import evaluate<br>from datasets import Dataset<br>import requests<br>import fitz<br>from pathlib import Path<br>from Evaluate_Rag import PDFIngestor<br>from ragas.testset.generator import TestsetGenerator<br>from ragas.testset.evolutions import simple, reasoning, multi_context<br>from langchain_openai import ChatOpenAI, OpenAIEmbeddings<br>import os<br>from dotenv import load_dotenv<br>from ragas.metrics import (<br>    answer_relevancy,<br>    faithfulness,<br>    context_recall,<br>    context_precision,<br>)</pre><pre>load_dotenv()<br>openai_api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)<br>assert openai_api_key is not None, &quot;Please set OPENAI_API_KEY in .env file&quot;<br><br># generator with openai models<br>embeddings = OpenAIEmbeddings(model=&#39;text-embedding-3-small&#39;, openai_api_key=openai_api_key)<br>generator_llm = ChatOpenAI(model=&quot;gpt-3.5-turbo&quot;, openai_api_key= openai_api_key)<br>critic_llm = ChatOpenAI(model=&#39;gpt-4o&#39;, openai_api_key= openai_api_key)</pre><pre>pdf_path = &#39;write your path here&#39;<br>ingester = PDFIngestor(embed_fn=embeddings)<br>ingester.ingest_pdf_with_metadata(pdf_path)<br><br>langchain_docs = None<br>langchain_docs = ingester.complete_doc_list<br>assert langchain_docs is not None, &quot;No documents found in the pdf&quot;</pre><pre>generator = TestsetGenerator.from_langchain(generator_llm=generator_llm, critic_llm=critic_llm, embeddings=embeddings)<br>testset = generator.generate_with_langchain_docs(langchain_docs, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})</pre><pre>testset_pd = testset.to_pandas()<br>testset_pd.to_csv(&#39;write your CSV path&#39;, index=False)</pre><pre>This approach involves sending a post request to my RAG chain endpoint. You are welcome to call the function directly, or any other approach you want to use.</pre><pre>answerList=[]<br>for q in testset_pd[&#39;question&#39;]:<br>    payload = {<br>    &#39;query&#39;: q,<br>    &#39;vectordb_name&#39;:&#39;symmetry-12-01882.pdf2024-07-02&#39;<br>    }<br>    headers = {<br>    &#39;accept&#39;: &#39;application/json&#39;,<br>    }<br>    response = requests.post(url, headers=headers,params=payload)#<br>    genrated=eval(response.text)[&#39;Response&#39;]<br>    answerList.append(genrated)</pre><pre>testset_pd[&#39;answer&#39;] = answerList<br>HF_dataset=Dataset.from_pandas(testset_pd)</pre><p><strong>Now</strong> <strong>By running this below command. you will get “Answer Relevance, Context Relevance, Groundedness and context</strong> <strong>precision”</strong></p><pre>result = evaluate(<br>    HF_dataset,<br>    metrics=[<br>        context_precision,<br>        faithfulness,<br>        answer_relevancy,<br>        context_recall,<br>    ],<br>)</pre><pre>print(result)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X4J-fSOvHpcj947-jVQuGg.png" /></figure><p><strong>I’ve endeavored to provide a clear overview of how Ragas and TruLens can be employed to comprehensively evaluate RAG models. To further explore specific use cases, challenges, or potential advancements, I invite you to connect with me for in-depth discussions.</strong></p><h3>For contact:</h3><p>Gmail: hassan.mahmood8605@gmail.com</p><p>contact number: 03314348417</p><p>#RAG #Trulens #Ragas #Langchain #Python #pandas #numpy #ragChain #LLM #evaluation</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=356c96a937e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migration Guide: Upgrading Your Pinecone Serverless Index to the New Serverless Index]]></title>
            <link>https://medium.com/@hassan.mahmood1/migration-guide-upgrading-your-pinecone-serverless-index-to-the-new-serverless-index-39d5245851d8?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/39d5245851d8</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Thu, 16 May 2024 11:05:01 GMT</pubDate>
            <atom:updated>2024-05-16T11:05:01.138Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is pinecone:</h3><p>In simple words pinecone is vector database to build the knowledgeable AI projects.</p><p>Pinecone helps computers quickly find similar things. Imagine you have lots of pictures of animals and you want the computer to find similar ones fast. Pinecone helps with that!</p><h3>Migration</h3><h3>Approach:</h3><p>The migration process is straightforward. First, you need to initialize both the source and destination indexes where you want to transfer the data. Next, retrieve all the vector IDs from the source index. Once you have obtained all the IDs from the source index, gather their corresponding information and organize it into a dictionary format required by the Pinecone vector-upsert function. Then, invoke the upsert function to transfer the vectors along with their metadata to the destination index. Code is given below:</p><h3>Code:</h3><pre>from pinecone import Pinecone<br>from dotenv import load_dotenv<br>load_dotenv()<br>import os<br><br>api_key=os.getenv(&quot;PINECONE_API_KEY&quot;)<br>pc = Pinecone(api_key=api_key)<br><br>source_index_name=&#39;name&#39;<br>destination_index_name=&#39;name&#39;<br>dest_index = pc.Index(destination_index_name) <br>source_index = pc.Index(source_index_name)<br>all_ids = []<br><br>for ids in source_index.list():<br>    all_ids.append(ids)<br>merged_list = [item for sublist in all_ids for item in sublist]<br></pre><p><strong>After running this code, you will have all the vector IDs of the Source index that you specified.</strong></p><pre>for vec_id in merged_list:<br>    try:<br>        vector_info = index.fetch([vec_id])<br>        #get vector info <br>        pageNo = vector_info[&#39;vectors&#39;][vec_id][&#39;metadata&#39;][&#39;page_number&#39;]<br>        pageUrl=vector_info[&#39;vectors&#39;][vec_id][&#39;metadata&#39;][&#39;page_url&#39;]<br>        text=vector_info[&#39;vectors&#39;][vec_id][&#39;metadata&#39;][&#39;text&#39;]<br>        title=vector_info[&#39;vectors&#39;][vec_id][&#39;metadata&#39;][&#39;title&#39;]<br>        embeddings=vector_info[&#39;vectors&#39;][vec_id][&#39;_data_store&#39;][&#39;values&#39;] <br>        #Now lets call the upsert function <br>        upsert_response = dest_index.upsert(vectors=[<br>        {<br>            &quot;id&quot;: vec_id,<br>            &quot;values&quot;: embeddings,<br>            &quot;metadata&quot;: {<br>                &quot;page_number&quot;:pageNo,<br>                &quot;page_url&quot;:pageUrl,<br>                &quot;text&quot;:text,<br>                &quot;title&quot;:title<br>            }<br>        }])<br>    except:<br>        continue</pre><p>Congratulations! You have completed the migration of the serverless index to the new serverless index.</p><blockquote><strong>Note: The vector information may vary depending on your source index. You can also achieve this by batching. This can be helpful for large Pinecone Index.</strong></blockquote><blockquote><strong>I try my best to explain the logic which i made. May be it will helpful for you.</strong></blockquote><p>#pinecone #Migration #serverless #vectorstores #python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39d5245851d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Maximizing Efficiency: Updating metadata on Pinecone]]></title>
            <link>https://medium.com/@hassan.mahmood1/maximizing-efficiency-updating-metadata-on-pinecone-cc74f9f613ec?source=rss-87c3bef8b57a------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc74f9f613ec</guid>
            <dc:creator><![CDATA[Hafiz Hassan Mahmood ]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 08:51:30 GMT</pubDate>
            <atom:updated>2024-03-25T08:51:30.925Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is pinecone:</h3><p>In simple words pinecone is vector database to build the knowledgeable AI projects.</p><p>Think of Pinecone as a super helpful tool for computers. It’s like a magical guide that helps computers understand and organize tons of information really, really fast. So, when you’re searching for something online, Pinecone makes sure you find what you’re looking for super quickly, even if there’s a whole lot of stuff to look through. It’s like having a super-smart friend who knows exactly where everything is!</p><h3>Metadata:</h3><p>Metadata is like a special label that tells you important things about other information. It’s like the details on a book cover that help you decide if you want to read it. In the world of computers, metadata describes stuff like what a file is about, when it was made, who made it, and how it’s related to other things.</p><h3>Updating metadata:</h3><h3>Approach:</h3><p>If I talk about the approach, it’s simple. First, you should initialize the index on which you want to update the metadata. Then, fetch all the vector IDs of the specific index. After that, fetch the specific field from the metadata on which basis you want to update the metadata on the specific vector ID, and then save both the vector ID and source in a <strong>CSV </strong>to avoid repeating these steps again and again for later use. You can also save in <strong>DB</strong>. In my case, I want to update metadata based on the ‘Filename,’ which is already in the metadata of vectors. The code is given below.</p><pre>import csv<br>from pinecone import Pinecone<br>index_name=&#39;index name&#39;<br>pinecone_key=&#39;specifiy your key&#39;<br>pc = Pinecone(api_key=pinecone_key)<br><br>all_ids = []<br>index = pc.Index(index_name)<br>for ids in index.list():<br>    all_ids.append(ids)<br>merged_list = [item for sublist in all_ids for item in sublist]</pre><p><strong>After running this code, you will have all the vector IDs of the index that you specified earlier.</strong></p><pre>all_source= []<br>csv_filename = &#39;file name&#39;<br>with open(csv_filename, &#39;w&#39;, newline=&#39;&#39;) as csvfile:<br>    writer = csv.writer(csvfile)<br>    writer.writerow([&#39;ids&#39;, &#39;Source&#39;])<br>    for vec_id in merged_list:<br>            try:<br>                vector_info = index.fetch([vec_id])<br>                source = vector_info[&#39;vectors&#39;][vec_id][&#39;metadata&#39;][&#39;source&#39;]<br>                writer.writerow([vec_id, source])<br>                all_source.append(source)<br>            except Exception as e:<br>                continue</pre><p><strong>Now you have all the IDs and their respective sources. Again In my case, it is ‘source,’ but it may vary in your case.</strong></p><h4><strong>Now, let’s call the update function to update the metadata.</strong></h4><pre>from pinecone import Pinecone<br>import pandas as pd<br><br>index_name=&#39;index name&#39;<br>pinecone_key=&#39;Your pinecone key&#39;<br>pc = Pinecone(api_key=pinecone_key)<br>index = pc.Index(index_name)<br><br>df=pd.read_csv(&quot;file name&quot;)<br>value=df.loc[df[&#39;Source&#39;].isin([filename])]<br>for id in value[&#39;ids&#39;]:<br>            try:<br>                update_state = index.update(id=id, set_metadata=metadata)<br>            except Exception as e:<br>                continue</pre><blockquote><strong>“Note: The logic to call the update function may vary in your case.”</strong></blockquote><blockquote><strong><em>I try my best to explain the logic which i made. May be it will helpful for you.</em></strong></blockquote><p>#pinecone #metadata #langchain #LLM #python #vectorstores #Embeddings</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc74f9f613ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>