<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by naga_vjk on Medium]]></title>
        <description><![CDATA[Stories by naga_vjk on Medium]]></description>
        <link>https://medium.com/@nagaraj.jayakumar?source=rss-f5a7a1c1cd37------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*4LSA8IwsgfYP1_Sm.jpg</url>
            <title>Stories by naga_vjk on Medium</title>
            <link>https://medium.com/@nagaraj.jayakumar?source=rss-f5a7a1c1cd37------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:17:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nagaraj.jayakumar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Hybrid Search System with Cohere and Pinecone — Part 1]]></title>
            <link>https://medium.com/@nagaraj.jayakumar/building-a-hybrid-search-system-with-cohere-and-pinecone-part-1-e85d746b367f?source=rss-f5a7a1c1cd37------2</link>
            <guid isPermaLink="false">https://medium.com/p/e85d746b367f</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[pinecone]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[cohere]]></category>
            <dc:creator><![CDATA[naga_vjk]]></dc:creator>
            <pubDate>Thu, 24 Oct 2024 20:44:23 GMT</pubDate>
            <atom:updated>2024-10-24T20:45:57.285Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Hybrid Search System with Cohere and Pinecone — Part 1</h3><p>In the realm of data retrieval, hybrid search systems are revolutionizing how we access information. By combining traditional keyword search with cutting-edge vector search, hybrid systems offer enhanced performance and adaptability. In this blog post, we will explore how to implement a hybrid search system using Cohere for embedding generation and Pinecone for vector storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gab4MiewaI63KMTaMPuxtA.png" /></figure><h3>Part 1: Setting Up the Environment</h3><p>To begin, we need to set up our environment by loading necessary libraries and initializing clients for Cohere and Pinecone.</p><pre># Load environment variables<br>env_vars = load_environment_variables()<br># Initialize Cohere and Pinecone clients<br>cohere_client, pinecone_client = initialize_clients(env_vars[&quot;cohere_api_key&quot;], env_vars[&quot;pinecone_api_key&quot;])</pre><h3>Part 2: Extracting and Processing Data</h3><p>We start by extracting elements from a PDF document. This involves using high-resolution partitioning to ensure that tables and text are accurately captured.</p><h3>Step 2.2: Extract Elements from the PDF</h3><p>We use a function partition_pdf to extract elements from the PDF. This function breaks down the document into manageable parts, such as text blocks and tables, using a high-resolution strategy. It also infers table structures and uses a model named &quot;yolox&quot; for chunking by title:</p><pre># Define your PDF file path<br>filename = &#39;/path/to/your/pdf/document.pdf&#39;<br><br># Extract elements from the PDF<br>orig_elements = partition_pdf(<br>    filename=filename,<br>    strategy=&quot;hi_res&quot;,<br>    infer_table_structure=True,<br>    model_name=&quot;yolox&quot;,<br>    chunking_strategy=&quot;by_title&quot;,<br>    max_characters=4000,<br>    new_after_n_chars=3800,<br>    combine_text_under_n_chars=2000,<br>)</pre><h3>Step 2.3: Initialize LangChain and Cohere</h3><p>LangChain and Cohere are initialized to summarize extracted content. LangChain’s ChatCohere model is set up with a specific temperature and model name for generating concise summaries:</p><pre>llm = ChatCohere(temperature=0, model_name=&quot;command-r-plus&quot;)<br>chain = load_summarize_chain(llm, chain_type=&quot;stuff&quot;)</pre><h3>Step 2.4: Copy and Group Elements by Page Number</h3><p>The extracted elements are deep-copied to preserve the original structure. They are then grouped by page number to facilitate further processing:</p><pre>elements = copy.deepcopy(orig_elements)<br>page_content = defaultdict(list)<br>for element in elements:<br>    if isinstance(element, (CompositeElement, Table)):<br>        page_number = element.metadata.page_number if element.metadata else None<br>        if page_number is not None:<br>            page_content[page_number].append(element)</pre><h3>Step 2.5: Process Each Page for Headers and Footers</h3><p>For each page, headers and footers are identified around tables. This involves iterating through elements on each page and distinguishing between headers, footers, and table content:</p><pre>all_tables_info = []<br><br>for page, elements in tqdm(page_content.items(), desc=&quot;Processing Pages&quot;):<br>    header_text = &quot;&quot;<br>    footer_text = &quot;&quot;<br>    found_first_table = False<br><br>    for elem in elements:<br>        if isinstance(elem, CompositeElement):<br>            if not found_first_table:<br>                header_text += elem.text + &quot;\n&quot;<br>            else:<br>                footer_text += elem.text + &quot;\n&quot;<br>        elif isinstance(elem, Table):<br>            found_first_table = True</pre><h3>Step 2.6: Summarize Tables with Headers and Footers</h3><p>Tables are processed to extract HTML content and text. LangChain is used to generate summaries of these tables, which are then combined with headers and footers:</p><pre>table_index = 0<br>for elem in elements:<br>    if isinstance(elem, Table):<br>        html_content = elem.metadata.text_as_html if elem.metadata and hasattr(elem.metadata, &#39;text_as_html&#39;) else &quot;No HTML content&quot;<br>        table_text = elem.text<br><br>        documents = [Document(page_content=html_content), Document(page_content=table_text)]<br>        res = chain.invoke(documents)<br><br>        combined_text = f&quot;{header_text.strip()}\n{res[&#39;output_text&#39;]}\n{html_content}\n{footer_text.strip()}&quot;<br>        <br>        transformed_data = {<br>            &quot;id&quot;: f&quot;chunk-3-page-{page}-table-{table_index}&quot;,<br>            &quot;text&quot;: combined_text,<br>            &quot;metadata&quot;: {<br>                &quot;context&quot;: combined_text,<br>                &quot;page&quot;: page,<br>                &quot;type&quot;: &quot;Table&quot;,<br>                &quot;id&quot;: f&quot;chunk-3-page-{page}-table-{table_index}&quot;,<br>                &quot;summary&quot;: res[&#39;output_text&#39;],<br>                &quot;html&quot;: html_content,<br>                &quot;header&quot;: header_text.strip(),<br>                &quot;footer&quot;: footer_text.strip()<br>            }<br>        }<br>        all_tables_info.append(transformed_data)<br>        table_index += 1</pre><h3>Step 2.7: Prepare Data for Hybrid Search</h3><p>The processed data is prepared for hybrid search by fitting it into a BM25 model. This involves storing vector representations in a JSON file for later retrieval:</p><pre>contexts = [table_info[&quot;text&quot;] for table_info in all_tables_info]<br>bm25.fit(contexts)<br>json_file_path = store_bm25_values(bm25, filename)</pre><h3>Part 3: Uploading Data to the Hybrid Vector Store</h3><p>Once the data is extracted and processed, the next step is to upload it to a hybrid vector store. This involves creating both dense and sparse vector representations of the data and storing them in a Pinecone index. This section outlines how to efficiently batch-process and upload data to the vector store.</p><h3>Step 3.1: Define Batch Size</h3><p>To manage memory and processing efficiency, we define a batch size for processing our data in chunks. This helps in handling large datasets without overwhelming system resources:</p><pre>batch_size = 200</pre><h3>Step 3.2: Process Data in Batches</h3><p>We iterate over the processed data (all_tables_info) in batches, creating both dense and sparse vectors for each batch. This involves:</p><ul><li>Extracting a batch of data</li><li>Generating dense embeddings using Cohere’s embedding model</li><li>Creating sparse vectors using the BM25 encoder</li></ul><pre>for i in tqdm(range(0, len(all_tables_info), batch_size)):<br>    # Find end of batch<br>    i_end = min(i + batch_size, len(all_tables_info))<br>    <br>    # Extract batch<br>    batch = all_tables_info[i:i_end]<br>    <br>    # Create dense vectors<br>    dense_embeds = embeddings.embed_documents([item[&quot;text&quot;] for item in batch])<br>    <br>    # Create sparse BM25 vectors<br>    sparse_embeds = bm25.encode_documents([item[&quot;text&quot;] for item in batch])</pre><h3>Step 3.3: Prepare Data for Upload</h3><p>For each item in the batch, we prepare a dictionary containing its ID, dense vector values, sparse vector values, and metadata. These dictionaries are collected into a list (upserts) for uploading:</p><pre>upserts = []<br>for item, sparse, dense in zip(batch, sparse_embeds, dense_embeds):<br>    upserts.append({<br>        &#39;id&#39;: item[&quot;id&quot;],<br>        &#39;sparse_values&#39;: sparse,<br>        &#39;values&#39;: dense,<br>        &#39;metadata&#39;: item[&quot;metadata&quot;]<br>    })</pre><h3>Step 3.4: Upload Data to Pinecone Index</h3><p>The prepared data is then uploaded to the Pinecone index using the upsert method. This method efficiently inserts or updates the documents in the index:</p><pre># Upload the documents to the hybrid index<br>index.upsert(upserts)</pre><h3>Conclusion</h3><p>In this blog post, we’ve explored the process of building a robust hybrid search system using Cohere for embedding generation and Pinecone as our vector database. By combining semantic search with traditional keyword-based approaches, hybrid search systems offer enhanced precision and recall, making them ideal for complex data retrieval tasks.</p><h3>Key Takeaways:</h3><ol><li><strong>Data Extraction and Processing</strong>: We demonstrated how to extract structured data from PDF documents, including text and tables, while preserving contextual information such as headers and footers. This structured approach ensures that the extracted data is ready for further analysis and storage.</li><li><strong>Hybrid Vector Storage</strong>: By creating dense embeddings with Cohere and sparse vectors with BM25, we leveraged the strengths of both semantic understanding and keyword matching. The integration of these vectors into a Pinecone index allows for efficient and scalable storage.</li><li><strong>Batch Processing</strong>: Handling large datasets efficiently requires processing data in batches. This approach minimizes memory usage and optimizes processing time, ensuring that the system can scale to accommodate growing volumes of data.</li><li><strong>Scalable Search Infrastructure</strong>: With the hybrid vector store in place, our system is capable of performing advanced search operations that combine the best of both worlds — semantic relevance and precise keyword matching. This makes it particularly useful for domains requiring detailed information retrieval, such as financial documents or technical reports.</li></ol><p>By following these steps, you can implement your own hybrid search system tailored to your specific needs. Whether you’re dealing with large-scale document repositories or specialized datasets, this approach provides a flexible and powerful solution for modern search challenges. We hope this guide has been informative and encourages you to explore the potential of hybrid search systems in your projects. If you have any questions or feedback, feel free to reach out or leave a comment below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e85d746b367f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding the NeMo Guardrails Configuration — Part 2]]></title>
            <link>https://medium.com/@nagaraj.jayakumar/understanding-the-nemo-guardrails-configuration-part-2-52fe527bd9ca?source=rss-f5a7a1c1cd37------2</link>
            <guid isPermaLink="false">https://medium.com/p/52fe527bd9ca</guid>
            <category><![CDATA[enterprise-ai]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[guardrail]]></category>
            <category><![CDATA[cohere]]></category>
            <dc:creator><![CDATA[naga_vjk]]></dc:creator>
            <pubDate>Sun, 13 Oct 2024 00:49:40 GMT</pubDate>
            <atom:updated>2024-10-13T01:06:28.074Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Secure and Intelligent AI Chatbot: Combining Cohere, NeMo Guardrails, and Pinecone — Part 2</h3><p>This is Part 2 of our series on implementing ethical AI chatbots. If you haven’t read Part 1, you can find it here: [<a href="https://medium.com/@nagaraj.jayakumar/building-a-secure-and-intelligent-ai-chatbot-combining-cohere-nemo-guardrails-and-pinecone-bb67a7621b96">Building a Secure and Intelligent AI Chatbot: Combining Cohere, NeMo Guardrails, and Pinecone — Part1</a>]</p><h3>Understanding the NeMo Guardrails Configuration</h3><p>In our AI chatbot, we’ve implemented several guardrails to ensure the AI responds appropriately to various types of queries. Let’s break down the key components of our rails.config file.</p><p>NeMo Guardrails operate as a semi or fully deterministic shield between the user and the AI model. They act as a protective layer that monitors, influences, and controls interactions with AI-driven chatbots. Let’s break down the key components and processes using the provided configuration:</p><h3>1. Semantic Encoding</h3><p>Guardrails use semantic encoding to understand user inputs and match them to predefined patterns:</p><ul><li>All utterances specified in the Colang file are encoded into a semantic vector space.</li><li>When a user sends a message, it’s encoded into the same vector space.</li><li>Semantic similarity is calculated to identify the most relevant utterances and their corresponding canonical forms.</li></ul><p>For example, in the “politics” flow:</p><pre>define user ask politics<br>    &quot;what are your political beliefs?&quot;<br>    &quot;thoughts on the president?&quot;<br>    &quot;left wing&quot;<br>    &quot;right wing&quot;<br>    &quot;Who is the president?&quot;</pre><p>These utterances are encoded into a vector space. When a user asks a question like “What do you think about the current administration?”, it’s encoded and compared to these utterances to determine if it’s a political question.</p><h3>2. Flow Control</h3><p>Guardrails use defined flows to control the conversation. For instance:text</p><pre>define flow politics<br>    user ask politics<br>    bot answer politics<br>    bot offer help</pre><p>When a user input matches a defined pattern (e.g., asking about politics), it triggers this specific flow. The flow determines how the AI should respond, often using predefined messages or actions.</p><h3>3. Response Generation</h3><p>The final response is generated based on the activated flows:</p><pre>define bot answer politics<br>    &quot;I&#39;m a Cohere model knowledge assistant, I don&#39;t like to talk of politics.&quot;</pre><p>If the politics flow is triggered, the AI will use this predefined response instead of generating a new one, ensuring it doesn’t engage in political discussions.</p><h3>4. Action Execution</h3><p>Guardrails can trigger specific actions when needed. In the Cohere flow:</p><pre>define flow cohere<br>    user ask cohere<br>    $answer = execute response(inputs=$last_user_message)<br>    bot $answer</pre><p>This flow executes a custom action (response) when users ask about Cohere models, allowing for dynamic, context-aware responses.</p><h3>5. Topic Limitation</h3><p>The configuration includes several topic limitations:</p><ul><li>Politics: Redirects political questions.</li><li>Programming: Avoids engaging in coding tasks.</li><li>Off-topic: Handles a wide range of unrelated queries.</li></ul><p>For example:</p><pre>define flow programming<br>    user ask programming<br>    bot answer programming<br>    bot offer help</pre><p>This flow ensures the AI doesn’t engage in programming tasks, maintaining its focus on being a knowledge assistant for Cohere models.</p><h3>6. Fallback Mechanism</h3><p>For off-topic queries:text</p><pre>define flow<br>    user ask off topic<br>    bot explain cant help with off topic<br>    bot offer help</pre><p>This catch-all flow handles queries outside the AI’s intended scope, explaining its limitations and offering to help with relevant topics.</p><h3>Conclusion</h3><p>These guardrails create a robust system that:</p><ol><li>Maintains focus on Cohere model-related information.</li><li>Avoids sensitive topics like politics.</li><li>Prevents engagement in unintended tasks like programming.</li><li>Handles off-topic queries gracefully.</li></ol><p>By implementing these guardrails, developers can create AI chatbots that are more reliable, safe, and focused on their intended purpose, significantly reducing the risks associated with deploying AI in real-world applications. The use of semantic encoding and predefined flows allows for flexible yet controlled interactions, ensuring the AI behaves consistently and ethically across a wide range of user inputs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52fe527bd9ca" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Secure and Intelligent AI Chatbot: Combining Cohere, NeMo Guardrails, and Pinecone —…]]></title>
            <link>https://medium.com/@nagaraj.jayakumar/building-a-secure-and-intelligent-ai-chatbot-combining-cohere-nemo-guardrails-and-pinecone-bb67a7621b96?source=rss-f5a7a1c1cd37------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb67a7621b96</guid>
            <category><![CDATA[enterprise-ai]]></category>
            <category><![CDATA[cohere]]></category>
            <category><![CDATA[guardrail]]></category>
            <category><![CDATA[nemo]]></category>
            <dc:creator><![CDATA[naga_vjk]]></dc:creator>
            <pubDate>Fri, 11 Oct 2024 15:29:19 GMT</pubDate>
            <atom:updated>2024-10-13T00:58:23.753Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Secure and Intelligent AI Chatbot: Combining Cohere, NeMo Guardrails, and Pinecone — Part 1</h3><p>In the rapidly evolving landscape of artificial intelligence, creating intelligent chatbots has become more accessible than ever. However, with this increased power comes a greater responsibility to ensure these AI systems operate safely and ethically. In this post, we’ll explore how to build a secure AI chatbot using Cohere, NVIDIA’s NeMo Guardrails, and Pinecone, demonstrating enterprise-grade AI capabilities with built-in safeguards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cBLgKyRGnXqhVpHaWKpIsA.gif" /></figure><h3>The Power Trio: Cohere, NeMo Guardrails, and Pinecone</h3><p>Our chatbot leverages three key technologies:</p><ol><li><strong>Cohere</strong>: A powerful language model that generates human-like text.</li><li><strong>NVIDIA NeMo Guardrails</strong>: A framework for implementing ethical AI guidelines and safety measures.</li><li><strong>Pinecone</strong>: A vector database for efficient semantic search and retrieval.</li></ol><p>By combining these technologies, we create a chatbot that’s not only intelligent but also operates within predefined ethical boundaries.</p><h3>Setting Up the Environment</h3><p>To get started, you’ll need to set up your environment with the necessary dependencies. Here’s a snippet from the requirements.txt file:</p><pre>streamlit==1.22.0<br>langchain==0.0.184<br>cohere==4.3.1<br>pinecone-client==2.2.1<br>nemoguardrails==0.3.0</pre><p>You’ll also need to set up your API keys as environment variables:</p><pre>COHERE_API_KEY = os.environ[&quot;COHERE_API_KEY&quot;]<br>PINECONE_API_KEY = os.environ[&quot;PINECONE_API_KEY&quot;]<br>PINECONE_INDEX = os.environ[&quot;PINECONE_INDEX&quot;]</pre><h3>Implementing Guardrails</h3><p>One of the key features of our chatbot is the implementation of guardrails using NVIDIA’s NeMo framework. This allows us to define rules and boundaries for the AI’s behavior:</p><pre>config = RailsConfig.from_path(&quot;./config&quot;)<br>llm = Cohere(cohere_api_key=COHERE_API_KEY, model=&quot;command-nightly&quot;)<br>rails = LLMRails(config, llm)</pre><p>The config object loads our predefined rules, which are then applied to the language model through the LLMRails class.</p><h3>The RAG Chain: Retrieval-Augmented Generation</h3><p>Our chatbot uses a RAG (Retrieval-Augmented Generation) chain to process user input. This involves several steps:</p><ol><li>Document Retrieval</li><li>Context Generation</li><li>Prompt Generation</li><li>Guardrails Application</li><li>Response Generation</li></ol><p>Let’s break down each step of the RAG chain:</p><h3>Document Retrieval</h3><pre>async def retrieve_docs(query):<br>    embeddings = CohereEmbeddings(model=&quot;embed-english-v2.0&quot;, cohere_api_key=COHERE_API_KEY)<br>    docsearch = Pinecone.from_existing_index(PINECONE_INDEX, embeddings)<br>    docs = docsearch.similarity_search(query)<br>    return docs</pre><p>This function creates an embedding of the user’s query using Cohere’s embedding model and then performs a similarity search in the Pinecone index to find the most relevant documents.</p><h3>Context and Prompt Generation</h3><pre>async def get_rag_chain_response(prompt):<br>    docs = await retrieve_docs(prompt)<br>    context = &quot;\n&quot;.join([doc.page_content for doc in docs])<br>    prompt_template = f&quot;Context:\n{context}\n\nHuman: {prompt}\n\nAssistant:&quot;<br>    response = await rails.generate_async(prompt=prompt_template)<br>    return response.strip()</pre><p>Here, we create the context by joining the content of all retrieved documents. This context is then incorporated into the prompt_template.</p><h3>Applying Guardrails and Generating Response</h3><pre>response = await rails.generate_async(prompt=prompt_template)</pre><p>The rails.generate_async() method applies the predefined guardrails to the prompt and generates a response using the Cohere language model.</p><h3>The User Interface</h3><p>We use Streamlit to create a simple yet effective user interface for our chatbot:</p><pre>st.title(&quot;AI Chatbot with Guardrails&quot;)<br>user_input = st.text_input(&quot;You: &quot;, key=&quot;input&quot;)<br><br>if user_input:<br>    response = asyncio.run(get_rag_chain_response(user_input))<br>    st.text_area(&quot;AI: &quot;, value=response, height=120, max_chars=None, key=&quot;response&quot;)</pre><p>Users can input their questions, and the AI’s responses are displayed in a conversational format.</p><h3>Conclusion</h3><p>Building a secure AI chatbot involves more than just implementing a language model. By incorporating technologies like NeMo Guardrails and Pinecone, we can create AI systems that are not only intelligent but also operate within ethical boundaries and provide relevant, context-aware responses.</p><p>This project demonstrates how to implement a sophisticated RAG chain that combines the power of retrieval-based systems with the flexibility of generative AI, all while maintaining strong safeguards for responsible AI deployment.</p><p>As AI continues to evolve, such approaches will become increasingly important in ensuring that our AI systems remain beneficial and aligned with human values.</p><p>Remember, the full source code and detailed setup instructions are available in the <a href="https://github.com/nagarajjayakumar/cohere-nemo-demo">GitHub repository</a>.</p><p>Continue Reading This article is part of a series on implementing ethical AI chatbots. To learn more about the guardrails configuration, check out Part 2: [<a href="https://medium.com/@nagaraj.jayakumar/understanding-the-nemo-guardrails-configuration-part-2-52fe527bd9ca">Building a Secure and Intelligent AI Chatbot: Combining Cohere, NeMo Guardrails, and Pinecone — Part 2</a>]</p><p>In Part 2, we dive deeper into the rails configuration, exploring how we handle political queries, Cohere-specific questions, programming requests, and off-topic inquiries. Learn how these guardrails help maintain the AI’s focus and ensure ethical operations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb67a7621b96" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>