I have been in the Tech Industry for over seven years. I have worked both in consulting firms and as an in-house employee. From when I was a junior to when I became a senior, and now as a manager, I have observed the people around me and always believed that “a successful manager must be good at multitasking.”

What is Multi-tasking?

When I typed “multi-tasking” into Google search, the AI ​​instantly helped me define it. (Thanks, Gemini.)

Let me highlight some keywords:

Multitasking is the act of performing 02 or MORE tasks at the SAME time, or rapidly SWITCHING between tasks.

The first promotion from a Junior to a Mid-level engineer, I went from constantly working on my tasks with senior support to doing my tasks while also supporting 1–2 members. It was not so overwhelming. I was able to handle that. Bingo!

The second promotion, when I became a Senior, changed everything dramatically. I was now managing engineers with responsibility, not just completing tasks. I worked long hours (and sometimes unpaid overtime) to ensure everything was completed as perfectly as possible (I’m a bit OCD).

Thanks to hard work, I became a Manager responsible for engineering teams, stakeholders, and business owners. After successfully working long hours and completing everything in one go during my second promotion, I tried to apply the same. Here’s how it went👇.

A very small percentage of people in the world are good at Multi-tasking.

Even great people don’t choose to multitask.

1. Bill Gates’ Mastery of ‘Deep Work’ Fueled His Success (Source: https://www.businessinsider.com/)
2. Steve Jobs Quote: “Focus comes from saying NO to a thousand things so you don’t go in the wrong direction or do too much.”
3. Elon Musk created his own “Time Blocking” Method to boost productivity.

You and I are probably not a good fit for “Multiple-tasking” either.

An average of 900 people die every day trying to multitask while driving. Think before you answer a text or change your Spotify music station.

This is the Alternative

What I have learned from a job changes as my mindset changes. Here are my practices for improving productivity and quality:

𝗛𝗮𝗻𝗱𝗹𝗲 𝗠𝗨𝗟𝗧𝗜-𝗧𝗔𝗦𝗞𝗜𝗡𝗚 𝗯𝘆 𝘀𝘂𝗰𝗰𝗲𝗲𝗱𝗶𝗻𝗴 𝗶𝗻 𝗦𝗜𝗡𝗚𝗟𝗘-𝗧𝗔𝗦𝗞𝗜𝗡𝗚 𝗮𝗻𝗱 𝗗𝗘𝗘𝗣-𝗧𝗔𝗦𝗞𝗜𝗡𝗚.

What is Single-tasking?

It’s exactly what it sounds like. Single-tasking is simply the act of focusing on ONE task at a time.

What is Deep-working?

The term was coined by Cal Newport, a computer science professor at Georgetown University and author of “Deep Work: Rules for Focused Success in a Distracted World.” In his book, Newport defines deep work as a state of distraction-free concentration when your brain works at its maximum potential. — https://asana.com/resources/what-is-deep-work

How to Practice Single Tasking and Deep Tasking

Let’s set a schedule for each hour of the day and stick to it while avoiding distractions like social media, games, and emails. Put your mind in 100% focus mode.

Something like this, with the main focus on the afternoon.

𝗠𝗼𝗿𝗻𝗶𝗻𝗴:
9:00–9:20: Plan the day
9:20–9:30: Check and prioritize emails
9:30–10:30: Focus on Task A
10:30–11:30: Focus on Task B
11:30–12:00: Check and prioritize emails
12:00–1:00: Lunch break

𝗔𝗳𝘁𝗲𝗿𝗻𝗼𝗼𝗻:
1:00–2:00: Meetings
2:00–2:30: Check and prioritize emails
2:30–3:30: Task C
3:30–4:30: Meetings
4:30–5:00: Check and prioritize emails
5:00–6:00: Task D and wrap up the day

Some tools that support Single-tasking and Deep Work:

• Pomodoro technique: Work for 25–50 minutes, then take a 5–10 minute break.
• Focus apps: Block distractions with tools like Forest, Focus@Will, and Freedom.

Many creative ways exist to manage multiple projects in software engineering and other fields.

Got any good tips? Sharing is caring.

💡Data Migration, Analytics, AI & Automation | Sharing the Journey on Medium || Github || Kaggle || Linkedin. 🔔 Follow “Nhi Yen” for future updates!

A successful Manager must excel at Multi-tasking was originally published in Code Like A Girl on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Web scraping automates the extraction of data from websites using programming or specialized tools. Required for tasks such as market research, data analysis, content aggregation, and competitive intelligence. This efficient method saves time, improves decision making, and allows businesses to study trends and patterns, making it a powerful tool for extracting valuable information from the Internet.

Since I received a lot of positive comments on my last article, I would like to delve into this topic that is of interest to many people. Besides BeautifulSoup, Scrapy, and Selenium, have you considered web scraping using LLM? It’s a technique worthy of your curiosity.

👉 Check out my earlier articles for more information:

• $4000 Freelance Income: Exploring More Than Just Web Scraping
• Before You Freelance: 5 Must-Know Truths

💡I write about Machine Learning on Medium || Github || Kaggle || Linkedin. 🔔 Follow “Nhi Yen” for future updates!

👉 Get UNLIMITED access to every story on Medium with just $1/week — HERE

Let’s start!!!

If you know Python but not HTML, you should first understand the basics of HTML.

I. Understand Basic HTML

HTML is the basic component of web pages. A set of tags is used to structure content on the Internet. Tags are like containers that wrap various elements, defining their structure and appearance on a web page.

1. Structure of HTML

An HTML document consists of 2 main parts: header and body.

• The header contains metadata such as the page title and links to external resources.
• The body contains the content of the web page, including text, images, and other multimedia elements.

Here’s a simple HTML template:

<!DOCTYPE html>
<html>
<head>
    <title>My First Web Page</title>
</head>
<body>
    <h1>Hello, World!</h1>
    <p>This is a simple webpage.</p>
</body>
</html>

2. HTML Tags

Tags are the heart of HTML. They define different elements on a webpage. Some common tags include:

• <h1> to <h6>: Headings, with <h1> being the largest and <h6> the smallest.
• <p>: Paragraphs of text.
• <a>: Hyperlinks, allowing you to navigate to other pages.
• <img>: Images.
• <ul>, <ol>, <li>: Lists, both unordered and ordered.

3. Attributes:

Attributes provide additional information about an HTML element. These are always included in the opening tag and specified as name/value pairs. For example, the href attribute in an <a> tag defines the hyperlink's destination.

<a href="https://www.example.com">Visit Example</a>

4. How to Inspect Elements in a Website

Most web browsers provide built-in tools for inspecting elements. To open this tool, right-click the web page and select Inspect. Here you can view the HTML code and identify the tags containing the data you want to extract.

Selectors help pinpoint specific elements on a webpage. Common selectors include:

• Element Selector: element selects all instances of the specified HTML element.
• Class Selector: .class selects all elements with the specified class.
• ID Selector: #id selects a unique element with the specified ID.

For example:

II. How to “Web Scrape”

❗Disclaimer

This article is not about web scraping for illegal purposes. Make sure you have permission before extracting content. Ways to check include:

1. Review the website’s terms of use. Look for mentions of scraping permissions.
2. Use APIs when available. APIs provide legal access to data.
3. Please contact the website owner to clarify your permission directly.

OK, assume you are aware of your actions. Let’s start.

Here’s the URL for Action Movies of 2022: https://www.imdb.com/list/ls566941243/. I’ll use 05 different methods to scrape [‘Title’, ‘Genre’, ‘Stars’, ‘Runtime’, ‘Rating’] data from this website.

# Define the URL
url = "https://www.imdb.com/list/ls566941243/"

👉 You can find the entire code in this GitHub Repository. To begin, please install the required Python packages listed in requirements.txt.

Method 1: BeautifulSoup and Requests

The first method uses the popular BeautifulSoup and Requests libraries. These tools make it easy to analyze HTML and navigate web page structure. Below is a sample Python code.

import requests
from bs4 import BeautifulSoup
import pandas as pd

# Step 1: Send a GET request to the specified URL
response = requests.get(url)

# Step 2: Parse the HTML content of the response using BeautifulSoup
soup = BeautifulSoup(response.text, 'html.parser')

# Step 3: Save the HTML content to a text file for reference
with open("imdb.txt", "w", encoding="utf-8") as file:
    file.write(str(soup))
print("Page content has been saved to imdb.txt")

# Step 4: Extract movie data from the parsed HTML and store it in a list
movies_data = []
for movie in soup.find_all('div', class_='lister-item-content'):
    title = movie.find('a').text
    genre = movie.find('span', class_='genre').text.strip()
    stars = movie.find('div', class_='ipl-rating-star').find('span', class_='ipl-rating-star__rating').text
    runtime = movie.find('span', class_='runtime').text
    rating = movie.find('span', class_='ipl-rating-star__rating').text
    movies_data.append([title, genre, stars, runtime, rating])

# Step 5: Create a Pandas DataFrame from the extracted movie data
df = pd.DataFrame(movies_data, columns=['Title', 'Genre', 'Stars', 'Runtime', 'Rating'])

# Display the resulting DataFrame
df

In each web scraping method, I save the HTML in an imdb.txt file for a clearer view of the targeted HTML elements.

Final Output:

Method 2: Scrapy

Scrapy is a powerful and flexible web scraping framework. Below is a code snippet showing how to use Scrapy, with explanations in the comments.:

# Import necessary libraries
import scrapy
from scrapy.crawler import CrawlerProcess

# Define the Spider class for IMDb data extraction
class IMDbSpider(scrapy.Spider):
    # Name of the spider
    name = "imdb_spider"
    # Starting URL(s) for the spider to crawl
    start_urls = ["https://www.imdb.com/list/ls566941243/"]
    # start_urls = [url]

    # Parse method to extract data from the webpage
    def parse(self, response):
        # Iterate over each movie item on the webpage
        for movie in response.css('div.lister-item-content'):
            yield {
                'title': movie.css('h3.lister-item-header a::text').get(),
                'genre': movie.css('p.text-muted span.genre::text').get(),
                'runtime': movie.css('p.text-muted span.runtime::text').get(),
                'rating': movie.css('div.ipl-rating-star span.ipl-rating-star__rating::text').get(),
            }
# Initialize a CrawlerProcess instance with settings
process = CrawlerProcess(settings={
    'FEED_FORMAT': 'json',
    'FEED_URI': 'output.json',  # This will overwrite the file every time you run the spider
})


# Add the IMDbSpider to the crawling process
process.crawl(IMDbSpider)
# Start the crawling process
process.start()

import pandas as pd

# Read the output.json file into a DataFrame (jsonlines format)
df = pd.read_json('output.json')

# Display the DataFrame
df.head()

In this demo, I’m using Jupyter Notebook. If you encounter a ReactorNotRestartable error, just restart the kernel and run the code again.

Display the data in a dataframe:

import pandas as pd

# Read the output.json file into a DataFrame (jsonlines format)
df = pd.read_json('output.json')

# Display the DataFrame
df.head()

Method 3: Selenium

Selenium is often used for dynamic web scraping. Here’s a basic example:

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd

# URL of the IMDb list
url = "https://www.imdb.com/list/ls566941243/"

# Set up Chrome options to run the browser in incognito mode
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")

# Initialize the Chrome driver with the specified options
driver = webdriver.Chrome(options=chrome_options)

# Navigate to the IMDb list URL
driver.get(url)

# Wait for the page to load (adjust the wait time according to your webpage)
driver.implicitly_wait(10)

# Get the HTML content of the page after it has fully loaded
html_content = driver.page_source

# Parse the HTML content with BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')

# Save the HTML content to a text file for reference
with open("imdb_selenium.txt", "w", encoding="utf-8") as file:
    file.write(str(soup))
print("Page content has been saved to imdb_selenium.txt")

# Extract movie data from the parsed HTML
movies_data = []
for movie in soup.find_all('div', class_='lister-item-content'):
    title = movie.find('a').text
    genre = movie.find('span', class_='genre').text.strip()
    stars = movie.select_one('div.ipl-rating-star span.ipl-rating-star__rating').text
    runtime = movie.find('span', class_='runtime').text
    rating = movie.select_one('div.ipl-rating-star span.ipl-rating-star__rating').text
    movies_data.append([title, genre, stars, runtime, rating])

# Create a Pandas DataFrame from the collected movie data
df = pd.DataFrame(movies_data, columns=['Title', 'Genre', 'Stars', 'Runtime', 'Rating'])

# Display the resulting DataFrame
print(df)

# Close the Chrome driver
driver.quit()

Final Output:

The key to Selenium is in its Chrome options, which are settings for customizing the behavior of the Chrome browser controlled by Selenium WebDriver. These options enable control over aspects like incognito mode, window size, notifications, and more.

Here are some important Chrome options that you might find useful:

# Runs the browser in incognito (private browsing) mode.
chrome_options.add_argument("--incognito")

# Runs the browser in headless mode, i.e., without a graphical user interface. 
# Useful for running Selenium tests in the background without opening a visible browser window.
chrome_options.add_argument("--headless")

# Sets the initial window size of the browser.
chrome_options.add_argument("--window-size=1200x600")

# Disables browser notifications.
chrome_options.add_argument("--disable-notifications")

# Disables the infobar that appears at the top of the browser.
chrome_options.add_argument("--disable-infobars")

# Disables browser extensions.
chrome_options.add_argument("--disable-extensions")

# Disables the GPU hardware acceleration.
chrome_options.add_argument("--disable-gpu")

# Disables web security features, which can be useful for testing on localhost without CORS issues.
chrome_options.add_argument("--disable-web-security")

These options can be combined or used individually based on your requirements. When you create a webdriver.Chrome instance, you can pass these options using the options parameter:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--headless")

driver = webdriver.Chrome(options=chrome_options)

These options provide flexibility and control over the browser’s behavior when using Selenium for web automation or testing. Choose options based on your specific use case and requirements.

Method 4: Requests and lxml

lxml is a Python library linking with C libraries libxml2 and libxslt, combining speed and XML features with a simple native Python API, similar to ElementTree but with added benefits.

import requests
from lxml import html
import pandas as pd

# Define the URL
url = "https://www.imdb.com/list/ls566941243/"

# Send an HTTP request to the URL and get the response
response = requests.get(url)

# Parse the HTML content using lxml
tree = html.fromstring(response.content)

# Extract movie data from the parsed HTML
titles = tree.xpath('//h3[@class="lister-item-header"]/a/text()')
genres = [', '.join(genre.strip() for genre in genre_list.xpath(".//text()")) for genre_list in tree.xpath('//p[@class="text-muted text-small"]/span[@class="genre"]')]
ratings = tree.xpath('//div[@class="ipl-rating-star small"]/span[@class="ipl-rating-star__rating"]/text()')
runtimes = tree.xpath('//p[@class="text-muted text-small"]/span[@class="runtime"]/text()')

# Create a dictionary with extracted data
data = {
    'Title': titles,
    'Genre': genres,
    'Rating': ratings,
    'Runtime': runtimes
}

# Create a DataFrame from the dictionary
df = pd.DataFrame(data)

# Display the resulting DataFrame
df.head()

Method 5: How to Use LangChain for Web Scraping

I bet all the readers who are interested this article know about ChatGPT, BARD. LLM makes life simpler. You can use it for various tasks, like asking ‘Who is Donald Trump?’ or ‘Translate sentences from German to English,’ and get quick answers. The good news is you can also use them for Web Scraping. Here’s how.

👉 Helpful LangChain resources for this demo:

• LangChain Beautiful Soup
• LangChain Extraction

Brief explanations for each code line are provided in the comments.

import os
import dotenv
import time

# Load environment variables from a .env file
dotenv.load_dotenv()

# Retrieve OpenAI and Comet key from environment variables
MY_OPENAI_KEY = os.getenv("MY_OPENAI_KEY")
MY_COMET_KEY = os.getenv("MY_COMET_KEY")

In my LLM Project, I usually record outputs in a Comet Project. In this demo, I’m using just 01 URL. However, if you need to loop through multiple URLs, Comet LLM’s experiment tracking is very useful.

👉 Read more about Comet LLM.

👉 How to obtain the API keys: OpenAI Help Center — Where can I find my API key?; CometLLM — Obtaining your API key

import comet_llm

# Initialize a Comet project
comet_llm.init(project="langchain-web-scraping",
               api_key=MY_COMET_KEY,
               )

# Resolve async issues by applying nest_asyncio
import nest_asyncio
nest_asyncio.apply()

# Import required modules from langchain
from langchain_openai import ChatOpenAI
from langchain_community.document_loaders import AsyncChromiumLoader
from langchain_community.document_transformers import BeautifulSoupTransformer
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import create_extraction_chain

# Define the URL
url = "https://www.imdb.com/list/ls566941243/"

# Initialize ChatOpenAI instance with OpenAI API key
llm = ChatOpenAI(openai_api_key=MY_OPENAI_KEY)

# Load HTML content using AsyncChromiumLoader
loader = AsyncChromiumLoader([url])
docs = loader.load()

# Save the HTML content to a text file for reference
with open("imdb_langchain_html.txt", "w", encoding="utf-8") as file:
    file.write(str(docs[0].page_content))
print("Page content has been saved to imdb_langchain_html.txt")

# Transform the loaded HTML using BeautifulSoupTransformer
bs_transformer = BeautifulSoupTransformer()
docs_transformed = bs_transformer.transform_documents(
    docs, tags_to_extract=["h3", "p"]
)

# Split the transformed documents using RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(chunk_size=1000, chunk_overlap=0)
splits = splitter.split_documents(docs_transformed)

Please note that depending on your needs, you might want to explore the HTML layout to select the right tags_to_extract. In this demo, I am extracting movie title, genre, rating, and runtime, so I’ll go with the <h3> and <p> tags.

Basically after obtaining the required HTML, we will ask LLM: “Hey LLM, with this HTML, please fill in the information according to the schema below.”

# Define a JSON schema for movie data validation
schema = {
    "properties": {
        "movie_title": {"type": "string"},
        "stars": {"type": "integer"},
        "genre": {"type": "array", "items": {"type": "string"}},
        "runtime": {"type": "string"},
        "rating": {"type": "string"},
    },
    "required": ["movie_title", "stars", "genre", "runtime", "rating"],
}

def extract_movie_data(content: str, schema: dict):
    """
    Extract movie data from content using a specified JSON schema.

    Parameters:
    - content (str): Text content containing movie data.
    - schema (dict): JSON schema for validating the movie data.

    Returns:
    - dict: Extracted movie data.
    """
    # Run the extraction chain with the provided schema and content
    start_time = time.time()
    extracted_content = create_extraction_chain(schema=schema, llm=llm).run(content)
    end_time = time.time()

    # Log metadata and output in the Comet project for tracking purposes
    comet_llm.log_prompt(
        prompt=str(content),
        metadata= {
            "schema": schema
        },
        output= extracted_content,
        duration= end_time - start_time,
    )

    return extracted_content

And finally, we’ve got the result:

# Extract movie data using the defined schema and the first split page content
extracted_content = extract_movie_data(schema=schema, content=splits[0].page_content)

# Display the extracted movie data
extracted_content

Final Output:

There’s a lot to discover about LLMs. Want more topics on LLM? Share your thoughts in the comments!

• How to Create a Simple Chatbot for E-commerce Using OpenAI
• How to Enhance Conversational Agents with Memory in Lang Chain

❗A Heads-up

Web scraping is not as simple as you might think. You may face challenges along the way.

1. Dynamic website structures: Modern websites use dynamic JavaScript structures and require tools like Selenium for accurate data extraction.
2. Anti-scraping: IP blocks and CAPTCHAs are common deterrents. Strategies such as IP address rotation can help overcome these obstacles.
3. Legal and Ethical Considerations: To avoid legal repercussions, it is important to follow the website’s terms of use. Prioritize ethical practices and ask for permission.
4. Data quality and consistency: Maintaining data quality while updating a website is an ongoing challenge. Update your scraping script regularly to ensure accurate and reliable extraction.

Conclusion

Web scraping is worth a try. It may not seem like a formal job, but it’s actually a lot of fun using Python, JavaScript, and HTML. There are many ways to collect information from websites. I hope you will try more than just web scraping by implementing LLM techniques.

#WebScraping #DataExtraction #DataMining #WebDevelopment #LLM #LangChain

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletter (Deep Learning Weekly), check out the Comet blog, join us on Slack, and follow Comet on Twitter and LinkedIn for resources, events, and much more that will help you build better ML models, faster.

Web Scraping With 5 Different Methods: All You Need to Know was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.

Table of Contents

I. LLM Chains

II. What is exactly Chaining in LLMOps and is it essential?

1. The Basics of Chaining

2. Core Chains in LLMOps

2.1. Prompt Chain

2.2. Tools Chain

2.3. Data Indexing Chain

2.4. Best Practices for Core Chains

III. Lang Chain: The Glue that Connects

💡I write about Machine Learning on Medium || Github || Kaggle || Linkedin. 🔔 Follow “Nhi Yen” for future updates!

👉 Get UNLIMITED access to every story on Medium with just $1/week — HERE

You may have come across terms like OpenAI, LLM, LLM Chain, LangChain, Llama Index, etc., but this is causing a bit of confusion. Let’s try to simplify things by understanding the basics of LLM and LLM Chains, what Chaining is.

I. LLM Chains

Using OpenAI models as an example, there are two distinct approaches to interacting with language models: Direct LLM Interface and the LLMChain Interface. Let’s look at the characteristics of each and decide which one to choose.

1. Direct LLM Interface: Simplifying Ad-Hoc Tasks

Essentially, LLM is a foundational class for interacting with language models. LLM streamlines the process by handling important low-level tasks such as rapid tokenization, API calls, and error handling.

Let’s illustrate this simplicity with a snippet:

llm = OpenAI(temperature=0.9)    
if prompt:            
    response = llm("Who is Donald Trump?")
    st.write(response)

LLM is ideal for basic language processing tasks because you can pass messages and start interacting with language models in just a few lines. By using instances of OpenAI classes directly, users can enter prompts with a variety of structures, making them suitable for flexible or ad hoc tasks. This model generates responses without requiring a predefined request format.

👉 The article below is a showcase of using Direct LLM Interface. In the project, I used OpenAI GPT-3.5 Turbo model for natural language processing and comet_llm to keep track of what users ask, how the bot responds, and how long each interaction takes.

How to Create a Simple Chatbot for E-commerce Using OpenAI

👉 Check out Comet in action by looking through my previous hands-on projects:

• Enhancing Customer Churn Prediction with Continuous Experiment Tracking
• Hyperparameter Tuning in Machine Learning: A Key to Optimize Model Performance
• The Magic of Model Stacking: Boosting Machine Learning Performance
• Logging — The Effective Management of Machine Learning Systems

2. LLMChain Interface: Level up the Game with Additional Logic

Now let’s use LLMChain to level up. LLMChain is a sophisticated chain that encapsulates LLM and enhances its functionality with additional features. Think about message formatting, input/output analysis, and conversation management. LLMChain covers everything.

The snippet below showcases its prowess:

from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

template = "Write me something about {topic}"
topic_template = PromptTemplate(input_variables=['topic'], template=template)
topic_chain = LLMChain(llm=llm, prompt=topic_template)
if prompt:    
 response = topic_chain.run(question)
 st.write(response)

LLMChain is like a very smart tool that works in conjunction with LLM to facilitate complex language tasks. In the example above, using LLMChain with the PromptTemplate class adds an extra layer to keep things organized, especially for structured tasks. This method is best when you want a consistent format for your prompts. PromptTemplate allows you to define a structured message with specific details, and LLMChain uses this template to generate a response using the underlying model.

Choose LLMChain if you have a task that requires a specific request format. In addition to PromptTemplate, LLMChain provides additional functionality and allows you to enhance your tasks with features such as pre-processing and post-processing steps. I will explain it in detail in the next paragraphs in this article.

II. What is exactly Chaining in LLMOps and is it essential?

1. The Basics of Chaining

Chains are the secret sauce that transforms ordinary LLM interactions into powerful applications. When using LLMs such as ChatGPT, most people stick to a simple question and answer scenario. But to take things to the next level, developers are turning to chains, the process of connecting different components to enhance the functionality of an LLM.

2. Core Chains in LLMChain

In LLMChain, 03 core chains form the backbone of the entire process: prompt chain, tools chain, and data indexing chain. These chains work together seamlessly to create a matrix-like structure that allows developers to connect and power different components.

2.1. Prompt Chain

Prompt chains are the starting point for LLMOps, where a developer creates specific instructions or prompts for LLM. It guides the language model to understand the desired user interaction and sets the context for subsequent chains. Creating effective prompts is critical to getting accurate and appropriate responses from your LLM.

👉 I also mentioned about “Guidelines for Developers on Prompting” in the article “Create a Simple E-commerce Chatbot Using OpenAI + CometLLM.” It’s useful for getting a deeper understanding of the Prompt Chain.

2.2. Tool Chain

The Tool Chain includes various development tools that enhance the functionality of LLM.

Components:

• Extractive Tools: Used to extract specific information from the LLM’s responses.
• Transformative Tools: Responsible for transforming and refining the output generated by LLMs.
• Utility Tools: Additional tools that add functionalities like formatting, summarization, etc.
• Integration: Tools are integrated to ensure that the output from LLMs aligns with the desired application requirements.

2.3. Data Indexing Chain

The data indexing chain is the backbone for efficiently storing and retrieving information from documents.

Process:

• Chunking: Documents are split into manageable chunks for efficient processing.
• Embeddings: These chunks are converted into embeddings, which are numerical representations suitable for storage in vector databases.
• Vector Database: Specialized databases such as Pinecone, Chroma Database, Weaviate, FAISS, Milvus, Qrant, and Elasticsearch store these embeddings for fast and effective retrieval. These databases store document embeddings in a way that allows for fast and efficient retrieval. Although it may seem complicated, it is essentially a specialized database that is optimized to handle large amounts of data.

Role:

• You can search large amounts of data quickly, contributing to the efficiency of the LLMOps applications.

2.4. Best Practices for Core Chains

• Balanced Prompts: Creating prompts that balance specificity and generality will help LLMs provide useful and diverse responses.
• Tool Selection: Selecting and integrating tools that complement the required functionality of your application is critical to a smooth LLMOps workflow.
• Optimized indexing: Tuning the data indexing process for efficient storage and retrieval can significantly improve application performance.

III. LangChain: The Glue that Connects

I believe you all have heard of LangChain. It is an innovative tool for LLMOps that serves as a general-purpose interface for interacting with LLM. This acts as a connector, allowing developers to combine LLMs with other tools and databases. The key idea behind Lang Chain is to go beyond simple question-and-answer interactions and create applications that leverage the full potential of LLM.

LLMOps seamlessly integrates LLM, vector databases, and other tools around Lang Chain. The chaining process involves using a developer tool stack to connect models, databases, etc. to ultimately create a powerful LLM application.

Let’s take a look at a simplified version of LangChain below.

In addition to LangChain, the following important Chain players should be kept in mind:

• LangChain: Raised approximately $10 million in seed funding.
• Llama Index: A remarkable open source tool.
• HayStack: Received approximately $9.2 million in seed funding and debt financing.
• AgentGPT : Level AI’s open source project, backed by a generous $20 million Series B funding in 2022, focused on call center applications.

👉 You might be interested in “LangChain Conversation Memory Types” in another article.

How to Enhance Conversational Agents with Memory in Lang Chain

Conclusion

To wrap it up, Chaining is the secret sauce to make LLM super powerful in everyday uses. LangChain is the top pick for developers wanting to make the most out of LLMOps, thanks to its flexible interface and easy connections. Digging deep into Chaining and LLM Developer tools can set you apart in job searches. Keep following for more insights and learning opportunities. Cheers!

#LLM #LLMChain #LLMOps #ChainingDecoded #DifferentiateLLM

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletter (Deep Learning Weekly), check out the Comet blog, join us on Slack, and follow Comet on Twitter and LinkedIn for resources, events, and much more that will help you build better ML models, faster.

Decoding the Significance of Chaining in LLMOps was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.

When it comes to chatbots and conversational agents, the ability to retain and remember information is critical to creating fluid, human-like interactions. This article describes the concept of memory in Lang Chain and explores its importance, implementation, and various strategies for optimizing conversation flow.

💡I write about Machine Learning on Medium || Github || Kaggle || Linkedin. 🔔 Follow “Nhi Yen” for future updates!

👉 Get UNLIMITED access to every story on Medium with just $1/week — HERE

👉 I previously shared relevant articles on creating a basic chatbot without using Conversation Memory. You might find it interesting.

Create a Simple E-commerce Chatbot With OpenAI

Why Memory Matters in Conversational Agents

When users interact with chatbots, they often expect a level of continuity and understanding similar to human conversations. This expectation includes the ability to refer to past information, which leads to the conversational agent’s need for memory. Memory allows the system to remember previous interactions, process abbreviations, and perform co-reference resolution, ensuring consistent, context-aware conversations.

The Memory Challenge in Large Language Models

Large language models like Lang Chain do not have their own memory. Unlike humans, who naturally retain information during conversations, these models operate on a rapid response mechanism. Attempts have been made to integrate memory into its Transformer model, but large-scale practical implementation remains a challenge.

Memory Strategies in Lang Chain

Given a context that when a customer inquires about the customer service of a fashion store and expresses a problem with the jeans. This issue involves a stuck zipper and is similar to a hardware issue. I’ll ask the bot a list of questions for each LangChain memory type:

1. Hey! I am Nhi.
2. How are you today?
3. I'm doing well, thank you. I need your assistance.
4. I bought this pair of jeans, and there's a problem. 
5. When I tried them on, the zipper got stuck, and now I can't unzip it.
6. It seems to be a problem with the zipper.

In this experiment, I’ll use Comet LLM to record prompts, responses, and metadata for each memory type for performance optimization purposes. This allows me to track response duration, tokens, and cost for each interaction. View the full project HERE.

Comet LLM provides additional features such as UI visualization, detailed chain execution logs, automatic tracking with OpenAI chat model, and user feedback analysis.

Find the complete code in this GitHub Repository.

To start, bring in the common libraries needed for all 6 memory types. Make sure you’ve installed the necessary Python packages in requirements.txt and have your OpenAI API and Comet API keys ready. (Reference: OpenAI Help Center — Where can I find my API key?; CometLLM — Obtaining your API key)

import os
import dotenv
import comet_llm
from langchain.callbacks import get_openai_callback
import time
dotenv.load_dotenv()

MY_OPENAI_KEY = os.getenv("YOUR_OPENAI_KEY")
MY_COMET_KEY = os.getenv("YOUR_COMET_KEY")

# Initialize a Comet project
comet_llm.init(project="YOUR COMET PROJECT NAME",
               api_key=MY_COMET_KEY)

1. Conversation Buffer Memory

The simplest form of memory involves the creation of a talk buffer. In this approach, the model keeps a record of ongoing conversations and accumulates each user-agent interaction into a message. While effective for limited interactions, scalability becomes an issue for long conversations.

🎯 Implementation: Include the entire conversation in the prompt.

✅ Pros: Simple and effective for short interactions.

❌ Cons: Limited by token span; impractical for lengthy conversations.

from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain_openai import OpenAI
from langchain.chains import ConversationChain
from langchain.callbacks import get_openai_callback

llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
             temperature=0,
             max_tokens = 256)

buffer_memory = ConversationBufferMemory()

conversation = ConversationChain(
    llm=llm,
    verbose=True,
    memory=buffer_memory
)

Interact with the bot by inputting each prompt in conversation.predict(). Example:

conversation.predict(input="Hey! I am Nhi.")

conversation.predict(input="How are you today?")

conversation.predict(input="I'm doing well, thank you. I need your assistant.")

Output after the 3rd prompt:

> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: Hey! I am Nhi.
AI:  Hi Nhi! My name is AI. It's nice to meet you. What can I do for you today?
Human: How are you today?
AI:  I'm doing great, thanks for asking! I'm feeling very excited about the new projects I'm working on. How about you?
Human: I'm doing well, thank you. I need your assistant.
AI:

> Finished chain.
' Sure thing! What kind of assistance do you need?'

ConversationBufferMemory allows conversations to grow with each turn and allows users to see the entire conversation history at any time. This approach allows ongoing interactions to be monitored and maintained, providing a simple but powerful form of memory for language models, especially in scenarios where the number of interactions with the system is limited.

print(conversation.memory.buffer)

Output:

Human: Hey! I am Nhi.
AI:  Hi Nhi! My name is AI. It's nice to meet you. What can I do for you today?
Human: How are you today?
AI:  I'm doing great, thanks for asking! I'm feeling very excited about the new projects I'm working on. How about you?
Human: I'm doing well, thank you. I need your assistant.
AI:  Sure thing! What kind of assistance do you need?

Use Comet LLM to complete the code and record relevant information for analysis purposes.

def conversation_memory_buffer(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {conversation.memory.buffer}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_buffer_memory",
            "history": conversation.memory.buffer,
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

Call the function conversation_memory_buffer():

Example:
conversation_memory_buffer("Hey! I am Nhi.")
conversation_memory_buffer("How are you today?")
conversation_memory_buffer("I'm doing well, thank you. I need your assistant.")

How the data is logged in Comet LLM:

2. Conversation Summary Memory

To overcome the scalability challenge, Conversation Summary Memory provides a solution. Rather than accumulating each interaction, the model generates a condensed summary of the essence of the conversation. This reduces the number of tokens and increases the sustainability of long-term interactions.

🎯 Implementation: Summarize the conversation to save tokens.

✅ Pros: Efficient token usage over time.

❌ Cons: May lose fine-grained details; suitable for concise interactions.

from langchain.chains.conversation.memory import ConversationSummaryMemory
from langchain import OpenAI
from langchain.chains import ConversationChain
from langchain.callbacks import get_openai_callback

# Create an instance of the OpenAI class with specified parameters
llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
            temperature=0,
            max_tokens = 256)

# Create an instance of the ConversationSummaryMemory class
summary_memory = ConversationSummaryMemory(llm=OpenAI(openai_api_key=MY_OPENAI_KEY))

# Create an instance of the ConversationChain class, combining OpenAI, verbose mode, and memory
conversation = ConversationChain(
    llm=llm,
    verbose=True,
    memory=summary_memory
)

Similar to ConversationBufferMemory, we’ll also interact with the bot by inputting each prompt in conversation.predict(). Example:

conversation.predict(input="Hey! I am Nhi.")

conversation.predict(input="How are you today?")

Output:

> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Nhi introduces themselves to the AI and the AI responds with a greeting, revealing its name to be AI. The AI offers to assist Nhi with any tasks and asks how it can help them today. The human then asks how the AI is doing, to which it responds that it is doing great and asks how it can be of assistance.
Human: I'm doing well, thank you. I need your assistant.
AI:

> Finished chain.
' Absolutely! How can I help you today?'

The ConversationSummaryMemory generates the conversation summary over multiple interactions.

Current conversation:
Nhi introduces themselves to the AI and the AI responds with a greeting, revealing its name to be AI. The AI offers to assist Nhi with any tasks and asks how it can help them today. The human then asks how the AI is doing, to which it responds that it is doing great and asks how it can be of assistance.

The summaries capture key points of the conversation, including introductions, queries, and responses. As the conversation progresses, the model generates summarized versions, emphasizing that it tends to use fewer tokens over time. The summary’s ability is to retain essential information, making it useful for reviewing the conversation as a whole.

Additionally, we can access and print out the conversation summary.

print(conversation.memory.buffer)

Output:

Nhi introduces themselves to the AI and the AI responds with a greeting, revealing its name to be AI. The AI offers to assist Nhi with any tasks and asks how it can help them today. The human asks how the AI is doing, to which it responds that it is doing great and asks how it can be of assistance. The human informs the AI that they need its assistance and the AI eagerly offers to help.

Use Comet LLM to log important information for analysis purposes in your code.

def conversation_summary_memory(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {conversation.memory.buffer}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_summary_memory",
            "history": conversation.memory.buffer,
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

Call the function conversation_memory_buffer():

Example:
conversation_summary_memory("Hey! I am Nhi.")
conversation_summary_memory("How are you today?")
conversation_summary_memory("I'm doing well, thank you. I need your assistant.")

3. Conversation Buffer Window Memory

Conversation Buffer Window Memory is an alternative version of the conversation buffer approach, which involves setting a limit on the number of interactions considered within a memory buffer. This balances memory depth and token efficiency, and provides flexibility to adapt to conversation windows.

🎯 Implementation: Retain the last N interactions.

✅ Pros: Balances memory and token constraints.

❌ Cons: Potential loss of early conversation context.

from langchain.chains.conversation.memory import ConversationBufferWindowMemory
from langchain import OpenAI
from langchain.chains import ConversationChain
from langchain.callbacks import get_openai_callback

# Create an instance of the OpenAI class with specified parameters
llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
            model_name='text-davinci-003',
            temperature=0,
            max_tokens = 256)

# Create an instance of the ConversationBufferWindowMemory class
# We set a low k=2, to only keep the last 2 interactions in memory
window_memory = ConversationBufferWindowMemory(k=2)

# Create an instance of the ConversationChain class, combining OpenAI, verbose mode, and memory
conversation = ConversationChain(
    llm=llm,
    verbose=True,
    memory=window_memory
)

Note that in this demo we will set a low value k=2 , but you can adjust it depending on the memory depth you need. Four prompts are introduced into the model in the following order:

conversation.predict(input="Hey! I am Nhi.")

conversation.predict(input="I'm doing well, thank you. I need your assistant.")

conversation.predict(input="I bought this pair of jeans, and there's a problem.")

conversation.predict(input="When I tried them on, the zipper got stuck, and now I can't unzip it.")

In the demo, the prompt contains an ongoing conversation, and the model tracks a set number of recent interactions. However, only the last 02 interactions are considered, so the previous interactions are not preserved.

Memory depth can be adjusted based on factors such as token usage and cost considerations. We compare our approach with previous approaches and emphasize selectively preserving recent interactions.

print(conversation.memory.buffer)

Output:

Human: I bought this pair of jeans, and there's a problem.
AI:  Oh no! What kind of problem are you having with the jeans?
Human: When I tried them on, the zipper got stuck, and now I can't unzip it.
AI:  That sounds like a tricky problem. Have you tried lubricating the zipper with a bit of oil or WD-40? That might help loosen it up.

Use Comet LLM to log important information for analysis in your code:

def conversation_buffer_window_memory(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {conversation.memory.buffer}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_buffer_window_memory",
            "history": conversation.memory.buffer,
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

Call the function conversation_buffer_window_memory():

Example:
conversation_buffer_window_memory("Hey! I am Nhi.")
conversation_buffer_window_memory("How are you today?")
conversation_buffer_window_memory("I'm doing well, thank you. I need your assistant.")

4. Conversation Summary Buffer Memory: A Combination of Conversation Summary and Buffer Memory

Conversation Summary Buffer Memory keeps a buffer of recent interactions in memory, but compiles them into a digest and uses both, rather than just removing old interactions completely.

🎯 Implementation: Merge summary and buffer for optimal memory usage.

✅ Pros: Provides a comprehensive view of recent and summarized interactions.

❌ Cons: Requires careful tuning for specific use cases.

from langchain.chains.conversation.memory import ConversationSummaryBufferMemory
from langchain import OpenAI
from langchain.chains import ConversationChain
from langchain.callbacks import get_openai_callback


# Create an instance of the OpenAI class with specified parameters
llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
            temperature=0,
            max_tokens = 512)

# Create an instance of the ConversationSummaryBufferMemory class
# Setting k=2: Retains only the last 2 interactions in memory.
# max_token_limit=40: Requires the installation of transformers.
summary_buffer_memory = ConversationSummaryBufferMemory(llm=OpenAI(openai_api_key=MY_OPENAI_KEY), max_token_limit=40)

# Create an instance of the ConversationChain class, combining OpenAI, verbose mode, and memory
conversation = ConversationChain(
    llm=llm,
    memory=summary_buffer_memory,
    verbose=True
)

Feed the model with 4 prompts in an order:

conversation.predict(input="Hey! I am Nhi.")

conversation.predict(input="I bought this pair of jeans, and there's a problem.")

conversation.predict(input="When I tried them on, the zipper got stuck, and now I can't unzip it.")

conversation.predict(input="It seems to be a problem with the zipper.")

The model retains the last 2 interactions and summarize the older ones as below:

> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
System: 
Nhi introduces herself to the AI, and the AI responds by introducing itself and expressing pleasure in meeting Nhi. Nhi mentions a problem with a pair of jeans she bought, and the AI asks for more details. Nhi explains that the zipper got stuck and now she can't unzip it.
AI:  That sounds like a tricky situation. Have you tried lubricating the zipper with a bit of oil or soap? That might help it move more smoothly.
Human: It seems to be a problem with the zipper.
AI:

> Finished chain.
' It sounds like the zipper is stuck. Have you tried lubricating it with a bit of oil or soap? That might help it move more smoothly.'

Print out the result:

print(conversation.memory.moving_summary_buffer)

Output:

Nhi introduces themselves to the AI and the AI responds by introducing itself and offering assistance. Nhi explains a problem with a pair of jeans they bought, stating that the zipper is stuck and they can't unzip it. The AI suggests lubricating the zipper with oil or soap to help it move more smoothly or replacing it if that doesn't work.

Use Comet LLM in your code to log important information for analysis:

def conversation_summary_buffer_memory(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {conversation.memory.moving_summary_buffer}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_summary_buffer_memory",
            "history": conversation.memory.moving_summary_buffer,
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

Call the function conversation_summary_buffer_memory():

Example:
conversation_summary_buffer_memory("Hey! I am Nhi.")
conversation_summary_buffer_memory("I bought this pair of jeans, and there's a problem.")
conversation_summary_buffer_memory("When I tried them on, the zipper got stuck, and now I can't unzip it.")
conversation_summary_buffer_memory("It seems to be a problem with the zipper.")

5. Conversation Knowledge Graph Memory

Lang Chain goes beyond just conversation tracking and introduces knowledge graph memory. It builds a mini knowledge graph based on related information, creating nodes and connections to represent key entities. This method improves the model’s ability to understand and respond to situations.

🎯 Implementation: Extract relevant information and construct a knowledge graph.

✅ Pros: Enables structured information extraction.

❌ Cons: May require additional processing for complex scenarios.

Python Code example:

from langchain.chains.conversation.memory import ConversationKGMemory
from langchain import OpenAI
from langchain.chains import ConversationChain
from langchain.prompts.prompt import PromptTemplate
from langchain.callbacks import get_openai_callback

# Create an instance of the OpenAI class with specified parameters
llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
            temperature=0,
            max_tokens = 256)

# Define a template for conversation prompts
template = """
This is a friendly chat between a human and an AI. The AI shares details from its context. \n
If it doesn't know the answer, it honestly admits it. The AI sticks to information in the 'Relevant Information' section and doesn't make things up. \n\n

Relevant Information: \n

{history}

Conversation: \n
Human: {input} \n
AI:"""

# Create an instance of the PromptTemplate class with specified input variables and template
prompt = PromptTemplate(input_variables=["history", "input"], template=template)

# Create an instance of the ConversationKGMemory class with the OpenAI instance
kg_memory = ConversationKGMemory(llm=llm)

# Create an instance of the ConversationChain class, combining OpenAI, verbose mode, prompt, and memory
conversation = ConversationChain(
    llm=llm,
    verbose=True,
    prompt=prompt,
    memory=kg_memory
)

def conversation_kg_memory(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {str(conversation.memory.kg.get_triples())}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_kg_memory",
            "history": str(conversation.memory.kg.get_triples()),
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

6. Entity Memory

Similar to Knowledge Graph memory, entity memory extracts specific entities from the conversation, such as names, objects, or locations. This focused approach aids in understanding and responding to user queries with greater precision.

🎯 Implementation: Identify and store specific entities from the conversation.

✅ Pros: Facilitates extraction of key information for decision-making.

❌ Cons: Sensitivity to entity recognition accuracy.

from langchain import OpenAI, ConversationChain
from langchain.chains.conversation.memory import ConversationEntityMemory
from langchain.chains.conversation.prompt import ENTITY_MEMORY_CONVERSATION_TEMPLATE
from pydantic import BaseModel
from typing import List, Dict, Any
from langchain.callbacks import get_openai_callback

# Print the template used for conversation prompts
ENTITY_MEMORY_CONVERSATION_TEMPLATE.template
print(ENTITY_MEMORY_CONVERSATION_TEMPLATE.template)

# Create an instance of the OpenAI class with specified parameters
llm = OpenAI(openai_api_key=MY_OPENAI_KEY,
            temperature=0,
            max_tokens = 256)

# Create an instance of the ConversationEntityMemory class with the OpenAI instance
entity_memory = ConversationEntityMemory(llm=llm)

# Create an instance of the ConversationChain class, combining OpenAI, verbose mode, prompt, and memory
conversation = ConversationChain(
    llm=llm,
    verbose=True,
    prompt=ENTITY_MEMORY_CONVERSATION_TEMPLATE,
    memory=entity_memory
)

Create a function to get the response and log important metadata using CometLLM:

def conversation_entity_memory(prompt):
    with get_openai_callback() as cb:
        start_time = time.time()
        response = conversation.predict(input=prompt)
        end_time = time.time()
        print(f"Response: {response}")
        print(f"History: {conversation.memory.entity_store.store}")

        # Log to comet_llm
        comet_llm.log_prompt(
            prompt=prompt,
            output=response,
            duration= end_time - start_time,
            prompt_template = conversation.prompt.template,
            metadata={
            "memory_type": "conversation_entity_memory",
            "entity_cache": conversation.memory.entity_cache,
            "history": conversation.memory.entity_store.store,
            "total_tokens": cb.total_tokens,
            "prompt_tokens": cb.prompt_tokens,
            "completion_tokens": cb.completion_tokens,
            "total_cost_usd": cb.total_cost
            },
        )

Call the function:

conversation_entity_memory("Hi, this is Nhi. I bought this pair of jeans, and there's a problem. When I tried them on, the zipper got stuck, and now I can't unzip it.")

The function returns the history, the matching response using Entity memory, and also logs metadata in Comet LLM as shown below.

Reference

• Find the complete code on this GitHub repository.
• Experiment Tracking in Commet LLM Project.

👉 Explore Comet in action by reviewing my past hands-on projects.

• Enhancing Customer Churn Prediction with Continuous Experiment Tracking
• Hyperparameter Tuning in Machine Learning: A Key to Optimize Model Performance
• The Magic of Model Stacking: Boosting Machine Learning Performance
• Logging — The Effective Management of Machine Learning Systems

Conclusion

Memory plays a fundamental role in enhancing the capabilities of conversational agents. By understanding and implementing Lang Chain’s different memory strategies, you can create more dynamic and context-sensitive chatbots. Whether you choose a buffer-based approach or leverage a knowledge graph, the key is to tailor your memory mechanisms to your specific use case and user expectations.

As the field of Natural Language Processing (NLP) continues to evolve, integrating memory into conversational agents remains a promising avenue for improving user experience.

#ConversationalAgents #LangChainMemory #ChatbotTips #LangChainConversations #MemoryOptimization

How to Enhance Conversational Agents with Memory in Lang Chain was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.