<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Meena on Medium]]></title>
        <description><![CDATA[Stories by Meena on Medium]]></description>
        <link>https://medium.com/@meena.08.ch?source=rss-e309faf143a9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YB5ZIzmD5ZMXIuglxWAUlg.jpeg</url>
            <title>Stories by Meena on Medium</title>
            <link>https://medium.com/@meena.08.ch?source=rss-e309faf143a9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:17:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@meena.08.ch/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[EDA and Feature Engineering in Machine Learning]]></title>
            <link>https://medium.com/@meena.08.ch/eda-and-feature-engineering-in-machine-learning-3ec0d1e5026d?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ec0d1e5026d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Tue, 17 Jun 2025 08:58:26 GMT</pubDate>
            <atom:updated>2026-09-21T04:01:09.722Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploratory Data Analysis and Feature Engineering in Machine Learning</h3><p>Let’s start with a general scenario. A person who is a beginner in baking wants to make peanut butter chocolate chip cookies for his daughter and he follows the baking steps.</p><p>He mindfully gets all the ingredients in the right quantity. Here getting ingredients is called the<strong> collection of raw data</strong>.</p><p>He changed his mind and decided to bake just the peanut butter cookies. Now he may not need all the ingredients, he would remove chocolate from the raw ingredients. This is called <strong>data cleansing</strong>.</p><p>He separates out the dry ingredients and wet ingredients and then starts mixing them separately. This is called the <strong>preprocessing (transformation) of data</strong> before the actual baking step.</p><p>Finally he mixed the preprocessed portions together and baked the cookies at right temperature in an oven.</p><p>We will not say anything about the taste of the baked cookies :)</p><blockquote>But we can clearly see that the output has a big dependency on the <strong>preprocessing step</strong> (with data cleansing)</blockquote><h3>7 Steps of Machine Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*j__Cwe_YV9eheGy_mMThmA.png" /></figure><p>Before applying a machine learning model, one needs to first clearly understand what is the given problem statement or identify the problem within the given dataset. What are the key contributing features in the raw data that are also called independent variables and what is the dependent variable(s) or output feature.</p><h3>Preprocessing Steps</h3><p>In the data preprocessing step, we identify and remove data inconsistencies like <strong>outliers or null removal</strong> from the data. Handle data imbalance, missing data and data skewness issues. Convert categorical variables into the numerical format for example (0, 1).</p><p>In this step we also <strong>identify the relevant independent features</strong>, <strong>filter out the unnecessary features</strong>, t<strong>ransform or standardize some of the relevant features.</strong></p><blockquote>In order to achieve this first we do <strong>exploratory data analysis</strong> and then perform <strong>feature engineering.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*NIh5YZXjGthWBvcgMygJug.png" /></figure><h3><strong>1st — Exploratory Data Analysis</strong></h3><p><strong>EDA provides data and visuals</strong> that clearly shows the data distribution trends, central tendencies, variability and correlation between features.</p><p>One can visualize data distribution and relationship between features using various types of charts, such as histograms, correlation heatmap, scatter plot, box plots and bar chart etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*bNeDfaQPsYkQBXrbkK6qAA.png" /></figure><p><strong>EDA helps in generation of new hypothesis</strong> use cases as it reflects the trends and relationships amongst the features.<strong> </strong>It helps the business strategist to build the right strategies early on.</p><blockquote>Note: Hypothesis describes <strong>a tentative statement about the relationship between two or more features.</strong></blockquote><blockquote><strong>Examples</strong></blockquote><blockquote>More study hours can contribute to better academic performance or better marks.</blockquote><blockquote>A strong GDP (Gross Domestic Product) growth rate correlates with higher property prices due to increased job creation, rising incomes, and stronger consumer confidence</blockquote><p><strong>EDA helps in identifying data quality issues</strong> such as missing values, outliers, and inconsistencies like data skewness and imbalances.</p><p>We can perform <strong>EDA </strong>using any of these data analysis and visualization tools like <strong>Excel, Tableau, Power BI, Python (</strong>use libraries such as <em>pandas</em> for d<strong>ata manipulation</strong>, <em>numpy</em> for <strong>numerical operations</strong>, <em>matplotlib and seaborn</em> for <strong>data visualization) or Apache Spark</strong> (For Big Data Analysis) etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*I_nPNgQx0k9GRDYWWPvOPA.png" /></figure><h3>2nd — Feature Engineering</h3><p>Definition wise feature engineering in machine learning is the process of transforming raw data into a format that machine learning algorithms can effectively use to solve prediction or classification problems.</p><blockquote>It involves extracting and transforming relevant features from raw data to improve model performance and solve issues like missing values or non-numeric categorical data.</blockquote><h4><strong>Key aspects of feature engineering</strong></h4><p><strong>a) Feature Creation</strong><br>Generating new features from existing ones. It can be done by transformation of independent variables to capture more relevant information.</p><p><strong>b) Feature Transformation</strong><br>Modifying existing features to make them suitable for the machine learning model. This includes data scaling, normalization, or applying one hot encoding on the categorical data.</p><p>To handle categorical variables convert them into a numerical format by using one hot encoding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LPfl3hDXfXV1YvRkFencbA.png" /></figure><p>We can handle missing values in the dataset by using <strong>data imputation techniques</strong> like filling missing values with the mean or median.</p><blockquote><strong>Note: Data imputation is the process of replacing missing values in a dataset with substituted values.</strong></blockquote><p>To handle Scaling and Normalization, apply techniques like standardization or min-max scaling to ensure all features have a similar scale.</p><p>Data with varying scales needs to be standardized as the significantly changing range of values of different features may impact a machine learning model performance and accuracy. It should be handled properly.</p><blockquote>The features with larger ranges can influence the model, leading to biased or inaccurate predictions. Hence we apply feature scaling techniques like normalization or standardization.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*Sd844N31EkWNU8iTRYi1cQ.png" /></figure><p>In the model training and testing phase we use the standardized column instead of the original salary column.</p><p>By addressing issues like missing values and encoding categorical data, feature engineering can improve a model’s ability to generalize to new or unseen data in the prediction stage.</p><p><strong>c) Feature Extraction and Selection</strong></p><p>Use Principal Component Analysis (PCA) algorithm to avoid overcrowding of features. Using PCA, we can reduce the number of data dimensions and extract only relevant features.</p><p>We can perform feature engineering on a dataset using python libraries like Pandas, Scikit-learn, FeatureTools, Feature-engine, Category encoders, and more.</p><p><a href="https://www.udemy.com/course/exploratory-data-analysis-in-data-science/?referralCode=C4FAEC5E4A4BF3B54CFC&amp;couponCode=26BBPFEO2MXTRMT">https://www.udemy.com/course/exploratory-data-analysis-in-data-science/?referralCode=C4FAEC5E4A4BF3B54CFC&amp;couponCode=26BBPFEO2MXTRMT</a></p><h3>Benefits of Feature Engineering in ML Model Performance</h3><blockquote><strong>It improves the model performance and prevents overfitting problems in the test phase of model execution.</strong></blockquote><blockquote><strong>It helps in reducing the model training time and improves the model accuracy.</strong></blockquote><p><a href="https://youtu.be/uNfc7n2UqYU">Linear Regression Model</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ec0d1e5026d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a LLM ChatBot using Streamlit]]></title>
            <link>https://medium.com/@meena.08.ch/create-a-llm-chatbot-using-streamlit-7a5043005f39?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a5043005f39</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[streamlit]]></category>
            <category><![CDATA[build-a-chatbot-using-llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Sat, 03 May 2025 07:21:10 GMT</pubDate>
            <atom:updated>2025-07-11T07:29:24.900Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to make ChatBot with Gen-AI and Streamlit</h3><h3><strong>What are ChatBots</strong></h3><p>Chatbots are like virtual assistants that can answer simple questions which helps the user to handle wide range of tasks. The technical purpose of chatbot is to automate the conversation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*18ZfZ_FpavHObLF29iYotg.png" /></figure><h3>Day-to-day use of ChatBots</h3><p>Booking online appointments, handling routine customer queries</p><p>Order processing — managing orders, tracking shipments, status updates to customers</p><p>Personalized experience — based on the user queries, bots can provide personalized recommendations, product information etc.</p><h3>How to make a ChatBot</h3><p>A ChatBot can be build using large language model (LLMs), LangChain Framework and Streamlit open source Python Library.</p><p>Reference — <a href="https://medium.com/@meena.08.ch/langchain-for-llm-application-development-f1c94d0bdcf1"><em>LangChain Story</em></a></p><p><strong>Challenge with LLM Model Execution — </strong>LLM process each question as a standalone task, it generates output solely based on the current input question. It simplifies the model architecture but poses challenges for LLM applications that require context continuity. For instance a chatbot requires conversation state management. The problem is — “<strong>How to do session state management in a chat-bot application</strong>”</p><h3>Use Streamlit to solve the session state management problem</h3><p>Streamlit provides an easy and faster way to create custom machine learning and data-science apps. It is widely used by AI and ML engineers in the development of data applications.</p><p>State management or conversation persistence of User questions and AI responses can be very easily handled using <strong>Streamlit <em>session_state</em></strong><br>For each user session, it provides a way to share variables between reruns.</p><pre>if &#39;chat_history&#39; not in st.session_state:<br>    st.session_state[&#39;chat_history&#39;] = []<br>    st.session_state[&quot;chat_history&quot;].append(&quot;You: &quot;+ input)<br>    st.session_state[&quot;chat_history&quot;].append(&quot;Bot: &quot;+ response)</pre><p><strong>Configuration Steps</strong></p><ol><li>Configure the default settings of the page, <br><em>st.set_page_config(page_title = “QnA Demo”)</em></li><li>Use <em>st.subheader()</em> function for generating subheadings of the application <em>st.subheader(“The response is: “)</em></li><li>Use <em>write()</em> function to display almost anything, including text, data frames, charts, and more. All with a single line of code<br><em>st.write(response)</em></li></ol><h4>ChatBot.py code</h4><pre>from langchain.chains.llm import LLMChain<br>from langchain_core.prompts import ChatPromptTemplate<br>from langchain_core.output_parsers import StrOutputParser<br><br>from langchain_groq import ChatGroq<br>import streamlit as st<br><br>def main():<br><br>    ##initialize the llm model with the api key<br>    groq_chat_llm = ChatGroq(model=&quot;Gemma2-9b-It&quot;, groq_api_key=groq_api_key)<br>    ## Create Prompt template<br>    prompt_template = ChatPromptTemplate.from_messages(<br>        [(&quot;human&quot;, &quot;{input}&quot;)]<br>    )<br><br>    def get_groq_response(user_question):<br>        ## Create LLM Chain and get response<br>        chain = prompt_template | groq_chat_llm | StrOutputParser()<br>        response = chain.invoke(input=user_question, stream = True)<br>        return response<br>    <br>    ## Streamlit steps<br>    st.set_page_config(page_title = &quot;QnA Demo&quot;)<br>    st.header(&quot;GenAI LLM Application&quot;)<br><br>    if &#39;chat_history&#39; not in st.session_state:<br>        st.session_state[&#39;chat_history&#39;] = []<br><br>    input = st.text_input(&quot;Input:&quot;, key=&quot;input&quot;)<br>    submit = st.button(&quot;Ask the question&quot;)<br>    if submit and input:<br>        response = get_groq_response(input)<br>        st.session_state[&quot;chat_history&quot;].append(&quot;You: &quot;+ input)<br>        st.subheader(&quot;The response is: &quot;)<br><br>        st.write(response)<br>        st.session_state[&quot;chat_history&quot;].append(&quot;Bot: &quot;+ response)<br>        st.subheader(&quot;The chat history is: &quot;)<br><br>        for text in st.session_state[&quot;chat_history&quot;]:<br>            st.write(text)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><h4>How to run Streamlit app</h4><p>To run a Streamlit app, simply use the command <em>streamlit run [your_app.py]</em> in your terminal.<br>This command will launch a local Streamlit server, and your app will open in a new browser tab. <br>C:\..\anaconda3&gt;streamlit run C:\project\llm_apps\chatbot.py</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fvglij_-ix9za4sfoELyEQ.png" /><figcaption><strong>ChatBot App</strong></figcaption></figure><blockquote><em>With little customization, we can build a RAG based LLM ChatBot that solves a business purpose.</em></blockquote><p><a href="https://medium.com/@meena.08.ch/rag-technique-combine-gen-ai-llms-external-knowledge-base-15f455a21ca2">RAG Technique</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a5043005f39" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create Documents Chain in LangChain]]></title>
            <link>https://medium.com/@meena.08.ch/create-documents-chain-in-langchain-e871100bd255?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e871100bd255</guid>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Fri, 28 Mar 2025 14:28:41 GMT</pubDate>
            <atom:updated>2025-06-27T16:33:48.767Z</atom:updated>
            <content:encoded><![CDATA[<p>Large language models (LLMs) can <strong><em>process the text in the form of tokens</em></strong>. They have<strong> <em>context window</em> or<em> context length</em> </strong>for these tokens that the model can remember at the time of execution.</p><h4>Background</h4><p><em>LLM pricing is based on token usage</em>, where a token is a unit of text the model process, that means the cost of using an LLM depends on the number of input and output tokens used.</p><blockquote><strong>A prompt, natural language conversations (eg. Q and A chat bots), document summarization or code base shall not exceed LLM model’s context window size.</strong></blockquote><p><em>Note : LLM token limit can be set. It vary depending on the specific model and its provider.</em></p><h4><strong>1. create_stuff_documents_chain</strong></h4><blockquote>Create Stuff Documents Chain takes a list of <strong>small</strong> documents and first combines them into a single context window. It is a simple way to combine documents for question-answering, summarization, and other purposes.</blockquote><p><em>from langchain.chains.combine_documents import create_stuff_documents_chain</em></p><p>It can be easily be integrated with the other langchain components using <a href="https://medium.com/@meena.08.ch/lcel-to-build-chain-of-langchain-components-87235335e7f5">LCEL</a>.</p><h4><strong>Usecase — Text summarization with create_stuff_documents_chain</strong></h4><p>Create Stuff Documents Chain takes a list of documents and combines them into a single string.</p><p><strong>A)</strong> It does this by formatting each document into a string with the document_prompt and then joining them together with document_separator.</p><p><strong>B)</strong> It then adds that new string to the inputs with the variable name set by <em>document_variable_name</em>. The <em>document_variable_name</em> is used for the formatted documents in the prompt. Default name is <em>“context”.</em> These inputs are then passed to the llm_chain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*PvPI9xaTUWTmbmyk3uYlFA.png" /><figcaption>Design of Stuff Documents Chain</figcaption></figure><pre>from langchain.chains.combine_documents import create_stuff_documents_chain<br>from langchain_core.prompts import ChatPromptTemplate<br><br>loader=WebBaseLoader(<br>    web_paths=(&quot;https://lilianweng.github.io/posts/2023-06-23-agent/&quot;,),<br>    bs_kwargs=dict(parse_only=bs4.SoupStrainer(class_=(&quot;post-tags&quot;))),<br>    )<br>document = loader.load()<br><br>##initialize llm model with the api key<br>llm = ChatGroq(model=&quot;Gemma2-9b-It&quot;, groq_api_key=&lt;&lt;groq_api_key&gt;&gt;)<br><br># Define prompt, the prompt contains a task that you ask from the Large Language Model<br># &quot;context&quot; is the document_variable_name<br>prompt_template = ChatPromptTemplate.from_messages(<br>    [(&quot;system&quot;, &quot;Write a concise summary of the following:\\n{context}&quot;)]<br>)<br><br>## Instantiate stuff document chain and pass llm and prompt to it<br>llmchain = create_stuff_documents_chain(llm, prompt_template)<br><br># Invoke the llm chain with the document object<br>result = llmchain.invoke({&quot;context&quot;: document})<br>print(result)</pre><h4>2. map_reduce_documents_chain</h4><p>For a large or multiple documents, stuff documents chain may result in a prompt that exceeds the context length.</p><blockquote><strong><em>MapReduce</em></strong> is a programming model design that processes large volume of data in a distributed, parallel manner. The <strong><em>map_reduce_documents_chain</em></strong> is a scalable way to handle large volume of text documents.</blockquote><p>With this approach, any task is broken down into multiple sub-tasks. Each sub-task is processed in parallel. Summary result is aggregated across all the completed tasks.</p><h4>Usecase — Text Summarization with MapReduceDocumentsChain</h4><p><strong>A)</strong> To summarize long documents, first split the documents into smaller parts called chunks that are suitable for token context length of the LLM</p><p><strong>B)</strong> Then a <em>map-reduce-chain</em> can summarize these chunks independently before recombining. This scales summarization to any length of text while controlling chunk size. In the recombining step as part <em>ReduceDocumentsChain</em>, we use the StuffDocumentsChain to combine the summary results. This step runs iteratively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*F0kGY4NT2WLlQPb5Xtfw0Q.png" /><figcaption>Design of Map Reduce Documents Chain</figcaption></figure><pre>from langchain.chains import MapReduceDocumentsChain, ReduceDocumentsChain<br>from langchain.chains.combine_documents.stuff import StuffDocumentsChain<br>from langchain.chains.llm import LLMChain<br>from langchain_core.prompts import ChatPromptTemplate<br>from langchain_community.document_loaders import PyPDFLoader<br><br>pdf_file_path = &quot;Orders_Feedback.pdf&quot;<br>loader = PyPDFLoader(pdf_file_path)<br>documents = loader.load_and_split()<br><br>##initialize llm model with the api key<br>llm = ChatGroq(model=&quot;Gemma2-9b-It&quot;, groq_api_key=&lt;&lt;groq_api_key&gt;&gt;)<br><br># We define the prompt templates and chain for the map and reduce steps<br># Instantiate separate chains for these steps, and finally instantiate <br># the MapReduceDocumentsChain<br>map_prompt_template = ChatPromptTemplate.from_messages(<br>    [(&quot;human&quot;, &quot;Write a concise summary of the following:\\n{context}&quot;)]<br>)<br><br>reduce_prompt_template = ChatPromptTemplate.from_messages(<br>    [(&quot;human&quot;, &quot;The following is a set of summaries: {final_summary} <br>              Take these and summarize it into a consolidated summary<br>              of the main themes.&quot;)])<br><br>map_chain = LLMChain(llm = llm, prompt = map_prompt_template)<br>reduce_chain = LLMChain(llm = llm, prompt = reduce_prompt_template)<br><br># Initialize StuffDocumentsChain for aggregating the summary results by using <br># reduce_chain with reduce_prompt_template<br># The document_variable_name is used for the formatted documents in the prompt.<br>stuff_documents_chain = StuffDocumentsChain(<br>    llm_chain = reduce_chain, document_variable_name = &quot;final_summary&quot;<br>)<br><br># Initialize ReduceDocumentsChain that combines and iteratively reduces<br># the mapped documents using stuff_documents_chain<br>reduce_documents_chain = ReduceDocumentsChain(<br>    # This is final chain that is called.<br>    combine_documents_chain = stuff_documents_chain,<br>    # If documents exceed context for `StuffDocumentsChain`<br>    collapse_documents_chain = stuff_documents_chain,<br>    # The maximum number of tokens to group documents into.<br>    token_max=1000,<br>)<br><br># Initialize MapReduceDocumentsChain by using map_chain and<br># reduce_documents_chain. It combines documents by mapping a chain over them, <br># then combines results<br># The document_variable_name is used for the formatted documents in the prompt.<br>map_reduce_chain = MapReduceDocumentsChain(<br>    # Map chain<br>    llm_chain = map_chain,<br>    # Reduce chain<br>    reduce_documents_chain = reduce_documents_chain,<br>    # The variable name in the llm_chain to put the documents in<br>    document_variable_name=&quot;context&quot;,<br>    # Return the results of the map steps in the output<br>    return_intermediate_steps=False,<br>)<br><br>result = map_reduce_chain.invoke(documents)<br>print(result[&quot;output_text&quot;])</pre><p>Reference: <a href="https://python.langchain.com/docs/versions/migrating_chains/map_reduce_chain/">Map Reduce Chains</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e871100bd255" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LCEL : To build chain of Langchain components]]></title>
            <link>https://medium.com/@meena.08.ch/lcel-to-build-chain-of-langchain-components-87235335e7f5?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/87235335e7f5</guid>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Fri, 21 Mar 2025 02:12:27 GMT</pubDate>
            <atom:updated>2025-05-23T12:35:48.378Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to build a chain of LangChain components : LCEL</h3><h3>Background</h3><p>An application based on LangChain framework is composed of multiple elements starting from the user’s question in the form of prompts to the LLM model’s output which is context based.</p><blockquote>In order to handle AI components of the language model together in the form of a chain we follow declarative programming approach.</blockquote><blockquote>This approach is called <strong><em>LangChain Expression Language (LCEL).</em></strong></blockquote><p><strong>It is a program control flow solution used for handling runtime execution of custom chain components in an optimal manner.</strong></p><h3>LCEL as Runnable Sequence</h3><p>The chain elements in a declarative programming approach declares only the high level components without describing the component’s control flow.</p><blockquote>The primitive component here is a <em>Runnable</em> interface which acts as the building block of the chain.</blockquote><blockquote>The LCEL adapts declarative approach to build new runnable components from the existing runnable(s). These runnable components are combined using a pipe operator<em> ( | ).</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*Z26_1WguBygS9VtB9SI0HA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*MRZWxhGyuqcN8rPWQOpa0g.png" /></figure><p>The Runnable interface can be chained together into sequences. <br>The output of the previous runnable.invoke() call is passed as input to the next runnable. This can be done using the <em>pipe </em>operator.</p><p>In image-1,</p><p><em>PromptTemplate</em> implements the standard <em>Runnable i</em>nterface.</p><p><em>ChatGroq </em><strong><em>LLM </em></strong>implements the standard <em>Runnable i</em>nterface.</p><p><em>StrOutputParser</em> implements the standard <em>Runnable i</em>nterface.</p><h4>Code</h4><pre>from langchain_groq import ChatGroq<br><br># Create LLM object and prompt<br>llm = ChatGroq(model=&quot;&lt;&lt;model_name&gt;&gt;&quot;,groq_api_key=&quot;&lt;&lt;api_key&gt;&gt;&quot;)<br><br>from langchain.prompts import PromptTemplate<br>prompt = PromptTemplate.from_template(&quot;Tell me more about the given topic: {topic}.&quot;)<br><br># A LCEL chain combines prompt, llm, and output parser<br>from langchain_core.output_parsers import StrOutputParser<br>lcelchain = prompt | llm | StrOutputParser()<br><br># Execute the chain to generate content about the given topic<br>response = lcelchain.invoke({&quot;topic&quot;: &quot;Climate Change Causes&quot;})<br>print(response)</pre><h3>Important Points</h3><p>LCEL provides ease to break down the application steps into logical components that follows single responsibility principle with well defined purpose.</p><p>These components are to be made stateless to maximize reusability.</p><p>These components should be configurable so that the customization of the component is feasible.</p><p>Exception, error handling, logging and monitoring process should be integrated to improvise performance and reliability KPIs of an application.</p><h3>LCEL Advanced Features</h3><blockquote>LCEL can handle async requests, optimized parallel execution, retries and fallback mechanism to make reliable chains, access to intermediate results, input output validations based on schema definition.</blockquote><p><a href="https://python.langchain.com/docs/concepts/lcel/">https://python.langchain.com/docs/concepts/lcel/</a></p><h3>LCEL integration with LangSmith</h3><blockquote>Observability is one of the KPI (key performance indicator) of a business application. It is very important to trace the response time, error, resource usage of application components.</blockquote><blockquote>All steps of <strong><em>LCEL </em></strong>are logged to <strong><em>LangSmith </em></strong>for observability purpose.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87235335e7f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG Technique : Combine Gen AI LLMs & External Knowledge Base]]></title>
            <link>https://medium.com/@meena.08.ch/rag-technique-combine-gen-ai-llms-external-knowledge-base-15f455a21ca2?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/15f455a21ca2</guid>
            <category><![CDATA[code]]></category>
            <category><![CDATA[generative-ai-solution]]></category>
            <category><![CDATA[generative-ai-application]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Mon, 17 Mar 2025 12:37:40 GMT</pubDate>
            <atom:updated>2025-06-28T14:41:14.931Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build a Retrieval-Augmented Generation (RAG) system</h3><h3><strong><em>Challenge with Large Language Models</em></strong></h3><p><strong>LLMs</strong> have huge reliance on the training data. Training data-set is static in nature and it often causes bias in the LLM response.</p><p>Outdated data which is not from a reliable source-system shows biased or wrong output response and it is a critical problem.</p><p>Business domains rely on their own data as they are best positioned to understand, manage, and utilize it effectively. Such domain oriented approach is used in the context of data management. Business centric domain teams own the latest dataset and are responsible for the data quality.</p><p>In order to use LLM capabilities with their own knowledge base they can leverage RAG design approach.</p><h3>RAG — Combine LLMs &amp; External Knowledge Base</h3><blockquote><strong>RAG</strong> approach forces the LLM model to provide response based on the provided content or the knowledge base.</blockquote><blockquote>The knowledge base is in the form of vector data.</blockquote><p>The user’s perspective of an application is mostly domain centric and RAG approach solves the business user’s objectives.</p><h4>Features of RAG Based Applications</h4><p><strong>A) RAGs </strong>are very useful in building bots or apps that can answer the <em>business users</em> or <em>end users</em> questions that are relevant to the specific domain(s).</p><p>B) RAGs design approach provides raw <em>business centric data, feeds or frequently updated and trusted knowledge</em> to the Generative-AI LLMs.</p><p>C) The LLM takes the updated knowledge and provides response in the form of natural language to the users.</p><h4>RAG Vector Embeddings</h4><p>The raw information resides in a vector store in the form of embeddings. The technique is split into two parts:</p><ol><li><em>Retriever based response from the </em><strong><em>Knowledge Base</em></strong></li><li><em>Generator based response from LLM endpoint in natural language</em></li></ol><p>The below mentioned design is<strong> a <em>simple sequential demonstration of a basic RAG.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*2M3mWLj2uW79thxz3z5njA.png" /><figcaption><strong>RAG Design</strong></figcaption></figure><h4>Sample Code</h4><pre>!pip install langchain_community<br>!pip install langchain_huggingface<br>!pip install jq<br>!pip install chromadb<br>! pip install langchain_groq<br>from langchain_community.document_loaders import JSONLoader<br>from langchain_community.vectorstores import Chroma<br>from langchain_huggingface import HuggingFaceEmbeddings<br>from langchain_core.output_parsers import StrOutputParser<br>from langchain_core.runnables import RunnableLambda, RunnablePassthrough<br><br>from langchain_core.prompts import ChatPromptTemplate<br>## provide HF_TOKEN for the embeddings<br>os.environ[&#39;HF_TOKEN&#39;] = HF_TOKEN<br>embeddings = HuggingFaceEmbeddings(model_name=&quot;all-MiniLM-L6-v2&quot;)<br><br>## provide groq_api_key<br>from langchain_groq import ChatGroq<br>llm=ChatGroq(model=&quot;Llama3-8b-8192&quot;,groq_api_key=&quot;groq_api_key&quot;)<br><br>## use a json file as input data-set<br>loader = JSONLoader(file_path=&quot;shopping.json&quot;,jq_schema=&quot;.[]&quot;, text_content=False)<br><br>documents = loader.load()<br>## print(documents)<br><br>##populate the document in the Vector DB using the HuggingFaceEmbeddings model<br>db = Chroma.from_documents(documents, embeddings)<br><br>##user query <br>query = &quot;What are the grocery items?&quot;<br><br>retriever = db.as_retriever()<br><br>template = &quot;&quot;&quot;Answer the question based only on the following context:{context}<br>              Question: {question} &quot;&quot;&quot;<br>prompt = ChatPromptTemplate.from_template(template)<br>## integrate groq model to retrieve query output from the vector db<br>chain = (<br>    {&quot;context&quot;: retriever, &quot;question&quot;: RunnablePassthrough()}<br>    | prompt<br>    | llm<br>    | StrOutputParser()<br>    )<br>print(chain.invoke(query))</pre><h3>Advantages</h3><blockquote>Integration of LLM with the trusted knowledge base response enables up-to-date LLM response for the functional purpose.</blockquote><blockquote>Its design can be enhanced to restrict sensitive information retrieval.</blockquote><blockquote>It can be configured, to reuse for the broad range of use-cases which is scalable across domains.</blockquote><p><strong><em>RAG basic design can be enriched with native cloud services or cloud agnostic services, LLM models and vector stores.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15f455a21ca2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use LangChain Framework In LLM Applications]]></title>
            <link>https://medium.com/@meena.08.ch/langchain-for-llm-application-development-f1c94d0bdcf1?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1c94d0bdcf1</guid>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Mon, 10 Mar 2025 13:16:20 GMT</pubDate>
            <atom:updated>2025-05-23T13:01:34.446Z</atom:updated>
            <content:encoded><![CDATA[<h3>Application Components &amp; LLM challenges</h3><p><strong>Any application</strong> is comprised of some of the basic <strong>key components</strong> and <strong>features</strong> such as</p><ol><li>It must be well integrated with the input and output data source and sink.</li><li>Data source must be reliable without any bias <strong>otherwise LLMs may struggle with accuracy issue.</strong></li><li>Output validation is required, <strong>to prevent non-compliance and its associated risks. </strong>LLM outputs should be rigorously validated for correctness in order<strong> to achieve compliance and standards.</strong></li><li>Application parts mostly require to solve series of steps to arrive at the final solution. <strong>While LLMs cannot directly solve problems requiring multiple reasoning steps due to its inability to take actions.</strong></li></ol><h3><strong>LangChain</strong></h3><blockquote>LangChain is an open-source python framework that provides the key capabilities to solve above LLM limitations and makes LLM powered application development easier.</blockquote><p><strong>Main components</strong></p><p><strong>P</strong>rompt Templates, <strong>C</strong>hains, <strong>I</strong>ndexes, <strong>M</strong>emory, <strong>T</strong>ools and <strong>A</strong>gents</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*cXMC2_3Hz5QR2wpzS_Db7w.png" /></figure><h4><strong>a) Prompts and Prompt Templates</strong></h4><p>User’s can interact with LLMs using questions and these are called <strong>prompts.</strong></p><p><strong>Prompt template</strong> class in LangChain helps in building context based instructions or prompts. Prompt template strings have placeholders that helps in construction of dynamic inputs.</p><blockquote>prompt = PromptTemplate.from_template(“Tell me more about the given topic: {topic}.”)</blockquote><h4>b) <strong>Chains</strong></h4><p>Chains are workflows of LangChain that combine prompt template, LLM and other components as a sequence of functions or steps for execution purpose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*p3bNolPjBzr4pBEX.png" /></figure><p><a href="https://medium.com/@meena.08.ch/lcel-to-build-chain-of-langchain-components-87235335e7f5"><strong>Refer : LCEL</strong></a></p><h4>c) Agents</h4><p>LangChain helps in building conversational context and its persistence through <strong>Agents and Memory.</strong></p><blockquote><strong><em>Agents uses LLMs to request an action based on one or a series of prompt instructions.</em></strong></blockquote><blockquote><strong><em>Agents can decide which tool to use and when.</em></strong></blockquote><p>In the reference code below,</p><p>Step 1: Create an agent using LCEL<br>Step 2: The input to LCEL are: <em>tool</em>, <em>input </em>questions, <em>agent scratch pad</em> information, and <em>chat history</em> in the form of dictionary object</p><blockquote>Note : Chat history is a record of the conversation between the user and the chat model. It is used to maintain context and state throughout the conversation. The chat history is sequence of messages, each of which is associated with a specific role for instance “user” or “system”</blockquote><p>Step 3: Invoke <strong><em>agent LCEL</em></strong> using <strong><em>agent_executor </em></strong>runnable</p><h4>d) Tools</h4><p>Tools have broad categories like tools for online Search, Web browsing, Code Interpretation, Productivity and Databases tools to automate tasks in databases etc. These are used by Agents or LLMs to interact with the world.</p><blockquote>Eg : LLMs can extract information, perform data analysis using Wikipedia Retrievers and LLM powered search engines.</blockquote><h4><strong>LangChain Code — To show basic usage of agents and tools</strong></h4><pre>from langchain_groq import ChatGroq<br>from langchain.agents import create_react_agent<br>from langchain.agents import Tool<br>from langchain_core.prompts import PromptTemplate<br>from langchain_experimental.utilities import PythonREPL<br><br>from langchain.agents.format_scratchpad import format_to_openai_function_messages<br>from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser<br>from langchain.agents import AgentExecutor<br><br>##ChatGroq model - as LLM processing or language processing unit<br>##Langchain framework provides the python library to run ChatGroq model &quot;Llama3-8b-8192&quot;<br><br>llm=ChatGroq(model=&quot;Llama3-8b-8192&quot;,groq_api_key=&lt;&lt;groq_api_key&gt;&gt;)<br><br>## Use REPL as LLM agent tool<br>## REPL stands for Read Eval Print Loop, an interactive programming<br>## environment that allows us to run python code interactively in real time<br>python_repl = PythonREPL()<br>repl_tool = Tool(<br>    name=&quot;python_repl&quot;,<br>    description=&quot;A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.&quot;,<br>    func=python_repl.run,<br>)<br><br>## LLMs are stateless — meaning each incoming query is processed independently of other interactions.<br>## The only thing that exists for a stateless agent is the current input, nothing else.<br>## The agent scratchpad is where the agent writes its notes as it works through multiple internal thought<br>## and tool-use steps to produce a final output for the user<br>## Hence in the prompt template specify the tool, input, also placeholders for <br>## chat_history messages with agent scratchpad<br>prompt = ChatPromptTemplate.from_messages([<br>    (&quot;user&quot;, &quot;Answer the following questions as best you can.<br>                         You have access to the following tools: {tools}&quot;),<br>    MessagesPlaceholder(variable_name=&quot;chat_history&quot;),<br>    (&quot;user&quot;, &quot;{input}&quot;),<br>    MessagesPlaceholder(variable_name=&quot;agent_scratchpad&quot;),<br>])<br><br>## Create an agent using LCEL<br>## The input to LCEL are: tool name, input questions, agent scratch pad <br>## information, chat history in the form of dictionary<br>agent = ({&quot;tools&quot;: lambda x: x[&quot;tool_name&quot;], &quot;input&quot;: lambda x: x[&quot;input&quot;],<br>        &quot;agent_scratchpad&quot;: lambda x: format_to_openai_function_messages(x[&quot;intermediate_steps&quot;]),<br>        &quot;chat_history&quot;: lambda x: x[&quot;chat_history&quot;],} | prompt | llm | OpenAIFunctionsAgentOutputParser())<br><br>if __name__ == &quot;__main__&quot;:<br>  agent_executor = AgentExecutor(agent=agent, tools=[repl_tool], verbose=True)<br>  user_task = [&quot;What is 12900+ 19999?&quot;, &quot;Double the previous output&quot;,<br>                    &quot;Summarize chat history questions and answers&quot;]<br>  # Initialize chat_history to an empty list<br>  chat_history = []<br>  # Invoke agent LCEL using agent_executor runnable<br>  output = agent_executor.invoke({&quot;tool_name&quot;: &quot;repl_tool&quot;, &quot;input&quot;: user_task, &quot;chat_history&quot;: chat_history})<br>  print(output[&#39;output&#39;])</pre><h4>e) Memory</h4><p>One of the key limitation of Agents and Chains is their statelessness. In order to persist conversation information LLM applications require memory.</p><blockquote><em>ConversationalBufferMemory can be used </em>to store messages in the model history, Database SQL and NoSQL options, In-memory database for caching support.</blockquote><p><strong>RAG (<em>Retrieval-Augmented Generation</em>)</strong> based applications in LLMs require memory for storing the knowledge base or document data in the form of embeddings in a <strong>Vector Store.</strong></p><blockquote><em>Embeddings are numerical representation of vectors</em></blockquote><blockquote><em>Vector Store is a form of memory </em>(Check the RAG Story)</blockquote><blockquote>Such options helps in building context aware <strong><em>LLM Agents</em>.</strong></blockquote><p><a href="https://medium.com/@meena.08.ch/rag-technique-combine-gen-ai-llms-external-knowledge-base-15f455a21ca2"><strong><em>Refer: the Design of RAG (Retrieval Augmented Generation)</em></strong></a></p><h4>f) Indexing</h4><p>LangChain indexing helps you to load and sync document information from any data-source into a vector store.</p><p>It avoids writing duplicate content, avoids re-writing unchanged content, avoids recomputing embeddings over unchanged content. It keeps track of document writes in the Vector Store.</p><h3>LangGraph</h3><p>LangGraph is an control flow framework of LangChain for creating agents and multi-agent workflows.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1c94d0bdcf1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Natural Language Processing works — LLM]]></title>
            <link>https://medium.com/@meena.08.ch/llms-for-natural-language-processing-4268b312c4b2?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/4268b312c4b2</guid>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Sun, 09 Mar 2025 12:24:07 GMT</pubDate>
            <atom:updated>2025-05-27T12:46:51.338Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Generative AI Large Language Models (LLMs)</strong> are <strong>advanced statistical tools</strong> as it follows the principles of statistics to analyze vast amount of data and identify patterns in natural language.</p><blockquote>LLM can predict the next word in a sentence. To accomplish this, statistical probabilities are determined for each of the words that came before, to make its next prediction.</blockquote><blockquote>In natural language processing the prebuilt statistical language models play vital role <strong>to predict future words.</strong></blockquote><h3>Natural Language Processing (NLP)</h3><p>The<strong> language models</strong> have <strong>deep learning transformer based architecture.</strong> These models are <em>context centric</em> and are <em>formed from self-attention mechanism</em>. The attention mechanism enables the model to focus on various parts of the natural language input sequence.</p><p>In this process, importance is assigned to every word or phrase in the sentence.</p><p>Let’s consider this importance criteria as numerical weight value and it calculates weight for a word in the sentence with respect to the weights of every other word in the sentence. It processes input word sequence in parallel both during training and while predicting the output words.</p><blockquote>Example - In a sentiment analysis, weights are assigned to words like <strong>peaceful</strong> or <strong>horrible</strong> to indicate their positive or negative sentiment.</blockquote><p>Technically, the transformer based architecture follows the fundamental process of <strong>encoding and decoding of sentences.</strong></p><blockquote><em>Encoding</em> is the process of putting words into a vector representation.</blockquote><blockquote>Decoding is the reverse process of conversion of an encoded format back into the original sequence of characters.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*DMwQ-bKObkGCjBKYBnnoUQ.png" /><figcaption><strong>Transformer Model</strong></figcaption></figure><p>T<strong>ransformer based language models</strong> is the <strong>model of choice for NLP tasks,</strong> mainly due to the efficiency of these models.</p><p><em>Before transformers, RNNs particularly LSTMs were the preferred approach to deal with sequential data like natural language.</em></p><h3><strong>LLMs used in Natural Language Understanding, Processing and Generation tasks</strong></h3><p><strong>Use-cases</strong> — Conversational Chatbots, Sentiment Analysis, Language Translation, Content Creation, Code Generation and NLP</p><h3>LLM Providers</h3><blockquote>OpenAI’s GPT series, Google’s BERT, HuggingFace Open Source Text Generation LLM Ecosystem, MetaAI’s LLaMA, Anthropic Claude etc</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4268b312c4b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generative AI]]></title>
            <link>https://medium.com/@meena.08.ch/generative-ai-56e1ac363259?source=rss-e309faf143a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/56e1ac363259</guid>
            <category><![CDATA[beginner]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Meena]]></dc:creator>
            <pubDate>Wed, 05 Mar 2025 07:03:17 GMT</pubDate>
            <atom:updated>2025-09-10T05:37:50.109Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>What is Generative AI</strong></h3><p>Generative AI is a type of artificial intelligence that solves the purpose of creating new content. It is used for automating a series of complex tasks that are repetitive in nature and thus helps in increasing efficiency and productivity of organization business.</p><h4>How Generative AI Works</h4><p>Generative AI work by using models that learn patterns from vast amounts of data. The data can be of any modality that is <strong><em>text, images, audio, video, and time series data</em></strong><em>.</em></p><p><strong>Gen AI </strong>is a type of artificial intelligence that generates new content.</p><ol><li>Summarizing document, content creation like article, stories and generates virtual world</li><li>To generate music, audio and video. Creation of synthetic data from the existing data.</li><li>Automate repetitive and administrative tasks and many more</li></ol><p>“A question may arise”, what are conventional AI models ?</p><h3>Conventional AI</h3><p><strong>In Conventional AI, </strong>we use mostly labelled data. As an important step we optimize the machine learning model algorithm. So that the ML model is used for human like decision making tasks.</p><p>In general terms, <strong>these models </strong>are optimized to perform specific task. Examples :</p><ol><li><strong>ML models </strong>used for regression, classification.</li><li>D<strong>eep learning</strong> neural network models that are used for the classification, regression and other complex tasks.</li></ol><h3><strong>Advancement in AI</strong></h3><h4><strong>Conventional ML models</strong> analyze and act on the existing data in a <strong>task centric approach</strong>.</h4><p>These models are generalized as,<strong> <em>P(y | x)</em></strong> where<em> ‘</em><strong><em>x’ </em></strong>stands for features and ‘<strong><em>y’</em> </strong>stands for the output or dependent variables.</p><p>A) <strong>Regression tasks </strong>— By using the historical sales data of a product, train a model that can forecast the future sales by analyzing and formulating the data patterns.</p><p>In the regression example forecasting the future sales is the task.</p><p>B)<strong> Classification tasks </strong>— For an online retailer, predict which product a particular customer is most likely to buy. For this we receive previous orders data of the customers.</p><p>In the classification example predicting the likelihood of a customer’s choice of product is the task.</p><p>C) <strong>Deep learning models like transformer models</strong> are a type of neural network models that process sequential data, like text. A self-attention mechanism used in order to understand the relationships and context between different parts of the sequence text.</p><p>One of the usage is in <strong>natural language processing</strong>, the transformer model enable a machines to understand, interpret, and generate human language. Example: virtual assistants like Alexa use transformer technology to understand and respond to voice commands.</p><h4><strong>Generative AI ML models learn from the entire feature space, these are designed for multipurpose applications.</strong></h4><p>They are also known as <strong>comprehensive models </strong>as the<strong> </strong>model work by learning features from vast dataset.</p><p><strong>**Generative AI models are a subset of deep learning models.</strong> As<strong> transformer</strong> which is a type of deep learning neural network model forms the foundation for most modern generative AI models. Majorly due to their ability to process large sequences of data in parallel.</p><h3><strong>LLMs</strong></h3><p>The improvements in the <strong>transformer models</strong> has enabled creation of Generative AI models, particularly <strong>Large Language Models also known as LLMs</strong>.</p><p><strong><em>Example </em></strong>— <strong><em>The GPTs of the world</em></strong></p><p><strong>LLMs</strong> are <strong><em>foundational models</em></strong> or <strong>language models</strong> that understands and generates natural language and content of other modalities. These models are trained on the vast amount of data and hence they have many parameters.</p><p>These language models are series of G<strong>enerative Pre-trained Transformers (GPTs)</strong> that are finetuned for<strong> conversational applications</strong>.</p><p>LLM based AI systems perform tasks like text translation, writing different creative content, answering questions ( chatbot), and engaging in conversations etc.</p><h4>Concept</h4><ul><li>ML is a subset of AI where machines learn from data</li><li>DL is a further specialized subset of ML focusing on complex data modeling using neural networks</li><li>Generative AI Models like LLMs are the modified form of DL transformers that creates new, original content based on vast amount of data</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*9VeNXUAEgacWhGYc-3qAdw.png" /></figure><h3>LLMs</h3><p>OpenAI’s GPT models (like GPT-3.5 and GPT-4)</p><p>Google’s Gemini and PaLM</p><p>Meta’s LLaMA models</p><p>Anthropic’s Claude and many more</p><h3>Finetuning</h3><blockquote><strong><em>Large Language models can be fine-tuned for specific tasks and can be guided by prompt engineering.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56e1ac363259" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>