<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ashadi Methasa De Silva on Medium]]></title>
        <description><![CDATA[Stories by Ashadi Methasa De Silva on Medium]]></description>
        <link>https://medium.com/@methasa?source=rss-21260e3197df------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*z5osw5XJg72NwijV.jpg</url>
            <title>Stories by Ashadi Methasa De Silva on Medium</title>
            <link>https://medium.com/@methasa?source=rss-21260e3197df------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 11:14:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@methasa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Rethinking Concurrency for I/O with Virtual Threads]]></title>
            <link>https://medium.com/@methasa/rethinking-concurrency-for-i-o-with-virtual-threads-123eea56cd53?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/123eea56cd53</guid>
            <category><![CDATA[virtual-threads]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[jdk21]]></category>
            <category><![CDATA[project-loom]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 13:42:33 GMT</pubDate>
            <atom:updated>2026-09-11T13:42:33.782Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/734/0*FJc3tlDM8LLc7DWG.jpg" /></figure><p>Concurrency has been a key component in ensuring high performance and scalability of today’s backend application systems. To allow the parallel execution of these tasks traditionally thread pools with platform threads have been adapted. However, as the concurrency increases this can lead to pool saturation of I/O bound threads in waiting state causing the increase of queued requests and ultimately increasing the tail latency which can eventually became a bottleneck when large number of requests are waiting on I/O.</p><p><strong>Then if the application is spending most of its time waiting, why should every concurrent operation require an expensive platform thread?</strong></p><p>Java 21 introduced virtual threads as a new concurrency model designed to make high levels of concurrency more accessible. Virtual threads are lightweight threads managed by the JVM, allowing applications to create substantially more concurrent tasks without requiring an equivelant number of operating system threads.</p><p><strong>But does this actually make applications faster?</strong></p><p>The answer is more nuanced than simply saying that virtual threads are “faster” than platform threads. Virtual threads do not reduce the time required for a network request, database query, or downstream API call. Instead, they change the cost of waiting and therefore how an application can scale its concurrency.</p><p>This artical explores the difference between platform and virtual threads, focusing specifically on I/O-bound workloads. It examines the model and execution structure of virtual threads and its impact in efficient resource usage. Finally, we provide a decision framework to select the most suitable approach for your system.</p><p>The goal is not to determine a universal winner, but to understand when virtual threads provide a meaningful advantage, when traditional platform threads remain appropriate, and what limitations still exist regardless of the threading model.</p><h3>Understanding Platform threads</h3><h4>What is a platform thread</h4><p>A platform thread is a Java thread that is backed by an operating-system thread. When a java application creates a platform thread, the JVM typically maps that thread to a native thread managed by the operating system.</p><p>This gives platform threads direct access to the CPU and allows multiple tasks to execute concurrently. However, platform threads are relatively heavywight resources. Creating and maintaining large numbers of them requires memory and operating-system resources, while scheduling between threads also introduces a context-switching overhead.</p><p>This means that although an application can create many platform threads, creating an unlimited number of them is neither practical nor desirable.</p><p>For this reason, Java applications commonly use thread pools to control the number of platform threads available for executing tasks.</p><h4>Traditional thread pool model</h4><p>A thread pool maintains a fixed or bounded number of worker threads that can execute submitted tasks.</p><p>For example, an application might have a pool containing four platform threads:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bJ8EJuJdZqje_OqwCvsEBg.png" /><figcaption>Figure 1 : Thread pool model</figcaption></figure><p>When a task arrives, an available thread executes it. Once the task finishes, the thread becames available to execute another task.</p><p>This model model provides an important benefit: controlled resource usage. Rather than allowing every incoming requests to create its own thread, the application limits the number of concurrent tasks to the size of the pool.</p><p>However, this becomes more complicated when tasks perform blocking I/O.</p><h4>The problem with blocking I/O</h4><p>Consider an API aggregation service that needs to retrieve information from three downstream services:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*7mQBgTjMS187VdScpuH_1w.png" /><figcaption>Figure 2: API aggrgator architecture</figcaption></figure><p>The aggregator can execute these requests concurrently. However, each call may spend a significant portion of its lifetime waiting for a response.</p><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*VwLDwZnfZbyASubF9TNujg.png" /><figcaption>Figure 3 : Platform thread execution workflow</figcaption></figure><p>While the downstream service is processing the request, the platform thread is blocked waiting for the response.</p><p>The important distinction is that the thread is occupied, but the CPU is not necessarily being utilised by that task.</p><p>This becames a scalability problem as the number of concurrent requests increases.</p><p>Suppose an application has a thread pool containing 100 platform threads and each request spends most of its time waiting for downstream services. Once all 100 threads are occupied waiting for I/O, a new request cannot immediately acquire a thread. Depending on the executor configuration, it may have to wait in a queue until another thread becames available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*PbK5T9ck5qjGFAW1rv2BWw.png" /><figcaption>Figure 4: Thread-pool saturation with waiting state threads</figcaption></figure><p>The bottleneck therefore isn’t necessarily the CPU. The application may have available computational capacity while its platform threads are tied up waiting for I/O.</p><p>Increasing the size of the thread pool can provide more concurrency, but it does not solve the fundamental problem. Larger number of platform threads also consume more resources, and beyond a certain point, increasing the pool can introduce additional scheduling and memory overhead.</p><p>This creates a fundamental trade-off:</p><blockquote>More threads allow more concurrent work, but platform threads themselves are relatively expensive resources.</blockquote><p>For CPU-bound workloads, this trade-off is often reasonable because threads are actively using the CPU. For I/O-bound workloads, however, a large proportion of the thread’s lifetime may be spent waiting.</p><p>This is the problem that virtual threads are designed to address: how can an application support a large number of concurrent, potentially blocking operations without requiring a large number of heavyweight platform threads?</p><h3>Virtual Threads</h3><p>Rather than making applications abandon the familiar thread-per-task programming model, virtual threads aim to make that model significantly more lightweight.</p><h4>What are virtual threads</h4><p>A virtual thread is a thread managed by the <strong>Java Virtual Machine(JVM) </strong>rather than being directly represented by a dedicated operating-system thread.</p><p>Unlike platform threads, virtual threads are designed to be extremely lightweight. An application can therefore create a very large number of virtual threads without requiring an equivalent number of operating-system threads.</p><p>For example, instead of restricting an application to a relatively small pool of platform threads, the application can create a virtual thread for each concurrent task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EHe_Exemd70zhvqUwjeuhw.png" /><figcaption>Figure 5: thread per task model with virtual threads</figcaption></figure><p>The important idea is that a<strong> virtual thread is not equivalent to an operating-system thread. </strong>The JVM schedules virtual threads onto a much smaller number of underlying platform threads known as carrier threads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ErIQdY9qyMQ_gWhqNRvW7A.png" /><figcaption>Figure 6: Virtual thread and carrier thread connection architecture</figcaption></figure><p>The JVM is responsible for scheduling these virtual threads onto carrier threads as they became runnable.</p><p>This seperation between logical thread executing the task and the platform thread executing the code at a particular moment is what allows virtual threads to scale concurrency differently from thread pools.</p><h4>How the model differs</h4><p>The key difference is therefore not simply that virtual threads are “faster threads”. <strong>They change the relationship between task, Java threads and operating-system threads.</strong></p><p>With a traditional platform-thread model, an application might have</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ts4M4kteQBeTLc_Dj5RZg.png" /></figure><p>If those tasks are blocked waiting for I/O, those platform threads remain occupied during the wait.</p><p>With virtual threads model, the JVM schedular acts as the primary actor responsible for handling how long each virtual thread is pinned to a carrier thread.</p><p>Initially the a virtual thread is <strong>mounted </strong>to a carrier thread by the JVM schedular. When the virtual thread task reaches a blocking operation, the scheduler would then <strong>unmount</strong> that virtual thread from the carrier thread and mount a different virtual thread. Once the blocking operation is completed, the scheduler would <strong>re-mount</strong> the the virtual thread back into the carrier thread and complete the rest of the execution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vZOzRnSeRIGEzjggrEo8iw.png" /></figure><p>This means the number of concurrent operations no longer needs to correspond directly to the number of platform threads available to the application.</p><p>That is the fundamental shift:</p><blockquote>Platform threads can make threads relatively scarce resources. Virtual threads make threads cheap enough to represent individual concurrent tasks.</blockquote><p>This also means that the programming model can remain relatively straightforward. Developers can continue writing synchronous-looking code that performs blocikng operations rather than having to restructure every operation around asynchronous callbacks or reactive pipelines.</p><h4>Why blocking becomes less expensive</h4><p>Unlike a platform thread that remains tied to the task while it waits, with a virtual thread, the JVM can park the virtual thread while the I/O operation is in progress. The carrier thread that was executing the virtual thread can meanwhile be used to execute other runnable virtual threads.</p><p>This is where the efficiency comes from.</p><p>It is not that the network request finishes faster. If a downstream service task 500milliseconds to response, virtual threads do not magically reduce that to 100 milliseconds.</p><p>Instead, the application can have many operations spending those 500 milliseconds waiting without requiring a dedicated platform thread for every operation.</p><p>For an I/O-bound workload, this can make a significant difference.</p><p>Consider an application recieving over 10,000 concurrent requests, where each request spends most of its lifetime waiting for downstream services.</p><p>A platform-thread approach would require a very large number of platform threads to keep all of those operations concurrently active, which introduces substantial resource costs.</p><p>Virtual threads allow the application to represent those operations as lightweight concurrent tasks while the JVM manages a much smaller set of carrier threads.</p><p>This is why virtual threads are particularly well suited to workloads where the ratio of waiting time to CPU execution is high.</p><p>However, this distinction is important:</p><blockquote>Virtual threads improve the scalability of concurrency; they do not increase the processing speed of the underlying I/O operation.</blockquote><p>If a downstream service, database , or network connection is the actual bottleneck, virtual thread cannot remove that bottleneck. They simply prevent the applicatino’s threads from becaming the bottleneck as quickly.</p><p>This makes virtual threads especially interesting for applications such as API aggregators, web servers, and services that perform many concurrent network of database operations.</p><h3>The Trade-offs</h3><h4>Advantages</h4><ul><li>High concurrency: Thousands of concurrent tasks can be created without requiring an equivelant number of platform threads.</li><li>Lower resource overhead: Virtual threads require significantly less memory than platform threads.</li><li>Simpler programming model: Developers can continue using familiar blocking , sequential-style code rather than rewriting everything using asynchronous callbacks.</li><li>Well suited to I/O-bound workloads: Applications that frequently wait for HTTP requests, databases, or other external services can benefit significantly.</li></ul><h4>Limitations</h4><ul><li>They do not make I/O faster: A slow database or downstream service remains slow.</li><li>CPU-bound workloads still need CPU: Virtual threads cannot create additional CPU capacity for operation intensive tasks. In such cases platform threads are a better solution.</li><li>Some operations can pin a carrier thread: Certain blocking situations can prevent a virtual thread from being unmounted, reducing the expected benefit.</li><li>External resources remain limited: A database may only support a fixed number of connections, regardless of how many virtual threads the application creates.</li></ul><h3>Practical Desision framework</h3><p>Choosing between platform and virtual threads does not need to be complicated. Start by asking what the application spends most of its time doing.</p><ol><li><strong>Is the workload I/O bound?</strong></li></ol><ul><li>Yes: Virtual threads are a strong candidate</li><li>No, mostly CPU bound: Platform threads may be more appropriate, as the bottlneck is more CPU rather than thread availability</li></ul><p><strong>2. Does the application perform many concurrent blocking operations?</strong></p><ul><li>Yes: Virtual threads can significantly improve concurrency.</li><li>No: The benefit may be limited</li></ul><p><strong>3. Are external resources the bottleneck?</strong></p><ul><li>Yes: Virtual threads alone will not solve the problem. Consider connection pools, rate limits, semaphores, or other concurrency controls</li></ul><p>A simple rule of thumb is:</p><blockquote>If your application spends a lot of time waiting, virtual threads are worth considering. If it spends most of its time computing , focus on CPU utilisation instead.</blockquote><h3>Conclusion</h3><p>Virtual threads represent a significant shift in how Java applicatinos can handle concurrency. By allowing many lightweight threads to share a smaller number of carrier threads, they make high levels of concurrency more practical, particularly for I/O-bound applications where tasks spend much more of thier time waiting. However, virtual threads do not make the underly I/O or CPU-intensive operations faster, so they should not be viewed as a universal replacement for platform threads. Instead, they are another tool for developers to choose based on the workload , with their greatest value coming from making concurrent, blocking I/O more scalable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=123eea56cd53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAFT: Revolutionizing Domain-Specific Language Models]]></title>
            <link>https://medium.com/@methasa/raft-revolutionizing-domain-specific-language-models-d61c9c3cee77?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/d61c9c3cee77</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Wed, 11 Jun 2025 14:27:47 GMT</pubDate>
            <atom:updated>2025-06-11T14:27:47.220Z</atom:updated>
            <content:encoded><![CDATA[<p>In the fast-evolving world of artificial intelligence, large language models (LLMs) have become powerful tools for tasks like answering questions, summarizing texts, and generating content. However, adapting these general-purpose models to specialized domains — such as healthcare, legal, or software documentation — remains a challenge. Enter Retrieval-Augmented Fine-Tuning (RAFT), a groundbreaking method developed by researchers at UC Berkeley, in collaboration with Microsoft and Meta, to enhance LLMs for domain-specific tasks. RAFT combines the strengths of Retrieval-Augmented Generation (RAG) and fine-tuning, enabling models to not only access external documents but also learn how to use them effectively. This blog explores how RAFT works, its benefits, real-world applications, and how you can experiment with it yourself.</p><h3>Understanding RAFT</h3><p>To grasp RAFT, we first need to understand its building blocks: RAG and fine-tuning.</p><h4>What is RAG?</h4><p>Retrieval-Augmented Generation (RAG) allows LLMs to answer questions by pulling relevant information from external documents, much like an open-book exam. When a user asks a question, the model retrieves documents that seem related and uses them to generate an answer. However, RAG can struggle with irrelevant or misleading documents, leading to less accurate responses.</p><h4>What is Fine-Tuning?</h4><p>Fine-tuning involves training a pre-trained LLM on a specific dataset to adapt it to a particular domain, like medical literature or legal texts. This process is akin to a student studying a subject deeply but can lead to overfitting, where the model becomes too focused on the training data and struggles with new information and real data.</p><h4>RAFT: The Best of Both Worlds</h4><p>RAFT, or Retrieval-Augmented Fine-Tuning, bridges these two approaches. It trains the model to use RAG more effectively by teaching it to focus on relevant documents and ignore distractions. Think of RAFT as preparing a student for an open-book exam by having them study the textbook beforehand, so they know exactly where to look for answers.</p><h3>How RAFT works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*PxO-ZyDBn_-eDtrTUQQgNQ.png" /></figure><h4>1. Training data synthesis</h4><p>RAFT’s training process relies on a synthetic dataset designed to mimic real-world RAG scenarios. The synthetic data divides into the following main components:</p><ul><li><strong>Question(Q) </strong>— A query that mirrors what the model will encounter in practice, such as “What are the latest treatments for Type 2 diabetes?” in a healthcare context.</li><li><strong>Oracle Documents (D*) — </strong>These contain the relevant information needed to answer the question. There may be one or more oracle documents per question.</li><li><strong>Distractor Documents (Di) — </strong>These are irrelevant documents that do not contain useful information for answering the question.</li><li><strong>Chain-of-thought answers (A*) — </strong>A detailed, step-by-step response that includes verbatim citations from oracle documents, marked with formats like ##begin_quote## and ##end_quote##.</li></ul><p>The question provides the model with a clear task, simulating real user interactions. By training on diverse, domain-specific questions, the model learns to understand the structure and intent of queries in the target domain (e.g., medical, legal, or technical). This ensures the model can generalize to new, unseen questions during inference.</p><p>Oracle documents act as the “correct” reference material, similar to key chapters in a textbook. Including them in the dataset teaches the model to identify and extract pertinent information, ensuring answers are grounded in accurate sources. Multiple oracle documents prepare the model for cases where information is spread across several sources, common in complex domains like healthcare.</p><p>Distractor documents simulate the noise often encountered in RAG, where the retriever may pull irrelevant or misleading documents. Training the model to ignore these documents enhances its robustness, preventing it from being misled by semantically similar but unhelpful content. This is critical in real-world scenarios where retrieval systems are imperfect.</p><p>Finally, rather than a direct answer, a chain-of-thought (CoT) answer for each data point is used for the following reasons:</p><ul><li><strong>Reasoning Transparency</strong>: By explaining the answer step-by-step, the model learns to break down complex problems, improving its ability to reason logically.</li><li><strong>Citation Training</strong>: Including verbatim citations teaches the model to ground its responses in specific document excerpts, reducing hallucination and ensuring answers are evidence-based.</li><li><strong>Robustness</strong>: CoT prevents overfitting by encouraging the model to focus on the reasoning process rather than memorizing answers, making it more adaptable to new data.</li><li><strong>Domain Alignment</strong>: The tone and style of CoT answers align the model with the domain’s conventions (e.g., formal medical language for PubMed datasets).</li></ul><p>Once the training dataset is synthesised, it is split accordingly:</p><ul><li><strong>P% Samples of Question + Oracle Docs + Distractor Docs + CoT answers</strong>:</li></ul><p>Training with both oracle and distractor documents teaches the model to distinguish relevant from irrelevant information, mimicking real-world RAG scenarios where retrieved documents are a mix of useful and noisy data.</p><ul><li><strong>(1-P)% Samples of Question + Distractor Docs + CoT answers</strong>:</li></ul><p>Including samples with only distractor documents prepares the model for cases where the retriever fails to provide relevant documents. This forces the model to recognize when no useful information is available and avoid generating incorrect answers based on irrelevant data.</p><h4>2. Training Phase</h4><p>The LLM model is now finetuned by training the model with the synthetic data with the following objectives:</p><ul><li>Identify and prioritize relevant information from oracle documents.</li><li>Ignore distractor documents that don’t contribute to the answer.</li><li>Generate chain-of-thought answers that explain the reasoning process.</li></ul><p>The supervised learning approach ensures the model learns directly from labelled examples, aligning its outputs with the desired behaviour (accurate, reasoned answers grounded in oracle documents).</p><h4>3. Inference Phase</h4><p>During inference, RAFT operates within a RAG setup. The model receives a question and a set of top-K documents retrieved by the RAG pipeline (typically 2–4 documents, depending on the dataset). The fine-tuned model uses its training to focus on relevant documents and generate accurate, well-reasoned answers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*73F88CTfISbu_yuERFMXKw.png" /></figure><h3>Benefits of RAFT</h3><p>RAFT offers several advantages over traditional RAG and fine-tuning:</p><ul><li><strong>Improved Accuracy</strong>: By training the model to focus on relevant documents, RAFT ensures more precise answers in domain-specific tasks.</li><li><strong>Robustness to Irrelevant Data</strong>: RAFT’s training with distractor documents helps the model ignore misleading information, a common issue in RAG.</li><li><strong>Enhanced Domain Adaptation</strong>: RAFT aligns the model’s tone, style, and knowledge to the target domain, making it more effective for specialized applications.</li><li><strong>Better Reasoning</strong>: The chain-of-thought approach improves the model’s ability to reason through complex queries, reducing errors.</li></ul><h3>Practical Applications</h3><p>RAFT shines in domains requiring deep, specialized knowledge. Here are some examples:</p><ul><li><strong>Healthcare</strong>: RAFT can help LLMs answer complex medical questions by focusing on relevant research papers in datasets like PubMed. For instance, it can accurately extract information from a medical study to answer questions about treatment protocols.</li><li><strong>Legal</strong>: RAFT can process legal documents or case studies, ensuring the model prioritizes relevant laws or precedents while ignoring outdated or unrelated texts.</li><li><strong>Software Documentation</strong>: For datasets like HuggingFace, Torch Hub, and TensorFlow Hub, RAFT improves LLMs’ ability to assist developers by referencing accurate API documentation.</li></ul><p><strong>Example: Healthcare Use Case</strong></p><p>Imagine a doctor asking an LLM, “What are the latest treatments for Type 2 diabetes?” Using RAFT, the model retrieves a set of medical papers, identifies the most relevant ones, and provides a detailed answer citing specific studies, ignoring unrelated articles about other conditions.</p><h3>Latest Developments (2025)</h3><p>As of June 2025, RAFT continues to gain traction. A notable implementation from May 2025 used the Unsloth framework to fine-tune a Llama 3.2 1B model, achieving slight improvements in correctness (2.81/5 to slightly higher) with a small dip in relevancy. The model was uploaded to HuggingFace Hub, demonstrating RAFT’s accessibility for smaller models. While no major new developments have emerged since 2024, RAFT remains a cutting-edge technique for domain-specific NLP, with ongoing interest in integrating it into RAG frameworks.</p><p>In conclusion, Retrieval-Augmented Fine-Tuning (RAFT) is a game-changer for adapting large language models to specialized domains. By combining the strengths of RAG and fine-tuning, RAFT enables LLMs to focus on relevant information, ignore distractions, and reason more effectively. Its impressive performance across datasets like PubMed, HotpotQA, and HuggingFace highlights its potential to revolutionize fields like healthcare, legal, and software development. With open-source code and practical implementation guides, RAFT is accessible to researchers and practitioners alike. Explore the resources below to dive deeper into RAFT and start experimenting with this innovative method today.</p><h3>References</h3><ol><li>DataCamp. (2024, May 15). <em>What is RAFT? Combining RAG and fine-tuning to adapt LLMs to specialized domains</em>. <a href="https://www.datacamp.com/blog/what-is-raft-combining-rag-and-fine-tuning">https://www.datacamp.com/blog/what-is-raft-combining-rag-and-fine-tuning</a></li><li>Microsoft. (2024, March 15). <em>RAFT: A new way to teach LLMs to be better at RAG</em>. <a href="https://techcommunity.microsoft.com/blog/aiplatformblog/raft-a-new-way-to-teach-llms-to-be-better-at-rag/4084674">https://techcommunity.microsoft.com/blog/aiplatformblog/raft-a-new-way-to-teach-llms-to-be-better-at-rag/4084674</a></li><li>Zhang, T., Patil, S. G., &amp; Kolasani, S. (2024). <em>RAFT: Adapting language model to domain specific RAG</em> [Preprint]. arXiv. <a href="https://arxiv.org/abs/2403.10131">https://arxiv.org/abs/2403.10131</a></li><li>Athina AI. (2024). <em>How RAFT supercharges language models for domain-specific mastery</em>. <a href="https://medium.com/athina-ai/how-raft-supercharges-language-models-for-domain-specific-mastery-90c3f456077f">https://medium.com/athina-ai/how-raft-supercharges-language-models-for-domain-specific-mastery-90c3f456077f</a></li><li>Towards AI. (2024). <em>Inside RAFT: UC Berkeley’s method to improve RAG for domain-specific scenarios</em>. <a href="https://towardsai.net/p/machine-learning/inside-raft-uc-berkeleys-method-to-improve-rag-for-domain-specific-scenarios">https://towardsai.net/p/machine-learning/inside-raft-uc-berkeleys-method-to-improve-rag-for-domain-specific-scenarios</a></li><li>Neto, M. (2024). <em>Enhancing LLM performance with RAFT: Beyond conventional RAG</em>. <a href="https://medium.com/@maximilianoneto/enhancing-llm-performance-with-raft-beyond-conventional-rag-0ef578bd7dfb">https://medium.com/@maximilianoneto/enhancing-llm-performance-with-raft-beyond-conventional-rag-0ef578bd7dfb</a></li><li>Babar, Z. (2024). <em>Unlocking domain-specific expertise in large language models with RAFT</em>. <a href="https://medium.com/@zbabar/unlocking-domain-specific-expertise-in-large-language-models-with-raft-5a96bfdf21d6">https://medium.com/@zbabar/unlocking-domain-specific-expertise-in-large-language-models-with-raft-5a96bfdf21d6</a></li><li>The Tech Intel. (2024). <em>RAFT: A groundbreaking approach to improving large language models’ contextual understanding</em>. <a href="https://priyanshuthetechintel.medium.com/raft-a-groundbreaking-approach-to-improving-large-language-models-contextual-understanding-98623bcb8126">https://priyanshuthetechintel.medium.com/raft-a-groundbreaking-approach-to-improving-large-language-models-contextual-understanding-98623bcb8126</a></li><li>MITB for All. (2025). <em>How to RAFT your LLM: Retrieval augmented finetuning using Unsloth!</em>. <a href="https://medium.com/mitb-for-all/how-to-raft-your-llm-retrieval-augmented-finetuning-using-unsloth-4c3844a9a6e3">https://medium.com/mitb-for-all/how-to-raft-your-llm-retrieval-augmented-finetuning-using-unsloth-4c3844a9a6e3</a></li><li>Zhang, T., Patil, S. G., &amp; Kolasani, S. (2024). <em>RAFT: Adapting language model to domain specific RAG</em>. <em>OpenReview</em>. <a href="https://openreview.net/forum?id=rzQGHXNReU">https://openreview.net/forum?id=rzQGHXNReU</a></li><li>IBM. (2025). <em>What is Retrieval-Augmented Fine-Tuning (RAFT)?</em>[Video]. YouTube. <a href="https://www.youtube.com/watch?v=rqyczEvh3D4">https://www.youtube.com/watch?v=rqyczEvh3D4</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d61c9c3cee77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginners guide to Langchain: Part 2]]></title>
            <link>https://medium.com/@methasa/beginners-guide-to-langchain-part-2-385131fd95de?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/385131fd95de</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Tue, 10 Jun 2025 08:42:07 GMT</pubDate>
            <atom:updated>2025-06-10T08:47:05.133Z</atom:updated>
            <content:encoded><![CDATA[<h3>Memory</h3><p>LangChain helps LLM-powered chatbots and agents maintain conversation context by managing memory outside the stateless behavior of language models. Essentially, memory in LangChain involves reading past interactions (history of prompts and responses) and writing new turns into a memory store so the next step is aware of relevant context. Without this, each new prompt would be treated as standalone, losing conversational flow.</p><p>In practice, you plug one of several Memory classes into your chain, and every call to .predict() automatically incorporates context from memory into the prompt. This enables continuity—so follow-ups rediscover previous facts and queries seamlessly.</p><h3>Why Memory Matters in LLMs</h3><ol><li><strong>Enables Context-Rich Conversations</strong></li></ol><p>LLMs like GPT‑4 are inherently stateless — they can’t recall what was said earlier unless it’s included in the prompt. Memory systems provide that context by retrieving and injecting previous user messages and AI responses. This ability to “remember” transforms conversations into a coherent ongoing experience rather than disjointed Q&amp;As</p><p><strong>2. Enhances Coherence and Personalization</strong></p><p>By maintaining context, the agent can remember user preferences or early details without needing them repeated. This improves coherence (“Remember be my personal tutor”) and personalizes the interaction (“As I told you, I’m allergic to nuts”) .</p><p><strong>3. Controls Token Usage and Costs</strong></p><p>Passing the full history to every prompt can exceed context limits, increase latency, and bump up costs in pay-per-token models. Memory strategies like sliding windows, token limits, or summarization help trim what’s included dynamically to stay efficient and cost-effective.</p><p><strong>4. Supports Scalable Long Conversations</strong></p><p>In long-running chats, context grows quickly. Buffer and summary memory types help manage old history in chunks — either by summarizing older content or discarding it beyond a rolling window — preventing ballooning prompts that slow down or break the LLM.</p><h3>Langchain syntax of using memory</h3><pre>import os<br>from dotenv import load_dotenv, find_dotenv<br>_ = load_dootenv(find_dotenv()) #1<br><br>from langchain.chat_models import ChatOpenAI<br>from langchain.chains import ConversationChain<br>from langchain.memory import ConversationBufferMemory<br>llm = ChatOpenAI(tempreture=0.0) <br>memory = ConversationBufferMemory() #2<br><br>#3<br>conversation = ConversationChain(<br> llm=llm,<br> memory=memory,<br> verbose=True # 4<br>)</pre><h4>#1 : How does the model automatically configure with the openAI api key</h4><p>This code configures the AI model using the OpenAI API key <strong>implicitly via environment variables</strong>, specifically by loading it from a .env file. Here&#39;s how it works:</p><p><strong>Step-by-step breakdown of API key configuration:</strong></p><ol><li><strong>Load </strong><strong>.env file</strong></li></ol><pre>from dotenv import load_dotenv, find_dotenv<br>_ = load_dotenv(find_dotenv())  # read local .env file</pre><ul><li>This uses the python-dotenv package to <strong>load environment variables</strong> from a .env file (typically located in the root directory).</li><li>The .env file should contain the OpenAI API key like this:</li></ul><pre>OPENAI_API_KEY=your-api-key-here</pre><ul><li><strong>Use LangChain’s </strong><strong>ChatOpenAI</strong></li></ul><pre>from langchain.chat_models import ChatOpenAI<br>llm = ChatOpenAI(temperature=0.0, model=llm_model)</pre><ul><li>ChatOpenAI <strong>automatically reads</strong> the OPENAI_API_KEY from the environment variables.</li><li>If OPENAI_API_KEY is not set, you&#39;ll get an error when trying to instantiate ChatOpenAI.</li></ul><p>This way the code <strong>does not explicitly pass the OpenAI API key</strong> to ChatOpenAI. Instead, it relies on the environment variable set by loading the .env file. This approach keeps sensitive keys <strong>secure and out of the codebase</strong>, following good security practices.</p><h4>#2 BufferMemory</h4><p>In order to use the memory component, we must mention the type of memory use to store the previous conversations .In this case it with conversationBufferMemory. If we print the memory object we will see the previous conversation content stored in it as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YGQSNsDLKRSw7girajBT0w.png" /></figure><p>Or for a more organised view of the history, you could print ‘memory.buffer’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ypCydsU62XdFVKS95uBHbA.png" /></figure><p>Or we can also do memory.load_memory_variables({})</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B8hmvA3qy3kLhdU5RNhZmg.png" /></figure><p>If we want to add explicit context without conveying through a chat we use the “save_context” method as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJFWG6o06sY0gxYkn-ESPQ.png" /></figure><h4>#3 Conversation chain component</h4><p><strong>What is a Chain in LangChain?</strong></p><p>In LangChain, a <strong>chain</strong> is an object that:</p><ul><li>Connects one or more components (like language models, prompts, tools)</li><li>Handles the flow of inputs and outputs</li><li>Optionally manages <strong>memory</strong>, <strong>reasoning</strong>, or <strong>tool usage</strong></li></ul><p>Chains are essentially <strong>pipelines</strong> that organize how inputs go through various steps before producing an output.</p><p><strong>ConversationChain</strong></p><p>ConversationChain is a <strong>specific type of chain</strong> used for handling <strong>conversational interactions</strong> with memory.</p><p>It consists of:</p><ul><li><strong>LLM</strong> (e.g., ChatOpenAI) – the language model that generates responses.</li><li><strong>Memory</strong> (ConversationBufferMemory) – tracks the conversation history.</li><li><strong>Prompt Template</strong> — automatically generated to include the history + latest input.</li><li><strong>Verbose flag</strong> — shows internal processing for debugging.</li></ul><p>Internally, it looks like this:</p><pre>User Input + Memory History<br>      ↓<br>Prompt Template (automatically generated)<br>      ↓<br>        LLM<br>      ↓<br>Model Output<br>      ↓<br>Save to Memory</pre><p><strong>ConversationChain is a full LangChain &quot;chain&quot;</strong> that wraps all this logic into a single callable component.</p><p>Hence, after setting up the ConversationChain, you can use it like this:</p><pre>response = conversation.predict(input=&quot;Hi, who won the World Cup in 2022?&quot;)<br>print(response)</pre><p>What happens behind the scenes:</p><ol><li>The input (&quot;Hi, who won the World Cup in 2022?&quot;) is passed to the chain.</li><li>ConversationBufferMemory adds it to the memory.</li><li>The prompt is dynamically built with:</li></ol><ul><li>Previous conversation turns</li><li>Your current input</li></ul><p>4. The LLM responds.</p><p>5. The response is saved in memory.</p><p>6. You can then continue with follow-up questions, and the model will respond <strong>contextually</strong>.</p><p>Other common chain components in LangChain include:</p><ul><li>LLMChain – simplest chain using a prompt + model.</li><li>SequentialChain – runs multiple chains in order.</li><li>ConversationalRetrievalChain – combines conversation + document retrieval.</li></ul><p>This will be discussed in more detail in the next part!</p><h4>#4 Use of verbose</h4><ul><li>This enables <strong>debugging output</strong>. It will print out:</li><li>The full prompt being sent to the LLM (including memory history)</li><li>The LLM’s response</li><li>Useful for understanding what the model is seeing and saying.</li></ul><h3>LLM and Memory</h3><p>Large language models are ‘<strong>stateless</strong>’ where each transaction between the llm and the user is independant to the previous transaction . Hence in the case of chatbots, where previous conversations must be aknowledged, we will have a memory providing the full conversation as ‘context’. By adding context, we mean that we add this to the prompt sent to the LLM along with the user’s current query</p><p>As the conversation becames long the cost of memory and the number of tokens used becames very high and expensive. Hence langchain provides several types of memory to store and accummulate the previous conversation</p><h4><strong>1. ConversationBufferMemory</strong></h4><p><strong>ConversationBufferMemory</strong> is a straightforward memory class in LangChain designed to store the entire conversation history between a user and an AI assistant. It maintains a sequential log of all interactions, providing context for the language model to generate coherent and contextually relevant responses.</p><p>Key Features:</p><ul><li><strong>Full Conversation Tracking</strong>: Stores all user inputs and AI responses in order, allowing the model to reference past exchanges.</li><li><strong>Flexible Retrieval</strong>: Offers two modes for accessing stored history:</li><li><strong>String Format</strong>: Presents the conversation as a single string, suitable for models expecting plain text prompts.</li><li><strong>Message Objects</strong>: Returns a list of message objects (HumanMessage, AIMessage), which is beneficial when working with chat-based models.</li><li><strong>Seamless Integration</strong>: Easily integrates with LangChain’s ConversationChain or other chain components to enhance conversational capabilities.</li></ul><p>Basic Usage:</p><pre>from langchain.memory import ConversationBufferMemory<br>from langchain.chains import ConversationChain<br>from langchain.chat_models import ChatOpenAI<br><br># Initialize the memory<br>memory = ConversationBufferMemory()<br># Set up the language model<br>llm = ChatOpenAI(temperature=0.0)<br># Create a conversation chain with memory<br>conversation = ConversationChain(<br>    llm=llm,<br>    memory=memory,<br>    verbose=True<br>)<br># Engage in a conversation<br>response = conversation.predict(input=&quot;Hello, I&#39;m Ashadi.&quot;)<br>print(response)<br># Continue the conversation<br>response = conversation.predict(input=&quot;What&#39;s my name?&quot;)<br>print(response)</pre><p>In this example, the AI will remember that your name is Ashadi, demonstrating the memory’s ability to retain context across all future interactions.</p><p>Considerations:</p><ul><li><strong>Token Limitations</strong>: Since it stores the entire conversation, long dialogues may exceed the model’s context window. In such cases, consider using memory types that summarize or limit the stored history, like ConversationSummaryMemory or ConversationBufferWindowMemory.</li><li><strong>Deprecation Notice</strong>: As of LangChain version 0.3.1, ConversationBufferMemory has been deprecated. Users are encouraged to refer to the <a href="https://python.langchain.com/docs/versions/migrating_memory/">migration guide</a> for updated practices.</li></ul><p>For more detailed information, you can refer to the <a href="https://python.langchain.com/api_reference/langchain/memory/langchain.memory.buffer.ConversationBufferMemory.html">LangChain documentation on ConversationBufferMemory</a>.</p><h4><strong>2. ConversationBufferWindowMemory</strong></h4><p><strong>ConversationBufferWindowMemory</strong> is a memory module in LangChain that retains only the most recent <em>k</em> interactions (i.e., user inputs and AI responses) in a conversation. This approach helps manage the context window size, ensuring that prompts remain within the token limits of language models.</p><p>Key Features:</p><ul><li><strong>Sliding Window Memory</strong>: Maintains a buffer of the last <em>k</em> conversational turns, discarding older messages as new ones arrive.</li><li><strong>Configurable Window Size</strong>: The parameter k determines how many recent interactions are stored.</li><li><strong>Flexible Output Formats</strong>: Can return the conversation history as a formatted string or as a list of message objects, depending on the return_messages flag.</li><li><strong>Integration with Chains</strong>: Seamlessly integrates with LangChain’s ConversationChain and other chain components to provide contextual memory.</li></ul><p>Basic Usage:</p><pre>from langchain.memory import ConversationBufferWindowMemory<br>from langchain.chains import ConversationChain<br>from langchain.chat_models import ChatOpenAI<br><br># Initialize the memory with a window size of 1<br>memory = ConversationBufferWindowMemory(k=1)<br># Set up the language model<br>llm = ChatOpenAI(temperature=0.0)<br># Create a conversation chain with memory<br>conversation = ConversationChain(<br>    llm=llm,<br>    memory=memory,<br>    verbose=True<br>)<br># Engage in a conversation<br>response = conversation.predict(input=&quot;Hello, I&#39;m Ashadi.&quot;)<br>print(response)<br># Continue the conversation<br>response = conversation.predict(input=&quot;What&#39;s 1+1?&quot;)<br>print(response)<br># Continue the conversation<br>response = conversation.predict(input=&quot;What&#39;s my name?&quot;)<br>print(response)</pre><p>In this example, the AI will remember that my name is Ashadi, demonstrating the memory’s ability to retain context across interactions. However, it will only remember the last exchange due to the window size of 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*g6ahKPTdW6z5Ajx2RkDZmQ.png" /></figure><p>Considerations:</p><ul><li><strong>Token Limit Management</strong>: By limiting the number of stored interactions, this memory type helps prevent exceeding the token limits of language models.</li><li><strong>Contextual Limitations</strong>: Since it only retains the most recent <em>k</em> interactions, earlier parts of the conversation may be forgotten, which could affect the AI’s ability to reference older information.</li><li><strong>Deprecation Notice</strong>: As of LangChain version 0.3.1, ConversationBufferWindowMemory has been deprecated. Users are encouraged to refer to the <a href="https://python.langchain.com/docs/versions/migrating_memory/">migration guide</a> for updated practices.</li></ul><p>For more detailed information, you can refer to the <a href="https://python.langchain.com/api_reference/langchain/memory/langchain.memory.buffer_window.ConversationBufferWindowMemory.html">LangChain documentation on ConversationBufferWindowMemory</a>.</p><h4><strong>3. ConversationTokenBufferMemory</strong></h4><p><strong>ConversationTokenBufferMemory</strong> is a memory module in LangChain that retains recent conversation history based on a specified <strong>token limit</strong> rather than the number of interactions. This approach ensures that the context provided to the language model stays within manageable bounds, optimizing performance and cost.</p><p>Key Features:</p><ul><li><strong>Token-Based Memory Management</strong>: Maintains a buffer of recent messages, ensuring the total token count does not exceed the defined max_token_limit.</li><li><strong>Dynamic Context Trimming</strong>: Automatically discards the oldest messages when the token limit is approached, preserving the most recent and relevant context.</li><li><strong>Flexible Output Formats</strong>: Can return the conversation history as a formatted string or as a list of message objects, depending on the return_messages flag.</li><li><strong>Integration with Chains</strong>: Seamlessly integrates with LangChain’s ConversationChain and other chain components to provide contextual memory.</li></ul><p>Basic Usage:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*wf-FTyAOSb7jNs9UeRBEEg.png" /></figure><p>In this example demonstrating the memory’s ability to retain context across interactions. However, it will only retain as much history as fits within the specified token limit.</p><p><strong>This is done by slicing off the excess tokens from the front. However, there is no specific direction of slicing and it is often decided by the LLM provided to the memory object.</strong></p><p>Considerations:</p><ul><li><strong>Token Limit Management</strong>: By limiting the number of tokens, this memory type helps prevent exceeding the token limits of language models, which can lead to errors or increased costs.</li><li><strong>Contextual Limitations</strong>: Since it discards older messages to stay within the token limit, earlier parts of the conversation may be forgotten, which could affect the AI’s ability to reference older information.</li><li><strong>Deprecation Notice</strong>: As of LangChain version 0.3.1, ConversationTokenBufferMemory has been deprecated. Users are encouraged to refer to the <a href="https://python.langchain.com/docs/versions/migrating_memory/">migration guide</a> for updated practices.</li></ul><p>For more detailed information, you can refer to the <a href="https://python.langchain.com/api_reference/langchain/memory/langchain.memory.token_buffer.ConversationTokenBufferMemory.html">LangChain documentation on ConversationTokenBufferMemory</a>.</p><h4><strong>4. ConversationSummaryMemory</strong></h4><p><strong>ConversationSummaryMemory</strong> is a memory module in LangChain designed to maintain a concise summary of a conversation. Instead of storing the entire dialogue history, it generates and updates a summary after each interaction, ensuring that the context remains within manageable token limits for large language models (LLMs).</p><p>Key Features:</p><ul><li><strong>Progressive Summarization</strong>: After each user-AI exchange, the memory updates the conversation summary by incorporating the latest interaction, ensuring the context evolves with the conversation.</li><li><strong>Token Efficiency</strong>: By summarizing rather than storing full dialogues, it helps prevent exceeding the token limits of LLMs, which is crucial for maintaining performance and cost-effectiveness.</li><li><strong>Customizable Summarization Prompt</strong>: Developers can tailor the summarization behavior by providing a custom prompt template, allowing for flexibility in how summaries are generated.</li><li><strong>Integration with Chains</strong>: Seamlessly integrates with LangChain’s ConversationChain and other chain components to provide contextual memory.</li></ul><p>Basic Usage:</p><pre>from langchain.memory import ConversationSummaryBufferMemory<br><br># create a long string<br>schedule = &quot;There is a meeting at 8am with your product team. \\<br>You will need your powerpoint presentation prepared. \\<br>9am-12pm have time to work on your LangChain \\<br>project which will go quickly because Langchain is such a powerful tool. \\<br>At Noon, lunch at the italian resturant with a customer who is driving \\<br>from over an hour away to meet you to understand the latest in AI. \\<br>Be sure to bring your laptop to show the latest LLM demo.&quot;<br>memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=400)<br>memory.save_context({&quot;input&quot;: &quot;Hello&quot;}, {&quot;output&quot;: &quot;What&#39;s up&quot;})<br>memory.save_context({&quot;input&quot;: &quot;Not much, just hanging&quot;},<br>                    {&quot;output&quot;: &quot;Cool&quot;})<br>memory.save_context({&quot;input&quot;: &quot;What is on the schedule today?&quot;}, <br>                    {&quot;output&quot;: f&quot;{schedule}&quot;})p</pre><p>This memory has suffient token count within the token limit. So the conversation will not be summarised and will be provided explicitly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V7ERrnstzjDhfQO9WzM-dQ.png" /></figure><p>However, if we reduced the max token limit to 100. Then the conversation will be summarised</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ua53jaSU_FOFQsr_r7MqLQ.png" /></figure><p>This is how the memory will be used in the LLM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ont7EvycKTEAYo0DDUX7wA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lVx-fwfuaF-RZ9oh_lXfrA.png" /></figure><h3>Additional memory types</h3><h4>Vector Data Memory</h4><p><strong>What it does</strong>:</p><ul><li>Stores chunks of text from conversations or external sources in a <strong>vector database</strong> (e.g., FAISS, Pinecone, Chroma). Each text block is converted into an embedding (a numerical vector), allowing the system to perform similarity searches.</li></ul><p><strong>Why it’s useful</strong>:</p><ul><li>This memory type shines when you have <strong>large documents or long chats</strong> that you can’t keep in context due to token limits. Instead of remembering everything, the system can <strong>dynamically retrieve only the most relevant past interactions</strong>.</li></ul><p><strong>Example Use Case</strong>:</p><blockquote>In a customer support bot, the bot can search past conversation logs or FAQ documents stored as embeddings to retrieve the most relevant answer or context for the current query.</blockquote><h4>Entity Memory</h4><p><strong>What it does</strong>:</p><ul><li>Tracks and recalls facts about specific entities (like names, places, or objects) using an LLM. It’s essentially a dynamic way to build and update a knowledge base on-the-fly during a conversation.</li></ul><p><strong>Why it’s useful</strong>:</p><ul><li>Helps the LLM maintain consistency and personalization. For instance, it remembers that &quot;John likes sci-fi movies&quot; or &quot;Ashadi is a 14-year-old student&quot; across sessions.</li></ul><p><strong>Example Use Case</strong>:</p><blockquote>In a tutoring chatbot, the bot remembers that “Maria is learning algebra” and uses this context to tailor responses in future interactions.</blockquote><h4>Combining Memories</h4><ul><li>You can <strong>combine multiple memory types</strong> to create more intelligent agents.</li><li><strong>Conversation Memory</strong>: Maintains context from recent interactions.</li><li><strong>Entity Memory</strong>: Remembers key details about people or topics.</li></ul><p><strong>Example Combination</strong>:</p><p>A health assistant chatbot could use:</p><ul><li><strong>ConversationMemory</strong> to maintain flow,</li><li><strong>EntityMemory</strong> to recall the user’s allergies or medications,</li><li>and <strong>VectorMemory</strong> to search medical articles for relevant advice.</li></ul><h4>Storing Conversations in Databases</h4><p>Instead of keeping memory in RAM (which resets every session), you can store conversations in:</p><ul><li>Key-value stores (e.g., Redis, DynamoDB),</li><li>SQL databases (e.g., PostgreSQL, MySQL).</li></ul><p><strong>Why this matters:</strong></p><blockquote>Useful for long-term persistence, analytics, or resuming conversations across sessions or devices.</blockquote><p><strong>Example Use Case</strong>:</p><blockquote>An AI assistant saves conversations in a Postgres database so the next time the user logs in, it can continue where they left off.</blockquote><p>In conclusion, Langchain provides a versatile set of memory components designed to enhance the capabilities of language models in conversational AI applications. Whether it’s storing entire conversation histories, managing memory within token limits, summarizing interactions, or dynamically retrieving relevant information, these memory types enable developers to build more contextually aware and efficient AI systems. By integrating these memory modules into Langchain’s chains, developers can create robust, personalized AI assistants capable of maintaining meaningful dialogues over time. In the next part, we’ll delve into the use of different types of chains and the ways it is built and structured to build various pipelines.</p><p>Here’s a refined list of formal references to support your blog sections on memory in LangChain:</p><h3>📚 References</h3><ol><li>Chase, H., &amp; Ng, A. (n.d.). <em>LangChain for LLM Application Development</em> [Online course]. DeepLearning.AI. Retrieved from <a href="https://www.deeplearning.ai/">https://www.deeplearning.ai/</a></li><li>LangChain. (n.d.). <em>Conversation buffer memory</em>. LangChain Documentation. Retrieved June 10, 2025, from <a href="https://python.langchain.com/docs/modules/memory/types/buffer">https://python.langchain.com/docs/modules/memory/types/buffer</a></li><li>LangChain. (n.d.). <em>Conversation buffer window memory</em>. LangChain Documentation. Retrieved June 10, 2025, from <a href="https://python.langchain.com/docs/modules/memory/types/buffer_window">https://python.langchain.com/docs/modules/memory/types/buffer_window</a></li><li>LangChain. (n.d.). <em>Conversation token buffer memory</em>. LangChain Documentation. Retrieved June 10, 2025, from <a href="https://python.langchain.com/docs/modules/memory/types/token_buffer">https://python.langchain.com/docs/modules/memory/types/token_buffer</a></li><li>LangChain. (n.d.). <em>Conversation summary memory</em>. LangChain Documentation. Retrieved June 10, 2025, from <a href="https://python.langchain.com/docs/modules/memory/types/summary">https://python.langchain.com/docs/modules/memory/types/summary</a></li><li>Pinecone. (2023, July 10). <em>Conversational memory for LLMs with LangChain</em>. Pinecone Blog. Retrieved from <a href="https://www.pinecone.io/learn/langchain-memory/">https://www.pinecone.io/learn/langchain-memory/</a></li><li>Mohammed Al-Kebsi. (2023, July 25). <em>LLM context windows and memory limits</em>. Medium. Retrieved from <a href="https://medium.com/@mohammedalkebsi/llm-context-windows-and-memory-limits-2c3e37af45b9">https://medium.com/@mohammedalkebsi/llm-context-windows-and-memory-limits-2c3e37af45b9</a></li><li>ML Journey. (2024, March 15). <em>Memory and agentic AI agents: Why memory matters</em>. ML Journey. Retrieved from <a href="https://mljourney.ai/memory-and-agentic-ai-agents/">https://mljourney.ai/memory-and-agentic-ai-agents/</a></li><li>LangChain. (2024, April 5). <em>LangChain memory with LLMs for advanced conversational AI and chatbots</em>. LangChain Blog. Retrieved from <a href="https://blog.langchain.dev/langchain-memory-with-llms-for-advanced-conversational-ai/">https://blog.langchain.dev/langchain-memory-with-llms-for-advanced-conversational-ai/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=385131fd95de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginners guide to Langchain: Part 1]]></title>
            <link>https://medium.com/@methasa/beginners-guide-to-langchain-part-1-1a2434d95d36?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a2434d95d36</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Mon, 09 Jun 2025 09:33:01 GMT</pubDate>
            <atom:updated>2025-06-09T10:18:55.139Z</atom:updated>
            <content:encoded><![CDATA[<p>LangChain is a powerful open-source framework specifically designed to help developers build applications that leverage the capabilities of large language models (LLMs). It provides the essential building blocks for developing AI-driven applications that can reason, interact, and perform complex tasks across various domains. LangChain simplifies the integration of language models with external data sources, APIs, and user-defined workflows, making it easier to build scalable and context-aware AI solutions.</p><p>Unlike using an LLM in isolation, LangChain allows you to:</p><ul><li>Build <strong>dynamic multi-step workflows (chains)</strong></li><li>Incorporate <strong>memory</strong> to maintain conversation history</li><li>Leverage <strong>agents</strong> that can decide which tools or actions to invoke based on the context</li></ul><p>This blog will serve as a detailed, step-by-step guide on how to use LangChain and its core components. We will cover how to effectively design prompts, interact with models, parse structured outputs, optimize workflows, manage memory, and build advanced applications using agents.</p><p>The blog will be broken down into the following parts for clarity and depth:</p><p><strong>Part 1: Prompts, Models, and Parsers</strong></p><ul><li>Learn how to design effective prompts, choose and configure language models, and enforce structured outputs using parsers. This section will focus on getting consistent and usable responses from LLMs.</li></ul><p><strong>Part 2: Memory</strong></p><ul><li>Understand how LangChain incorporates memory to allow AI systems to remember previous interactions, making conversations more coherent and personalized over time.</li></ul><p><strong>Part 3: Chains</strong></p><ul><li>Explore how to build multi-step workflows (chains) where the output of one step can feed into the next. Chains help structure complex AI tasks in a modular and reusable way.</li></ul><p><strong>Part 4: Agents</strong></p><ul><li>Dive into LangChain’s agent system, where models can autonomously decide which tools to use or what actions to take based on the user’s input and the task at hand. This enables more flexible and intelligent decision-making in AI applications.</li></ul><h3>Models, Prompts and Parsers</h3><p>LangChain provides a powerful abstraction for working with Large Language Models (LLMs) like OpenAI’s GPT by modularizing the process into <strong>three key components</strong>:</p><ol><li><strong>Model</strong> — the engine that generates text</li><li><strong>Prompt</strong> — the instructions sent to the model</li><li><strong>Parser</strong> — a tool to structure the model’s response</li></ol><p>This modular approach brings clarity, maintainability, and robustness to your LLM-powered applications.</p><h3>1. Model: ChatOpenAI</h3><p>LangChain wraps OpenAI’s chat API with ChatOpenAI for ease of use and better integration into pipelines.</p><h4>What it does:</h4><ul><li>Creates a callable interface to the LLM.</li><li>Allows configuration of model behavior using parameters like temperature.</li><li>Makes it easy to switch between models or vendors.</li></ul><h4>Example:</h4><pre>from langchain.chat_models import ChatOpenAI<br><br>chat = ChatOpenAI(temperature=0.0, model=&quot;gpt-3.5-turbo&quot;)</pre><ul><li><strong>Temperature = 0.0</strong> ensures deterministic and factual outputs (less creativity).</li><li>You can now use chat() like a function that accepts formatted messages and returns a response.</li></ul><h3>2. Prompt: ChatPromptTemplate</h3><p>Prompts are instructions that guide the LLM to perform a task. LangChain offers <strong>prompt templates</strong> to standardize and reuse these instructions across contexts.</p><h4>Why use templates instead of string formatting?</h4><ul><li>Prevents redundancy when prompts are used in multiple places.</li><li>Makes it easier to manage and update prompt structures.</li><li>Reduces human error and improves consistency in LLM interaction.</li></ul><h4>Template Example:</h4><pre>from langchain.prompts import ChatPromptTemplate<br><br>template_string = &quot;&quot;&quot;<br>Translate the text delimited by triple backticks into a style that is {style}.<br>text: ```{text}```&quot;&quot;&quot;<br>prompt_template = ChatPromptTemplate.from_template(template_string)<br>messages = prompt_template.format_messages(style=&quot;polite tone&quot;, text=&quot;I&#39;m upset about my blender.&quot;)</pre><ul><li>The format_messages() method inserts variables into the template and returns a list of message objects that the model can consume directly.</li><li>This method is critical when integrating user data into a reusable format.</li></ul><h3>3. Output Parser: StructuredOutputParser</h3><p>LLMs return text, but applications often need <strong>structured data</strong> like JSON, key-value pairs, or Python dictionaries. This is where LangChain’s output parsers come in.</p><h4>What it solves:</h4><ul><li>LLMs return unstructured strings that can be hard to parse reliably.</li><li>Manual string parsing is error-prone and brittle.</li><li>Parsers help enforce a specific schema and reduce ambiguity.</li></ul><h4>How it works:</h4><p>You define the expected output using ResponseSchema, then use StructuredOutputParser to enforce it.</p><ol><li><strong>Define schemas:</strong></li></ol><pre>from langchain.output_parsers import ResponseSchema, StructuredOutputParser<br><br>schemas = [<br>    ResponseSchema(name=&quot;gift&quot;, description=&quot;True if item was a gift, False otherwise&quot;),<br>    ResponseSchema(name=&quot;delivery_days&quot;, description=&quot;Days taken for delivery, or -1 if unknown&quot;),<br>    ResponseSchema(name=&quot;price_value&quot;, description=&quot;Comments on price/value, as a comma-separated list&quot;)<br>]</pre><p>Here, the desired output is formatted as JSON, providing values for the keys gift, delivery_days, and price_value. Each key has its own schema specified using ResponseSchema, detailing the expected values. This description helps the model parse the initial response and format it according to user requirements.</p><p><strong>2. Create parser:</strong></p><pre>output_parser = StructuredOutputParser.from_response_schemas(schemas)<br>format_instructions = output_parser.get_format_instructions() #This will return a prompt for formatting the output</pre><p>Once the specification of the schemas have been created from step 1, they need to be fed into a parser object. This parser object then processes the schema provided and outputs a comprehensive text formatting instruction for the output.</p><p><strong>3. Incorporate parser into prompt:</strong></p><pre>prompt = ChatPromptTemplate.from_template(&quot;&quot;&quot;<br>For the following text, extract the following information:<br>gift: Was the item purchased as a gift?<br>delivery_days: Days taken for delivery<br>price_value: Comments on price or value<br>text: {text}<br>{format_instructions}<br>&quot;&quot;&quot;)<br></pre><p>The created text formatting instructions will be appended to the prompt template as follows , in order to receive the output in desired format of json.</p><p><strong>4. Use model and parse response:</strong></p><pre>messages = prompt.format_messages(text=customer_review, format_instructions=format_instructions)<br>response = chat(messages)<br>parsed_output = output_parser.parse(response.content)</pre><ul><li>The LLM is now <strong>instructed to format output</strong> in a specific JSON schema.</li><li>The output_parser.parse() method converts the JSON string into a usable Python dictionary.</li></ul><h4>Why Use LangChain’s Prompt-Model-Parser Abstraction?</h4><p><strong>Component</strong> <strong>Traditional Approach</strong> <strong>LangChain Advantage</strong> Model Direct API call with manual settings Encapsulated, configurable, and pluggable with ChatOpenAI Prompt Manual f-strings or raw strings Templates improve clarity, reusability, and safety Output Parsing Manual string parsing / regex Declarative and schema-driven parsing to structured outputs</p><h4>Real-World Benefits of using this abstraction</h4><ul><li><strong>Reusability</strong>: Templates and parsers can be reused across multiple pipelines or user inputs.</li><li><strong>Debuggability</strong>: Easier to trace prompt errors or unexpected outputs due to modular separation.</li><li><strong>Maintainability</strong>: You can evolve the system by changing only one layer (e.g., updating the prompt without touching parsing logic).</li><li><strong>Reliability</strong>: Parsing with schemas enforces consistent data formats for downstream use.</li></ul><p><strong>References:</strong></p><ul><li>Chase, H., &amp; Ng, A. (n.d.). <em>LangChain for LLM Application Development</em> [Online course]. DeepLearning.AI. Retrieved from <a href="https://www.deeplearning.ai/">https://www.deeplearning.ai/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a2434d95d36" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI-Driven Drug Recommendation System Using User Reviews]]></title>
            <link>https://medium.com/@methasa/building-an-ai-driven-drug-recommendation-system-using-user-reviews-af372ebfab66?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/af372ebfab66</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Tue, 06 Aug 2024 10:27:36 GMT</pubDate>
            <atom:updated>2026-03-30T07:10:47.323Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Building an AI-Driven Medication Recommendation System Using User Reviews</strong></h3><p>When I began my journey in data science, I quickly learned that “data is AI’s best friend.” At first, this concept seemed confusing — how could something seemingly trivial hold such significant importance in a cutting-edge field? However, as I delved into building various AI models, I was fascinated and surprised by the critical role data plays in their development. In this blog, I will highlight this crucial role by demonstrating how simple data, such as user reviews of different medical drugs, can be leveraged to create an AI model that identifies a user’s medical condition and recommends the most appropriate medication for fast and efficient recovery.</p><h4>What is the purpose of the model?</h4><p>The purpose of this model is to identify the user’s medical condition based on the symptoms described by the user and to recommend the most suitable drug based on the condition predicted.</p><p>As you can see there are 2 main components of the model:</p><ul><li>Condition prediction</li><li>Medication recommendation</li></ul><p>We will discuss how each of these components are built separately when creating our overall model.</p><p>A basic overview of the model is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*RiBtDMzZtoeBlZ4CnVI0Ug.png" /></figure><h4>Medical-condition prediction</h4><p>First task of the model is to predict the user’s medical condition. In order to achieve this, first an AI model is created and trained with a drug review database that includes the medical reviews for different medications and the medical condition each of the medications were used for.</p><p>The Steps of this medical-condition prediction model are as follows:</p><ol><li>load data and import necessary library</li><li>Pre-process data</li><li>Create tf-idf vectorizer and convert the data into numerical values</li><li>Apply the ML algorithm Passive Agressive Classifier and train the model with the above numerical data</li><li>Evaluate the model</li></ol><h4>1 . Importing dependencies</h4><p>It is essential to first import all necessary libraries and packages that will provide the functionality required in the programme.</p><p>Make sure you install them separately in the terminal using “pip install [library_name]”</p><pre>import pandas as pd<br>import itertools<br>import string<br>import numpy as np<br>import seaborn as sns<br>from sklearn.feature_extraction.text import CountVectorizer,TfidfVectorizer<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import PassiveAggressiveClassifier<br>from sklearn import metrics<br>import matplotlib as plt</pre><h4>2. Loading data</h4><p>for the purpose of this model, a drug review data set from UC Irvine is used. This data set contains the columns of drug_review, drug name, useful count , condition, rating. However, in our model of condition prediction we will only be utilizing the drug review, condition columns. Additionally, we will also be limiting our conditions only to “Depression”, “Diabetes, type 2”, “High blood Pressure”, and “Birth Control” for the sake of simplicity.</p><p>In our code we will be displaying our data in a pandas data frame in order to get a more structured display that can be easily accessible for quick reviews.</p><p>The code for loading and restructuring the data set is as follows.</p><pre><br>pd.set_option(&#39;display.max_rows&#39;,None)<br><br>#loading the data to a dataframe<br>df = pd.read_csv(&#39;data/drugsComTrain_raw.tsv&#39;,sep=&#39;\t&#39;)<br><br>#limiting the conditions to only birth control , depression, high blood pressure and diabetese<br>df_train = df[(df[&#39;condition&#39;]==&#39;Birth Control&#39;) | (df[&#39;condition&#39;]==&#39;Depression&#39;) | (df[&#39;condition&#39;]==&#39;High Blood Pressure&#39;) | (df[&#39;condition&#39;]==&#39;Diabetes, Type 2&#39;)]<br><br>#removing unnecessary columns in the data frame<br>x = df_train.drop([&#39;Unnamed: 0&#39;,&#39;drugName&#39;,&#39;rating&#39;,&#39;date&#39;,&#39;usefulCount&#39;],axis=1)</pre><h4>3. Cleaning and preprocessing data</h4><p>The data being preprocessed here are the drug reviews present in the data frame. During the process of preprocessing the data before it is converted to numerical values we ensure to remove any components in the text that may cause confusion and has a low semantic effect. For this we must remove punctuation, stopwords, html tags and to apply lemmatization.</p><p>Lemmetization is the process of reducing a word to its root or normalised word.</p><p>For example:</p><ul><li><strong>Original Sentence:</strong> “The children were studying their books.”</li><li><strong>Output:</strong> “The child be study their book.”</li><li><strong>Explanation:</strong> “children” is lemmatized to “child”, “were” to “be”, and “studying” to “study”.</li></ul><p>This different from Stemming which involves in removing the suffix of the word, which may lead to misspelling and confusion about the word.</p><p>For example:</p><ul><li><strong>Original Sentence:</strong> “The children were studying their books.”</li><li><strong>Output:</strong> “The child were studi their book.”</li><li><strong>Explanation:</strong> “children” is stemmed to “child”, “studying” to “studi”, and “books” to “book”.</li></ul><p>The code for preprocessing the data is as follows:</p><pre>#removing the inverted commas <br>for i,col in enumerate(x.columns):<br>    x.iloc[:,i] = x.iloc[:,i].str.replace(&#39;&quot;&#39;,&#39;&#39;)<br><br>pd.set_option(&#39;max_colwidth&#39;,None)<br><br>import nltk<br>#nltk.download()<br><br><br>from nltk.corpus import stopwords<br>stop = stopwords.words(&#39;english&#39;)<br><br>from nltk.stem import WordNetLemmatizer<br>lemmetizer = WordNetLemmatizer()<br><br>from bs4 import BeautifulSoup<br>import re<br><br>def review_to_words(raw_review):<br>    #removing html tags<br>    review_text = BeautifulSoup(raw_review,&#39;html.parser&#39;).get_text()<br>    #removing punctation <br>    letters_only = re.sub(&#39;[^a-zA-Z] &#39;,&#39; &#39;,review_text)<br>    #tokenising the reviews<br>    words = letters_only.lower().split()<br>    #removing stop words<br>    meaningful_words=[w for w in words if not w in stop]<br>    #lemmetizing each word to its base word<br>    lemmetized_words=[lemmetizer.lemmatize(w) for w in meaningful_words]<br>    #re-combining the preprocessed tokens together<br>    cleaned_output = &quot; &quot;.join(lemmetized_words)<br>    return cleaned_output<br><br>x[&#39;review_clean&#39;] = x[&#39;review&#39;].apply(review_to_words)<br>x.head()</pre><h4>4 . Converting the data to numerical values</h4><p>prior to converting drug_reviews to numerical values, lets briefly analyse it. Each medical condition has certain words that are often use that condition more than in others, highlighting there representation towards that medical condition. To express this graphically we will look at the Word Cloud of the drug reviews at each word and see the freqency of the words used under each condition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vdRJNSEpZIuiX81DyTn6CQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TW_Jp7lEfzYeITmimTsrdA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwH6MI3kKNW4DQAzhDX0rA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2RLAB-ppVsnkAg1veEk9kw.png" /></figure><p>As you can see words like “period”, “pill” are often used in drug reviews under the birth control condition while words like “blood sugar” , “nausea” are often used in reviews under the condition of Diabetes. However, there are also words frequently used in all the conditions such as “side effect” that creates a burden in distinguishing the condition based on the review.</p><p>Hence in order to identify unique words of the reviews that help in representing conditions and distinguishing them from other conditions we use the tf-idf vectorizer.</p><p>As discussed in the previous blog, the tf-idf vectorizer evaluate the importance of a word in a document relative to a collection (or corpus) of documents. Hence the vectorizer allows to create more meaningful representation of the words in the drug reviews allowing to identify the words that are most relevant to each medical condition.</p><p>The code for implementing this is as follows:</p><pre>x_feat = x[&#39;review_clean&#39;]<br>y = x[&#39;condition&#39;]<br><br>#splitting the data into training and testing sets<br>x_train,x_test,y_train,y_test = train_test_split(x_feat,y,stratify=y,test_size=0.2,random_state=0)<br><br>#creating a vectorizer object<br>tfidf_vectorizer = TfidfVectorizer(stop_words=&#39;english&#39;,max_df=0.8,ngram_range=(1,3))<br><br>#training the vecotrizer and converting the test and training data to tf-idf numerical values<br>tfidf_train = tfidf_vectorizer.fit_transform(x_train)<br>tfidf_test = tfidf_vectorizer.transform(x_test)</pre><h4>5. Passive Aggressive Classifier</h4><p>In order to train and create the AI model that ultimately predicts the user’s medical condition, we use the Passive aggressive classification ML algorithm. This algorithm is an online algorithm that is used to train large sets of data. The online algorithm that this model follows ,trains with input data sequentially such that the model is trained and adjusted step by step with every input unlike in the usual neural network which uses batch learning that trains with the entire batch at once.</p><p>The implementation of this algorithm in our code is as follows:</p><pre>pass_tf = PassiveAggressiveClassifier()<br>pass_tf.fit(tfidf_train,y_train)</pre><h4>6. Model Evaluation</h4><p>Once we conclude the process of creating our model, we will proceed to evaluating the performance of it. For this the metric used is the accuracy metric. Accuracy is the ratio of correct predictions to the total number of predictions made by a model. Hence, it is easy to understand and provides a quick snapshot of a model’s performance. In our programme we will evaluate the models accuracy with the use of the test data as follows:</p><pre>pred = pass_tf.predict(tfidf_test)<br>score = metrics.accuracy_score(y_test,pred)<br>print(f&quot;accuracy: {score}&quot;)</pre><p>When we execute this, we get an accuracy of 96% showcasing that our model operates exceptionally.</p><h4>Recommendation system</h4><p>The task that follows after predicting the user’s medical condition is utilizing that predicted condition to recommend the most appropriate medication .This recommendation system is simply a function that takes in the condition and a data frame and extracts the top 3 drugs present in the data frame under the provided medical condition.</p><p>In order to select the top 3 drugs, the 2 main components analysed is the useful count and the rating in each drug review of the data frame. Having a high useful count suggest that review has been helpful to many other users and having a high rating suggests the efficiency of the drug mentioned in the review. Hence, we utilize these attributes of the data frame to extract the top 3 recommended to the user.</p><pre>#creating the drug recommendation function<br>def top_drugs_extractor(condition,df):<br>    df_top = df[(df[&#39;rating&#39;]&gt;=9) &amp; (df[&#39;usefulCount&#39;]&gt;= 100)].sort_values(by =[&#39;rating&#39;,&#39;usefulCount&#39;],ascending=[False,False])<br>    drug_lst = df_top[df_top[&#39;condition&#39;]== condition][&#39;drugName&#39;].head(3).tolist()<br>    return drug_lst<br><br>#experimenting the function<br>drugs = top_drugs_extractor(&#39;Depression&#39;,df)<br>print(drugs)</pre><p>This concludes the structure and functionality of the 2 main components of our model. To access the to the full complete code and see how both the model operates, visit <a href="https://github.com/ashadiiii/dr_recommendation">this</a> github repo.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af372ebfab66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Practice Made Perfect with AI]]></title>
            <link>https://medium.com/@methasa/practice-made-perfect-with-ai-2e935d19e675?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e935d19e675</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Mon, 08 Jul 2024 13:01:23 GMT</pubDate>
            <atom:updated>2024-07-08T13:01:23.477Z</atom:updated>
            <content:encoded><![CDATA[<p>Since the breakthrough of the ‘Attention is all you need’ paper, significant advancements have surged in AI’s Natural Language Processing (NLP) domain. Transformers leveraging this attention mechanism have paved the way for renowned Large language models like ChatGPT and Gemini, extending their utility beyond mere text generation.</p><p>In this blog, I delve into how I harnessed these Large Language Models (LLMs) to revolutionize education. As a student myself, the adage “practice makes perfect” resonates deeply. However, applying this in practice, especially when falling behind in certain subjects, can be daunting. With the vast expanse of online resources, finding relevant material aligned with the curriculum becomes a challenge, often demotivating students.</p><p>And comes AI to the rescue. With the current exponential developments and discoveries in AI, we can utilize these tools to develop a quiz generator that is tailored based on the topic and the educational resources provided by the user.</p><p>Questions generated in this model will be in the MCQ format. The topic informs the model what those questions must be based on. But rather than making the questions under that topic utilizing the knowledge and information extracted from the internet, we limit the knowledge to only the educational resources provided by the user such that it becomes more specific and relevant to the user requirement. Additionally, the user is also required to input the number of questions expected to generate by the model.</p><p>A basic overview of the model is as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GtgOq-iYdapg6FWZ.png" /></figure><p>According to the above image. The model begins by retrieving all the educational documents provided by the user. Then these documents will be preprocessed such that all the pages of the documents are converted to text embeddings and from those embeddings the one’s that have close semantic relationship with relationship to the topic provided by the user are extracted. Next, they are fed into a Large language model along with the topic, allowing the LLM to generate the MCQ question by utilizing the input data and a customized prompt template that specifies the output the LLM must produce. Once the question is generated it will go through some validation checks in order to ensure that the same question had not been generated by the model and to ensure that the question is in the desired MCQ format. After validating with these checks, the question will be stored in question bank list type variable and the process of generating a new question will be repeated until the number of questions in the question bank is equal to the number of questions requested by the user.</p><p>A few key limitations in the traditional Large Language Models is it’s trained with the data relevant to a certain time period and does not update its knowledge base further with time. Another limitation is that as the data used to train the LLM is based on a wide range of information, the LLM is unable to capture specialized information that the user may require(eg: Name of a soldier that fought in the world war). This causes a huge barrier in formulating quiz questions that are up-to-date and specific to the user’s school curriculum . Hence, in order to address this issue, the RAG AI framework to formulate the suitable prompt for the LLM that contains information about the student’s curriculum as context for the LLM to utilize. This preprocessing with RAG is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*N4kUdWQs60WoALVU.png" /></figure><p>The RAG framework begins by dissecting the large document to smaller chunks. For this, the DocumentProcessor class was used to split the document to individual pages that consisted of text in it(the blank pages were discarded out). Next, the EmbeddingClient class was used to convert the convert each page to a corresponding text embedding. All the text embeddings were then stored in vectorStore database collection, in this case the Chroma Collection. Finally, from all the embeddings present in the database the embeddings that had a high cosine similarity with the topic embedding(query embedding) were extracted.</p><p>The main framework utilized in creating this model was the Langchain framework. Langchain is an open-source framework that allows to combine external data and computations with the LLM. The Langchain framework consisted of various tools and libraries that assisted in the functionality of various sub processes. Examples of such libraries and tools used from langchain are as follows.</p><ul><li>PyPDFLoader — To split the document</li><li>VertexAI — To utilize the ‘gemini-pro’ LLM</li><li>PromptTemplate — To build the prompt template that provides the formulation of the output</li><li>VertexAIEmbeddings — To create vector embeddings</li></ul><p>After building these sub processed, langchain is used to also create links between these subprocesses such that it creates a chain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*NrYVVWBxo6cDjGGR.png" /></figure><p>Hence, leveraging AI and Large Language Models to generate customised quizzes is a game-changer in education. By tailoring practice questions to specific topics and resources, students can focus their efforts more effectively, ultimately making the age-old adage “practice makes perfect” a reality. These advancements not only streamline the learning process but also ensure that the material is relevant and aligned with the curriculum, saving valuable time and reduce frustration.</p><p>In the next blog, I am eager to further delve on the code and workings behind this model, discussing the optimizing strategies and other modifications made to make our learning journey as students much more efficient!</p><p>1</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e935d19e675" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling emotions with AI]]></title>
            <link>https://medium.com/@methasa/unveiling-emotions-with-ai-6c87bebf75d9?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c87bebf75d9</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Thu, 04 Jul 2024 11:41:14 GMT</pubDate>
            <atom:updated>2024-07-04T14:41:20.342Z</atom:updated>
            <content:encoded><![CDATA[<p>After completing the university assignment mentioned in my previous blog, I became intrigued by how well machines can simulate human writing behaviour. This led me to explore the capabilities of Natural Language Processing (NLP) in AI, particularly its application in sentiment analysis. Sentiment analysis uses NLP to understand the underlying sentiment in text, which can then be used for statistical data analysis.</p><p>In this blog, I will explore how sentiment analysis can be used to determine users’ attitudes on various topics based on their tweets. The way the model works is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*l2XVr4Nf4M1XAELqBxvpig.png" /></figure><p>Here’s a step-by-step guide on how the model is structured and built.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*McbQ5R-vhBlz623I8CxjNg.png" /></figure><p>The model is built and trained under the supervised learning technique. Hence, it requires two main components: labels and the corpus. The corpus is a list of sentences (in this case would be a list of tweets) and the labels&#39; dataset would contain the class each sentence in the corpus belongs to. These 2 datasets are used to train the neural network/decision tree model.</p><p>Once the corpus is retrieved, it needs to be pre-processed prior to feeding it to the Neural Network/decision tree for training purposes. This is done by tokenizing the sentences into individual words, followed by dropping the stop words and punctuations from the tokens list. This is because the impact of punctuation and stop words(such as at, be, but, by, etc.) in understanding the sentiment of a sentence is relatively low and would only cause a noise for the model to understand the underlying pattern for each sentiment. Hence, we remove them.</p><p>Machines can only understand numerical values. Hence, in order to allow the machine to understand the natural language, we must convert each token to a corresponding numerical value. In this application each word is represented numerically using the Term Frequency-Inverse Document Frequency (TF-IDF) AI framework.</p><p>Among the many AI frameworks used in NLP, the most suitable framework for classifying a text based on its sentiment is TF-IDF framework, it highlights the words that are most often used in a specific sentiment which allows classifying it to the respective category. The TF-IDF does not necessarily give a semantic representation of each word of a text but instead would provide a statistical measure by evaluating the importance of a word in a document relative to a collection of documents(corpus).</p><p>There are 2 main components that we must calculate in order to calculate the TF-IDF value of a word.</p><ul><li>Term Frequency(TF) — Measures how frequently a term appears in a document. The more times a term appears in a document, the higher its TF value is.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W17KezUprbJZp-ly3irE3w.png" /></figure><ul><li>Inverse Document Frequency(IDF) — Measures how important a term is by considering the number of documents in which the term appears. Words that are common across many documents have lower IDF values, whereas words that only appear in a few documents, have a higher IDF value.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*_yKj8jpd27jaSmY0s9Nm5g.png" /></figure><p>Hence the product of these 2 components give the TF-IDF value for the word in a document.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c1HapgdWxPfsfkkBKujxmw.png" /></figure><p>In this equation, words that are only present in a few documents but are present frequently in those document, have a higher TF-IDF value indicating higher importance in comparison to other commonly used words which have a low TF-IDF value. Therefore, indicating unique values, with greater importance, provides more distinguishing information about the documents in which they appear.</p><p>Once the numerical values for each token are set, we need to understand the non-linear or linear relationship between the features (words). In order to do so there are several approaches that can be taken.</p><ul><li>Random Forest Decision Tree</li><li>Neural Network</li></ul><p>In this case, we use this Random Forest decision tree due to a few main reasons.</p><ul><li><strong>Interpretability:</strong> Random Forests are more interpretable than neural networks, as they allow for the examination of individual decision trees and feature importance.</li><li><strong>Complexity and Flexibility:</strong> Neural networks can model more complex and non-linear relationships in data, making them potentially more powerful for certain tasks.</li><li><strong>Data Requirements:</strong> Neural networks typically require larger datasets to train effectively, while Random Forests can perform well with smaller datasets.</li></ul><p>The way a random forest operates is as follows:</p><ol><li><strong>Bootstrapping</strong>: Random Forest Creates subsets of the original training data by randomly sampling with replacement(bootstrapping). Each subset is used to train a separate decision tree. This process ensures each tree gets a slightly different view of the data. In the image below xi are the features and id are the lines of data.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*ZDqOUP_lHyjnF2fnXJa8Ew.png" /></figure><p>2. <strong>Training Multiple decision trees: </strong>For each decision tree created using each of the subset of data created above, a random subset of features is chosen to split the data at each node. This introduces additional randomness and ensures the trees are not overly correlated. It helps to reduce overfitting and increases the model diversity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*923BOW9DN19IRXP8HUlbag.png" /></figure><p>3. <strong>Aggregating Prediction:</strong> Once all the trees are trained, predictions are made for new instances by passing them through each tree. For classification, the final prediction is the mode(most frequent class) of the individual trees’ predictions. For regression, the final prediction is the mean of the individual tree’s predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*cpQKCKaeK4lZplW2angxPQ.png" /></figure><p>Although this concludes the basic structure of the sentiment analysis model in order to evaluate the accuracy of the model once it is trained, we make a small adjustment to the above structure. And that is to split a section of the cleaned data, vectorized data for testing purposes prior to training the rainforest model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KP2Vr36kF6vqxPi2PMyJ5Q.png" /></figure><p>The extracted test data and labels are then used to evaluate the accuracy of the model as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*tuSx_Ovgk5tB7Yl1OQIhhA.png" /></figure><p>Hence, this model can be used to unveil the underlying emotion of a user’s tweet. In the next blog, I will explain the code and libraries behind building this model.</p><p>This revolution from performing simple calculations with a machine to using it to understand sentiments exemplifies the drastic development of technology and its potential for far greater and impactful use cases in society. Although the journey of NLP and sentiment analysis is still evolving, its applications are already proving to be transformative.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c87bebf75d9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The ‘hello world’ in AI]]></title>
            <link>https://medium.com/@methasa/the-hello-world-in-ai-2244adf555d7?source=rss-21260e3197df------2</link>
            <guid isPermaLink="false">https://medium.com/p/2244adf555d7</guid>
            <dc:creator><![CDATA[Ashadi Methasa De Silva]]></dc:creator>
            <pubDate>Fri, 14 Jun 2024 23:38:03 GMT</pubDate>
            <atom:updated>2024-06-19T09:58:22.364Z</atom:updated>
            <content:encoded><![CDATA[<p>And it begins. On the 14th of February 2023, I created my first AI model. It began with a project that was given by my university professor, where we were assigned to build a text prediction model using functions that he had provided to us.</p><p>Initially, looking at so many equations in the code felt like reading gibberish, not understanding what component is responsible for what task in the model. After critical analysis with my best friend ChatGPT, I was able to break down the functionality of each of the methods provided to us.</p><p>We were given 2 main methods to work with, file2prob and file2pairs. File2prob is a method that creates a probability distribution of the next letter to be predicted based on the number of occurrences of the letter in the whole text. In file2pairs, the conditional probability distribution for the next possible character was created based on a previous character. When experimenting with these 2 methods I was able to understand that the use of probability and context(use of the previous characters) created a more semantically relevant prediction in file2pairs than in file2prob.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Rwa6Kuo2-A0itL6JejXRQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vRPPZc-KIv3l2ZqDBuc6sw.png" /></figure><p>The initial part of the assignment was as simple as creating the model such that it only predicts one character based on the previous character for a given number of iterations. For this, the main method used was file2pairs. This method creates a probability distribution of the possible next character for a given previous character and returns it as a dictionary. By selecting the character that has the highest probability in that distribution, the next character is selected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*QkAxMjl0xpVfeCOqxYPLpQ.png" /></figure><p>Through this part of the assignment, I attempted to utilize the same algorithm to predict the next word instead of character. For this, there was more preprocessing of data than the previous task. Initially, the words the text predicts were collected from a given text. I used the nltk model to parse the sentences into words and to remove any punctuation that may be present in it . Later we used the same method as above file2pairs to create a probability distribution and predict the next word.</p><blockquote>#!/usr/bin/env python</blockquote><blockquote>import nltk<br>import sys<br>import numpy<br>import string</blockquote><blockquote>nltk.download(‘punkt’)</blockquote><blockquote>def tokenise_strings(filename):<br> f = open(filename,”r”)<br> sentences = f.read().split(“.”)<br> tokens = []<br> for s in sentences:<br> t = nltk.word_tokenize(s)<br> tokens = t + tokens</blockquote><blockquote>tokens_dict=set(tokens)<br> new_words_set = []<br> punc = string.punctuation</blockquote><blockquote>for t in tokens_dict:<br> t.lower()<br> if t not in punc:<br> new_words_set.append(t)<br> <br> return new_words_set</blockquote><blockquote>def tonkenise_w_repeating_words(filename):<br> f = open(filename,”r”)<br> sentences = f.read().split(“.”)<br> tokens = []<br> for s in sentences:<br> t = nltk.word_tokenize(s)<br> tokens = t + tokens</blockquote><blockquote>new_words_set = []<br> punc = string.punctuation</blockquote><blockquote>for t in tokens:<br> t.lower()<br> if t not in punc:<br> new_words_set.append(t)</blockquote><blockquote>return new_words_set</blockquote><blockquote>def file2prob(filename):<br> words = tokenise_strings(filename)<br> repreating_words = tonkenise_w_repeating_words(filename)<br> words_dict = { c: 0 for c in words}<br> words_total = 0</blockquote><blockquote>for c in repreating_words:<br> if c.lower() not in words_dict:<br> continue<br> words_dict[c.lower()] += 1<br> words_total += 1</blockquote><blockquote>probs = { c: words_dict[c]/words_total for c in words_dict }<br> return probs</blockquote><blockquote>def file2pairs(filename):<br> “””<br> Read a file and return a dictionary of words and<br> the probabilities of following word. That is, the<br> conditional probability of a word given its<br> predecessor.<br> “””<br> words = tokenise_strings(filename)<br> repeating_words = tonkenise_w_repeating_words(filename)<br> words_dict = { c: { a: 0 for a in words } for c in words }</blockquote><blockquote>previous = None<br> for c in repeating_words:<br> if c.lower() not in words_dict:<br> continue<br> c = c.lower()<br> if previous is None:<br> previous = c<br> continue<br> words_dict[previous][c] += 1<br> previous = c <br> probs = { c: { d: words_dict[c][d]/sum(words_dict[c].values()) if sum(words_dict[c].values()) != 0 else 0 for d in words } for c in words }<br> <br> return probs</blockquote><blockquote>if __name__ == ‘__main__’:<br> import sys<br> <br> if len(sys.argv) != 3:<br> print(“Usage: %s filename letter” % sys.argv[0])<br> sys.exit(-1)<br> <br> filename = sys.argv[1]<br> letter = sys.argv[2]</blockquote><blockquote>probs = file2prob(filename)<br> print(probs[letter])</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vPu_O3ABKF7869tKnhDraw.png" /></figure><p>This model created is a very basic model that can be used to produce text that follow a certain writing style(create Shakespeare poetry). And there it is. The first step to a new journey.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2244adf555d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>