<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jasmeet on Medium]]></title>
        <description><![CDATA[Stories by Jasmeet on Medium]]></description>
        <link>https://medium.com/@sjasmeet135?source=rss-1f96777f55b7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mqj81mFT8pZ_zIYFub0hGw.png</url>
            <title>Stories by Jasmeet on Medium</title>
            <link>https://medium.com/@sjasmeet135?source=rss-1f96777f55b7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:46:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sjasmeet135/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What I Learned from Google’s “Explore Generative AI with Gemini API in Vertex AI” Skill Badge]]></title>
            <link>https://medium.com/@sjasmeet135/what-i-learned-from-googles-exploring-generative-ai-with-gemini-api-in-vertex-ai-skill-badge-7037ed0970fb?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7037ed0970fb</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[gen-ai-exchange-program]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Tue, 01 Jul 2025 20:54:48 GMT</pubDate>
            <atom:updated>2025-07-01T20:56:23.172Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*CkTx-xix56nLV9HG" /><figcaption>Skill Badge</figcaption></figure><p>I recently completed the Intermediate Skill Badge: Exploring Generative AI with Gemini API in Vertex AI, as part of the #GenAIExchange initiative by Google Cloud. This intermediate-level badge offers hands-on practice with some of the most exciting capabilities of the <strong>Gemini models</strong> via <strong>Vertex AI</strong>, including <strong>text generation, multimodal content analysis</strong>, and <strong>function calling</strong>.</p><p>Whether you’re a developer, ML engineer, or GenAI enthusiast, this badge is a great way to get up to speed with Google’s latest generative AI tools and apply them to real-world problems.</p><p>The badge is structured around four labs, each building on top of the previous one to help you develop advanced capabilities in interacting with the Gemini API.</p><p>Here’s a quick breakdown of each lab:</p><h3>Lab 1: Getting Started with Google Generative AI SDK</h3><p><strong>Objective:</strong> Learn to interact with Gemini via the Python SDK.</p><p>This lab is a comprehensive intro to the GenAI SDK. It covers:</p><ul><li>Installing the SDK and setting up authentication</li><li>Sending both <strong>text</strong> and <strong>multimodal prompts</strong></li><li>Using features like <strong>system instructions</strong>, <strong>temperature settings</strong>, <strong>safety filters</strong>, and <strong>multi-turn chats.</strong></li></ul><h3>Lab 2: Multimodality with Gemini</h3><p><strong>Objective:</strong> Use Gemini Flash for <strong>image and video analysis</strong>.</p><p>This was one of the most exciting parts of the badge. I worked with:</p><ul><li>Uploading and referencing <strong>images and videos</strong> in prompts</li><li>Analyzing visual content</li><li>Getting <strong>descriptive and informative outputs</strong> in response to media inputs.</li></ul><h3>Lab 3: Introduction to Function Calling with Gemini</h3><p><strong>Objective:</strong> Turn natural language into structured actions.</p><p>This lab focused on a hot feature in LLMs today: <strong>function calling</strong>. I learned to:</p><ul><li>Generate dynamic function calls from plain English</li><li>Use Gemini to interact with external APIs (like geocoding)</li><li>Extract structured entities from logs or product databases</li></ul><h3>Lab 4: The Challenge Lab</h3><p><strong>Objective:</strong> Build a real-world AI-powered video platform prototype.</p><p>In this final capstone lab, I put all the skills to test in a simulated startup scenario. Challenge was to:</p><ul><li>Implement text generation for video captions</li><li>Enable function calls for metadata extraction</li><li>Use Gemini to analyze and describe video content.</li></ul><h3>Key Learnings</h3><ul><li><strong>Versatility:</strong> The Gemini API is not just for chat. It handles text, images, and video.</li><li><strong>Function Calling:</strong> Adding structured logic to LLM outputs opens doors for automation and integration.</li><li><strong>Developer-Friendly:</strong> The Python SDK makes building and testing seamless within notebooks or apps.</li><li><strong>Real-World Ready:</strong> The challenge lab brought all features together into a use-case that mirrors real product development.</li></ul><h3>Conclusion</h3><p>This badge helped me move beyond basic prompt engineering and start building <strong>multimodal</strong>, <strong>functionally capable</strong>, and <strong>context-aware</strong> GenAI applications.</p><p>If you’re exploring how to integrate powerful AI into your workflows, products, or even passion projects, I highly recommend giving this badge a shot.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7037ed0970fb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What I Learned from Google’s “Build Real World AI Applications with Gemini and Imagen” Course]]></title>
            <link>https://medium.com/@sjasmeet135/what-i-learned-from-googles-build-real-world-ai-applications-with-gemini-and-imagen-course-bd99ee5f7721?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd99ee5f7721</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gen-ai-exchange-program]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Sat, 21 Jun 2025 11:07:45 GMT</pubDate>
            <atom:updated>2025-06-21T11:07:45.735Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*VT2nmv42LHTmmuKd" /><figcaption>Skill Badge</figcaption></figure><p>I Recently completed the <strong>“</strong>Build Real World AI Applications with Gemini and Imagen<strong>”</strong> skill badge as part of the #GenAIExchange initiative by Google Cloud. It was a wonderful experience that helped me dive deeper into some exciting AI tools like Gemini and Imagen.</p><p>The badge consisted of four labs, each designed to build real applications using image recognition, image generation, and natural language processing. Here’s a quick look at what I built and learned.</p><h4><strong>1. Image Recognition with Gemini on Vertex AI</strong></h4><p>In the first lab, I learned how to build an AI image recognition app using the Gemini model. The lab guided me through connecting to Vertex AI and loading a pre-trained generative AI model. I provided both image and text as input and received meaningful text responses.</p><p>This helped me understand how multi-modal models work and how they can be used to power smart applications that see and understand images.</p><h4><strong>2. Image Generation with Imagen on Vertex AI</strong></h4><p>The second lab was all about generating images from text. I used the Imagen model, which is trained to produce realistic images based on text prompts. After connecting to Vertex AI again, I sent a few creative prompts and got back beautiful images.</p><p>This lab showed how text-to-image models are opening new ways to bring imagination to life with just a few lines of code.</p><h4><strong>3. Building a Chat Application with Gemini</strong></h4><p>The third lab focused on natural language processing. I built a simple chat app that uses Gemini to handle conversations. I provided text prompts and received intelligent and contextual replies.</p><p>This helped me see how easy it is to build interactive applications using large language models without needing to train them from scratch.</p><h4><strong>4. Challenge Lab: A Multi-Modal AI Bouquet Generator</strong></h4><p>The final lab was a<strong> challeng lab </strong>and by far the most fun. The task was to create a multi-modal application for a fictional flower bouquet company.</p><p>Clients would describe the kind of bouquet they wanted, and the app would generate an image using Imagen. Then, using Gemini, the app would analyze the image and generate a birthday greeting based on the visual. I wrote two Python functions: one to create the bouquet image and another to analyze it and write a custom message. It was a perfect way to bring together everything I had learned.</p><h4>Conclusion</h4><p>This Skill Badge helped me explore the future of AI development. Using Google’s pre-trained models made it easier to focus on building rather than training. It also showed how generative models can work together combining vision and language in a single app.</p><p>If you’re curious about GenAI and want to build real apps using image and text models, I highly recommend checking out the labs on Google Cloud.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd99ee5f7721" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What I Learned from Google’s “Inspect Rich Documents with Gemini Multimodality and Multimodal RAG”…]]></title>
            <link>https://medium.com/@sjasmeet135/what-i-learned-from-googles-inspect-rich-documents-with-gemini-multimodality-and-multimodal-rag-a03138970bec?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a03138970bec</guid>
            <category><![CDATA[genai-exchange-program]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Fri, 06 Jun 2025 11:29:01 GMT</pubDate>
            <atom:updated>2025-06-06T11:32:49.406Z</atom:updated>
            <content:encoded><![CDATA[<h3>What I Learned from Google’s “Inspect Rich Documents with Gemini Multimodality and Multimodal RAG” Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Rag2ESbYd0tKUz2d" /></figure><p>I recently completed the Intermediate Skill Badge: Inspect Rich Documents with Gemini Multimodality and Multimodal RAG, as part of the #GenAIExchange initiative by Google Cloud.</p><p>Over the course of four hands-on labs, I explored Gemini’s capabilities via the Gemini API in Vertex AI, diving deep into visual understanding, multimodal recommendations, and the innovative application of Retrieval-Augmented Generation (RAG). Here’s what I learned:</p><h3>1. Multimodality with Gemini: Exploring Visual and Video Intelligence</h3><p>In this lab I used the <strong>Gemini Flash</strong> model to understand how it interprets and generates responses based on different modalities: text, images, and video.</p><p>Key capabilities I explored included:</p><ul><li><strong>Image analysis</strong>: detecting objects, interpreting diagrams, understanding UI layouts.</li><li><strong>Video processing</strong>: generating contextual descriptions, identifying highlights, and answering detailed questions.</li></ul><p>It was fascinating to see how a single prompt could guide the model across multiple data types seamlessly.</p><h3>2. Using Gemini for Multimodal Retail Recommendations</h3><p>In this lab, I played the role of a recommender system designer. Starting with a <strong>scene image</strong> (like a living room), I used Gemini-2.0-Flash to perform visual analysis and recommend suitable items (e.g. choosing the best chair from a list).</p><p>Highlights:</p><ul><li>Prompt engineering for <strong>multimodal context</strong>.</li><li>Creating a <strong>visual recommendation pipeline</strong>.</li><li>Understanding how visual cues affect textual inference and vice versa.</li></ul><p>This exercise displayed the real-world potential for Gemini in e-commerce, interior design, and augmented product discovery.</p><h3>3. Multimodal Retrieval Augmented Generation (RAG) with Gemini</h3><p>This lab introduced Multimodal RAG, an innovative method to ground LLM responses using both textual and visual data. I worked with a financial report filled with tables, charts, and paragraphs.</p><p>The benefits of Multimodal RAG over traditional RAG became immediately clear:</p><ul><li><strong>Richer context</strong>: Gemini could “see” and “read” documents, pulling insights from both images and text.</li><li><strong>Improved reasoning</strong>: Visual cues enhanced the accuracy of answers and reduced hallucination risks.</li></ul><h3>4. Inspect Rich Documents with Gemini Multimodality and Multimodal RAG (Challenge Lab)</h3><p>The final challenge lab placed me in a realistic scenario as a Marketing Campaign Coordinator. I was tasked with understanding Google’s brand identity through a rich mix of data: branding guides, past campaigns, ads, testimonials, and financials.</p><p>Tasks completed:</p><ul><li>Extracting insights using <strong>multimodal prompts</strong>.</li><li>Generating a <strong>video description</strong> using just the content.</li><li>Building <strong>document metadata</strong> and retrieving relevant information with <strong>Multimodal RAG</strong>.</li><li>Printing citations to ensure source credibility.</li></ul><p>It was the perfect culmination of everything I’d learned.</p><h3>Conclusion</h3><p>This course was a great introduction to <strong>multimodal generative AI</strong> and how platforms like <strong>Gemini in Vertex AI</strong> are revolutionizing how we extract, analyze, and reason with complex data.</p><p>Whether it’s retail recommendations, document search, or marketing analysis, <strong>Gemini’s multimodal and RAG features</strong> empower us to unlock insights faster, more accurately, and more meaningfully.</p><p>I’m excited to continue building in this space and explore real-world applications of these skills.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a03138970bec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What I Learned from Google’s “Prompt Design in Vertex AI” Course]]></title>
            <link>https://medium.com/@sjasmeet135/what-i-learned-from-googles-prompt-design-in-vertex-ai-course-4313ccb507da?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4313ccb507da</guid>
            <category><![CDATA[certification]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[gen-ai-exchange-program]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[genai]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Thu, 05 Jun 2025 10:14:20 GMT</pubDate>
            <atom:updated>2025-06-05T10:14:20.246Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*wzhpjwvqmgH58tRz" /><figcaption>Course Badge</figcaption></figure><p>I recently completed the Google Cloud Skill Boost’s Introductory Prompt Design in Vertex AI skill badge as part of the #GenAIExchange initiative by Google Cloud, and it has been a hands-on, enriching experience in the fast-evolving world of generative AI. This course is about mastering the ability to shape AI outputs effectively and responsibly using Google’s Vertex AI platform and Gemini models.</p><h4>Course Overview</h4><p>The course is designed to provide foundational and practical skills in prompt engineering, image analysis, and multimodal generative techniques using Vertex AI.</p><p>This course demonstrates how to:</p><ul><li>Craft well-designed prompts</li><li>Guide and control generative outputs</li><li>Apply Gemini models to real-world business problems</li></ul><h4>Lab 1: Getting Started with Prompt Engineering</h4><p>The First Self-Paced Lab in this course covers learning prompt design basics using the Google Gen AI SDK. This lab emphasized best practices in crafting prompts, keeping them concise, specific, and clearly defined.</p><p>I explored use cases such as:</p><ul><li>Ideation for product concepts</li><li>Question answering</li><li>Text summarization</li><li>Text extraction and classification</li></ul><h4>Lab 2: Building Applications in Vertex AI Studio</h4><p>Next, Lab covers a no-code interface for designing and testing powerful generative AI applications. In a simulated scenario, I assisted an insurance company in building a risk analysis assistant.</p><p>The hands-on work included:</p><ul><li>Turning prompt ideas into deployable prototypes</li><li>Using multimodal prompts to analyze images</li><li>Iterating quickly, all within the Google Cloud Console, with no API or SDK requirements</li></ul><h4>Lab 3: Using the Gen AI SDK with Python</h4><p>This lab brought the code into focus. I used the Gen AI SDK for Python to send both text and multimodal prompts.</p><p>Key skills included:</p><ul><li>Configuring model parameters and safety filters</li><li>Managing multi-turn conversations and asynchronous requests</li><li>Exploring features like token counting, function calling, and text embeddings</li><li>Interacting with Gemini models programmatically</li></ul><h4>Challenge Lab: A Real-World Marketing Scenario</h4><p>The final test was a real-world challenge. As part of a startup creating educational content for nature lovers, I collaborated with a fictional retailer <strong>Cymbal Direct </strong>to generate marketing content for outdoor gear.</p><h4>My Tasks:</h4><ul><li>Evocative product descriptions based on image inputs</li><li>Catchy taglines focused on audience appeal and emotional connection</li><li>Applying all my prompt engineering knowledge to build useful tools in Vertex AI Studio</li><li>Exporting and testing Python code in Jupyter Notebooks</li></ul><p>This challenge tied together everything from earlier labs and emphasized the practical potential of <strong>generative AI in marketing</strong>.</p><h4>Conclusion</h4><p>The Introductory Prompt Design in Vertex AI course offered a well-structured and hands-on learning experience. Each lab built on the last, guiding me through core concepts like prompt engineering, text and image analysis, and multimodal interaction using Vertex AI and Gemini models.</p><p>The combination of no-code tools and Python-based SDKs made it easy to understand both the creative and technical sides of generative AI. Overall, the course was practical and is an excellent starting point for anyone looking to explore generative AI with Google Cloud.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4313ccb507da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What I Learned from Google’s “Develop GenAI Apps with Gemini and Streamlit” Course]]></title>
            <link>https://medium.com/@sjasmeet135/what-i-learned-from-googles-develop-genai-apps-with-gemini-and-streamlit-course-996b4f8e6891?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/996b4f8e6891</guid>
            <category><![CDATA[streamlit]]></category>
            <category><![CDATA[genai-exchange-program]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[google]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Wed, 28 May 2025 16:43:53 GMT</pubDate>
            <atom:updated>2025-05-28T16:43:53.130Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*V41PNSrL2wUJvsERexJUTg.png" /></figure><p>I Recently completed the <strong>“Develop GenAI Apps with Gemini and Streamlit”</strong> skill badge as part of the #GenAIExchange initiative by Google Cloud and it was a fun deep dive into building real-world Generative AI applications.</p><p>In this blog, I’ll Walk you through what I learned, the hands-on labs, and how this course helped sharpen my skills in function calling, app deployment with Cloud Run, and integrating Gemini via Gen AI SDK. If you’re exploring GenAI development, this is a suitable place to start.</p><h3>Why This Course?</h3><p>The course is designed to be <em>intermediate </em>difficulty level, meaning you’re expected to have some familiarity with Python and cloud concepts.</p><p>It’s a practical guide to creating GenAI-powered applications using <strong>Google’s Gemini API</strong> with <strong>Streamlit</strong> which is a popular Python framework for building web apps.</p><p>What sets this course apart is the <strong>hands-on approach</strong>. Each module offers focused labs on Google Cloud Platform to cover each concept thoroughly and a <strong>challenge lab</strong> where you’re on your own to solve tasks using everything you’ve learned.</p><h3>Key Concepts Covered in the Course</h3><h3>1. Text Generation with Gemini 2.0 Flash</h3><p>In the first lab, I used <strong>cURL</strong> to interact directly with the <strong>Gemini 2.0 Flash model</strong> on Vertex AI. I explored:</p><ul><li>Sending simple prompts</li><li>Tuning model parameters</li><li>Multimodal prompting (including text from images and videos)</li></ul><p>This reinforced how flexible and fast the Gemini API can be for various text generation tasks.</p><h3>2. Function Calling with Gemini</h3><p>In this lab I learned about function calling using Python. Instead of relying on basic prompts, you can define functions with schemas. Gemini can then suggest which function to invoke and what arguments to pass. It’s like giving your AI assistant a toolbox and letting it decide which tool to use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wqwVK7hvimn1boAxRgE0RQ.png" /><figcaption>Function calling with maps API example in hands-on lab</figcaption></figure><h3>3. Using the GenAI SDK</h3><p>The <strong>Gen AI SDK</strong> simplifies communication with Gemini and removes a lot of boilerplate code.</p><p>I could:</p><ul><li>Set up context-aware prompts</li><li>Control temperature and token limits</li><li>Handle streaming responses</li></ul><p>This SDK makes it easier to embed generative AI features into any Python app.</p><h3>4. Streamlit + Cloud Run Deployment</h3><p>In this lab I used Streamlit to build an interactive interface where users could input prompts and receive AI-generated stories in real time. I then containerized the app with <strong>Docker</strong> and deployed it on <strong>Cloud Run, </strong>enabling scalability and a production-ready endpoint with just a few commands.</p><p>Cloud Shell made development smooth, letting me test everything without leaving the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nixv8BgUDZ0DZG3U" /><figcaption>Streamlit application deployed on Cloud Run</figcaption></figure><h3>5. Challenge Lab</h3><p>The final task of the course was a hands-on <strong>Challenge Lab</strong>, simulating a real-world scenario at <strong>Cymbal Health</strong>, a healthcare network focused on wellness and preventive care.</p><p>The goal of this lab was to build a Proof of Concept for an <strong>AI-powered Chef app</strong> that provides personalized recipe suggestions based on user preferences, allergies, and available ingredients.</p><h4>Tasks:</h4><ul><li>Use <strong>cURL</strong> to test prompts with the <strong>Gemini API</strong></li><li>Complete the <strong>Streamlit app (</strong><strong>chef.py)</strong> to generate recipes</li><li>Test the app locally in <strong>Cloud Shell</strong></li><li><strong>Containerize</strong> the app and push it to <strong>Artifact Registry</strong></li><li>Deploy it to <strong>Cloud Run</strong></li></ul><p>This lab tied everything together, from prompt design to app deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ahhZ6Yux7GEmJboNA-8Znw.png" /><figcaption>AI Chef application running on Streamlit</figcaption></figure><h3>Conclusion</h3><p>After completing all labs and the challenge, I earned a <strong>Google Cloud Skill Badge</strong> — a digital credential that recognizes proficiency in building GenAI applications with Gemini, Streamlit, and Cloud Run.</p><p>If you’re looking to get started with building <strong>generative AI applications </strong>beyond just calling an API this course hits the sweet spot between theory and application.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=996b4f8e6891" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Denoising Diffusion Model Implementation from Scratch]]></title>
            <link>https://medium.com/@sjasmeet135/denoising-diffusion-model-implementation-from-scratch-b0a1fc6ef5d8?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0a1fc6ef5d8</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[editors-pick]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Thu, 06 Jun 2024 21:37:42 GMT</pubDate>
            <atom:updated>2024-06-06T21:37:42.211Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lptkrdXPZ0rMqU0tcJr1A.jpeg" /></figure><h3>Introduction</h3><p>Denoising Diffusion Models are a powerful class of generative models that transform data distributions into noise distributions and then learn to reverse this process to generate new data. They build upon concepts from Hierarchical Variational Autoencoders (HVAEs) and utilize Markov chains for their forward and reverse diffusion processes. This article provides a detailed guide on implementing a denoising diffusion model from scratch, covering the theoretical background, training methodology, and sampling process.</p><h3>From HVAE to Diffusion Model</h3><p>Hierarchical Variational Autoencoders (HVAEs) generalize VAEs by introducing multiple hierarchies over latent variables. In the most general form, HVAEs have T hierarchy levels, with each latent variable allowed to condition on all previous latents. This complexity, while powerful, makes the network extremely intricate and challenging to manage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uwNBUURnLsJYf7dIxn-Z6Q.jpeg" /><figcaption>Latent variable models for VAEs and HVAEs</figcaption></figure><h4>Markovian Assumption Applied to HVAEs</h4><p>To simplify, the Markovian assumption is applied, where each latent variable at time step t is dependent only on the latent variable at time step t−1 kind of similar to RNNs. This assumption significantly reduces the complexity, making it more manageable.</p><h3>Diffusion Processes in Generative Models</h3><p>A diffusion process in generative models is a Markov chain that progressively transforms a data distribution into a known noise distribution. The model then learns to reverse this process to generate new data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*smNjsfvPQ__aZc54tycNJg.png" /><figcaption>directed graphical model for denoising diffusion process</figcaption></figure><h4>Forward Diffusion Process:</h4><p>Forward diffusion process is fixed to a Markov chain that gradually adds Gaussian noise to the data according to a variance schedule β1,β2,…,βT​.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*pp2e_ApkXnKNci4oLLdSdQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/0*IqOQoVZB1ROIUmd3.png" /><figcaption>Forward diffusion process of adding gaussian noise to input image</figcaption></figure><p>The forward process variances βt can be learned by reparameterization or held constant as hyperparameters for this implementation βt is set as an hyperparameter.</p><p>A notable property of the forward process is that it allows sampling xt at an arbitrary timestep ‘t’ in closed form using notation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*IsFLHeuCgAQmh-qrIsVrSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*rmAgvtnCVMhol3VYnrY0ZA.png" /><figcaption>for sampling xt at any timestep ‘t’</figcaption></figure><h4><strong>Reverse Diffusion process</strong>:</h4><p>The joint distribution pθ(x0:T ) is called the reverse process, and it is defined as a Markov chain with learned Gaussian transitions starting at xT∼ N (0, I) (Standard Normal):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*2aamJEiqAPr2l1m2w3pShQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*TCAvwAQdC5Qi0dnkfb6TVw.png" /><figcaption>Reverse Diffusion process</figcaption></figure><h4>Training</h4><p>Similar to VAE’s we develop an Evidence lower bound (ELBO) to get estimate of marginal distribution:</p><p><em>Training is performed by optimizing the usual variational bound on negative log likelihood:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*hqAdX-KtIwQ1Vs-HePfCyA.png" /><figcaption>Variational lower bound</figcaption></figure><p>Efficient training can be achieved by optimizing the random terms of variational bound using stochastic gradient descent. Additional improvements are obtained through variance reduction by rewriting L as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*dhjmhMp0o1GHpyBRbJsnvA.png" /></figure><ol><li><strong>Prior Matching Term (LT​):</strong> Measures the KL divergence between prior xT ~ N(0,1) and final forward noisified image q(xT | x0).</li><li><strong>Consistency Term (Lt-1)</strong>: Measure how consistent is forward nosification q(xt-1 | xt, x0) and reverse de-noisification pӨ(xt-1 | xt) by measuring the KL divergence between the two.</li><li><strong>Reconstruction term (L0)</strong>: Measure how well the image is being reconstructed through the reverse diffusion process.</li></ol><h4>Training steps and Algorithm</h4><p>When training, first, we set:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/272/1*ePVMJAjm_Grdxuxsme2h4A.png" /></figure><p>to untrained time dependent constants:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*QhZRmeMCpsB_Uzt0Hbt3ig.png" /><figcaption>Posterior Variance</figcaption></figure><p>Next, to represent the mean µθ (xt, t):</p><p>With pθ(xt−1|xt) = N (xt−1; µθ (xt, t), σ2t I), we can write L:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*bo03EmntuQ6nRYxkY3TGYA.png" /><figcaption>Consistency term</figcaption></figure><p>However it is much simpler to train on the following variant of the variational bound:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*cW7Mo6lo8ysmtFzrF9xxHA.png" /><figcaption>simpler variational lower bound</figcaption></figure><p>Where ϵ is the actual noise and ϵθ​(..) is the noise predicted by the model, UNet in this case.</p><p>Training Algorithm:</p><p><strong>Repeat until convergence:</strong></p><ul><li>Sample x0​ from the data distribution q(x0).</li><li>Randomly select a time step t from the uniform distribution {1,…,T}.</li><li>Sample noise ϵ from a standard normal distribution N(0,I)</li><li>Take a gradient descent step on the objective:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*cijUHDRx2XsLCNA6w3KOQQ.png" /><figcaption>Gradient of simpler variational bound</figcaption></figure><h4>Sampling</h4><p>The sampling process in the given algorithm involves generating samples from a probabilistic model. Here’s a step-by-step explanation:</p><ol><li><strong>Initialize xT</strong>: xT is drawn from a standard normal distribution xT​∼N(0,1). This serves as the starting point for the sampling process.</li><li><strong>Loop Over Time Steps t:</strong> The loop runs from T down to 1. At each time step t, the algorithm performs the following operations:</li><li><strong>Sample z:</strong> z is drawn from a standard normal distribution N(0,I). This introduces randomness into the sampling process.</li><li><strong>Update xt−1​:</strong> Compute xt−1 using the equation:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*AqpizQOHnppwD0-DnXUgXg.png" /></figure><p>5. <strong>Return the Final Sample x0​:</strong> After the loop completes, the algorithm returns x0​, which is the final sample generated by the process.</p><h3>Implementation</h3><h4>Imports</h4><p>To implement our denoising diffusion model from scratch, we start by importing essential libraries. These include numpy for numerical operations, tqdm for progress tracking, PIL for image processing, and matplotlib.pyplot for plotting. We utilize PyTorch (torch, torchvision, torch.nn, torch.optim) for deep learning tasks, including dataset handling, transformations, and neural network training. Additional utilities like ToPILImage for image conversion and SummaryWriter for logging to TensorBoard are also imported. Device configuration checks if a CUDA-enabled GPU is available, defaulting to the CPU if not, ensuring we leverage GPU acceleration for training on Google Colab.</p><pre>import math<br>import numpy as np<br>from tqdm import tqdm<br>from PIL import Image<br>import matplotlib.pyplot as plt<br><br>import torch<br>import torchvision<br>import torch.nn.functional as F<br>import torch.nn as nn<br>import torch.optim as optim<br>import torchvision.transforms as transforms<br>from torch.utils.data import DataLoader<br>import torchvision.datasets as datasets<br>from torchvision.transforms import ToPILImage<br>from torch.utils.tensorboard import SummaryWriter<br><br><br>device = &#39;cuda&#39; if torch.cuda.is_available() else &#39;cpu&#39;<br>print(device)</pre><h4>Dataset, Transforms and Dataloader:</h4><p>To prepare the dataset for the denoising diffusion model, we define a series of transformations and data loading procedures. First, we set the batch size to 128 and the image size to 32x32 pixels. We use a series of transformations (transforms.Compose) to preprocess the images from the CIFAR-10 dataset. These transformations include resizing, random horizontal flipping, converting images to tensors, and scaling tensor values from [0, 1] to [-1, 1]. The CIFAR-10 dataset is then loaded, applying these transformations, and a DataLoader is used to batch and shuffle the data for training. Additionally, we define a reverse_transforms function to convert processed images back to their original format for visualization. This function rescales the tensor values, permutes the dimensions, scales to the [0, 255] range, converts to a numpy array, and finally transforms the array back into a PIL image. This setup ensures efficient data preprocessing and loading, crucial for training our model.</p><pre>batch_size = 128<br>image_size = 32<br><br>transforms_image_to_tensor = transforms.Compose([<br>    transforms.Resize((image_size, image_size)),<br>    transforms.RandomHorizontalFlip(),  <br>    transforms.ToTensor(),  #Scales image to [0,1]<br>    transforms.Lambda(lambda t: (t * 2) - 1)  #Scale image to [-1, 1]<br>])<br><br>dataset = torchvision.datasets.CIFAR10(<br>    root=&#39;./data&#39;,<br>    transform = transforms_image_to_tensor,<br>    train=True,<br>    download=True,<br>)<br><br><br>dataloader = DataLoader(dataset, batch_size=batch_size,<br>                        shuffle=True,<br>                        drop_last=True,<br>                        num_workers=2)<br><br>def reverse_transforms(image):<br>    reverse_transforms = transforms.Compose([<br>        transforms.Lambda(lambda t: (t + 1) / 2), #Rescale the tensor values from [-1, 1] to [0, 1]<br>        transforms.Lambda(lambda t: t.permute(1, 2, 0)), ##CHW to HWC  (3, 32, 32) -&gt; (32, 32, 3)<br>        transforms.Lambda(lambda t: t * 255.), #Scale the tensor values from [0, 1] to [0, 255]<br>        transforms.Lambda(lambda t: t.numpy().astype(np.uint8)),   #Convert the tensor to a numpy array and change its data type to unsigned 8-bit integers<br>        transforms.ToPILImage(),  #transform numpy array to PIL Image<br>    ])<br><br>    if len(image.shape) == 4:<br>        image = image[0, :, :, :]<br>    return reverse_transforms(image)</pre><h4>Forward Diffusion Process</h4><p>The Forward_diffusion_process class defines the forward diffusion process crucial for the denoising diffusion model. The process starts by initializing with a range (start to end) and the number of timesteps. The noise_scheduler method creates a linearly spaced schedule for the noise variance (beta values) over the timesteps. The get_index method retrieves specific indexed values for the input tensor, reshaping them to match the input image dimensions. The mean_var_helper method computes the posterior variance and mean required for the sampling process, including various cumulative products and square roots of the alpha values. The forward_diffusion_sample method generates noisy samples of input images at a given timestep by adding Gaussian noise scaled by these computed values.</p><pre>class Forward_diffussion_process():<br>  def __init__(self, start, end, timesteps):<br>    self.start = start<br>    self.end = end<br>    self.steps = timesteps<br><br>  def noise_scheduler(self):<br>    beta = torch.linspace(self.start, self.end, self.steps)<br>    return beta<br><br>  def get_index(self, values, t, x_shape):  ##x_shape - shape of input images (32, 3, 64, 64)  (N - batch_size, channels, height, width)<br>    batch_size = t.shape[0] <br>    out = values.gather(-1, t.cpu())  #gather aplha_bar_t th index from alpha cumprod, from last-dim<br>    #print(out)<br>    return out.reshape(batch_size, *((1,) * (len(x_shape) - 1))).to(t.device)<br><br>  def mean_var_helper(self):  ##to get posterior variance and mean for sampling   [bar &lt;=&gt; cumprod]<br>    #Posterior variance<br>    beta = self.noise_scheduler()<br>    alpha = 1. - beta<br>    alpha_cumprod = torch.cumprod(alpha, axis=0)  #alpha_bar<br>    alpha_cumprod_shifted = F.pad(alpha_cumprod[:-1], (1, 0), value=1.0)  #alpha_bar_t_minus_one<br>    sqrt_recip_alpha = torch.sqrt(1.0 / alpha)  #1/sqrt(alpha)<br>    sqrt_alpha_cumprod = torch.sqrt(alpha_cumprod)  #sqrt(alpha_bar)<br>    sqrt_one_minus_alpha_cumprod = torch.sqrt(1. - alpha_cumprod) #sqrt(1- alpha_bar)|<br><br>    posterior_variance = beta * (1. - alpha_cumprod_shifted) / (1. - alpha_cumprod)<br>    return posterior_variance, sqrt_recip_alpha, beta, sqrt_one_minus_alpha_cumprod<br><br><br>  def forward_diffusion_sample(self, input_image, t):<br>    ##gaussian noise<br>    epsilon = torch.randn_like(input_image).to(device)<br>    beta = self.noise_scheduler()<br>    alpha = 1. - beta<br>    alpha_cumprod = torch.cumprod(alpha, axis=0)  #alpha_bar<br>    sqrt_alpha_cumprod = torch.sqrt(alpha_cumprod)  #sqrt(aplha_t)<br>    sqrt_one_minus_alpha_cumprod = torch.sqrt(1.0 - alpha_cumprod) #sqrt(1 - aplha_t)<br>    sqrt_acp_at_t = self.get_index(sqrt_alpha_cumprod,<br>                                   t,<br>                                   input_image.shape,).to(device)<br><br>    sqrt_one_minus_acp_at_t = self.get_index(sqrt_one_minus_alpha_cumprod,<br>                                   t,<br>                                   input_image.shape).to(device)<br>    ##xt = (Input_image) * sqrt(alphas_cumprod (till timestep t)) + (sqrt(1-alphas_cumprod (till timestep t))) * epsilon, epsilon ~ N(0,1)<br><br>    return sqrt_acp_at_t * input_image + sqrt_one_minus_acp_at_t * epsilon, epsilon</pre><h4>Forward Diffusion process Results</h4><p>To visualize the effects of the forward diffusion process, we set the total number of timesteps T to 300 and initialize the Forward_diffusion_process with a noise schedule ranging from 0.0001 to 0.02. Using a sample image from the CIFAR-10 dataset, we progressively apply noise to the image at intervals of 30 timesteps. The forward_diffusion_sample method is used to add Gaussian noise to the image, simulating the corruption process over time. The resulting noisified images are displayed using matplotlib, showing the gradual degradation of the image quality as the noise level increases. Each subplot represents the image at a specific timestep t, illustrating how the image transitions from its original form to a highly noised version through the linear noise scheduling process.</p><pre>T = 300<br>fdp = Forward_diffussion_process(0.0001, 0.02, T)<br>images, _ = next(iter(dataloader))<br>input_image = images[0].permute(1, 2, 0)<br><br>plt.figure(figsize=(10, 10))<br><br>for idx, i in enumerate(range(0, 300, 30)):<br>  t = torch.tensor([i]).type(torch.int64)<br>  noisified_image, _ = fdp.forward_diffusion_sample(input_image.to(device), t)<br>  #print(noisified_image.shape)<br>  plt.subplot(5, 5, idx + 1)<br>  img = show_tensor_image(noisified_image.detach().cpu().permute(2, 1, 0))<br>  plt.imshow(img)<br>  plt.title(f&#39;t = {t.item()}&#39;)<br>  plt.axis(&#39;off&#39;)<br><br>plt.suptitle(&quot;Forward diffusion process (Noising an Image via linear scheduling process)&quot;, fontsize=16)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*KqFXWXI9afB4QUHp_H_cWw.png" /><figcaption>Noising Image at different timesteps ‘t’ through forward diffusion process</figcaption></figure><h4>Reverse diffusion process via UNet with Position Embeddings:</h4><p><strong>Time Embeddings:</strong></p><p>The TimeEmbeddings class is a crucial component of the U-Net architecture used for predicting noise in images during the denoising diffusion process. This class generates time-dependent embeddings that help the model understand the progression of the diffusion process. It initializes with a specified dimension and computes embeddings for given time steps. The embeddings are calculated by first creating a range of values scaled exponentially, then applying sinusoidal functions (sine and cosine) to these scaled values. The resulting embeddings capture periodic patterns in the time steps, which are then concatenated and returned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*Fg1I-GEZ00ZSBRvCm9HWIw.png" /><figcaption>Time Embedding Architecture</figcaption></figure><pre>class TimeEmbeddings(nn.Module):<br>    def __init__(self, dim):<br>        super().__init__()<br>        self.dim = dim<br><br>    def forward(self, time):<br>        device = time.device<br>        half_dim = self.dim // 2<br>        embeddings = math.log(10000) / (half_dim - 1)<br>        embeddings = torch.exp(torch.arange(half_dim, device=device) * -embeddings)<br>        embeddings = time[:, None] * embeddings[None, :]<br>        embeddings = torch.cat((embeddings.sin(), embeddings.cos()), dim=-1)<br>        return embeddings</pre><p><strong>Block (for UNet):</strong></p><p>The Block class in the U-Net architecture processes image features with temporal information. It supports up-sampling and down-sampling modes, using convolutional layers, batch normalization, and ReLU activation. A linear layer integrates time embeddings into the feature map. The forward method processes the input tensor through these layers and adds the reshaped time embeddings, allowing the network to use temporal context effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*-YQ9KJqUAC5adh_1K5DGKQ.png" /><figcaption>Block Architecture</figcaption></figure><pre>class Block(nn.Module):<br>    def __init__(self, in_ch, out_ch, time_emb_dim, up=False):<br>        super().__init__()<br>        self.time_mlp =  nn.Linear(time_emb_dim, out_ch)<br>        if up:<br>            self.conv1 = nn.Conv2d(2*in_ch, out_ch, 3, padding=1)<br>            self.transform = nn.ConvTranspose2d(out_ch, out_ch, 4, 2, 1)<br>        else:<br>            self.conv1 = nn.Conv2d(in_ch, out_ch, 3, padding=1)<br>            self.transform = nn.Conv2d(out_ch, out_ch, 4, stride = 2, padding= 1)<br>        self.conv2 = nn.Conv2d(out_ch, out_ch, 3, padding=1)<br>        self.bnorm1 = nn.BatchNorm2d(out_ch)<br>        self.bnorm2 = nn.BatchNorm2d(out_ch)<br>        self.relu  = nn.ReLU()<br><br>    def forward(self, x, t, ):<br>        h = self.bnorm1(self.relu(self.conv1(x)))<br>        time_emb = self.relu(self.time_mlp(t))<br>        time_emb = time_emb[..., None, None]<br>        h = h + time_emb<br>        return self. Transform(h)</pre><p><strong>UNet Model:</strong></p><p>The Unet class represents a simplified U-Net architecture. It features a sequence of down-sampling and up-sampling layers for encoding and decoding image features. The model incorporates time embeddings using a PositionEmbeddings layer and a linear transformation with ReLU activation. The down-sampling (downs) and up-sampling (ups) paths are constructed using Block layers, which integrate temporal information. The forward method embeds the time step, processes the input through initial convolutions, and iteratively applies down-sampling and up-sampling blocks while preserving skip connections. The final convolutional layer produces the output image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*VqL5QhQqOdfSAmjSWGlGeQ.png" /><figcaption>UNet Architecture</figcaption></figure><pre>class Unet(nn.Module):<br>    &quot;&quot;&quot;<br>    A simplified variant of the Unet architecture.<br>    &quot;&quot;&quot;<br>    def __init__(self):<br>        super().__init__()<br>        image_channels = 3<br>        down_channels = (64, 128, 256, 512, 1024)<br>        up_channels = (1024, 512, 256, 128, 64)<br>        out_dim = 3<br>        time_emb_dim = 32<br><br>        # Time embedding<br>        self.time_embed = nn.Sequential(<br>                PositionEmbeddings(time_emb_dim), #o/p: (64,32) position embeddings<br>                nn.Linear(time_emb_dim, time_emb_dim), #inputs: (64,32), outputs: (64,32)<br>                nn.ReLU()<br>            )<br><br>        self.conv0 = nn.Conv2d(image_channels, down_channels[0], 3, padding=1)  ##output after this layer: (64 or 32, 3, 64, 64)<br><br><br>        self.downs = nn.ModuleList([Block(down_channels[i], down_channels[i+1], \<br>                                    time_emb_dim) \<br>                    for i in range(len(down_channels)-1)])<br><br>        self.ups = nn.ModuleList([Block(up_channels[i], up_channels[i+1], \<br>                                        time_emb_dim, up=True) \<br>                    for i in range(len(up_channels)-1)])<br><br>        self.output = nn.Conv2d(up_channels[-1], out_dim, 1)<br><br>    def forward(self, x, timestep):<br>        # Embedd time<br>        t = self.time_embed(timestep)<br><br>        x = self.conv0(x)<br><br>        residual_inputs = []<br>        for down in self.downs:<br>            x = down(x, t)<br>            residual_inputs.append(x)<br><br>        for up in self.ups:<br>            residual_x = residual_inputs.pop()<br>            x = torch.cat((x, residual_x), dim=1)<br>            x = up(x, t)<br>        out = self.output(x)<br>        return out<br>unet_model = Unet().to(device)</pre><h4>Sampling Function</h4><p>The sampling process involves generating an image by reversing the diffusion steps. The sample_image_at_t function predicts the denoised image at each time step t using the U-Net model. It calculates the mean and variance for the posterior distribution and adds noise based on the timestep. If t is 0, only the mean is returned; otherwise, noise is added. The sample_image function initializes a noisy image and iteratively applies the sample_image_at_t function from the final timestep back to the initial timestep. The process progressively denoises the image, visualizing the intermediate steps at regular intervals.</p><pre>@torch.no_grad()<br>def sample_image_at_t(x_t, t, posterior_variance, sqrt_recip_alpha, beta, sqrt_one_minus_alpha_cumprod):<br><br>    beta_t = fdp.get_index(beta, t, x_t.shape)<br>    sqrt_one_minus_alphas_cumprod_t = fdp.get_index(<br>        sqrt_one_minus_alpha_cumprod, t, x_t.shape<br>    )<br>    sqrt_recip_alphas_t = fdp.get_index(sqrt_recip_alpha, t, x_t.shape)<br><br><br>    model_mean = sqrt_recip_alphas_t * (<br>        x_t - beta_t * unet_model(x_t, t) / sqrt_one_minus_alphas_cumprod_t<br>    )<br>    posterior_variance_t = fdp.get_index(posterior_variance, t, x_t.shape)<br><br>    if t == 0:<br>        return model_mean<br>    else:<br>        noise = torch.randn_like(x_t)<br>        return model_mean + torch.sqrt(posterior_variance_t) * noise<br><br>@torch.no_grad()<br>def sample_image():<br>    # Sample noise<br>    img = torch.randn((1, 3, image_size, image_size), device=device)<br>    plt.figure(figsize=(15,15))<br>    plt.axis(&#39;off&#39;)<br>    num_images = 10<br>    stepsize = int(T/num_images)<br>    posterior_variance, sqrt_recip_alpha, beta, sqrt_one_minus_alpha_cumprod = fdp.mean_var_helper()<br><br>    for i in range(0,T)[::-1]:<br>        t = torch.full((1,), i, device=device, dtype=torch.long)<br>        x_t_minus_1 = sample_image_at_t(img, t, posterior_variance, sqrt_recip_alpha, beta, sqrt_one_minus_alpha_cumprod)  <br>        img = torch.clamp(x_t_minus_1, -1.0, 1.0)<br>        if i % stepsize == 0:<br>            plt.subplot(1, num_images, int(i/stepsize)+1)<br>            plt.imshow(reverse_transform(img.detach().cpu()))<br>            plt.title(f&#39;t = {t.item()}&#39;)<br>    plt.show()</pre><h4>Training</h4><p><strong>Loss and Optimizer:</strong></p><p>To train the U-Net model, we use the Adam optimizer with a learning rate of 1e-3, and the loss function, defined as F1_loss, computes the L1 loss between the actual noise and the predicted noise.</p><pre>from torch.optim import Adam<br>optimizer = Adam(unet_model.parameters(), lr = 1e-3)<br>def F1_loss(noise, noise_pred):<br>  return F.l1_loss(noise, noise_pred)</pre><p><strong>Training Loop:</strong></p><p>The training loop runs for 50 epochs, using a SummaryWriter to log the training progress. For each epoch, the model processes batches of images from the dataloader. In each batch, random timesteps are generated, and the images are noisified accordingly. The U-Net model predicts the noise, and the L1 loss between the predicted and actual noise is computed. This loss is then backpropagated to update the model&#39;s weights using the optimizer. After processing all batches, the average loss for the epoch is logged. Additionally, the sample_image function is called to generate and visualize the denoising process at the end of each epoch, allowing for monitoring the model&#39;s performance over time.</p><pre>num_epochs = 50<br>writer = SummaryWriter(log_dir=&#39;/content/logs&#39;)<br><br>for epoch in range(num_epochs):<br>  epoch_loss = 0<br>  num_batches = 0<br>  for input_image, _ in tqdm(dataloader):<br>    optimizer.zero_grad()<br>    t = torch.randint(0, T, (batch_size,), device=device).long()<br>    input_image = input_image.to(device)<br>    #print(input_image.shape)<br><br>    x_noisy, noise = fdp.forward_diffusion_sample(input_image, t)  #Get noisy image and corresponding noise<br>    model_noise_pred = unet_model(x_noisy, t)  #Predict the noise using the UNet model<br>    loss = F1_loss(noise, model_noise_pred)  #Compute L1 loss between actual noise and predicted noise<br>    loss.backward()<br>    optimizer.step()<br><br>    epoch_loss += loss.item()<br>    num_batches += 1<br><br>  ##logging average loss   <br>  avg_loss = epoch_loss / num_batches<br>  writer.add_scalar(&#39;Loss/train&#39;, avg_loss, epoch)  #Logging average loss per epoch to tensorboard<br>    <br>  print(f&#39;Loss: {loss.item()}&#39;)<br>  sample_image()<br><br>writer. Close()</pre><h4>Results:</h4><p>The results depict the denoising process starting from t=300 down to t=0. As shown in the sequence of images, the initial noisy image (t=270) progressively becomes clearer as t decreases, eventually reconstructing the original image at t=0. This demonstrates the model’s effectiveness in reversing the diffusion process.</p><p>Generated Images:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hrt8v8MnemwJp2hknWlL0w.png" /><figcaption>Generated Image after reverse diffusion process at t = 0</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/64/1*92uT82kXVHQaHpdpt81pYg.gif" /><figcaption>GIF depicting Image generation from noise</figcaption></figure><p>Additionally, the training progress is tracked through a plot of the average loss per epoch, indicating that the model’s performance improves over time, as evidenced by the decreasing loss.</p><p>Loss plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EkU6etaY8elcqmvLIGSeXA.png" /><figcaption>Average loss per epoch</figcaption></figure><h3>Conclusion</h3><p>In this article, we delved into the mechanics of denoising diffusion models and implemented one using a UNet-based architecture. We explored how these models function by gradually adding noise to images and then training a network to reverse this process, effectively removing the noise.</p><p>Diffusion models are revolutionizing generative AI by offering immense capabilities in generating high-quality, diverse samples across various domains. Their ability to model complex data distributions efficiently has led to breakthroughs in tasks such as image generation, text synthesis, and more. By leveraging diffusion models, researchers and practitioners can push the boundaries of creativity and realism in AI-generated content, paving the way for new applications and advancements in the field.</p><h3>References</h3><p>[1] Paper link: <a href="https://arxiv.org/pdf/2006.11239">https://arxiv.org/pdf/2006.11239</a></p><p>[2] Github Implementation: <a href="https://github.com/lucidrains/denoising-diffusion-pytorch">https://github.com/lucidrains/denoising-diffusion-pytorch</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0a1fc6ef5d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Text to Action: Building LLM Applications]]></title>
            <link>https://blog.gopenai.com/from-text-to-action-building-powerful-llm-applications-f9b3ed6d147a?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9b3ed6d147a</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Sun, 19 May 2024 17:01:22 GMT</pubDate>
            <atom:updated>2024-05-28T12:21:36.848Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GxWkRP979aeoC6x3xSKmAQ.jpeg" /></figure><p>Large Language Models (LLMs) have taken the world by storm with their ability to generate human-quality text, translate languages, and answer questions in an informative way. However, for all their power, LLMs come with inherent limitations. This article explores the key issues hindering LLM applications and delves into techniques to overcome these challenges, paving the way for responsible and effective LLM integration.</p><p>Key issues with using LLMs in applications:</p><ul><li><strong>Out of date knowledge</strong>: Internal knowledge of an LLM cuts off at the moment of pretraining.</li><li><strong>Struggles with complex math and reasoning</strong>: Since LLMs usually predict the next best token it is extremely hard for them to carry out mathematical and reasoning based tasks on their own.</li><li><strong>Hallucinations</strong>: LLMs can create seemingly coherent text, even when the information is factually incorrect. This phenomenon, known as hallucination, can lead to misleading outputs.</li></ul><p>There’s a way to break through these roadblocks. By connecting them to external data sources and applications, we can unlock a new level of capability. This allows LLMs to leverage the strengths of external tools, overcoming their limitations and enabling them to tackle a wider range of tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*9KeDpIHrvVUL3zd8dvtdng.png" /><figcaption>Connecting LLMs to External Data sources and Applications</figcaption></figure><h4>Bridging the Gap: Techniques for Enhanced LLMs</h4><p><strong>RAG — Retrieval augmented generation:</strong></p><p>RAG is the process of optimizing the outputs of a LLM, so it references an authoritative and mutable knowledge base outside of the data it was trained on before generating responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*AQjYqYeDXO7-yny9BTiwHw.png" /><figcaption>RAG Architecture</figcaption></figure><p><strong>Setting Vector store for RAG:</strong></p><ul><li><strong>Data Chunking:</strong> Large datasets are broken down into smaller pieces to fit within the LLM’s context window.</li><li><strong>Vector Store Creation:</strong> Information is stored in a format that facilitates efficient retrieval based on relevance to user queries.</li></ul><p><strong>The Retrieval Process:</strong></p><ul><li>The RAG system employs a retriever component that utilizes text embedding-based search to find relevant information from the external knowledge base.</li><li>This retrieved knowledge is then incorporated into an augmented prompt and fed to the LLM for response generation.</li></ul><p><strong>Benefits of RAG:</strong></p><ul><li><strong>Enhanced Accuracy and Relevance:</strong> LLMs leverage external data sources, leading to more accurate and relevant responses.</li><li><strong>Power of Similarity Search:</strong> RAG employs vector similarity for efficient information retrieval.</li></ul><p><strong>Chain-of-Thought Prompting for Reasoning:</strong></p><p>Chain-of-thought prompting tackles the complex reasoning challenges by providing prompts that illustrate logical reasoning steps required to solve a problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*36KP0_coXTgPv3Qb3YixkQ.png" /><figcaption>Chain-of-Thought Prompting</figcaption></figure><p>By breaking down complex problems into step-by-step solutions, these prompts guide the LLM through the thought process, enabling it to approach similar reasoning tasks more effectively.</p><p><strong>PAL- Program aided language models:</strong></p><p>PAL models bridge the gap between LLMs and code interpreters. This allows LLMs to tackle intricate mathematical and reasoning problems by leveraging the strengths of code interpreters in handling complex reasoning steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*retfQVnk2VC3J-HtCQEXGg.jpeg" /><figcaption>PAL Model Architecture</figcaption></figure><p><strong>PAL Prompt Format:</strong> A PAL prompt typically includes multiple examples, each containing a question followed by lines of Python code that demonstrate the reasoning steps to solve the problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*4YQ7AMMkhXLAxv9DhO3mXw.png" /><figcaption>PAL prompt format</figcaption></figure><p><strong>ReAct (Reason + Act): Synergizing Reasoning with Action in LLMs:</strong></p><p>ReAct is a prompting method that combines reasoning and acting with LLMs. ReAct prompts LLMs to generate verbal reasoning traces and actions for a task.</p><p>ReAct uses structured examples to show a large language model how to reason through a problem and decide on actions to take that move it closer to a solution.</p><p>ReAct Prompt format:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*tP6P5IqjPpZmMvr0w3X5Dg.png" /><figcaption>ReAct prompt format</figcaption></figure><p>Question: Problem that requires advanced reasoning and multiple steps to be solved.</p><p>Thought: A reasoning step that identifies how the model will tackle the problem and identify an action to take.</p><p>Action: An external task that the model can carry out from an allowed set of actions.</p><p>Observation: The result of carrying out the action.</p><p><strong>Challenges and Responsible Development for LLM based Applications:</strong></p><p>While LLMs offer immense potential, there are challenges to consider:</p><p><strong>Toxicity:</strong> LLM outputs can be biased or offensive towards certain groups.</p><p><strong>Addressing the Challenges:</strong></p><p>Steps to mitigate Toxicity:</p><ul><li><strong>Data Curation:</strong> Careful selection of training data to minimize bias.</li><li><strong>RLHF Fine-Tuning:</strong> Refining the LLM through a technique called Reinforcement Learning from Human Feedback (RLHF) and using diverse group of human annotators to further mitigate toxic outputs.</li></ul><p><strong>Hallucinations:</strong> As mentioned earlier, LLMs can generate factually or incorrect responses.</p><p>Steps to mitigate hallucination:</p><ul><li><strong>Verified Citation Integration:</strong> Augmenting LLMs with access to trustworthy citation databases.</li><li><strong>Transparency Through Disclaimers:</strong> Informing users about the limitations of generative AI models.</li><li><strong>User Education:</strong> Raising awareness about the capabilities and limitations of generative AI.</li></ul><p><strong>Intellectual Property:</strong> Concerns regarding the use of copyrighted content for LLM training.</p><p>Intellectual Property Considerations<strong>:</strong></p><ul><li><strong>Data Set Curation:</strong> Carefully curating training datasets to ensure compliance with copyright laws.</li><li><strong>Machine Unlearning:</strong> Utilizing machine unlearning techniques to minimize or remove the influence of copyrighted content on the LLM’s outputs.</li></ul><p><strong>Conclusion</strong></p><p>By harnessing the power of RAG, chain-of-thought prompting, and other emerging techniques, we can bridge the knowledge gap and reasoning hurdles that currently limit LLMs. Responsible development that prioritizes data curation and user education is essential to ensure these AI applications are used ethically and effectively. As we forge this LLM-to-world connection, we open doors to a future where these intelligent tools augment human capabilities across diverse fields.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9b3ed6d147a" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/from-text-to-action-building-powerful-llm-applications-f9b3ed6d147a">From Text to Action: Building LLM Applications</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Humanizing AI: How RLHF Bridges the Gap Between Machines and Human Values]]></title>
            <link>https://blog.gopenai.com/humanizing-ai-how-rlhf-bridges-the-gap-between-machines-and-human-values-1f0d340d21cf?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f0d340d21cf</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Fri, 10 May 2024 16:19:08 GMT</pubDate>
            <atom:updated>2024-05-16T16:07:32.467Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GX0wb3pS0zUx0Ji75hipdg.jpeg" /></figure><p>Large Language Models (LLMs) have become a powerful tool, capable of generating human-quality text, translating languages, and writing different kinds of creative content. However, their training on massive amounts of internet data can lead to unintended consequences. LLMs can generate toxic language, misleading information, and even dangerous content. This is where Reinforcement Learning from Human Feedback (RLHF) steps in, offering a way to bridge the gap between AI and human values.</p><h3>Personalizing LLMs from human feedback</h3><p>RLHF personalizes LLMs by incorporating human feedback into the fine-tuning process. Imagine an LLM as a student constantly learning. In traditional LLM training, the data acts as the teacher. With RLHF, humans become additional teachers, guiding the LLM towards generating text that is not only natural-sounding but also aligns with human values like helpfulness, honesty, and harmlessness.</p><h3>Why is RLHF Necessary?</h3><p>LLMs trained on massive datasets can exhibit biases and limitations. Here’s how:</p><ul><li><strong>Toxic Language:</strong> Exposure to hateful or offensive content online can be reflected in LLM outputs.</li><li><strong>Misleading Information:</strong> LLMs can struggle to distinguish between fact and fiction, potentially generating false or misleading information.</li><li><strong>Aggressive Responses:</strong> The impersonal nature of online interactions can lead to aggressive or confrontational language, impacting how LLMs communicate.</li><li><strong>Dangerous Information:</strong> LLMs may generate instructions or code that could be harmful if not carefully reviewed.</li></ul><p>RLHF helps mitigate these issues by incorporating human feedback to steer LLMs towards generating safe, unbiased, and helpful content.</p><h4>Understanding Reinforcement Learning</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*63wvWbA-SgVWsO1zqESaqQ.jpeg" /><figcaption>Reinforcement Learning loop</figcaption></figure><p>Reinforcement Learning involves Agent that tries to learn to make decisions by taking actions ‘at’ in the environment with the objective of maximizing notions of cumulative reward ‘rt’.</p><p>The strategy by which an agent makes decisions is called the <strong>RL policy</strong>.</p><p>The goal of the RL Agent is to learn the optimal policy so that when current state is passed to it, it outputs the action that maximizes the reward. Entire learning process is iterative and involves trial and error.</p><p>For Example: Imagine an agent navigating a maze. It takes actions, receives rewards for good choices, and penalties for bad ones. Over time, the agent learns the optimal policy to maximize rewards given the current state and reach the goal.</p><h3>Building the Foundation: Datasets for RLHF</h3><p>Two key datasets required for the RLHF process:</p><p><strong>Preference Dataset:</strong></p><p>Indicates human labelers preference between two responses generated by the LLM for the prompts in the prompt dataset. Each pair is accompanied by human judgments indicating which response is preferred or considered to be of higher quality. Preference dataset is later used for reward model training.</p><p>Example of <strong>preference dataset</strong>:</p><p>{“input_text”: “Write a review for the movie ‘The Shawshank Redemption’”,</p><p>“candidate_0”: “This movie is a masterpiece. The acting, story, and direction are all top-notch.”,</p><p>“candidate_1”: “I didn’t really like this movie. It was too slow and boring.”,</p><p>“choice”: 0}</p><p>The quality of the preference dataset is crucial. Labelers should be chosen carefully to represent diverse and global perspectives. This ensures the LLM learns preferences that resonate with a broad audience.</p><p><strong>Prompt Dataset:</strong> This dataset provides the prompts that will be given as input to the . A diverse set of prompts ensures the LLM is prepared for various scenarios and user intents.</p><p>Good Example of prompt dataset for tuning an LLM to be deployed on a website related to movie reviews could be a collection of diverse text prompts related to movie reviews, including positive and negative sentiments, different writing styles, and various movie genres.</p><h3>The Reward Model: Translating Preferences into Scores</h3><p>The reward model acts as a bridge between human feedback and the LLM. It is trained in a Supervised setting using the preference dataset, it assigns a score (logits) to each LLM output, indicating how well it aligns with human preferences.</p><p>Higher scores represent better alignment. This allows the LLM to understand which types of completions are considered desirable by humans.</p><h3>The RLHF Process: Aligning the LLM with Human Values</h3><p>The objective of RLHF is to train the LLM to generate text that humans perceive as good.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*8PJa86oWbe_CdJXtUOQvqA.jpeg" /><figcaption>RLHF Process</figcaption></figure><p>The LLM acts as the agent, constantly refining its policy to maximize the reward received from the reward model. Each time the LLM generates text, it receives a reward based on the reward model’s score.</p><p>Over time, the LLM’s internal weights are tuned via policy gradient method called Proximal Policy Optimization (PPO) to favor generating outputs that consistently receive high rewards.</p><p>This continuous feedback loop aligns the LLM with human preferences, enabling it to produce helpful, honest, and harmless content.</p><p>The entire process in action could look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*qMMy-Tu5oCKdO07gTGSXSA.jpeg" /><figcaption>RLHF in action</figcaption></figure><h3>Reward Hacking</h3><p>A potential challenge in RLHF is “reward hacking.” The LLM might learn to exploit loopholes in the reward system, generating outputs that maximize rewards without truly aligning with human values.</p><p>To prevent this, we can compare the LLM’s outputs after RLHF with its initial outputs and measure how much they diverge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*U9tWkcHcUvDackPPFDzHcw.jpeg" /><figcaption>RLHF with KL-divergence shift penalty</figcaption></figure><p>KL-divergence helps identify significant deviations, ensuring the LLM stays on track with human preferences.</p><h4>Conclusion</h4><p>RLHF is a powerful tool for shaping AI that aligns with human values. As research progresses, we can expect LLMs to become even more adept at generating text that is helpful, honest, and harmless. However, continuous refinement of reward models and attention to potential biases remain crucial. RLHF holds the key to unlocking the full potential of LLMs for good, paving the way for a more human-centric future of AI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f0d340d21cf" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/humanizing-ai-how-rlhf-bridges-the-gap-between-machines-and-human-values-1f0d340d21cf">Humanizing AI: How RLHF Bridges the Gap Between Machines and Human Values</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning Large Language Models Efficiently with QLoRA]]></title>
            <link>https://blog.gopenai.com/fine-tuning-large-language-models-efficiently-with-qlora-e3af8eedef36?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3af8eedef36</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Wed, 01 May 2024 14:35:26 GMT</pubDate>
            <atom:updated>2024-05-06T18:41:28.695Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WTbZDHfXcBVlncI3YopNTQ.png" /></figure><p>Large language models (LLMs) have revolutionized the field of natural language processing (NLP). However, their size often poses challenges in terms of memory consumption and computational cost. This is where techniques like QLoRA come in, enabling efficient fine-tuning of LLMs for specific tasks without sacrificing performance.</p><p>This article delves into the inner workings of QLoRA, exploring its key components:</p><p><strong>1. Quantization:</strong></p><p>At the heart of QLoRA lies <strong>quantization</strong>, a technique that reduces the memory footprint of LLM models. Traditionally, LLM parameters are stored in 32-bit floating-point format (FP32). QLoRA utilizes <strong>4-bit NormalFloat (NF4)</strong>, a format specifically designed for data with a normal distribution. This significantly reduces memory usage, making LLMs more deployable on resource-constrained devices.</p><p>Let’s understand what is Quantization and how does QLoRA quantizes FP32 format to 4-bit Normal Float(NF4)</p><p><strong>Quantization:</strong> This is a technique used to reduce the memory footprint of machine learning models, particularly large language models (LLMs). It achieves this by storing the model’s parameters (weights) using only 4 bits or 8 bits instead of the standard 32 bits (float32) or even 16 bits (float16). This significantly reduces memory usage.</p><p><strong>Quantization process:</strong></p><p>8 bits or a Byte gives us 8 binary digits, each digit being 0 or 1 so we get 2⁸ = 256 unique combinations so that each digit can be mapped in a range of [-127, 127].</p><p>So, a tensor X of FP32 format in model parameters can be quantized into INT8 so that the values are mapped within the range [-127, 127] by:</p><p><em>XINT8 </em>= <em>round((127 / absmax(XFP32) * XFP32) = round(cFP32* XFP32)</em></p><p>To dequantize tensor back to its original format FP32:</p><p><em>dequant(cFP32, XINT8) = XINT8 / cFP32</em></p><p>Where, (127 / absmax(XFP32) is the <strong>quantization constant</strong> <em>cFP32.</em></p><p><strong>4-bit Normal float (NF4):</strong></p><p>Introduced by the QLoRA fine-tuning method Hugging Face: <a href="https://huggingface.co/blog/4bit-transformers-bitsandbytes">https://huggingface.co/blog/4bit-transformers-bitsandbytes</a></p><p>This is a specific type of quantization format designed for data with a normal distribution (bell-shaped curve), which is particularly relevant for LLM weights. Since LLM weights are typically distributed normally with mean 0 and variance 1, instead of using an equally spaced quantization process, 4-bit NormalFloat (NF4) leverages this inherent distribution. This approach offers better performance compared to the alternative format, standard 4-bit float (FP4), for such data.</p><p>While 4-bit Normal Float quantization significantly reduces the memory footprint of LLMs, making them more deployable on devices with limited resources, it’s important to note that this efficiency comes with a potential trade-off: a slight loss in accuracy compared to using full precision (float32).</p><p><strong>2. Double Quantization:</strong></p><p>During quantization, we calculate a constant (cFP32) based on the absolute maximum value in the tensor (absmax(XFP32)). This process can be sensitive to outliers, as a single extreme value can significantly impact cFP32. To address this, we use <strong>block-wise quantization</strong>. Here, the input tensor is divided into smaller chunks (blocks), and each block gets its own quantization constant.</p><p>For example, consider a tensor XFP32 with the following values:</p><p>Tensor X: ([[0.9854, -0.045, 0.2454,-0.0678, 0.0072, 0.1278, -0.0087, 0.2739]])</p><p>If we calculate a single quantization constant <em>cFP32 = (127 / absmax(XFP32)</em> for the entire tensor, the outlier (0.9854) might dominate which may lead to suboptimal quantization for the remaining values.</p><p>We avoid quantizing the entire tensor at once. Instead, we subdivide it into smaller units (blocks) and determine a unique quantization constant for each block.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*sLNypZGHgFo1sPEdgeIE4Q.png" /><figcaption>Block-wise Quantization</figcaption></figure><p>We can further optimize this process with <strong>double quantization</strong>. Here, we quantize the individual quantization constants (CFP32) from each block into a lower-precision format.</p><p>CINT8 = <em>round((127 / absmax(CFP32) * CFP32)</em></p><p><strong>3. Paged Optimizers:</strong></p><p>Training large models often involves extensive computations on GPUs. However, memory limitations can arise during optimization.</p><ul><li>Paged optimizers operate on the principle of virtual memory used in operating systems.</li><li>They divide the training data and optimizer calculations into smaller units called “pages.”</li><li>Only the currently required pages are loaded onto the GPU memory, while the rest reside in the CPU’s main memory.</li><li>As the optimization progresses, paged optimizers swap pages between CPU and GPU efficiently. Pages containing outdated information are moved to CPU memory, freeing up space on the GPU for new calculations.</li></ul><p>4. <strong>LoRA:</strong></p><p>LoRA further reduces the number of trainable parameters to be trained by freezing most of the original LLM weights. It injects two rank decomposition matrices alongside original weights. Dimensions of these matrices are set so that their product is the same dimension as the weights they are modifying.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/0*hiqAzazlX5G_NqRh.png" /><figcaption>Low Rank Adaptation of weights by LoRA</figcaption></figure><p>For example:</p><p>A base transformer model has weights of dimension 512 * 512, which results in 262144 trainable parameters, instead of training all the parameter we inject two rank decomposition matrices A and B of rank = 8, which makes A and B of dimensions = 8 * 512 and 512 * 8, which totals to 8192 parameters giving us up to 97% reduction in parameters.</p><p><strong>In essence, LoRA acts as a fine-tuning accelerator within QLoRA.</strong></p><p>By combining these techniques, QLoRA offers a compelling solution for efficient fine-tuning of LLMs. It reduces memory footprint, minimizes training time, and delivers excellent performance, making LLMs more accessible for various applications..</p><p><strong>Impact of QLoRA:</strong></p><ul><li><strong>Reduced Memory Footprint:</strong> QLoRA significantly reduces the memory required to store and train LLMs. This allows for training LLMs on devices with limited memory, such as mobile phones and edge devices and fine-tuning pre-trained LLMs for specific tasks without requiring massive computational resources.</li><li><strong>Faster Training Times:</strong> Paged optimizers and efficient memory management by QLoRA lead to faster training cycles. Quicker development and deployment of LLM-powered applications and increased experimentation and exploration of various LLM fine-tuning possibilities.</li><li><strong>Excellent Performance:</strong> Despite the memory reduction, QLoRA maintains excellent performance with minimal accuracy loss compared to full-precision training.</li></ul><p>In <strong>conclusion</strong>, QLoRA is a game-changer for fine-tuning LLMs. By overcoming memory and training time hurdles, it unlocks the potential of these powerful language models for a wider audience and opens doors to future innovations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3af8eedef36" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/fine-tuning-large-language-models-efficiently-with-qlora-e3af8eedef36">Fine-Tuning Large Language Models Efficiently with QLoRA</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating Language Models: Metrices and Benchmarking]]></title>
            <link>https://blog.gopenai.com/evaluating-language-models-metrices-and-benchmarking-2cd18d5e893a?source=rss-1f96777f55b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cd18d5e893a</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jasmeet]]></dc:creator>
            <pubDate>Sat, 27 Apr 2024 19:51:00 GMT</pubDate>
            <atom:updated>2024-04-30T15:46:00.511Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wCFBrbJ5vY1rjaxtRYepNg.jpeg" /></figure><p>Language models (LLMs) have become indispensable tools in natural language processing (NLP), powering a wide range of applications from text summarization to machine translation. However, evaluating the performance of these models is significantly challenging due to their non-deterministic nature and the complexity of language understanding. In this article, we delve into various evaluation techniques and metrics employed in assessing the effectiveness of LLMs, including popular metrics like ROUGE and BLEU scores, as well as benchmark datasets.</p><p><strong>ROUGE Metrics for Text Summarization</strong></p><p>ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of metrics commonly used for evaluating text summarization models. It compares the generated summary against one or more reference summaries produced by humans.</p><p><strong>ROUGE-1</strong> measures unigram overlap between the generated summary and the reference summary. It calculates recall, precision, and F1 score based on the matching unigrams.</p><p>Example:</p><p>Reference summary: ‘They are playing outside.’</p><p>Generated Summary: ‘They are not playing outside.’</p><p>Rouge-1 (Recall) = <em>(no. of unigram matches)/(unigram in reference)</em> = 1.0</p><p>Rouge-1 (Precision) = <em>(no. of unigram matches)/(unigram in generated) </em>= 0.8</p><p>Rouge-1 (F-1) = <em>2∗(Precision × Recall)/(Precision + Recall)</em> = 0.89</p><p>ROUGE-2 extends this to bigram overlap, while ROUGE-L utilizes the longest common subsequence (LCS) between the two summaries.</p><p>Example:</p><p>Reference summary: ‘They are playing outside.’</p><p>Generated Summary: ‘They are not playing outside.’</p><p>Reference and generated summaries have 2 longest common subsequences: ‘They are’ and ‘playing outside’</p><p>Rouge-L (Recall) = <em>LCS(reference, summary)/(unigrams in reference) </em>= 0.5</p><p>Rouge-L (Precision) = <em>LCS(reference, summary)/(unigrams in generated) </em>= 0.4</p><p>Rouge-L (F-1) = <em>2∗(Precision × Recall)/(Precision + Recall) </em>= 0.44</p><p>An important consideration in ROUGE evaluation is the potential for inflated scores, especially in cases where the generated summary simply repeats words from the reference. To mitigate this, modified precision measures is used, clipping the count of a word in the generated summary by its maximum occurrence in the reference.</p><p>Example:</p><p>Reference summary: ‘They are playing outside.’</p><p>Generated summary: ‘playing playing playing playing.’</p><p>For this generation ROUGE-1 precision score would be = ⁴⁄₄ = 1.0</p><p>So instead, we use modified precision where we clip the count of a word in generated summary by maximum number of times it appears in the reference summary.</p><p>So, in this case ROUGE-1 (modified precision)</p><p>= <em>clip(no. of unigram matches)/(unigram in generated) = </em>¼ = 0.25</p><p><strong>BLEU Score for Machine Translation</strong></p><p>While ROUGE focuses on summarization, BLEU (Bilingual Evaluation Understudy) is a metric commonly used for evaluating machine translation systems.</p><p>BLEU calculates precision over a range of n-gram sizes, comparing the generated translation against one or more reference translations. As the generated translation aligns more closely with the reference, the BLEU score increases, ranging between 0 and 1.</p><p><strong>Benchmark Datasets for LLMs</strong></p><p>In addition to specific evaluation metrics, benchmark datasets play a crucial role in assessing the overall performance of LLMs across a diverse range of tasks. Benchmark datasets such as GLUE, Super GLUE or Helm cover a wide range of tasks on different scenarios. They do this by collecting and preparing datasets that test specific aspects of an LLM.</p><p><strong>GLUE (General Language Understudy Evaluation, 2018)</strong>:</p><p>GLUE is a tool for evaluating and analyzing the performance of models across a diverse range of existing natural language understanding tasks:</p><ul><li>Sentiment Analysis: Assessing the sentiment of a given text.</li><li>Text Classification: Categorizing text into predefined classes.</li><li>Text Similarity: Similarity measurement between two pieces of text.</li><li>Question Answering: Finding relevant answers to user questions based on a given context.</li><li>Named Entity Recognition: Identifying and classifying named entities in text, such as person names, organizations, and locations.</li></ul><p>The format of the GLUE benchmark is model-agnostic, so any system capable of processing sentence and sentence pairs and producing corresponding predictions is eligible to participate.</p><p><strong>Super GLUE (2019):</strong></p><p>Super GLUE builds upon GLUE by introducing more challenging tasks involving multi-sentence reasoning and reading comprehension.</p><p>The task formats in GLUE are limited to sentence- and sentence-pair classification Super GLUE expands on this by including tasks which involves multi sentence reasoning and reading comprehension.</p><p>Following are some tasks added in Super GLUE:</p><ul><li>BoolQ (Boolean Questions) is a QA task where each example consists of a short passage and a yes/no question about the passage.</li><li>COPA (Choice of Plausible Alternatives) is a causal reasoning task in which a system is given a premise sentence and must determine either the cause or effect of the premise from two possible choices.</li><li>WiC (Word-in-Context) is a word sense disambiguation task. The task is to determine whether the word in two sentences is used with the same sense in both.</li></ul><p><strong>Benchmarks for massive LLMs:</strong></p><p><strong>BIG BENCH (Beyond the Imitation Game Benchmark, 2022):</strong></p><p>Introduced as the Beyond the Imitation Game Benchmark, encompasses 204 tasks spanning areas like linguistics and childhood development. This comprehensive benchmark covers over 1000 written languages, including synthetic and programming languages, reflecting the diverse linguistic landscape. Additionally, it offers lite and Big Bench hard variants to cater to varying levels of task complexity and evaluation.</p><p><strong>HELM (Holistic Evaluation of Language Models):</strong></p><p>HELM, standing for Holistic Evaluation of Language Models, adopts a multimeric strategy by assessing seven metrics across 16 distinct scenarios. These metrics include accuracy, calibration, robustness, fairness, bias, toxicity score, and efficiency, collectively providing a comprehensive evaluation of LLM performance. By considering various aspects of model behavior, HELM enhances the transparency and accountability of language models.</p><p><strong>MMLU (Massie Multitask Language Understanding, 2021)</strong>:</p><p>A benchmark dataset designed to measure knowledge acquired during pretraining by evaluating models exclusively in zero-shot and few-shot settings. This benchmark dataset focuses primarily on topics such as Law, Computer Science, and US History.</p><p><strong>Conclusion</strong></p><p>Evaluating the performance of language models is a challenging task that requires careful consideration of metrics and benchmark datasets. Metrics like ROUGE and BLEU provide valuable insights into specific aspects of model performance, while benchmark datasets like GLUE and Super GLUE offer comprehensive evaluations across diverse NLP tasks. By leveraging a combination of evaluation techniques and benchmarks, researchers and practitioners can gain a deeper understanding of LLM capabilities and drive advancements in natural language processing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cd18d5e893a" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/evaluating-language-models-metrices-and-benchmarking-2cd18d5e893a">Evaluating Language Models: Metrices and Benchmarking</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>