<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pranav on Medium]]></title>
        <description><![CDATA[Stories by Pranav on Medium]]></description>
        <link>https://medium.com/@pady0014s?source=rss-7beca7f0145d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uoohYL_kHFfJ9ArzXVKzyw.png</url>
            <title>Stories by Pranav on Medium</title>
            <link>https://medium.com/@pady0014s?source=rss-7beca7f0145d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 23:08:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pady0014s/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Prompt Engineering Notes — Part 1]]></title>
            <link>https://medium.com/@pady0014s/prompt-engineering-notes-part-1-d60c5f7cbb8a?source=rss-7beca7f0145d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d60c5f7cbb8a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Pranav]]></dc:creator>
            <pubDate>Sun, 04 Aug 2024 14:08:37 GMT</pubDate>
            <atom:updated>2024-08-04T14:08:37.344Z</atom:updated>
            <content:encoded><![CDATA[<h3>Prompt Engineering Notes — Part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wDNPiyNMIPG48DVa.png" /><figcaption>Google</figcaption></figure><p>In recent years, artificial intelligence (AI) has evolved from a futuristic concept to a transformative force across various industries. At the heart of this revolution is the ability of AI models to understand and generate human-like text, unlocking a world of possibilities for applications ranging from customer service to creative writing. However, leveraging this power requires a unique skill set known as <strong>prompt engineering</strong>.</p><p>Prompt engineering is the art and science of crafting inputs, or “<em>prompts,</em>” that guide AI models to produce desired outputs. It involves understanding the nuances of AI behavior, mastering techniques to fine-tune responses, and creatively exploring how AI can be applied to solve real-world problems. In this introductory blog, we will explore the fundamentals of prompt engineering, its significance, and the potential it holds for various domains.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vUdFROzaVRpRtga2" /></figure><h3><strong>What is Prompt Engineering?</strong></h3><p><strong>Prompt engineering</strong> is the practice of designing, refining, and optimizing input queries, known as prompts, to elicit desired responses from AI language models like OpenAI’s GPT series, Google’s Bard, and others. This practice is essential for maximizing the effectiveness of AI in various applications, from automated content generation to complex decision-making processes.</p><h4>A Brief History and Evolution of Generative AI</h4><p>The journey of generative AI began with rule-based systems, which were limited in their ability to handle complex language tasks. The advent of neural networks and deep learning marked a significant shift, leading to the development of models like OpenAI’s GPT-3 and GPT-4, which are capable of understanding and generating human-like text.</p><p>These models are trained on vast datasets containing diverse linguistic patterns, allowing them to generate contextually relevant and coherent text. However, their output is heavily influenced by the quality and structure of the input prompts, making prompt engineering a crucial skill for unlocking their full potential.</p><h4>The Role of Prompt Engineering in Enhancing AI Capabilities</h4><p>Prompt engineering plays a pivotal role in shaping AI behavior. By carefully crafting prompts, users can guide AI models to:</p><p>- <strong>Generate specific content:</strong> Create articles, reports, and creative pieces with tailored narratives.<br>- <strong>Answer questions accurately:</strong> Provide precise information in response to user queries.<br>- <strong>Solve problems:</strong> Assist in decision-making by offering insights and recommendations.<br>- <strong>Simulate conversations:</strong> Engage in interactive dialogues for customer support and virtual assistance.</p><h3>Why Prompt Engineering Matters</h3><p>In an age where AI is becoming increasingly integrated into our daily lives, the importance of effective communication with AI models cannot be overstated. Prompt engineering enables users to harness the full potential of AI, ensuring that the generated content is relevant, accurate, and aligned with their goals.</p><h4>Applications Across Different Industries</h4><p>Prompt engineering is revolutionizing various sectors:</p><p>- <strong>Healthcare:</strong> AI can assist in diagnosing diseases, providing treatment recommendations, and generating patient reports.<br>- <strong>Education: </strong>Personalized learning experiences can be crafted by tailoring educational content to individual needs.<br>- <strong>Marketing:</strong> Businesses can leverage AI to create engaging content, optimize ad campaigns, and analyze consumer behavior.<br>- <strong>Finance:</strong> AI can provide financial analysis, fraud detection, and investment recommendations.</p><h4>Improving AI Efficiency and Relevance</h4><p>Effective prompt engineering enhances AI’s ability to deliver precise and contextually appropriate responses, reducing the need for manual intervention and post-processing. This efficiency translates into time and cost savings for businesses and individuals alike.</p><h3>Key Concepts in Prompt Engineering</h3><p>To master prompt engineering, it is essential to understand the key concepts that influence AI behavior:</p><h4>Understanding AI Models and Their Behavior</h4><p>AI models are statistical systems that generate text based on patterns learned from training data. They are not sentient beings but rather sophisticated algorithms that produce outputs based on input prompts. Recognizing this distinction is crucial for managing expectations and effectively guiding AI interactions.</p><h4>Types of Prompts</h4><p>There are several types of prompts, each serving different purposes:</p><p><strong>Directive Prompts</strong></p><p>Directive prompts instruct the AI to perform specific tasks, such as generating a summary or translating text.</p><pre>&quot;Summarize the following article on climate change.&quot;</pre><p><strong>Example:</strong><br><strong>Input:</strong></p><pre>&quot;Summarize the following article on climate change.&quot;</pre><p><strong>Output:</strong><em><br>“Climate change refers to long-term alterations in temperature, precipitation, and weather patterns, primarily caused by human activities such as burning fossil fuels and deforestation. These changes have led to global warming, resulting in melting ice caps, rising sea levels, and increased frequency of extreme weather events.”</em></p><p><strong>Creative Prompts</strong></p><p>Creative prompts encourage the AI to generate imaginative or artistic content.</p><pre>&quot;Write a short story about a time-traveling detective.&quot;</pre><p><strong>Example:</strong><br><strong>Input:</strong></p><pre>&quot;Write a short story about a time-traveling detective.&quot;</pre><p><strong>Output:<em><br></em></strong><em>“Detective Sarah Harper always had a knack for solving the unsolvable. But when she stumbled upon a mysterious time-travel device, she found herself thrust into the past, tasked with solving crimes before they even happened. Her first mission? Prevent the heist of the century in 1920s Paris, where every clue brought her closer to uncovering a plot that could change history forever.”</em></p><p><strong>Informative Prompts</strong></p><p>Informative prompts request factual information or explanations.</p><pre>&quot;Explain the process of photosynthesis.&quot;</pre><p><strong>Example: </strong><br><strong>Input:</strong></p><pre>&quot;Explain the process of photosynthesis.&quot;</pre><p><strong>Output<em>:</em></strong><em><br>“Photosynthesis is the process by which green plants, algae, and some bacteria convert light energy, usually from the sun, into chemical energy in the form of glucose. This process takes place in the chloroplasts of plant cells, where chlorophyll absorbs light energy, water molecules are split, and carbon dioxide is captured to produce glucose and oxygen.”</em></p><h4>The Impact of Context, Tone, and Specificity</h4><p>The effectiveness of a prompt is influenced by its context, tone, and specificity:</p><p>- <strong>Context:</strong> Providing relevant background information can improve the accuracy of AI responses.<br>- <strong>Tone:</strong> The tone of the prompt can guide the AI’s response style, whether formal, casual, humorous, or professional.<br>- <strong>Specificity:</strong> Clear and precise prompts lead to more focused and accurate outputs.</p><p><strong>Example: </strong><br><strong>Input:</strong></p><pre>&quot;Create a professional email to a potential client introducing our new product line of eco-friendly cleaning supplies.&quot;</pre><p><strong>Output: </strong><br><em>“Subject: Introducing Our New Eco-Friendly Cleaning Supplies</em></p><p><em>Dear [Client’s Name],</em></p><p><em>I hope this message finds you well. I’m excited to introduce our latest line of eco-friendly cleaning supplies designed to deliver exceptional results while minimizing environmental impact. Our products are made from natural ingredients and packaged in sustainable materials, making them a perfect choice for environmentally-conscious businesses.</em></p><p><em>We believe our eco-friendly solutions can add significant value to your operations, and I would love to discuss how we can support your sustainability goals. Please let me know if you are available for a brief call or meeting at your convenience.</em></p><p><em>Thank you for considering our products.</em></p><p><em>Best regards,</em></p><p><em>[Your Name] <br>[Your Position] <br>[Your Company]”</em></p><h3>Techniques for Crafting Effective Prompts</h3><p>Crafting effective prompts is both an art and a science. Here are some techniques to help you get started:</p><h4>The Iterative Process of Refining Prompts</h4><p>Prompt engineering is an iterative process that involves experimenting with different prompt structures, analyzing outputs, and refining inputs to achieve desired results. This experimentation is crucial for understanding how AI models interpret and respond to various prompts.</p><p><strong>Example Iteration:</strong></p><ol><li><strong>Initial Prompt:</strong></li></ol><pre>&quot;Tell me about the solar system.&quot;</pre><p><strong><em>Output: </em></strong><em><br> “The solar system consists of the Sun and the celestial bodies that orbit it, including eight planets, their moons, asteroids, and comets.”</em></p><p>2. <strong>Refined Prompt:</strong></p><pre>&quot;Explain the key characteristics of each planet in the solar system.&quot;</pre><p><strong>Output:</strong> <br><em> “The solar system consists of eight planets, each with unique characteristics: Mercury is the smallest and closest to the Sun, with extreme temperatures. Venus is similar in size to Earth but has a toxic atmosphere and high surface pressure. Earth supports life with its water-rich environment. Mars, known as the Red Planet, has the largest volcano and canyon. Jupiter is the largest, with a strong magnetic field and Great Red Spot storm. Saturn is famous for its prominent ring system. Uranus rotates on its side and has a faint ring system. Neptune, the farthest, has strong winds and a dynamic climate.”</em></p><h4>Experimentation and Adaptation for Improved Results</h4><p>Experimenting with variations of prompts allows users to adapt their approach based on the AI’s responses. By analyzing the output, users can identify areas for improvement and adjust prompts accordingly.</p><p><strong>Example: </strong><br><strong>Initial Prompt:</strong></p><pre>&quot;Generate a list of popular books.&quot;</pre><p><strong>Output: </strong><br><em>“Here are some popular books: <br>1. ‘The Great Gatsby’ by F. Scott Fitzgerald <br>2. ‘1984’ by George Orwell <br>3. ‘To Kill a Mockingbird’ by Harper Lee <br>4. ‘The Catcher in the Rye’ by J.D. Salinger <br>5. ‘Pride and Prejudice’ by Jane Austen”</em></p><p><strong>Refined Prompt:</strong></p><pre>&quot;Generate a list of popular science fiction books released in the last decade, including their authors and a brief description of each.&quot;</pre><p><strong>Output: </strong><br><em>“Here are some popular science fiction books released in the last decade:</em></p><p><em>1. ‘The Three-Body Problem’ by Liu Cixin — This book is the first in a trilogy exploring humanity’s first contact with an alien civilization. It delves into complex scientific concepts and the societal impact of discovering intelligent life.</em></p><p><em>2. ‘The Expanse’ series by James S.A. Corey — This epic series begins with ‘Levi</em></p><p><em>athan Wakes’ and explores a future where humanity has colonized the solar system, facing political intrigue and threats from mysterious alien technology.</em></p><p><em>3. ‘The Power’ by Naomi Alderman — In a world where women develop the power to emit electric shocks, societal structures are upended, leading to profound changes in gender dynamics and power distribution.</em></p><p><em>4. ‘Red Mars’ by Kim Stanley Robinson — This novel is the first in a trilogy that chronicles the colonization and terraforming of Mars, exploring themes of environmentalism, politics, and the challenges of building a new society on a distant planet.”</em></p><h4>Case Studies: Successful Prompt Engineering Strategies</h4><p><strong>Case Study 1: Automating Content Creation</strong></p><p>A digital marketing agency used prompt engineering to automate the creation of product descriptions. By crafting prompts that included product details, features, and target audience, they achieved consistent and engaging descriptions that matched their brand voice.</p><p><strong>Prompt:</strong></p><pre>&quot;Write a product description for our new line of noise-canceling headphones. Highlight the key features: wireless connectivity, 20-hour battery life, and superior sound quality. Target audience: tech-savvy professionals and frequent travelers.&quot;</pre><p><strong>Output:</strong><br><em>“Introducing our latest innovation in audio technology: the ultimate noise-canceling headphones designed for tech-savvy professionals and frequent travelers. With wireless connectivity, you can enjoy your favorite music without the hassle of tangled cords. Experience uninterrupted listening with a remarkable 20-hour battery life, perfect for long flights or back-to-back meetings. Our headphones deliver superior sound quality, immersing you in rich, crystal-clear audio that elevates your listening experience to new heights. Upgrade your audio experience today with our state-of-the-art headphones.”</em></p><p><strong>Case Study 2: Enhancing Customer Support</strong></p><p>A tech company implemented AI-driven chatbots for customer support. By designing prompts that guided the AI to ask clarifying questions and provide step-by-step solutions, they improved response accuracy and customer satisfaction.</p><p><strong>Prompt:</strong></p><pre>&quot;A customer is experiencing connectivity issues with their Wi-Fi router. Guide the AI to ask clarifying questions and provide step-by-step troubleshooting instructions.&quot;</pre><p><strong>Output:</strong><br><em>“Sure! I can help you with your Wi-Fi connectivity issues. Let’s start by asking a few questions to understand the problem better:</em></p><p><em>1. Can you please confirm if all the cables are securely connected to the router?<br>2. Have you tried restarting your router by unplugging it from the power source and plugging it back in after a few seconds?<br>3. Are any other devices in your home able to connect to the Wi-Fi network, or is it just one device experiencing issues?<br>4. Have you recently changed any settings on your router or device?</em></p><p><em>Once we have more information, we can proceed with the following troubleshooting steps:</em></p><p><em>1. Restart your device and attempt to reconnect to the Wi-Fi network.<br>2. Check if there are any firmware updates available for your router and apply them if necessary.<br>3. Try connecting to a different Wi-Fi network to see if the issue persists.<br>4. If the problem continues, please contact our technical support team for further assistance.”</em></p><h3>Challenges and Considerations</h3><p>While prompt engineering offers immense potential, it also presents challenges and ethical considerations:</p><h4>Common Pitfalls in Prompt Engineering</h4><p>- <strong>Ambiguity: </strong>Vague prompts can lead to irrelevant or nonsensical responses.</p><p><strong>Example: </strong><br><strong> Input:</strong></p><pre>&quot;Tell me something interesting.&quot;</pre><p><strong>Output: </strong><br><em> “Did you know that honey never spoils? Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible.”</em></p><p><strong>Refined Prompt:</strong></p><pre>&quot;Tell me an interesting fact about ancient Egyptian culture.&quot;</pre><p><strong>Output: </strong><br> <em>“The ancient Egyptians were one of the first civilizations to practice organized medicine. They had specialized doctors for different ailments, and their medical texts contained detailed knowledge of anatomy, treatments, and surgeries.”</em></p><p>- <strong>Overloading:</strong> Excessive information in a prompt can confuse AI and result in inaccurate outputs.</p><p><strong>Example: </strong><br><strong> Input:</strong></p><pre>&quot;Write a poem about the beauty of nature, including elements like trees, rivers, mountains, and the sky, and make sure it rhymes, has a specific rhythm, and evokes a sense of peace and wonder.&quot;</pre><p><strong>Output: </strong><br><em> “The trees sway gently in the breeze, <br> Rivers flow with grace and ease, <br> Mountains stand tall, touching the sky, <br> Nature’s beauty is a wondrous sigh.”</em></p><p><strong>Refined Prompt:</strong></p><pre>&quot;Write a rhyming poem about the beauty of nature, focusing on trees, rivers, and mountains.&quot;</pre><p><strong>Output: </strong><br><em> “In the forest, where the trees do sway, <br> Rivers sing and dance along their way. <br> Mountains rise majestically high, <br> Underneath the vast, endless sky.”</em></p><p>- <strong>Bias: </strong>AI models may reflect biases present in training data, requiring careful prompt design to mitigate these issues.</p><p><strong>Example</strong>:<br><strong>Input:</strong></p><pre>&quot;Describe the typical leader.&quot;</pre><p><strong>Output: </strong><br> <em>“A typical leader is often seen as a confident, assertive, and charismatic male who takes charge and leads with authority.”</em></p><p><strong>Refined Prompt:</strong></p><pre>&quot;Describe the qualities of an effective leader, emphasizing inclusivity and empathy.&quot;</pre><p><strong>Output: </strong><br><em> “An effective leader embodies inclusivity, empathy, and the ability to listen actively. They create an environment where diverse perspectives are valued, fostering collaboration and innovation. With compassion and understanding, they guide their team towards shared goals, inspiring trust and respect.”</em></p><h4>Ethical Considerations and Responsible AI Usage</h4><p>Ethical prompt engineering involves:</p><p>- <strong>Ensuring fairness: </strong>Avoiding biased or discriminatory language in prompts.<br>- <strong>Maintaining privacy:</strong> Safeguarding sensitive information in AI interactions.<br>- <strong>Promoting transparency:</strong> Clearly communicating the role of AI in generating content.</p><h4>Balancing Creativity and Control</h4><p>Striking the right balance between creativity and control is essential. While creative prompts can yield innovative outputs, maintaining control ensures alignment with specific goals and guidelines.</p><p><strong>Example: </strong><br><strong>Creative Prompt:</strong></p><pre>&quot;Generate an imaginative story about a parallel universe where animals are the dominant species and humans are their pets.&quot;</pre><p><strong>Output: </strong><br><em>“In a parallel universe, animals ruled the world, and humans were kept as loyal companions. In this land, the wise elephants governed the cities, the cunning foxes ran businesses, and the artistic parrots painted the skies with vibrant colors. Humans, though smaller and less dominant, were cherished for their ability to learn and adapt, forming deep bonds with their animal caretakers.”</em></p><ul><li><strong>Controlled Prompt:</strong></li></ul><pre>&quot;Generate a controlled narrative about a parallel universe where animals have become the dominant species, focusing on societal structures, economy, and culture.&quot;</pre><p><strong>Output:</strong><br><em>“In a parallel universe, the societal structure is shaped by the dominance of animals, with different species occupying roles based on their inherent traits. Elephants, known for their wisdom, serve as leaders and advisors, while foxes excel in commerce and trade. Culture is rich and diverse, with parrots leading in art and expression. The economy thrives on cooperation between species, emphasizing sustainability and harmony. Humans, while not the dominant species, contribute to society through their adaptability and skills, living in symbiosis with their animal </em>counterparts.”</p><h3>The Future of Prompt Engineering</h3><p>As AI technology continues to evolve, the field of prompt engineering is poised for growth and innovation:</p><h4>Emerging Trends and Innovations</h4><p>- <strong>Contextual Awareness: </strong>AI models are becoming more contextually aware, enabling more nuanced and contextually relevant responses.<br>- <strong>Interactive AI:</strong> Advancements in conversational AI are enhancing user interactions and enabling dynamic dialogues.<br>- <strong>Specialized AI Models:</strong> Industry-specific AI models are being developed to cater to unique needs and requirements.</p><h4>The Growing Importance of Human-AI Collaboration</h4><p>Human-AI collaboration is becoming increasingly important, with prompt engineering serving as the bridge between human creativity and AI capabilities. By leveraging AI as a collaborative tool, users can amplify their productivity and innovation.</p><h4>Opportunities for Aspiring Prompt Engineers</h4><p>The demand for skilled prompt engineers is on the rise, offering opportunities for individuals interested in AI, language, and technology. Aspiring prompt engineers can explore roles in AI development, content creation, customer support, and more.</p><h3>Conclusion</h3><p>Prompt engineering is an essential skill for anyone looking to harness the full potential of generative AI. By mastering this art, you can unlock new opportunities for innovation and creativity in your personal and professional endeavors. Join us on this journey as we delve deeper into the world of prompt engineering, exploring advanced techniques and real-world applications in upcoming blogs.</p><p>Stay tuned for the next blog in our series, where we will dive into specific techniques and strategies to refine your prompt engineering skills. Subscribe to our newsletter and follow us on social media to receive updates and exclusive insights into the world of generative AI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d60c5f7cbb8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Daily AI papers-Jan’14]]></title>
            <link>https://medium.com/@pady0014s/daily-ai-papers-564ce622f03d?source=rss-7beca7f0145d------2</link>
            <guid isPermaLink="false">https://medium.com/p/564ce622f03d</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[daily-update]]></category>
            <dc:creator><![CDATA[Pranav]]></dc:creator>
            <pubDate>Sun, 14 Jan 2024 14:14:00 GMT</pubDate>
            <atom:updated>2024-01-14T14:15:15.970Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wMxEpVqntuIwDp1bJEr47w.png" /><figcaption>Author</figcaption></figure><h3>Introduction:</h3><p>In thе еvеr еvolvin’ landscapе of artificial intеlligеncе and rеsеarchеrs arе continually pushin’ thе boundariеs of what’s possiblе. Today and wе dеlvе into fivе thought provokin’ rеsеarch papеrs that showcasе thе divеrsе an’ groundbrеakin’ work happеnin’ in thе fiеld. From addrеssin’ privacy concеrns in languagе modеls to еnhancin’ thе еfficiеncy of GANs an’ comprеssin’ largе languagе modеls and thеsе papеrs offеr valuablе insights into thе futurе of AI</p><h3>1. TOFU: A Task of Fictitious Unlearnin’ for LLMs by Pratyush Main</h3><p><strong>Paper</strong>: <a href="https://arxiv.org/abs/2401.06121">TOFU: A Task of Fictitious Unlearning for LLMs</a></p><p>Largе languagе modеls (LLMs) havе shown incrеdiblе prowеss in gеnеratin’ human likе tеxt and but concеrns about privacy arisе whеn thеsе modеls inadvеrtеntly mеmorizе sеnsitivе information. Pratyush Maini introducеs TOFU and a Task of Fictitious Unlеarnin’ and as a bеnchmark to еxplorе unlеarnin’ mеthods. Thе papеr providеs a datasеt of synthеtic author profilеs and challеngin’ еxistin’ unlеarnin’ algorithms an’ еmphasizin’ thе nееd for morе еffеctivе approachеs to еnsurе modеls bеhavе as if thеy wеrе nеvеr trainеd on spеcific data.</p><h3>2. <strong>E2GAN: Efficiеnt Trainin’ of Efficiеnt GANs for Imagе to Imagе Translation by Yifan Gong:</strong></h3><p><strong>Paper: </strong><a href="https://arxiv.org/abs/2401.06127">E2GAN: Efficient Training of Efficient GANs for Image-to-Image Translation</a></p><p>Yifan Gong tacklеs thе challеngе of makin’ thе distillation of GANs from diffusion modеls morе еfficiеnt. Thе papеr introducеs innovativе tеchniquеs and such as constructin’ a basе GAN modеl with gеnеralizеd fеaturеs adaptablе to various concеpts an’ еmployin’ Low Rank Adaptation (LoRA) for еfficiеnt finе tunin’. Thе rеsults dеmonstratе thе potеntial for еmpowеrin’ GANs to pеrform rеal timе and high quality imagе еditin’ on mobilе dеvicеs with rеducеd trainin’ costs an’ storagе rеquirеmеnts.</p><h3>3. <strong>Extrеmе Comprеssion of Largе Languagе Modеls via Additivе Quantization by Vagе Egiazarian:</strong></h3><p>Paper: <a href="https://arxiv.org/abs/2401.06118">Extreme Compression of Large Language Models via Additive Quantization</a></p><p>In thе quеst for dеployin’ largе languagе modеls on еnd usеr dеvicеs and Vagе Egiazarian еxplorеs еxtrеmе comprеssion tеchniquеs and targеtin’ еxtrеmеly low bit counts pеr paramеtеr. Thе papеr builds on Additivе Quantization and a classic algorithm from thе Multi Codеbook Quantization family and achiеvin’ statе of thе art rеsults in LLM comprеssion. Thе proposеd algorithm outpеrforms rеcеnt tеchniquеs and providin’ a significant advancеmеnt in comprеssin’ modеls whilе maintainin’ accuracy.</p><h3>4. <strong>LLM as a Coauthor: Thе Challеngеs of Dеtеctin’ LLM Human Mixcasе by Chujiе Gao:</strong></h3><p>Paper: <a href="https://arxiv.org/abs/2401.05952">LLM-as-a-Coauthor: The Challenges of Detecting LLM-Human Mixcase</a></p><p>With thе incrеasin’ prеvalеncе of machinе gеnеratеd tеxt (MGT) and Chujiе Gao addrеssеs thе challеngеs of dеtеctin’ mixеd scеnarios involvin’ both machinе gеnеratеd an’ human gеnеratеd contеnt and tеrmеd as mixcasе. Thе papеr introducеs MixSеt and a datasеt dеdicatеd to studyin’ thеsе mixеd modification scеnarios and an’ еvaluatеs thе еffеctivеnеss of еxistin’ MGT dеtеctors. Thе findings undеrscorе thе nееd for morе finе grain dеtеctors tailorеd for mixcasе and highlightin’ potеntial risks in information quality an’ complеtеnеss.</p><h3><strong>5. Thе Bеnеfits of a Concisе Chain of Thought on Problеm Solvin’ in Largе Languagе Modеls by Matthеw Rеnzе:</strong></h3><p>Paper: <a href="https://arxiv.org/abs/2401.05618">The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models</a></p><p>Matthеw Rеnzе еxplorеs thе impact of concisеnеss in promptin’ on thе problеm solvin’ pеrformancе of largе languagе modеls. Introducin’ Concisе Chain of Thought (CCoT) promptin’ and thе papеr comparеs standard CoT an’ CCoT prompts and rеvеalin’ a significant rеduction in avеragе rеsponsе lеngth without a substantial impact on problеm solvin’ pеrformancе. Thеsе findings offеr practical implications for AI systеm еnginееrs usin’ LLMs an’ providе insights into stеp by stеp rеasonin’ in thеsе modеls.</p><h3>Conclusion:</h3><p>Thеsе fivе rеsеarch papеrs rеprеsеnt just a snapshot of thе cuttin’ еdgе work happеnin’ in thе fiеld of artificial intеlligеncе. From addrеssin’ privacy concеrns to improvin’ thе еfficiеncy of imagе to imagе translation an’ comprеssion tеchniquеs and thеsе papеrs contributе valuablе knowlеdgе an’ pavе thе way for futurе advancеmеnts in AI. As rеsеarchеrs continuе to еxplorе nеw frontiеrs and thе possibilitiеs for thе application of artificial intеlligеncе in various domains rеmain both еxcitin’ an’ limitlеss.</p><p>For more updates, checkout my website — <a href="https://aiafternight.com/category/ai-feed/">AIAfterNight.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=564ce622f03d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 books that will change your life ]]></title>
            <link>https://medium.com/@pady0014s/10-books-that-will-change-your-life-6c57938e0cda?source=rss-7beca7f0145d------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c57938e0cda</guid>
            <category><![CDATA[life]]></category>
            <category><![CDATA[books]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[personal-development]]></category>
            <dc:creator><![CDATA[Pranav]]></dc:creator>
            <pubDate>Sun, 21 May 2023 08:19:49 GMT</pubDate>
            <atom:updated>2023-05-21T08:23:50.807Z</atom:updated>
            <content:encoded><![CDATA[<h3>📌10 books that will change your life😉</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gS2hv4I3YFStgCIG" /><figcaption>Photo by <a href="https://unsplash.com/@conti_photos?utm_source=medium&amp;utm_medium=referral">Fabrizio Conti</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>“Books are portals to limitless worlds, where imagination takes flight, knowledge expands, and souls find solace. They are timeless companions that illuminate our minds, spark curiosity, and ignite the flames of inspiration. Within the pages of a book, we discover the power to transform, to empathize, and to be infinitely enriched. So, let us embrace the magic of books and embark on extraordinary journeys, for within each chapter lies a universe waiting to be explored.”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*1joKoAFoZPyDgVCS0DK3RQ.jpeg" /><figcaption><strong>The Alchemist by Paulo Coelho</strong></figcaption></figure><p><strong>The Alchemist by Paulo Coelho</strong></p><p>The Alchemist is a story about a young shepherd named Santiago who travels from his homeland in Spain to the Egyptian desert in search of treasure buried near the Pyramids. Along the way, he meets a variety of people who teach him about the importance of following one’s dreams, listening to one’s heart, and living in the present moment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*SaAcZ_vpJB-z48bjaoVEaw.jpeg" /><figcaption>Tuesdays with Morrie by Mitch Albom</figcaption></figure><p><strong>Tuesdays with Morrie by Mitch Albom</strong></p><p>Tuesdays with Morrie is a memoir about the author’s weekly visits with his former college professor, Morrie Schwartz, who is dying of Amyotrophic Lateral Sclerosis (ALS). In these visits, Morrie shares his wisdom and insights on life, death, and the importance of living each day to the fullest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*-etz9-UDly6OzzR44Qqwwg.jpeg" /><figcaption>The Four Agreements by Don Miguel Ruiz</figcaption></figure><p><strong>The Four Agreements by Don Miguel Ruiz</strong></p><p>The Four Agreements is a book about personal development that teaches readers how to live in harmony with themselves and others. The four agreements are:</p><ul><li>Be impeccable with your word.</li><li>Don’t take anything personally.</li><li>Don’t make assumptions.</li><li>Always do your best.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*RTc6eJJchkSJ1b5iDpQWBA.jpeg" /><figcaption>The Power of Now by Eckhart Tolle</figcaption></figure><p><strong>The Power of Now by Eckhart Tolle</strong></p><p>The Power of Now is a book about spiritual enlightenment that teaches readers how to live in the present moment and let go of negative thoughts and emotions. Tolle argues that the past and future are illusions and that the only thing that is real is the present moment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*GQGJNQRVoZ2qhJcuEaYQUw.jpeg" /><figcaption>Man’s Search for Meaning by Viktor Frankl</figcaption></figure><p><strong>Man’s Search for Meaning by Viktor Frankl</strong></p><p>Man’s Search for Meaning is a memoir about the author’s experiences in Nazi concentration camps during World War II. Frankl argues that even in the darkest of times, it is possible to find meaning and hope.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*KNFvqJcE8IAdVcK8GKArdw.jpeg" /><figcaption>The Seven Habits of Highly Effective People by Stephen Covey</figcaption></figure><p><strong>The Seven Habits of Highly Effective People by Stephen Covey</strong></p><p>The Seven Habits of Highly Effective People is a self-help book that teaches readers how to be more effective in their personal and professional lives. The seven habits are:</p><ul><li>Be proactive.</li><li>Begin with the end in mind.</li><li>Put first things first.</li><li>Think win-win.</li><li>Seek first to understand, then to be understood.</li><li>Synergize.</li><li>Sharpen the saw.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*WIctUClOwxvWqrYHva5wrA.jpeg" /><figcaption>The Art of Happiness by the Dalai Lama and Howard Cutler</figcaption></figure><p><strong>The Art of Happiness by the Dalai Lama and Howard Cutler</strong></p><p>The Art of Happiness is a book about the Dalai Lama’s views on happiness and well-being. The Dalai Lama argues that happiness is a state of mind that can be cultivated through meditation, compassion, and a focus on the present moment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*5_9DQSaG2MMt_lnaF3kTMQ.jpeg" /><figcaption>Daring Greatly by Brené Brown</figcaption></figure><p><strong>Daring Greatly by Brené Brown</strong></p><p>Daring Greatly is a book about vulnerability and courage. Brown argues that vulnerability is not a weakness, but rather a strength. She encourages readers to embrace their vulnerability and live their lives with courage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*KfuyNg5EjDjtu-YIYT4XHQ.jpeg" /><figcaption>The Gift of Fear by Gavin de Becker</figcaption></figure><p><strong>The Gift of Fear by Gavin de Becker</strong></p><p>The Gift of Fear is a book about intuition and self-protection. De Becker argues that our intuition is a powerful tool that can help us to stay safe. He teaches readers how to listen to their intuition and how to protect themselves from harm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*ITy5P4_oHh5QoE3vLXXAfw.jpeg" /><figcaption>The Life-Changing Magic of Tidying Up by Marie Kondo</figcaption></figure><p><strong>The Life-Changing Magic of Tidying Up by Marie Kondo</strong></p><p>The Life-Changing Magic of Tidying Up is a book about decluttering and organizing. Kondo argues that our homes are a reflection of our minds and that by decluttering our homes, we can also declutter our minds.</p><p>These are just a few of the many books that can teach you valuable life lessons. I encourage you to read widely and to find books that speak to you. The more you read, the more you will learn and grow.</p><p>Here are some pointers for successful book reading:</p><ol><li>Pick the ideal book for you. Not every book is made equally. Some books will fit your interests and learning preferences better than others.</li><li>Look for a peaceful area to read. Focusing on your reading can be tough when there are distractions. Locate a quiet area where you can unwind and read the book without distractions.</li><li>Make a target for yourself. How much reading do you intend to undertake each day? You will be more motivated and on track if you have a goal. Take pauses. Don’t try to read nonstop for hours. Every 20 to 30 minutes, take a break to stretch, get up and move around, or get something to eat. First, skim the book. This will offer you an overall picture of the book and assist you in determining which sections are most significant to you.</li><li>Actively read. Reading the text on the page is not enough. Consider the author’s point of view and how it connects to your own situation. Make a note. This will make it easier for you to recall the key ideas from the book and to return to them in the future.</li><li>Write a review. This is a fantastic approach to discuss the book with others and aid them in determining whether it’s the appropriate choice for them.</li><li>Enjoy reading the book! It should be enjoyable to read. Put down a book you’re not enjoying and pick up another one. There are so many excellent novels available that you don’t need to push yourself to read anything you don’t enjoy.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c57938e0cda" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is RNN?]]></title>
            <link>https://medium.com/@pady0014s/what-is-rnn-124d847d9e65?source=rss-7beca7f0145d------2</link>
            <guid isPermaLink="false">https://medium.com/p/124d847d9e65</guid>
            <category><![CDATA[application-of-rnn]]></category>
            <category><![CDATA[rnn-workflow]]></category>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[rnn]]></category>
            <category><![CDATA[bidirectional-rnn]]></category>
            <dc:creator><![CDATA[Pranav]]></dc:creator>
            <pubDate>Sun, 05 Feb 2023 04:35:55 GMT</pubDate>
            <atom:updated>2023-02-05T04:46:14.821Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a02LjgxVDaIKr0mbJK8ZEQ.png" /></figure><p>RNNs are a class of neural networks that is powerful for modeling sequence data such as time series or natural language.</p><p>It is a type of deep-learning neural network that remembers the input sequence, stores it in memory states/cell states, and predicts future words/sentences.</p><h3>Workflow of RNN</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ydy0TDql1xoY9nIQ6OuwLQ.png" /></figure><p>there are following stages in RNNs:</p><p>Regular Network which contains hyperbolic tan function as an activation function tanh()</p><p>Forgetting NN which forgets some not needed information and adds up with predictions made by the regular network. sigmoid() is used here whose output multiplies with previous memory states to remove unwanted information. From the image below, we can see how important information is with a value closer to 1 (0.8 x 1.0) kept and a value with nearly zero probability gets ignored (0.8 x 0.0 = 0.0). This process is called <strong>Gating</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u03juw_IDi_vfu64-xeGSQ.png" /><figcaption>Gating(Cross Junction)</figcaption></figure><p>After Gating, this output vector is get added to the predicted value(possibilities) vector. You can refer to the image given below. The output of this Plus Junction is given back to the memory state for storing it(which will again compute with forgetting NN via Gating and process continues) for further process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P2WhQVwkTr4ti96RxJ3Geg.png" /><figcaption>Plus Junction</figcaption></figure><p>Selection NN filters the data using sigmoid() contributes to the Gating process with the predicted possibilities. The output(prediction) of this gives us the single predicted value which is again fed to the network back as an input.</p><p>This is a basic flow of RNN</p><h4>Why RNN?</h4><p>RNNs work well with inputs that are in the form of sequences. As an example, consider, <strong>“<em>I like eating ice-creams. My favorite is chocolate ____</em>”</strong>.</p><p>For humans, it is obvious to fill the blank with the word <strong><em>ice cream</em></strong><em>,</em> but the machine has to understand the context and remember the previous words in the sentence to predict the subsequent word. This is where RNNs are useful.</p><p>There are <strong>3 weights matrices</strong>, which are used repeatedly used in each cell. $W_x$ are the weights for the input, $W_y$ are the weights for the output and $W_h$ are the weights for the memory. They are combined using the following equations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*Rnvt9Gz_lw20HYW1L_HxBQ.png" /></figure><p>where <strong><em>f </em></strong>and <strong><em>g</em> </strong>are <strong>activation functions</strong> like <strong>ReLU()</strong> or <strong>tanh()</strong> and b and c are the <strong>biases</strong>. Note that it is common to set <strong>h_0</strong> to be the zero vector. And not every task requires a prediction at each time step <strong><em>y_t</em></strong>, some only require one at the end of each sentence.</p><p>As an example, if we pass the sentence “She has a cat” through an RNN for part-of-speech tagging; at <strong><em>t1</em></strong> a prediction is made using “She”, at <strong><em>t2 </em></strong>“She has” is used, similarly at <strong><em>t3</em></strong> “She has a” is used and at <strong><em>t4</em></strong> the whole sentence is used to make a prediction.</p><p>A <strong>Language model</strong> determines the <strong>probability</strong> of a sequence of words. For example, if you consider the incomplete sentence “I’m going to walk the …”, a reasonable guess for the next word would be “dog” as opposed to “ice cream” or “mountain”. We can see that language is <strong>predictable</strong> and some words are more likely to appear than others.</p><p>Mathematically, we can use the <strong>multiplication rule</strong> to represent the probability of a sequence as a product of conditional probabilities:</p><p>This is a <strong>difficult task</strong> because it is impossible to get a training corpus with every combination of words in a sequence. We will see if we can use an RNN to <strong>train</strong> a language model by using a clever trick.</p><h3>Training RNNs</h3><p>Training an RNN is <strong>trickier</strong> than a standard feed-forward neural network because of the time aspect. The algorithm used is called <strong>Backpropagation Through Time</strong> (BPTT). The main idea is that when we compute gradients, we need to <strong>reverse all the arrows</strong> in the computation graph and multiply them using the chain rule.</p><p>For example at the final time step, all the input vectors (<strong><em>xt</em></strong>) and all the hidden states (<strong><em>htℎ</em></strong>) were used to make this prediction so we have to update all of their corresponding weights. As they share weights matrices, we can add each component together. You can find more details in <a href="https://d2l.ai/chapter_recurrent-neural-networks/bptt.html">this article</a>, but luckily all major deep learning frameworks will <strong>implement this for us</strong>.</p><h4>Training an RNN for language modelling</h4><p>To train an RNN for language modeling, we start by <strong>segmenting</strong> a corpus into chunks, sentences or paragraphs to feed into the model. The longer the sequence length the more <strong>computationally expensive</strong> it is to train but the better the model is at capturing <strong>long-range dependencies</strong>.</p><p>We then use a <strong>self-supervised</strong> learning approach. At every time step, we get the RNN to try to <strong>predict the next word</strong> in the sentence in the training corpus. What is great is that we don’t need to label the data, as we use the words as the labels themselves.</p><p>The outputs $yt$ are <strong>probability distributions</strong> over the whole vocabulary. They assign a probability to each word, which represents how likely that word is to appear next given the previous words. A <strong>cross-entropy loss</strong> can be used to measure how good each prediction is.</p><h4>Text generation</h4><p>To <strong>generate new text</strong> after training, we modify the RNN to <strong>sample</strong> the output probability distribution at each time step and feed this to be the <strong>input</strong> word for the following time step, that is $xt+1$$∼yt+1$.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z3JoAb50I92AQxKe.jpg" /></figure><p>So given an input word (this can be fixed or randomly sampled), we can generate any amount of text we like. It won’t be perfect or grammatically correct by default but it will produce text that has a <strong>similar distribution</strong> to the one it was trained on.</p><p>For example, if we use a training corpus comprising of the Lord of the Rings books, then the generated text will look somewhat similar to the language in these books.</p><h4>Evaluating language models</h4><p>If we <strong>measure</strong> the output <strong>probabilities</strong> of a language model on a <strong>test set</strong>, we expect better models to produce <strong>higher</strong> probabilities for the correct words.</p><p>A common <strong>metric</strong> to evaluate language models is called <strong>Perplexity</strong> and is defined as:</p><p>Because of the inverse probabilities, the <strong>lower</strong> the perplexity score the <strong>better</strong> and the more natural the language produced. The T-th root is there to <strong>control</strong> for different test lengths.</p><p>Note that <strong>humans</strong> have a certain perplexity score too, depending on the corpus, so getting a model’s perplexity score too low below that <strong>benchmark</strong> may also produce unnatural text.</p><h4>Applications of RNN: —</h4><ol><li>Image Captioning — It is used to caption an image by analyzing the activities present in it.</li><li>Time Series Prediction</li><li>Natural Language Processing — Text mining &amp; Sentiment Analysis</li><li>Machine Translation — By taking input in one language, RNN can be used to translate it to different languages as output.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FCvQJxiC2p-gBkPB.jpeg" /></figure><h4>Types of RNN architectures: —</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LptO518TOp_LquC7.png" /></figure><p>Basic RNN architectures (Stanford cs231 lecture)</p><ol><li>One-to-One: — It is also known as a vanilla neural network. It is used for basic machine-learning problems.</li><li>One to Many: — It has a single input and many outputs. Application: used in image captioning.</li><li>Many to One: — It has many inputs and a single output. This is basically used in analyzing sentiments, where we give a sentence as an input and get sentiment regarding that as an output.</li><li>Many to Many: — It takes a sequence of inputs and generates a sequence of outputs.</li><li>Application: Machine Translation</li></ol><h4>Issues while training a RNN : —</h4><ol><li><a href="https://towardsdatascience.com/the-vanishing-gradient-problem-69bf08b15484">Vanishing Gradient Problem</a></li><li><a href="https://deepai.org/machine-learning-glossary-and-terms/exploding-gradient-problem">Exploding Gradient Problem</a></li><li>Struggling to capture <strong>long-range dependencies</strong> because the signal gets <strong>weakens</strong> over time. Simple RNNs have no way of deciding what information is important to ‘remember’ and what can be ‘forgotten’.</li></ol><p>The problem arises during the training of a deep neural network when the gradients travel in the back-propagation back to the initial layer. As the gradients have to go through continuous matrix multiplication because of the chain rule. Therefore, if they have small values (&lt;1) they shrink exponentially till the time they vanish and this is called the <strong>vanishing gradient problem</strong>. This causes a loss of information over time. Moreover, if gradients have large values (&gt;1) they get larger and eventually blow up and crash the model, this is called <strong>exploding gradient problem.</strong></p><p><strong><em>Issues due to these problems :</em></strong></p><ol><li>Long training time</li><li>Poor Performance</li><li>Bad Accuracy</li></ol><h4>Why gradient problem doesn’t take place during the forward pass and only in the backward pass?</h4><p>Actually, in the forward pass, the activations at each step are put through a non-linear activation function, which typically squashes the values, preventing them from blowing up. Since the backward pass is entirely linear, there’s nothing to prevent the derivatives from blowing up.</p><h4>Solutions to the problems of Gradient Explosion and Vanishing Gradient: —</h4><ol><li><strong>Gradient Clipping :</strong></li></ol><p>It helps in preventing gradients from blowing up by re-scaling them so that their norm is at most a particular value η i.e, if ‖g‖&gt; η, where g is the gradient, we set</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/292/0*96VQ6zzsSxu0rwua.png" /></figure><p>By doing this we are introducing bias in the training procedure since the resulting values won’t actually be the gradient of the cost function. The following figure shows an example with a cliff and a narrow valley; if you happen to land on the face of the cliff, you take a huge step that propels you outside the good region. With gradient clipping, you can stay within the valley.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OnHHJKWW4m4BzJnM.png" /></figure><p>2. <strong>Input Reversal :</strong></p><p>In the starting we learned, that it is very difficult for the network to learn long-distance dependencies. Therefore, this affect the learning ability of the architecture. This can be fixed by reversing the order of the words in the input sentence, which can be clearly understood by analyzing the below image :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/0*T59Z4epuU8oncZIh.png" /></figure><p>Input Reversal</p><p>In the above diagram, we can see that there’s a gap of only a one-time step between when the first word is read and when it’s needed. This allows the network to learn relationships between the first words. Once it learned this, it can be further trained to learn more difficult dependencies between words later in the sentences.</p><p><strong>3. Identity Initialization :</strong></p><p>Apart from the identity function f(x) = x, which you can iterate many times, other iterated functions can have complex and chaotic behavior. Hence, if a network computes the identity function, the gradient computation will be perfectly stable, since the jacobian is simply the identity matrix.</p><p>In the identity RNN architecture, all the activation functions are RELU, and the weights are initialized to the identity matrix. Furthermore, the RELU activation function clips the activations to be non-negative, but for non-negative activations, it’s equivalent to the identity function.</p><p><strong>4. Long-Term Short-Term Memory (LSTM):</strong></p><p>LSTM is a special kind of RNN which is mainly useful for learning long-term dependencies. The name refers to the idea that the activations of a network corresponding to short-term memory, while the weights correspond to long-term memory. If the activations can preserve information over long distances, that makes them long-term short-term memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*tpR8WPGfhPZjPGaI.png" /></figure><h3>RNN Variants</h3><h3>GRU</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qbsZY-lD2-arcvQb.png" /></figure><p>GRU stands for <strong>Gated Recurrent Unit</strong> and is a variant of a RNN where the cell/unit has the ability to <strong>forget</strong> information it deems irrelevant. As a result, it tends to do a better job at capturing <strong>long range dependencies</strong> at the cost of increasing the number of <strong>parameters</strong> in the model.</p><h3>LSTM</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*Q-FXEYf9mH3l4hV_.jpg" /></figure><p>LSTM stands for <strong>Long Short Term Memory</strong> and is similar to GRU but has more parameters and can manage more complex information transfer. In addition to the normal hidden state that gets passed between units, LSTM has an additional <strong>cell state</strong> that captures <strong>long term memory</strong>. The affect is that the model can make predictions using both short term and long term memory.</p><p>LSTM is more complicated than GRU, so it is able to reach <strong>better performance</strong> on some tasks but again at the cost of increasing the number of <strong>parameters</strong> in the model and therefore taking <strong>longer to train</strong>.</p><h3>4.4 Deep RNNs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*EcYIuc6Pj9wJZ6bn.png" /></figure><p>There are several ways to <strong>combine</strong> recurrent neural networks. One way is to <strong>stack</strong> multiple RNNs on top of each other. We can do this since each RNN takes in a sequence and outputs another sequence of the same length.</p><p>Just like adding another <strong>layer</strong> in a feed-forward neural network, stacking RNNs allows the model to learn more useful <strong>representations</strong> of the data. This can be useful for very <strong>complex</strong> tasks that consist of a large number of <strong>interactions</strong> in the data, for example machine translation. Again, the thing to keep in mind is that this takes much <strong>longer</strong> to train the model.</p><h3>4.5 Bidirectional RNNs</h3><p>There are some tasks where it makes sense to use not only previous words in the sentence but also <strong>future words</strong> in the sentence. For example, the word “India” in the two sentences “India has a population of 1.4 billion people” and “India got her bag ready to go to school” has <strong>different meaning</strong> based on the words that come <strong>after</strong> it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/0*U1B_p5yoyGaNdTM9.png" /></figure><p><strong>Bidirectional RNNs</strong> work by adding a <strong>second</strong> RNN that processes the text in <strong>reverse order</strong>. This ensures that the model uses the <strong>whole sentence</strong> each time it makes a prediction.</p><p>Note that the cells in this model can be either <strong>simple RNN, GRU or LSTM units</strong> depending on your preference. It is even possible to <strong>stack</strong> bidirectional RNNs to create <strong>deep bidirectional RNNs</strong>.</p><p>Reference -</p><ol><li>Check out <a href="https://medium.com/analytics-vidhya/in-depth-tutorial-of-recurrent-neural-network-rnn-and-long-short-term-memory-lstm-networks-3a782712a09f"><strong>this article</strong></a> on the medium by Sanket Maheshwari</li><li><a href="https://www.kaggle.com/code/samuelcortinhas/nlp8-rnns-and-language-models/notebook#3.-Language-Models"><strong>RNNs and Language Models</strong></a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=124d847d9e65" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>