<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Deepak Singh on Medium]]></title>
        <description><![CDATA[Stories by Deepak Singh on Medium]]></description>
        <link>https://medium.com/@ds.deepak?source=rss-382c0b0aab6e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*7dmvJV2AguB4eAM_5sjslg.jpeg</url>
            <title>Stories by Deepak Singh on Medium</title>
            <link>https://medium.com/@ds.deepak?source=rss-382c0b0aab6e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 11:09:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ds.deepak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Game Theory to Language Models: The Power of Planning in AI]]></title>
            <link>https://medium.com/@ds.deepak/from-game-theory-to-language-models-the-power-of-planning-in-ai-b4c4321d4200?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4c4321d4200</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[nash-equilibrium]]></category>
            <category><![CDATA[game-theory]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[ai-planning]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sun, 22 Sep 2024 15:50:04 GMT</pubDate>
            <atom:updated>2024-09-22T15:50:04.883Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hflW0mfgZqn1yKOluVKhMQ.png" /><figcaption>The One With All the Poker (S1.E18, F.R.I.E.N.D.S)</figcaption></figure><p>It was really surprising that many big companies like DeepMind, OpenAI, and others started by investing a lot of time in building AI to conquer games. What seemed like a detour into recreational computing has, in fact, paved the way for some of the most transformative AI technologies we see today. The strategies developed for mastering chess, Go, and even complex video games have become the building blocks for teaching language models intuition, judgment, and human-like reasoning. In this article, we’ll delve deep into why those game-playing efforts were not wasted and how they’ve become the cornerstone of AI’s path to more sophisticated cognitive abilities. From the zero-sum world of board games to the nuanced realm of natural language processing, we’ll explore how the principles of strategic thinking, pattern recognition, and adaptive learning have found surprising applications in creating AI systems that can understand, reason, and communicate in increasingly human-like ways.</p><blockquote>Game Theory, Nash Equilibrium, and Reinforcement Learning: A Powerful Trio</blockquote><p>At the core of many AI systems for strategic decision-making are the interrelated concepts of game theory, Nash equilibrium, and reinforcement learning (RL).</p><h4>The Foundations of Game Theory</h4><p>Game theory provides a framework for analyzing strategic interactions between rational decision-makers. It helps us model scenarios where the outcome for each participant depends on the actions of others. Games in this context can be classified into several categories:</p><ol><li><strong>Zero-Sum Games</strong>: In these games, one player’s gain is exactly balanced by the other player’s loss. The total benefit to all players in the game always adds up to zero. Classic examples include poker and chess. In AI, zero-sum games are often easier to model and solve because the interests of the players are directly opposed.</li><li><strong>General-Sum Games</strong>: These games allow for outcomes where all players can gain or lose simultaneously. They model more complex real-world scenarios where cooperation can be beneficial. Examples include business negotiations or climate change mitigation efforts. AI systems for general-sum games need to balance competitive and cooperative strategies.</li><li><strong>Cooperative Games</strong>: In these games, players can form coalitions to improve their outcomes. The challenge lies in determining how to fairly distribute the benefits of cooperation. AI applications in this area include resource allocation problems and team formation in multi-agent systems.</li><li><strong>Sequential vs. Simultaneous Games</strong>: Games can also be categorized based on whether players make decisions sequentially (like in chess) or simultaneously (like in rock-paper-scissors). This distinction is crucial for AI as it affects the type of algorithms and strategies that can be employed.</li></ol><h4>The <strong>Nash</strong> Equilibrium: A Cornerstone of Strategic Thinking</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0y-eGHo-8-csBc3IRpcChg.png" /><figcaption><em>Credits : Still from A Beautiful Mind (2001, Universal Pictures/Dreamworks).</em></figcaption></figure><p>The Nash equilibrium, a key concept in game theory, represents a stable state where no player can unilaterally improve their outcome by changing their strategy, given what others are doing. In AI, computing or approximating Nash equilibria often serves as a goal for creating robust strategies.</p><p>For zero-sum games, finding a Nash equilibrium is relatively straightforward and corresponds to finding an optimal strategy. However, for general-sum games, there may be multiple Nash equilibria, and finding them becomes computationally challenging as the complexity of the game increases.</p><h4>Reinforcement Learning: Learning Through Interaction</h4><p>Reinforcement learning is a type of machine learning where an agent learns to make decisions by interacting with an environment, receiving rewards or penalties based on its actions. RL algorithms often aim to find optimal policies that maximize long-term rewards.</p><p>Key concepts in RL include:</p><ol><li><strong>The Value Function</strong>: This estimates the expected cumulative reward from being in a particular state or taking a specific action.</li><li><strong>Policy</strong>: The strategy that the agent follows to determine its actions.</li><li><strong>Exploration vs. Exploitation</strong>: The balance between trying new actions to gather information and exploiting known good strategies.</li></ol><p>These concepts come together in AI systems designed for complex, multi-agent environments. By using RL techniques to approximate Nash equilibria, AI can develop strategies that are both adaptive and theoretically sound.</p><h4>Conquering Poker and Diplomacy: The Role of Search and Planning</h4><p>The integration of advanced search techniques with these theoretical foundations has led to landmark achievements in games like poker and Diplomacy.</p><blockquote>Poker: Imperfect Information and Opponent Modeling</blockquote><p>In poker, the breakthrough came with the realization that adding a simple search algorithm on top of pre-trained models could dramatically improve performance. By allowing the AI to “think” for a few seconds before each action, considering multiple possible futures, its play improved far more than simply scaling up the model size.</p><p>The challenge in poker lies in dealing with imperfect information — players don’t know their opponents’ cards. AI systems used techniques like:</p><ol><li><strong>Counterfactual Regret Minimization (CFR)</strong>: This algorithm allows the AI to learn optimal strategies by repeatedly playing against itself and minimizing “regret” for not choosing alternative actions.</li><li><strong>Information Abstraction</strong>: Grouping similar game states together to make the problem computationally tractable.</li><li><strong>Opponent Modeling</strong>: Adapting strategies based on perceived opponent tendencies.</li></ol><h4>Diplomacy: Strategic Planning with Natural Language</h4><p>For Diplomacy, a game involving both strategic planning and natural language negotiation, the AI system CICERO employed a sophisticated planning engine. This allowed it to reason about the likely actions of other players and plan its own moves accordingly, all while generating contextually appropriate messages.</p><p>CICERO’s architecture combined several key components:</p><ol><li><strong>Strategic Reasoning Module</strong>: Uses iterative planning and search to predict player actions and optimize its own strategy.</li><li><strong>Natural Language Processing Module</strong>: Generates and interprets messages in the context of the game state and planned actions.</li><li><strong>Consistency Module</strong>: Ensures that the AI’s messages align with its planned actions, maintaining trust with other players.</li></ol><h4>The Search for Better Solutions</h4><p>The power of search extends beyond just considering multiple futures. In both games, iterative search techniques allowed the AI to refine its predictions and strategies over multiple rounds, similar to how a human might mentally simulate different scenarios before making a decision.</p><p>This planning capability proved crucial in handling the complexities of imperfect information and multi-agent dynamics present in these games. By combining learned models with real-time search, the AIs could adapt to specific game states in ways that pure pattern recognition struggles to match.</p><h4>Self-Play vs. Human Data: Choosing the Right Training Approach</h4><p>An important consideration in developing AI for complex games and strategic scenarios is the choice between self-play and incorporating human data in the training process. Both approaches have their merits and are often used in combination:</p><ol><li><strong>Self-Play</strong>:</li></ol><ul><li><strong>Advantages</strong>: Can discover novel strategies beyond human knowledge; scales well with computational power.</li><li><strong>Disadvantages</strong>: May develop unrealistic or brittle strategies that don’t generalize well to human opponents.</li><li><strong>When to use</strong>: Ideal for games with clear rules and well-defined rewards, especially in zero-sum scenarios.</li></ul><p><strong>2. Incorporating Human Data</strong>:</p><ul><li><strong>Advantages</strong>: Helps develop more human-like strategies; can learn subtle nuances of human behavior.</li><li><strong>Disadvantages</strong>: May inherit human biases or suboptimal play; limited by the quality and quantity of available data.</li><li><strong>When to use</strong>: Crucial for games involving negotiation, cooperation, or complex social dynamics.</li></ul><p>In practice, many successful AI systems use a hybrid approach. For example:</p><ul><li>AlphaGo initially trained on human game records before refining its strategies through self-play.</li><li>CICERO for Diplomacy used human data to train its language model and initial strategy, then fine-tuned through self-play.</li></ul><p>The choice often depends on the specific domain:</p><ul><li>For chess or Go, pure self-play has proven highly effective.</li><li>For poker, a combination approach has worked well, with self-play for strategy and human data for opponent modeling.</li><li>In language-heavy games like Diplomacy, human data is crucial for developing natural communication abilities.</li></ul><h4>Bridging to Language Models</h4><p>The lessons from game-playing AI are now finding applications in the realm of large language models (LLMs). While LLMs have achieved remarkable results through scaling and massive pre-training, there’s growing recognition that adding planning and search capabilities could unlock even greater performance.</p><h4>Simple Search Techniques in LLMs</h4><p>Techniques like consensus (generating multiple solutions and choosing the most common) and best-of-n (generating multiple solutions and selecting the best according to a reward model) have shown promising results on challenging benchmarks like mathematical problem-solving.</p><h4>Advanced Planning in Language Tasks</h4><p>More sophisticated approaches, such as step-by-step verification with process reward models, allow for even more robust planning within the language domain. These methods enable LLMs to break down complex tasks, verify each step, and produce higher-quality outputs.</p><h4>The Critical Role of Reward Models</h4><p>Reward models play a crucial role in enabling effective search and planning in LLMs. These models serve as a proxy for human preferences, guiding the AI towards more desirable outputs. Here’s a deeper look at reward models:</p><ol><li><strong>Purpose</strong>: Reward models help evaluate the quality of LLM outputs, allowing the system to select the best among multiple generations or guide the generation process itself.</li><li><strong>Creation Process</strong>:</li></ol><p><strong><em>a) Data Collection:</em></strong><em> </em>Gather a dataset of LLM outputs with human ratings or preferences.</p><p><strong><em>b) Model Training:</em> </strong>Train a separate neural network to predict human preferences based on this data.</p><p><strong><em>c) Iteration and Refinement:</em></strong><em> </em>Continuously update the reward model with new data and feedback.</p><p><strong>3. Types of Reward Models</strong>:</p><ul><li><strong>Binary Classifiers</strong>: Distinguish between “good” and “bad” outputs.</li><li><strong>Regression Models</strong>: Provide a continuous score for output quality.</li><li><strong>Ranking Models</strong>: Compare multiple outputs and rank them by preference.</li></ul><p><strong>4. Challenges in Reward Modeling</strong>:</p><ul><li><strong>Subjectivity</strong>: Human preferences can be inconsistent or vary across individuals.</li><li><strong>Complexity</strong>: Capturing nuanced aspects of language quality is difficult.</li><li><strong>Bias</strong>: Reward models can perpetuate or amplify biases present in training data.</li></ul><p><strong>5. Importance for LLM Search</strong>: A well-crafted reward model is essential for making search in LLMs effective:</p><ul><li>It guides the search process towards more promising directions.</li><li>It helps filter out low-quality or inappropriate generations.</li><li>It enables more complex planning by evaluating intermediate steps.</li></ul><p><strong>6. Recent Advances</strong>:</p><ul><li><strong>Constitutional AI</strong>: Using reward models to align AI behavior with specific principles or values.</li><li><strong>Recursive Reward Modeling</strong>: Using LLMs themselves to assist in creating more sophisticated reward models.</li><li><strong>Multi-Objective Reward Modeling</strong>: Balancing multiple criteria (e.g., accuracy, safety, creativity) in the reward function</li></ul><p>As research progresses, we can expect to see more sophisticated reward models that can capture increasingly subtle aspects of language quality and task performance. This will be crucial in enabling LLMs to tackle even more complex and open-ended tasks through advanced planning and search techniques.</p><h4>The Road Ahead: Generalizing Planning for AI</h4><p>As AI continues to advance, developing more general and scalable planning techniques remains a critical frontier. While game-specific search algorithms have proven incredibly powerful, the challenge now is to create general planning methods that can work across a wide range of domains and tasks.</p><p>This pursuit of generality in planning parallels the development of large language models themselves. Just as we’ve seen tremendous gains from scaling up model size and training data, there’s immense potential in scaling up the amount of computation used at inference time for critical tasks.</p><h4>Future Directions</h4><ol><li><strong>Meta-Learning for Planning</strong>: Developing AI systems that can learn to plan efficiently across different domains.</li><li><strong>Hierarchical Planning</strong>: Combining high-level strategic planning with low-level tactical execution.</li><li><strong>Multimodal Planning</strong>: Extending planning capabilities to tasks involving multiple forms of data (text, images, audio, etc.).</li><li><strong>Ethical and Safe Planning</strong>: Ensuring that AI planning systems consider long-term consequences and align with human values.</li><li><strong>Explainable Planning</strong>: Developing techniques for AI systems to communicate their reasoning and planning process in human-understandable terms.</li></ol><p>The future of AI likely lies in finding the right balance between powerful pre-trained models and sophisticated real-time planning algorithms. By combining the pattern recognition capabilities of neural networks with the logical reasoning of search techniques, we may unlock levels of AI performance that neither approach could achieve alone.</p><p>As researchers continue to explore this space, we can expect to see AI systems that not only generate impressive outputs but also demonstrate more robust problem-solving abilities across an ever-widening range of challenges. The lessons learned from games like poker and Go may well shape the next generation of AI assistants, scientific discovery tools, and creative partners.</p><p>The journey from game theory to advanced language models illustrates a fundamental truth about artificial intelligence: the power of an AI system lies not just in its ability to recognize patterns, but in its capacity to reason, plan, and adapt. As we continue to refine these capabilities, we move closer to AI systems that can truly think and act in ways that match — and perhaps someday surpass — human-level performance across a wide range of cognitive tasks.</p><blockquote>and yes if you haven&#39;t guessed it already, o1 preview might be using the techniques what has been described above.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4c4321d4200" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monte Carlo Agents: Simulating Future Paths to Intelligent Foresight]]></title>
            <link>https://medium.com/@ds.deepak/monte-carlo-agents-simulating-future-paths-to-intelligent-foresight-8223d84417d6?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8223d84417d6</guid>
            <category><![CDATA[monte-carlo-simulation]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[ai-agents-army-bundle]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sat, 14 Sep 2024 09:56:20 GMT</pubDate>
            <atom:updated>2024-09-14T09:56:20.449Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NH64tHUzxZiW6z-3u7VK_A.jpeg" /></figure><p>The notion of artificial intelligence predicting the future, once confined to the realm of science fiction, is rapidly becoming a real possibility. A groundbreaking wave of research is leveraging the power of <strong>Monte Carlo simulations</strong>, a versatile tool borrowed from the fields of physics and mathematics, to grant AI agents a glimpse into the potential consequences of their actions.</p><p>Imagine a dartboard enclosed within a square. Now, imagine throwing darts randomly at this square. By analyzing the distribution of darts landing inside the circle versus those outside, we can surprisingly estimate the value of pi. This seemingly simple experiment lies at the heart of Monte Carlo methods: utilizing random sampling to unveil hidden patterns and extract meaningful estimations from complex systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*ALjYlyOLSqZOIoPY7KH8rw.png" /><figcaption><a href="https://en.wikipedia.org/wiki/Monte_Carlo_integration">https://en.wikipedia.org/wiki/Monte_Carlo_integration</a></figcaption></figure><p><strong>Beyond the Dartboard: AI Navigates the Labyrinth of Future Possibilities</strong></p><p>In the realm of AI decision-making, the dartboard transforms into a vast, multidimensional space representing a multitude of potential future actions and outcomes. Each dimension corresponds to a different variable influencing the future, creating a landscape of possibilities far more intricate than any physical dartboard. Each “dart” thrown into this space represents a simulated future scenario, allowing the AI to explore the potential consiquences of different choices.</p><p>However, this conceptual leap from dartboard to decision-making comes with a significant challenge: the sheer complexity of this multidimensional space. As the number of variables and potential actions grows, the possible outcomes multiply exponentially, quickly exceeding the capabilities of traditional analytical approaches.</p><p>This is where <strong>Dynamic Bayesian Networks (DBNs)</strong> step onto the stage, offering a structured framework for representing the intricate dance of cause and effect within this space of possibilities.</p><p>You can refer my previous article on Bayes Rule for basic understanding <a href="https://medium.com/@ds.deepak/demystifying-bayes-rule-with-an-example-to-remember-45848a7fa0cd">https://medium.com/@ds.deepak/demystifying-bayes-rule-with-an-example-to-remember-45848a7fa0cd</a></p><p><strong>Dynamic Bayesian Networks: A Blueprint for Evolving Intelligence</strong></p><p>Imagine a network of interconnected nodes, each representing a critical variable in the system, such as the AI agent’s skill level, the actions it takes, and the observed outcomes of those actions. The connections between these nodes represent the intricate web of dependencies between these variables, stretching across different moments in time.</p><p>DBNs provide a visual and mathematical language for understanding how an AI agent’s skills might evolve based on its actions and the environment’s feedback. They allow us to visualize how a specific action taken at one point in time can ripple through the network, influencing future skills and ultimately shaping the agent’s trajectory.</p><p>However, even with the elegance and structure of DBNs, we face a significant hurdle. The relationships between these variables, particularly in real-world scenarios, are rarely simple or linear. They are often characterized by non-linear dynamics, feedback loops, and a degree of inherent randomness that defies traditional analytical solutions.</p><p><strong>Taming Complexity: Monte Carlo Particle Filtering Illuminates the Path Forward</strong></p><p>This is where the true power of Monte Carlo methods shines through. Specifically, a technique known as <strong>particle filtering</strong> provides a way to navigate the turbulent waters of complex DBNs. Imagine releasing a swarm of “particles” into the network, each particle representing a hypothetical state of the system, including the AI agent’s skill levels.</p><p>These particles, like miniature probes, move through the network, their trajectories guided by the probabilistic rules encoded within the DBN. At each step, the particles are weighted based on how well they align with the observed data, allowing the system to focus on the most likely scenarios.</p><p>By analyzing the distribution and movement of these particles over time, we can glean valuable insights:</p><ul><li><strong>Skill Estimation:</strong> We can estimate the most probable skill levels of the AI agent at different points in time, understanding how its proficiency might evolve based on its actions and the environment’s feedback.</li><li><strong>Predicting Future Outcomes:</strong> We can project the particles forward in time, simulating a range of possible future scenarios and assessing the likelihood of different outcomes.</li></ul><p><strong>From Single Agents to Strategic Mastery: AI Learns to Play the Game</strong></p><p>Researchers have taken this a step further, deploying multiple AI agents, each equipped with unique decision-making strategies, within simulated competitive game environments. These “game-playing” AI agents exhibit varying skill levels and explore different approaches to achieve victory.</p><p>By observing their interactions, successes, and failures, researchers can gain valuable insights:</p><ul><li><strong>Comparing Strategies:</strong> Directly comparing the effectiveness of different decision-making strategies under varying conditions helps identify which approaches are most robust and adaptable.</li><li><strong>Understanding Skill Dynamics:</strong> Observing how different skill levels, combined with varying strategies, influence the outcome of the game provides a deeper understanding of how skill and strategy interact.</li><li><strong>Emerging Strategic Thinking:</strong> As AI agents become more sophisticated, there’s potential for them to develop higher-level strategic thinking, anticipating opponents’ moves and adapting their own strategies to maximize their chances of success.</li></ul><p>This area of research has already yielded promising results, with AI agents demonstrating an ability to learn and adapt their strategies in complex game environments. In a recent study, AI agents utilizing a combination of Monte Carlo tree search and reinforcement learning techniques achieved superhuman performance in games like Go and chess, showcasing the potential of these methods.</p><p><strong>A Future Shaped by Intelligent Foresight</strong></p><p>While still in its early stages, this research hints at a future where AI could possess a powerful form of foresight, allowing it to navigate complex, dynamic situations with a level of strategic thinking previously unseen.</p><p>Imagine AI systems capable of:</p><ul><li><strong>Predicting and Mitigating Risks:</strong> Foreseeing potential crises, from natural disasters to financial market crashes, and recommending preemptive actions to mitigate negative impacts.</li><li><strong>Optimizing Complex Systems:</strong> Managing intricate networks, such as traffic flow in megacities or energy distribution grids, adapting dynamically to changing conditions and optimizing for efficiency and sustainability.</li><li><strong>Accelerating Scientific Discovery:</strong> Exploring vast datasets in fields like medicine and materials science, identifying promising avenues for research and potentially accelerating the development of life-saving treatments and groundbreaking technologies.</li></ul><p>The ability to predict and shape the future, once considered the exclusive domain of human intuition and expertise, might soon be within reach of artificial intelligence. As we continue to refine these powerful techniques, we are embarking on a journey into a future where AI’s capacity for foresight could profoundly impact countless aspects of our lives.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8223d84417d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Thinking Like Us: A Deep Dive into the World of Reasoning with Foundation Models]]></title>
            <link>https://medium.com/@ds.deepak/thinking-like-us-a-deep-dive-into-the-world-of-reasoning-with-foundation-models-903493b01aa0?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/903493b01aa0</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Fri, 06 Sep 2024 18:28:41 GMT</pubDate>
            <atom:updated>2024-09-06T18:28:41.399Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M4Y9HvQPcs-RZRM_Ye4BLg.jpeg" /><figcaption><a href="https://unsplash.com/photos/a-circular-object-with-a-lot-of-different-things-on-it-qITnJecNF80">https://unsplash.com/photos/a-circular-object-with-a-lot-of-different-things-on-it-qITnJecNF80</a></figcaption></figure><p>Reasoning, the ability to analyze information, draw conclusions, and make informed decisions, is a hallmark of human intelligence. Now, with the advent of powerful AI systems known as foundation models, we’re witnessing a paradigm shift in artificial intelligence — one where machines are learning to reason in ways that were once considered the exclusive domain of humans.</p><p>This article delves deep into the fascinating realm of reasoning with foundation models, exploring the various types of reasoning tasks these models are tackling, the techniques driving their learning process, and the challenges and risks associated with their development.</p><p><strong>The Spectrum of Reasoning Tasks:</strong></p><p>Foundation models are being trained to engage with a broad range of reasoning tasks, mimicking the multifaceted nature of human thought. Here are some key areas of focus:</p><ul><li><strong>Commonsense Reasoning:</strong> This involves understanding and applying everyday knowledge that humans intuitively grasp. Foundation models are being trained to reason about social situations, predict human behavior, and interpret implied meanings in text, all tasks requiring a deep understanding of the unwritten rules and expectations that govern our world. Datasets like Social-IQA and PIQA are crucial for this, providing scenarios that test models’ ability to make logical assumptions and deductions in everyday contexts.</li><li><strong>Mathematical Reasoning:</strong> Foundation models are making significant strides in solving mathematical problems, ranging from simple arithmetic to complex geometry proofs. They are being trained to understand mathematical language, translate word problems into equations, and generate step-by-step solutions, mirroring the systematic approach humans take to solve mathematical challenges. Datasets like GSM8K and MathQA provide diverse and challenging problems to evaluate these capabilities.</li><li><strong>Logical Reasoning:</strong> This involves drawing conclusions based on formal logic, using rules and facts to deduce new knowledge. Foundation models are being trained to perform logical inference, understand the relationships between premises and conclusions, and verify the validity of logical arguments.</li><li><strong>Causal Reasoning:</strong> Understanding cause-and-effect relationships is essential for predicting outcomes and making informed decisions. Foundation models are learning to identify causal relationships, estimate the effects of interventions, and reason about hypothetical scenarios, enhancing their ability to understand complex systems and predict their behavior.</li><li><strong>Multimodal Reasoning:</strong> Real-world scenarios often involve information from multiple sources, such as text, images, and audio. Foundation models are being trained to integrate and reason across these different modalities, enhancing their understanding of complex environments and enabling more nuanced reasoning. This is especially crucial in fields like robotics, where agents need to interpret visual and linguistic information to interact with their surroundings.</li><li><strong>Embodied Reasoning:</strong> This involves understanding the world from the perspective of an embodied agent, such as a robot, and making decisions based on its sensory input and physical capabilities. Foundation models are being integrated with robotic systems, enabling them to reason about their environment, plan actions, and learn from their experiences.</li></ul><p><strong>Techniques Driving Foundation Model Reasoning:</strong></p><p>A diverse set of techniques is driving the advancement of reasoning capabilities in foundation models, shaping their learning process and enhancing their performance:</p><ul><li><strong>Pre-training:</strong> This is the foundation of most foundation models, involving training on massive datasets of text, images, code, and other data to learn general patterns and relationships. This pre-training stage equips models with a broad understanding of language and visual concepts, allowing them to be later fine-tuned for specific reasoning tasks.</li><li><strong>Fine-tuning:</strong> This involves training the pre-trained model on task-specific datasets, adjusting its parameters to improve performance on specific reasoning challenges. This targeted training helps the model adapt its general knowledge to the nuances of specific tasks, enabling more accurate and specialized reasoning.</li><li><strong>Alignment Training:</strong> Ensuring that foundation models align with human values and goals is a critical aspect of their development. This involves techniques like Reinforcement Learning from Human Feedback (RLHF), where models learn to generate outputs that are aligned with human preferences and feedback. This process helps mitigate unwanted behaviors and promotes ethical and safe reasoning.</li><li><strong>In-Context Learning:</strong> Foundation models exhibit remarkable abilities to learn by analogy. In-context learning involves providing the model with a few examples of how to reason through a task, allowing it to learn the underlying patterns and apply them to new instances. This capability mimics the human ability to learn from demonstrations and generalize knowledge to new situations.</li></ul><p><strong>Challenges and Risks on the Path to Thinking Machines:</strong></p><p>While the progress in foundation model reasoning is undeniable, significant challenges and risks need to be addressed:</p><ul><li><strong>Hallucinations:</strong> A major hurdle is the tendency of foundation models to generate fabricated or incorrect information. These “hallucinations” stem from the models’ limitations in fact-checking and their tendency to invent plausible-sounding but ultimately false statements. Addressing this requires developing techniques for fact verification, promoting transparency in model outputs, and fostering user awareness about the potential for inaccuracies.</li><li><strong>Limited Context Length:</strong> Current models struggle to maintain and reason about long sequences of information. This limits their ability to understand complex narratives, process large documents, or engage in extended reasoning processes. Expanding the context window of foundation models is an active area of research, with techniques like dilated attention and position interpolation showing promise.</li><li><strong>Efficiency and Cost:</strong> Training and using foundation models, especially those with billions of parameters, is computationally expensive and resource-intensive. This raises concerns about accessibility, environmental impact, and the potential for these technologies to be concentrated in the hands of a few well-resourced organizations. Developing more efficient model architectures, compression techniques, and parameter-efficient fine-tuning methods is crucial for democratizing access to these powerful tools.</li><li><strong>Ethical Considerations:</strong> The potential for foundation models to perpetuate biases, spread misinformation, and be misused for malicious purposes is a serious concern. Addressing this requires careful data curation, bias mitigation techniques, and frameworks for responsible development and deployment. Promoting transparency, accountability, and public discourse about the ethical implications of foundation models is paramount.</li></ul><p><strong>Charting the Future of Foundation Model Reasoning:</strong></p><p>Despite the challenges, the future of reasoning with foundation models is incredibly promising. Researchers are actively working to address these limitations and push the boundaries of what’s possible. Here are some exciting avenues for future exploration:</p><ul><li><strong>Super Alignment:</strong> As AI systems potentially surpass human intelligence, ensuring their alignment with human values and goals becomes even more critical. This requires developing new techniques for supervising and controlling these systems, ensuring their outputs remain beneficial and aligned with our intentions.</li><li><strong>Autonomous Language Agents:</strong> Foundation models are paving the way for the development of autonomous agents that can interact with the world, understand complex instructions, and complete tasks with minimal human intervention. These agents hold the potential to revolutionize fields like robotics, customer service, and personal assistants.</li><li><strong>Reasoning for Science:</strong> Foundation models can be transformative tools for scientific discovery, helping researchers analyze data, generate hypotheses, and accelerate scientific progress. This includes applications in fields like drug discovery, materials science, and climate modeling.</li></ul><p>Foundation models are rapidly evolving, and their reasoning capabilities are constantly improving. By addressing the challenges, mitigating the risks, and fostering responsible development, we can unlock the transformative potential of these technologies. Imagine a future where machines can collaborate with us, augment our intelligence, and help us solve some of humanity’s most pressing problems — a future where machines truly understand the world and reason like us.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=903493b01aa0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Multimodal Revolution: AI’s Next Leap Towards Human-Like Capabilities]]></title>
            <link>https://medium.com/@ds.deepak/the-multimodal-revolution-ais-next-leap-towards-human-like-capabilities-06e01e94a3ac?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/06e01e94a3ac</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[vlm]]></category>
            <category><![CDATA[multimodal]]></category>
            <category><![CDATA[chamaleon]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Fri, 21 Jun 2024 20:45:18 GMT</pubDate>
            <atom:updated>2024-06-21T20:45:18.092Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hzFJZvrGG0dEJ9u5lmMIPA.jpeg" /><figcaption><a href="https://unsplash.com/">https://unsplash.com/</a></figcaption></figure><p>The artificial intelligence landscape is transforming at a lightning pace, and at the heart of this transformation lies the rise of multimodal models. These advanced AIs are breaking free from the confines of text-only interaction, venturing into the realm of images, videos, and audio to achieve a more holistic and human-like understanding of the world. Recent research papers from tech giants like Google, Meta, and the open-source community, spearheaded by Hugging Face, offer a fascinating glimpse into this field and the exciting future it promises.</p><p>In the realm of artificial intelligence (AI), multimodality refers to the ability of models to process and generate information across multiple modes, such as text, images, audio, and more. This capability allows for more sophisticated and versatile AI systems that can understand and interact with the world in ways that closely mimic human perception and communication.</p><p><strong>The Components of Multimodal Models</strong></p><p><strong>Text Processing</strong>: The ability to understand and generate natural language text, enabling tasks like translation, summarization, and conversation.</p><p><strong>Image Processing:</strong> The capability to analyze and generate images, including recognizing objects, interpreting scenes, and creating visual content.</p><p><strong>Audio Processing:</strong> The ability to process and generate sounds and speech, facilitating tasks like speech recognition and audio synthesis.</p><p><strong>Integration and Context Understanding:</strong> Combining these modalities to understand context and provide more relevant and accurate responses.</p><p><strong>Why Multimodal Models Matter</strong></p><p>Multimodal models are significant because they reflect the way humans naturally process information. We don’t rely on just one type of sensory input; instead, we integrate sights, sounds, text, and other sensory data to form a comprehensive understanding of our environment. By mimicking this process, multimodal models can offer a more intuitive and effective user experience.</p><p>Now, Let’s take a look at some latest work happening in this area. I will be covering 3 state of the art models :</p><p>1. Idefix 2 (HuggingFace)</p><p>2. Chameleon(Meta)</p><p><strong>3.</strong> Mirasol (Google)</p><p><strong>The Open-Source Challenge: Resourceful Innovation in a World of Giants</strong></p><p>For the open-source community, progress in multimodal AI presents a unique challenge. With limited access to the vast computational resources enjoyed by corporations like Google and Meta, researchers must rely on ingenious solutions and pre-trained models to make meaningful contributions.</p><p>Hugging Face’s “Idefix 2” embodies this resourcefulness. This 8 billion parameter Vision Language Model (VLM) is a prime example of the “glue-together” approach that characterizes open-source multimodal models. By combining a pre-trained vision encoder (such as DINO or CLIP) with a pre-trained language model (like LLaMa), researchers can leverage existing advancements without the need for resource-intensive training from scratch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*T7CdQpmqtgViEji7ZhSiOA.png" /><figcaption><a href="https://arxiv.org/pdf/2405.02246">2405.02246 (arxiv.org)</a></figcaption></figure><p>The key component in this approach is the “adapter” layer. This relatively small set of parameters bridges the gap between the two pre-trained models, transforming the visual representations generated by the vision encoder into tokens compatible with the language model. Fine-tuning primarily focuses on optimizing this adapter layer, allowing for modest computational costs. Here is a drill down explanation of the compatibility part:</p><p><strong>Pretrained Vision Encoder:</strong> This part of the model (like CLIP or DINO) takes an image as input and processes it, ultimately producing a <em>vector</em> of numbers representing the image’s content. This vector captures features, objects, relationships, etc. Think of it as the image distilled into a mathematical code.</p><p><strong>PreTrained Language Model:</strong> This part (like LLaMa) works with <em>text tokens</em>. Tokens are discrete units, often representing words or parts of words. The language model has been trained on a massive text dataset, learning to predict and generate sequences of these tokens.</p><p>The problem is that the vision encoder’s output (a vector) and the language model’s input (a sequence of tokens) are fundamentally different. You can’t just feed the vector directly into the language model — it won’t know what to do with it.This is where that “small set of parameters” comes in, often called a <em>projection layer</em> or <em>adapter</em>. It acts as a translator: It takes the vision encoder’s vector as input and applies mathematical transformations (learned during training) to convert this vector into a sequence of numbers that <em>resemble</em> the tokens the language model understands. This sequence of “compatible tokens” can now be fed into the language model, allowing it to process visual information alongside text.</p><p>Imagine you have a French speaker and a Spanish speaker who need to communicate. They can’t directly understand each other’s languages. A translator (like our projection layer) takes the French message, converts it into Spanish, and now the Spanish speaker can understand. These compatible tokens are not <em>literally</em> the same tokens the language model was trained on. They are in the same “space” or format, allowing the model to treat them similarly, but they represent visual concepts instead of textual ones.</p><p>This is why this approach is often called “late fusion.” The modalities (vision and language) are only combined at the very end, right before being processed by the language model.</p><p>Hugging Face’s research highlights several key findings relevant to this approach:</p><p><strong>Language Model Quality Matters:</strong> Switching the language model from LLaMA-1–7B (Touvron et al., 2023) to the more powerful Mistral-7B (Jiang et al., 2023) resulted in a significant 5.1% improvement in performance, demonstrating the critical role of a strong language backbone in these hybrid models.</p><p><strong>Learned Pooling is Crucial:</strong> Traditional average pooling of visual features proved detrimental to performance. Adopting a trainable “Perceiver Resampler” for pooling visual tokens not only reduced computational burden but also enhanced downstream task performance.</p><p><strong>Challenges with Fine-Grained Visual Details:</strong> Like most VLMs, Idefix struggles to accurately extract text from images, especially when dealing with small fonts or intricate details, underscoring the limitations of current image tokenization techniques.</p><p>While Idefix 2 achieves state-of-the-art results for its size category, it can’t match the capabilities of models trained with significantly greater resources. This disparity highlights the growing divide between open-source and industry-led AI research, raising concerns about accessibility and the future of innovation in a world increasingly dominated by corporate giants.</p><p><strong>Meta’s Chameleon: Pushing the Boundaries of End-to-End Multimodality</strong></p><p>Meta, with its vast resources, is pushing the boundaries of multimodal AI by training models from scratch on an unprecedented scale. Their “Chameleon” model, a testament to this ambition, is a 34 billion parameter designed for “mixed-modal” understanding and generation.</p><p>Unlike the pieced-together approach of VLMs, Chameleon embraces an “early fusion” strategy, where text and image modalities are projected into a shared representation space from the very beginning. This allows for seamless interleaving of text and images during both training and generation, enabling a level of multimodal fluency never before achieved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*RFh1xOtjH4GhM5bNamj65Q.png" /><figcaption>Source : <a href="https://arxiv.org/pdf/2405.09818v1">https://arxiv.org/pdf/2405.09818v1</a></figcaption></figure><p>To achieve this groundbreaking capability, Meta had to overcome several significant challenges:</p><p><strong>Training from Scratch:</strong> Chameleon was trained from scratch on a massive dataset of 10 trillion tokens, including a mix of text-only data (from LLaMa 2), text-image pairs from public and licensed sources, and a dedicated corpus of interleaved text and image documents.</p><p><strong>Tokenization Innovation:</strong> Meta developed new tokenizers specifically for text and images, further optimizing the model for mixed-modal processing. The image tokenizer converts 512x512 images into 1024 discrete tokens, drawn from a codebook of 8,192 possibilities.</p><p><strong>Addressing Logit Drift:</strong> Training with multiple modalities introduced the problem of “logit drift,” where the model becomes biased towards predicting the modality with higher entropy (in this case, images). Meta addressed this through a combination of query-key normalization and dropout techniques.</p><p>Training Chameleon required a massive investment in infrastructure. The model was trained on Meta’s Research Super Cluster (RSC), consisting of over 3,000 NVIDIA A100 GPUs connected via a high-bandwidth Quantum InfiniBand network. This enabled the use of colossal global batch sizes of up to 8 million tokens.</p><p>The results are astounding. Chameleon not only surpasses existing VLMs like Flamingo and LAVAFix but also achieves state-of-the-art performance on unimodal text benchmarks, outperforming models like LLaMa 2. <strong>In human evaluations, Chameleon 34B was even preferred over GPT-4V in 46% of cases</strong>, marking a significant milestone in multimodal AI.</p><p>However, the immense computational cost of this approach poses a major obstacle to its widespread adoption. Open-sourcing Chameleon would be a game-changer for the research community, but its hefty resource requirements remain a significant barrier to entry for most researchers.</p><p><strong>Google’s Mirasol 3B: The Dawn of Audio-Visual AI and the Token Turing Machine</strong></p><p>Google DeepMind’s “Mirasol 3B” takes multimodal AI a step further, incorporating audio processing alongside video and text. While it currently outputs only text, this model offers a compelling preview of the capabilities that will be present in Google’s upcoming “Project Astra”.</p><p>Mirasol 3B employs several clever techniques:</p><p>First technique being the representation of audio and video. Instead of tokens, they used chunks of audio and video. They use a combiner to combine these modalities which is then being fed into the transformer as shown below. If we contrast this with the hugging face late fusion approach, combiner will be analogous to the “Adapter” layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*EjQpZmj-1WIjNWI6XxLkjQ.png" /></figure><p>Some other key innovation is summarized in below points:</p><p><strong>Spectogram Representation of Audio:</strong> By transforming raw audio waveforms into spectrograms (visual representations of frequency content over time), Google effectively converted audio into an image-like format, allowing it to be processed by the same Vision Transformer backbone used for video encoding. This reuse of existing architecture streamlines the model’s design.</p><p><strong>3D Tube Features for Video:</strong> The Vision Transformer analyzes video not as a sequence of individual frames but as “3D tubes,” capturing both spatial and temporal information. These tubes vary in their receptive fields, allowing the model to learn features representing both localized actions and broader scene context.</p><p><strong>Time-Aligned Multimodal Processing:</strong> The model leverages the inherent synchronization of audio and video in time to enhance joint learning, treating each modality as a sequential stream of information.</p><p>However, the truly revolutionary aspect of Mirasol 3B lies in its utilization of a “Token Turing Machine” (TTM). This component, somewhat understated in the paper, is essentially similar in idea to a recurrent neural network, built with Transformers. It allows the model to maintain an external memory, where input features can be stored, retrieved, and processed over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Ti4wF9tJHxW6GOdmQn-R7w.png" /><figcaption><a href="https://arxiv.org/pdf/2311.05698">2311.05698 (arxiv.org)</a></figcaption></figure><p>This memory component provides several key benefits:</p><p><strong>Enhanced Temporal Reasoning:</strong> The model can now recall information from previous time steps, enabling more complex reasoning about events and actions within the audio-visual input.</p><p><strong>Constant Time Complexity:</strong> Unlike traditional Transformers, which exhibit quadratic time complexity with respect to sequence length, the TTM achieves constant time complexity due to its fixed memory size, making it more efficient for processing long sequences of data.</p><p><strong>Potential for Long-Term Memory:</strong> The TTM hints at the possibility of implementing true long-term memory in AI models, where information from past interactions can be recalled and utilized in future conversations, creating a more personalized and context-aware experience.</p><blockquote><em>The demonstration of a memory-augmented multimodal model in both Mirasol 3B and models like gpt-4o signals a significant shift towards AI systems that can reason about events over time and build upon past experiences</em>. This move towards “stateful” AI, where the model maintains an internal state and memory, presents both exciting possibilities and new challenges for researchers.</blockquote><p><strong>The Multimodal Future: A World of AI-Generated Reality</strong></p><p>These groundbreaking advancements offer a glimpse into a future where AI seamlessly understands and generates all forms of human communication, seamlessly integrating with our physical world.</p><p>Imagine a world where:</p><p><em>AI assistants are no longer disembodied voices but interactive avatars, visually and aurally indistinguishable from real people, engaging in complex conversations and understanding our emotional cues.</em></p><p><em>Robots are controlled by end-to-end multimodal models, capable of interpreting our instructions, adapting to changing environments, and interacting with the world in a more nuanced and human-like manner.</em></p><p><em>VR and AR experiences are powered by AI that dynamically generates realistic environments, characters, and narratives, responding to our actions and creating truly immersive and personalized worlds.</em></p><p><em>While this future may seem both exhilarating and unnerving, it appears to be an inevitable outcome of the current trajectory of AI research.</em></p><p>References:</p><p>What matters when building vision-language models? <a href="https://www.youtube.com/redirect?event=video_description&amp;redir_token=QUFFLUhqazhObWVoMVFqNzAyRXNSTHQ1Ukp6X0xiZ0NkQXxBQ3Jtc0tsTi10bk9QSkJDLTkyaWpHZ0tQaFVobnY5ZERVMEFMRklXYXpoQlV1bG1WLXUwazZuM3RzME1NMlFxZjR5eWNhRTl1Smk1VXhmYmNJZ0RhLW1sSlZIeGNiME5TZkRqMXV6Yk0tRGFRMV91UHd3TzRQRQ&amp;q=https%3A%2F%2Farxiv.org%2Fpdf%2F2405.02246&amp;v=27cjzGgyxtw">https://arxiv.org/pdf/2405.02246</a></p><p>Mirasol3B: A Multimodal Autoregressive Model for Time-Aligned and Contextual Modalities <a href="https://www.youtube.com/redirect?event=video_description&amp;redir_token=QUFFLUhqbjJnVFk4SG5DWXUtb1ZGLXNkSGFnbGhpdENVQXxBQ3Jtc0tucV92SmNaOHFsTmU5Q09iWDl6R1pvSTJmZFF4dUlTQkhRTHFZUldFN29mRFdwcWRTUVdwQlJ5Z3NKX3dNNXpZXzdGRjByeEJHRDhvSE1qRXgxd05SSk1pdDlNRXJsMGh0TGVLS1JYYkZuanNwVVpycw&amp;q=https%3A%2F%2Farxiv.org%2Fpdf%2F2311.05698&amp;v=27cjzGgyxtw">https://arxiv.org/pdf/2311.05698</a></p><p>Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context <a href="https://www.youtube.com/redirect?event=video_description&amp;redir_token=QUFFLUhqbUVPamxSOElaNnUzZVNoTlVEOTlUWXBnczRkQXxBQ3Jtc0tuNndWVy1BYWhLcU41OVNNaHNSalFHbmtEeXprYVU1dkc4WVpiM3hHZERESHBVRmlkYzdVTGJ5d1ljYXdNTjlyNk95amhyaWh1VFpaQWFMZzRhZG94TDhTVFVrYnRQRXM2NEplajZDSFRvYTg2LVdZbw&amp;q=https%3A%2F%2Fstorage.googleapis.com%2Fdeepmind-media%2Fgemini%2Fgemini_v1_5_report.pdf&amp;v=27cjzGgyxtw">https://storage.googleapis.com/deepmi...</a></p><p>Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning <a href="https://www.youtube.com/redirect?event=video_description&amp;redir_token=QUFFLUhqbmNpam92b2x0VzY4RVREQUtSZjNyQm1OR3VMd3xBQ3Jtc0tub3h3Z0s0UlZXU2pRWUhnVEpUeGRtT080NTJfLXlQbmhpYUMzdTQtOVZNclNOVlk0UEVaUm1COV84dWhjMEtPZFp6YkM4ekFKRjFfWDNmX2ktc0dYREdVTnNfV016Q3hzdktqdVBzdWlZdGtxQnlzYw&amp;q=https%3A%2F%2Farxiv.org%2Fpdf%2F2309.02591&amp;v=27cjzGgyxtw">https://arxiv.org/pdf/2309.02591</a></p><p>Chamaleon : <a href="https://arxiv.org/pdf/2405.09818v1">https://arxiv.org/pdf/2405.09818v1</a></p><p>Youtube.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06e01e94a3ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling Transformers in Textual/Visual Domains: A Gateway to AGI?]]></title>
            <link>https://medium.com/@ds.deepak/scaling-transformers-in-textual-visual-domains-a-gateway-to-agi-254ee4f57dad?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/254ee4f57dad</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[agi]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[q]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sun, 03 Dec 2023 14:10:16 GMT</pubDate>
            <atom:updated>2023-12-03T14:29:38.818Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MmkT5i3qtbDJA4MNk733LA.jpeg" /><figcaption>Image generated by AI</figcaption></figure><p>AGI (Artificial General Intelligence) has become a hot topic of discussion lately, with leaked conversations (<em>referencing Q* from OpenAI</em>) adding significant momentum to the ongoing debate. Lets delve deep into several pivotal aspects of this evolving discourse.</p><blockquote>For the purpose of our discussions, AGI is ‘The ability of a computer or other machine to perform those human activities which are thought to require intelligence.’</blockquote><p>The concept of intelligence is inherently subjective and continuously evolving. Human perspectives on intelligence have shifted over time. Initially, when computers mastered chess and outperformed every grandmaster, we reconsidered the definition. ‘Perhaps playing chess isn’t the epitome of intelligence,’ we shifted. The next stop was to conquer a game like Go( The number of potential positions or moves in Go is astronomically high, its practically impossible to calculate or explore all of them).</p><p>Surprisingly, AlphaGo achieved a resounding victory of 4–1 against the Go world champion, Lee Sedol. Now, did that count as intelligence?</p><p>This continuous reevaluation of what constitutes intelligence underscores the complexity of the human mind and the challenge of replicating it within machines. Let’s explore the nuances of AGI and the evolving benchmarks of intelligence.</p><p>For this discussion, let’s stick to the intelligence which involves reasoning and logic like humans. Now don’t be mistaken GPT-4 did develop some reasoning capability as mentioned in the paper “Sparks of AGI”. I have written an article on the same : <a href="https://medium.com/@ds.deepak/the-theory-of-mind-has-it-emerged-in-llms-7571b89c1c1">The Theory of Mind, has it emerged in LLMs?</a></p><p>There are many scientists who believe that as we scale the model larger, it will become increasingly intelligent. (There is a race underway in the world to scale it larger, implementing various tweaks and adjustments along the way). Seeing what GPT-4 achieved , scaling with more and more data (human + AI Generated) definitely looks promising.</p><p>On the other side of this, there are legendary researchers like Yan Le Cunn who believes that language has nothing to do with intelligence. He argues that Orangutan can’t speak a language but they are almost as intelligent as humans. He suggests that the way to AGI will be through improved cognitive architecture and not just by scaling the transformers on texts, images, audios etc.</p><p><strong>Will AGI or Superintelligence destroy humanity?</strong></p><p>When asked whether AGI would attempt to eradicate humans, his emphatic response was a resounding “NO.” He went on to emphasize that intelligence is not synonymous with a desire for power. Citing historical examples, he noted that it has never been the intelligent individuals who sought to dominate the world; rather, it has often been the reverse.</p><blockquote><em>The Dual Process Theory</em></blockquote><p>Let’s delve into another very interesting topic: The dual process theory(discussed in the book Thinking Fast and Slow)</p><p>Our brains use two disparate modes of thinking:</p><p>1. Intuitive Understanding (Fast)</p><p>2. Logical Reasoning (Slow)</p><p>Brains spend only about 0.001% of its cycles on reasoning proving that reasoning is a very thin layer on top of understanding. Some people argue that 20th-century research was in the wrong direction, as it was focused more on the reasoning part and less on the understanding part (more on this in some other article where I will discuss the model-free methods)</p><p><em>“Doing what we do without thinking has to be the first step towards True AGI” Monica Anderson</em></p><p>She argues that solving these complex optimization problems( for example Minimax Tree Search etc) is not what constitutes an AGI. It is the capability to have a good intuitive understanding and reasoning capability is a right step towards AGI and the model-based methods that we followed in 21st Century are not the way to do it.</p><p>Monica Anderson’s perspective sheds light on a crucial aspect often overlooked in the pursuit of AGI. She posits that the emphasis on model-based methods in the 21st century might not align with the path towards True AGI. Instead, the core lies in developing intuitive understanding. In her view, the ability to navigate complex problems intuitively rather than relying solely on explicit reasoning is the hallmark of True AGI.</p><p>This concept correlates with the Dual Process Theory, underscoring the importance of intuitive understanding alongside logical reasoning. Our brains predominantly rely on intuitive understanding, allocating only a minuscule fraction of cognitive resources to logical reasoning. This underscores the significance of developing models that prioritize intuitive comprehension, mimicking the human brain’s cognitive processes more accurately.</p><p>Yan Le Cunn’s divergence from the mainstream belief highlights a critical debate within the AI community. His emphasis on improved cognitive architecture resonates with the idea that intelligence goes beyond mere language or computational scaling. While scaling models like transformers may enhance certain capabilities, the essence of achieving AGI may lie in the development of more sophisticated cognitive frameworks rather than sheer scaling alone.</p><p>Ultimately, the quest for AGI demands an understanding of intelligence, encompassing both reasoning and intuitive understanding. It requires a departure from conventional methods and a deeper exploration into cognitive architectures that mimic the human mind’s intricate workings. As the race towards AGI progresses, reconciling these differing viewpoints will be pivotal in shaping the future trajectory of artificial intelligence.</p><p>In conclusion, the path to Artificial General Intelligence might not solely rest on scaling models or adopting predetermined methodologies. Instead, it could be a fusion of scaling, cognitive architecture, intuitive understanding, and reasoning — a synergy that reflects the multi-dimensional facets of human intelligence. Only through the integration of these diverse perspectives we can pave the way toward a more comprehensive and closer realization of AGI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=254ee4f57dad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Theory of Mind, has it emerged in LLMs?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@ds.deepak/the-theory-of-mind-has-it-emerged-in-llms-7571b89c1c1?source=rss-382c0b0aab6e------2"><img src="https://cdn-images-1.medium.com/max/1024/1*m9tWTmZjqzVHa0gvYg69OA.png" width="1024"></a></p><p class="medium-feed-snippet">&#x201C;Theory of mind&#x201D; denotes the ability to comprehend and interpret the perspectives of others by attributing mental states to them. This&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@ds.deepak/the-theory-of-mind-has-it-emerged-in-llms-7571b89c1c1?source=rss-382c0b0aab6e------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@ds.deepak/the-theory-of-mind-has-it-emerged-in-llms-7571b89c1c1?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/7571b89c1c1</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[theory-of-mind]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sat, 30 Sep 2023 17:57:31 GMT</pubDate>
            <atom:updated>2023-09-30T18:08:56.016Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Demystifying Bayes Rule with an example to remember]]></title>
            <link>https://medium.com/@ds.deepak/demystifying-bayes-rule-with-an-example-to-remember-45848a7fa0cd?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/45848a7fa0cd</guid>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[bayes-theorem]]></category>
            <category><![CDATA[bayesian-machine-learning]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <category><![CDATA[bayes-rule]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sat, 17 Dec 2022 12:51:38 GMT</pubDate>
            <atom:updated>2022-12-17T12:54:50.831Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s talk about Probability today. It used to be the most dreaded subject during our high school days, mostly because of two reasons:</p><ol><li>Our mind, which has mostly been a part of a deterministic environment, simply could not connect with these probabilistic constructs.</li><li>You go near a new switchboard to turn on a fan. There are 6 switches and you still manage to switch it on, on the 1st attempt. (You are like, probability huh! and start thinking about whatever happened to that 1/6 ~16.67 % chance)</li></ol><p>Both, by the way, are super legit reasons :)</p><p>Rather than confusing our brains with notations and symbols, building correct intuition is important. That being said, let’s built a crystal clear intuition about Bayes Rule.</p><blockquote>Starting with a few history lessons which I came to know about from this beautiful book “<strong>The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, &amp; Emerged Triumphant from Two Centuries of Controversy</strong>”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kxSpVGg3AnXWAhFqOztTfQ.jpeg" /><figcaption>An Enigma machine, used for enciphering messages during World War II Image Credits: <a href="https://unsplash.com/@dchris">https://unsplash.com/@dchris</a></figcaption></figure><p>You must be like, what??? that P(A/B)=P(B/A)*P(A)/P(B) or something like this was able to crack Enigma Codes (How????).</p><p>I hope I have piqued your interest, Let’s begin.</p><p>The idea of Bayes was really simple “<strong>you start with an initial belief, keep updating it based on the new information and that is your new and improved belief</strong>”. That’s all! no more, no less.</p><p>It just teaches us how to be less and less wrong with more data.</p><p>Let’s say you travel to a new country. You are waiting on a signal to turn green so that you can cross the road. Suddenly, a car stops(out of the respect for a pedestrian) and asks you to cross the road. You crossed the road joyfully and now start thinking “Was that a nice person who let me cross” or “In this country, they always stop for the pedestrians”. Now, at this point, you have to take one of these prior beliefs. The next day while going to the office, the same thing happens (someone stops to let you cross). You update your belief and with every new occurrence, you acquire new information and keep updating your belief system. This is Bayes Rule.</p><blockquote>An Example to Remember</blockquote><p>Let’s assume, we are in the year 2050, Covid is very rare now and affects only 0.1% of the population. We have developed a home testing kit that can identify 99% of the time if you have Covid. One morning you wake up coughing and it feels like Covid so you decided to take that test.</p><p>It came positive, now if someone asks you what are the chances of you having Covid, I guess your answer will be 99%. Unfortunately, that is not correct and this is where you need Bayes theorem.</p><blockquote>Bayes theorem can tell you P(having the disease/given you tested positive)</blockquote><p>Let’s formally define Bayes Theorem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/1*_khmyL_YhJ6SE7i5KQP3gg.png" /></figure><p>For our example, this can be written as :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*IjLQm2uxdt9c8VhmtSO7DA.png" /></figure><p><strong>P(disease | test positive)</strong> represents the Probability of having the disease if your result is positive.</p><p><strong>P(test positive | disease)</strong> represents the Probability of testing positive, in case you have the disease.</p><p><strong>P(disease) </strong>represents the prior or initial belief about having the disease</p><p><strong>P(test positive)</strong> represents the total probability of the event occurring (testing positive)</p><p>P(test positive) can be represented as :</p><p>P(having the disease)*P(testing positive | having the disease) + P(not having the disease)*P(testing positive | not having the disease)</p><blockquote>Let’s do the math now:</blockquote><p><strong>P(disease | test positive)=(0.99*0.001)|(0.001*0.99+0.999*0.01)=~9%</strong></p><p>What! Just 9%, this seems low compared to our initial guess of 99%.</p><p>Let’s dig deeper to understand why this happened.</p><p>Let’s consider a sample population of 1000. Now, 0.1% of the population is likely to have this disease(1 person in 1000) and the test is likely to identify it correctly. Now, since the test is only 99% accurate, it will be wrong for the remaining 1% of the population i.e. for 10 persons.</p><p>If you were part of a group that has a positive test result, then you are part of a group where 1 out of 11 has the disease i.e. 9%. I hope the 9% which came mathematically makes sense now.</p><p>Consider a case, when you are not satisfied with the test result, and you do that test again with a kit you purchased at a different shop( so that these tests are independent) If the test came positive again, we need to update our belief accordingly. our prior which was 0.001 initially should now be updated to 0.09(~9%).</p><blockquote>Let’s do the math again:</blockquote><p><strong>P(disease | test positive)=(0.99*0.09)|(0.09*0.99+0.91*0.01)=~91%</strong></p><p>Now, this seems logical, if two independent samples are saying you have the disease, it must be right, and now the probability is 91%.</p><p>Surprisingly, it is still not 99%. You see the beauty of Bayes rule, it’s helping us update our beliefs as and when we have new data. In short, it’s helping us to be less and less wrong with more data.</p><blockquote>If the third test also comes positive, now the probability will get updated to 99.90%. (Readers! Please do the math and verify )</blockquote><p>Coming back to how Bayes Theorem helped in cracking Enigma codes. Alan Turing developed a mathematical model using the Bayesian Theory which helped him in guessing the letters in Enigma messages. The system was used to calculate the probabilities based on a prior(as we discussed) and the priors were updated as more evidence became available. Using a similar system the U-boats could be located with some degree of certainty.</p><blockquote>Excerpt from the book</blockquote><p>“Surprisingly, given Bayes’ success in fighting U- boats during the Second World War, the U.S. Navy embraced the method slowly and grudgingly during the Cold War.”</p><p>I hope this example helps in building a solid intuition.</p><p>References:</p><p>The Theory That Would Not Die</p><p>youtube.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45848a7fa0cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Going deep into Entropy, Cross-Entropy, and KL Divergence]]></title>
            <link>https://medium.com/@ds.deepak/going-deep-into-entropy-cross-entropy-and-kl-divergence-6c512c3767d8?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c512c3767d8</guid>
            <category><![CDATA[entropy]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[cross-entropy]]></category>
            <category><![CDATA[kl-divergence]]></category>
            <category><![CDATA[cross-entropy-loss]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Thu, 15 Dec 2022 20:42:32 GMT</pubDate>
            <atom:updated>2022-12-19T19:16:13.780Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Going deep into Entropy, Cross-Entropy, and KL Divergence</strong></p><p>In Machine Learning we hear a lot about Entropy, Cross-Entropy, and KL Divergence.</p><p>In Machine Learning literature, we can see these terms getting frequently used in the below constructs:</p><blockquote>Entropy in Decision Tree and Random Forest.</blockquote><blockquote>Cross Entropy in Cost Functions in Classification Problems.</blockquote><blockquote>KL Divergence in Knowledge Distillation.</blockquote><p>and many other places.</p><p>Now, if you start thinking about it you will recollect that:</p><blockquote>Entropy was taught in Information Theory during our undergrad days.</blockquote><p>and some other places.</p><p>Now, think about it how and why Machine Learning uses this concept from Information Theory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BHkoGIUjS7EkPzjo0tlqIQ.jpeg" /><figcaption>That’s me! Thinking deeply on why Machine Learning uses the concept of Entropy from Information Theory.</figcaption></figure><p>Today, we will go deep into these topics.</p><p>It all started when Claude Shannon wrote the seminal paper called “A Mathematical Theory of Communication” in 1948.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXC_3J8kUppoaHRbYMDDkw.png" /></figure><blockquote>With this paper, he put forward a solid foundation for Information Theory.</blockquote><p>The Goal was to reliably send a message from sender to recipient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*mmrHMQRaas27sl8Vg0YkRw.png" /></figure><p>In the digital age, messages are composed of bits (1 or 0). When we send information, not all bits are useful. Some bits are redundant, some are erroneous, etc. Shannon proposed that every bit of transferred information should reduce the recipient’s uncertainty by 2.</p><p>Let us take an example:</p><p>Suppose you have appeared for a Machine Learning Exam and are wondering if you passed or failed the exam. You called the teacher to enquire about the exam and the teacher informed you that you have passed. Now, it does not matter how this information was conveyed(“You have Passed”, “Pass”, “P”, 1, etc.), it reduced the uncertainty by a factor of 2(considering both cases were equally likely). We can say that the teacher sent you 1 Bit of information because it reduced the uncertainty by 2.</p><p>Now, consider a case where you would like to also know the actual grades which have 8 possible values(3,4,5,6,7,8,9,10).</p><p>Since you are studying Entropy and Cross-Entropy, I assume you will get 3 or higher grades :)</p><p>Now, If your teacher informs you about your grades, he reduces your uncertainty by a factor 0f 8 i.e. 2³, This can be considered as 3 bits of useful information. The number of bits communicated can be calculated by taking the binary logarithm of the uncertainty reduced e.g. log2(8)=3.</p><p>Consider a different case where you have prepared well for the exam and now the probability of passing is 75% and the probability of failing is 25%. Given the above conditions, if your teacher informs you that you have failed, any guesses on how many bits of information were conveyed?</p><p><strong>Answer</strong>: 2 bits(log2(4)), because he reduced your uncertainty by 4 (1/0.25).</p><blockquote>Note: Uncertainty reduction is the inverse of the event’s probability.</blockquote><p>The number of bits transferred can also be written as<strong> -log2(P(x)).</strong></p><p>If the teacher informs you that you have passed, the number of bits transferred will be -log2(0.75)=0.41</p><p>Given the conditions, on average your teacher is transferring :</p><blockquote>0.75*0.41 + 0.25*2=0.81 bits</blockquote><p>The above entity is nothing but Entropy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*Ty-cTygzFdtcl188clIqeA.png" /></figure><p>Formally, <strong>Entropy can be defined as the average amount of information that you get from any random sample of a probability distribution</strong>. It’s a measure of how unpredictable the distribution is. To explain this fact, let’s say Geoffrey Hinton appears for that exam. The probability of him passing is always 1, So the entropy, in this case, will be close to 0 (I guess, you get the idea where I am going with this :))</p><p><strong>Let’s talk about Cross-Entropy now</strong></p><p>It is just the average message length.</p><p>Let’s say you have 8 possible grades and your teacher uses the below code to convey this information:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*-kRjHKjjxGC4OtlThZ9_OA.png" /><figcaption>Probability Distribution of Grades</figcaption></figure><p>In this case, the cross entropy is 3 bits.</p><p>Suppose, you have put in a lot of effort and your chances of getting different grades are as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/1*IFVIZzrszPNuUSybhiXG2w.png" /><figcaption>Probability Distribution of Grades</figcaption></figure><p>In this scenario, the entropy is 0.35*log2(0.35) +0.35*log2(0.35)+………. =<strong>2.23 bits</strong></p><p>Now, the entropy of this distribution is only 2.23 bits but your teacher is using 3 bits to convey this Information. Now can we do better (using less than 3 bits)?</p><p>We can do better by using a different code for each message(preferably fewer bits for events having more probability). Let’s say we build a system like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/331/1*BzoX5b2J45nKzSd7ZqhLhA.png" /><figcaption>Probability Distribution of Grades</figcaption></figure><p>Let’s calculate the cross entropy:</p><p>0.35*2+0.35*2+0.1*3+0.1*3+0.04*4+0.04*4+0.01*5+0.01*5</p><p>The cross-entropy comes to <strong>2.42 bits.</strong> This is good but still not better than 2.23 bits.</p><blockquote>Exercise for the reader: Why have we chosen code in this arbitrary order (why no 1000 or 1111 after all they have less number of bits than 11101)? Put the answer in the comments.</blockquote><p>Now, if we reverse the order of probability, you will see that the total number of bits will drastically increase to <strong>4.58 bits.</strong></p><p>We are almost sending twice the number bits as is necessary. This is happening because the code we are using makes some implicit assumption about the distribution (.35 probability of getting grade 3 and so on..)</p><p>Let’s assume that the one with 2.42 bits was our actual distribution(p) and the one with 4.58 bits is our predicted distribution(q)</p><p>Formally, Cross-Entropy can be defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*22Jm3uBLBvntDNlYPOPlBA.png" /><figcaption>p is the true distribution and q is the predicted distribution</figcaption></figure><p>As observed, it is the same as Entropy, the only difference is we take the log of the predicted distribution instead of the true distribution.</p><blockquote>If our predictions are perfect, the Cross-Entropy will simply be equal to the Entropy.</blockquote><p>But, if the distributions differ then the Cross-Entropy will be greater than the Entropy by some bits. This amount is called KL Divergence.</p><blockquote>Cross-Entropy=Entropy + KL Divergence</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*w65wBagaRjY-eQUtpELyLg.png" /></figure><p>Now, Coming back to the questions I asked, why use Cross-Entropy Loss in classification problems?</p><p>For any given input there is a True Distribution. Let’s take the example of the Irish Dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*jljELOFSPHP7TpzAGgCYMw.png" /><figcaption><a href="https://archive.ics.uci.edu/ml/datasets/iris">https://archive.ics.uci.edu/ml/datasets/iris</a></figcaption></figure><p>We can use the Cross-Entropy between these two distributions as a cost function. This is also called log loss.</p><p>In this case, the Cross-Entropy Loss will be -log(0.8) =0.09. Instead, if the probability of Virginica was 25%, the loss would increase to log(0.25)=1.386.</p><p>As you can see the loss increased as the predicted probability distribution moved away from the actual probability distribution which is the whole point of choosing this Cross-Entropy as a cost function.</p><p>References:</p><p><a href="https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf">https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf</a></p><p>Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Géron.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c512c3767d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Hypothesis Testing and Statistical Tests]]></title>
            <link>https://medium.com/@ds.deepak/demystifying-hypothesis-testing-and-statistical-tests-6e8a3a8df158?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e8a3a8df158</guid>
            <category><![CDATA[statistical-test]]></category>
            <category><![CDATA[hypothesis-testing]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sat, 04 Jun 2022 11:45:30 GMT</pubDate>
            <atom:updated>2022-06-04T17:23:18.859Z</atom:updated>
            <content:encoded><![CDATA[<p>As a Machine Learning Scientist, during the initially days of my career I could not relate to constructs like Hypothesis Testing, p-value, T-Statistic etc. Tried reading lots of blogs/articles about it but it did not help. Finally, when I worked on some real use cases, that is when these ideas became clear.</p><p>If you are beginner, looking to start your Hypothesis Testing Journey, this article can serve as a good starting point.</p><p><strong>Now, let’s discuss what is Hypothesis Testing?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eLwMJ0XBMydwvQgAU_Hgsg.jpeg" /><figcaption><strong>Yup!! That’s me outside Holland Casino, Amsterdam</strong></figcaption></figure><p>One fine day, I went to play Blackjack in a Casino and won 3 out of 4 times. As any naïve gambler, I thought my winning probability is 3/4. Played 4 more times and lost all of the times.</p><p><strong>Moral of the story:</strong> You can’t randomly accept an observation as statistically Significant. It should be backed by a rigorous process of Hypothesis testing.</p><p>Basically, hypothesis testing is a way of proving/rejecting the idea that your observation is statistically significant. In simple terms, when a Random Experiment is performed, we already get the empirical data (the actual observations), now with the help of hypothesis testing we want to figure out if this(observation) was a fluke or it has some statistical significance.</p><p>On a very High Level, we can use one among the below techniques to perform a hypothesis testing:</p><p>On a very High Level, we can use one among the below techniques to perform a Hypothesis Testing:</p><p><strong>1. Statistical Tests (e.g., T-test, Z-test, Chi-Square test, ANOVA)</strong></p><p><strong>2. Permutation testing (Most things which statistical tests can do, can also be done by Permutation test)</strong></p><p>Note: All of these tests like Z test, T test etc. works when we know the distribution of test statistic under null Hypothesis.</p><p>Let’s get ourselves acquainted with some of the terms which will come handy in grasping the concepts in detail.</p><p><strong>Random Variable:</strong> A random variable is a function which assigns values to the outcome of a random experiment. For e.g., probability of getting a head in a coin toss is a random variable.</p><p><strong>p-value:</strong> Simply speaking, it is the probability of observing something which you have already observed (or something which is extremer than what you have observed).</p><p><strong>Test-Statistic:</strong> The test statistic is a number calculated from a statistical test of a hypothesis. It shows how closely your observed data match the distribution expected under the null hypothesis of that statistical test.</p><p><strong>Now, Let’s say your task is to determine if a coin is biased towards Heads.</strong></p><p>We need to perform an experiment with the coin and as I explained in the beginning, we will look at the observed data and then with the help of hypothesis testing we can figure out if this statement is statistically significant.</p><p><strong>Let’s design the experiment.</strong></p><p><strong>Ho(Null Hypothesis)</strong>: Coin is fair (T ~50)</p><p><strong>Ha(Alternate Hypothesis)</strong>: Coin is biased towards head (T &gt;&gt; 50)</p><p>Here, T denotes the #number of occurrence of Heads in 100 successive trials. Our test statistic is Number of heads observed in the 100-coin tosses. Now, The Knowledge of Random Variable and distributions will come handy.</p><p><strong>Some additional Theory: </strong>A coin toss is a Bernoulli Random Variable and the count of success or failure in N successive Bernoulli trials is a Binomial Random variable (i.e. A series of coin tosses is a perfect example of a binomial distribution)</p><p>We can see that the test statistic distribution under null hypothesis is known and a well-studied distribution (T ~ Binomial (n=100; p=1/2) )</p><p>Here n(100) denotes number of Bernoulli trials and p denotes the probability of success (1/2). Now, Let’s say in the experiment you observed 70 heads (This is the ground truth i.e., actual observed value).</p><p>P-Value = P (Probability of getting 70 or more heads in 100 coin toss | given : Coin is fair )</p><p>P(T&gt;=70) =&gt; 1 — P(T&lt;=69) =&gt; 1 — CDF(T=64) =0.003%</p><p>You can use below website to calculate the above value:</p><p><a href="https://stattrek.com/online-calculator/binomial.aspx">Binomial Calculator</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*9gPla_sbLtsnuGntS-qTZg.png" /></figure><p>As we can see this value (0.003%) is less than alpha (alpha is generally kept at 5%). Hence, we can reject the Null Hypothesis and Accept the Alternate Hypothesis.</p><p><strong>Now let’s analyze what just happened in simple terms.</strong></p><p>We just calculated that under the Null hypothesis (i.e., the coin is fair) the chances of 70 Heads in 100 Coin Toss is very less (~0.003%). But we have already observed 70 heads in 100-coin tosses.</p><p>Now, The ground truth(70 Heads in 100 Trials) can’t be wrong. So, only possible explanation is that the coin is not Fair and its Biased towards Heads. Hence the Null Hypothesis is Rejected and we accept the Alternate Hypothesis.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e8a3a8df158" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TensorFlow 2.0 : The complete picture]]></title>
            <link>https://medium.com/@ds.deepak/tensorflow-2-0-the-complete-picture-ada53fd2b477?source=rss-382c0b0aab6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ada53fd2b477</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[machine-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Deepak Singh]]></dc:creator>
            <pubDate>Sat, 13 Jun 2020 12:53:31 GMT</pubDate>
            <atom:updated>2020-06-13T13:11:08.691Z</atom:updated>
            <content:encoded><![CDATA[<h3>TensorFlow 2.0 : The complete picture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T70mkty9PLR2PbdiBnvx7Q.jpeg" /><figcaption>Image credits : <a href="http://forum.skill.jobs/">http://forum.skill.jobs/</a></figcaption></figure><p>There is so much buzz around TensorFlow 2.0 from the day it has been officially released. It is deeply loved and applauded by the ML community. There are numerous reasons for that and I would like to bring light to few of them.</p><p>In this blog series, I will be discussing new APIs/tools available in the TensorFlow2.0 and explain the valid use cases.</p><p>Let’s start by discussing reasons why TensorFlow 2.0 should be your partner in crime:</p><h4>1. It’s built By Google!</h4><p>As evident from various other innovations from Google, we will get seamless performance, quick updates and new releases as and when needed. It contains most state-of-the-art algorithmic implementations and it will keep on growing.</p><h4>2. It doesn’t cost a dime</h4><p>It has been open-sourced by Google and you can use this marvel for free. It does have a decent community support as well.</p><h4>3. It’s well documented</h4><p>With lots of code examples to learn, Google has done a decent job documenting it. It gives us an end-to-end understanding of this open source machine learning platform.</p><h4>4. It’s damn popular</h4><p>Among the top Machine Learning Libraries (TensorFlow 2.0, PyTorch, Keras and FastAI), TensorFlow 2.0 has the largest increase in job listings across the biggest online job boards. Companies which are recently moving to ML/AI are definitely opting for TensorFlow.</p><h4>5. It’s super compatible</h4><p>TensorFlow 2.0 provides support for CPU, TPU, desktops, web and mobile platforms. Given the tools and technology already available it can easily be scaled up for a production use. With the help of TensorFlow Serving(not limited to), ML applications can be easily integrated with an existing production setup and deployed easily.</p><h4>6. Ease of Integration</h4><p>It is integrated with High Level Libraries like Keras and Estimators. It has support for all the popular programming languages(Python, C#, Java, R, Go, Rust, Julia and many more) thereby helps in getting up to speed easily</p><h4>7. A boon for Research work</h4><p>Though TensorFlow 2.0 has made many high level APIs for its ease of use, Researchers/Developers needing a different flow design, can still go deep in the low level code design with the help of low level APIs and have a greater control.</p><h4>8. A Visual Delight</h4><p>When building and debugging new models, TensorFlow 2.0’s computational graph visualization helps you understand, inspect and debug via TensorBoard, which produces a visual representation of the model.</p><h4>9. Eager Execution</h4><p>The declarative programming approach used in TensorFlow 1.0 (static computational graphs, use of session variable etc) was a little confusing for many people. Now, with the presence of Eager Execution (It will be explained in great detail in future posts) there is no need to first statically define a computational graph.</p><h4>10. AutoGraph</h4><p>With AutoGraph, Tensorflow 2.0 now supports imperative python code like control flow and other Python-native features. As we know python code is very intuitive but converting this code to define a computational graph was a tedious task. AutoGraph takes eager-style Python code and automatically converts it to graph-generating code.</p><p>I will be discussing all these features in great detail in my coming posts. Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ada53fd2b477" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>