<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tejal Khade on Medium]]></title>
        <description><![CDATA[Stories by Tejal Khade on Medium]]></description>
        <link>https://medium.com/@tejalrk2000?source=rss-bba66688a986------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Ezq1Ool5hkspB3Y-2htieA.jpeg</url>
            <title>Stories by Tejal Khade on Medium</title>
            <link>https://medium.com/@tejalrk2000?source=rss-bba66688a986------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 22:35:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tejalrk2000/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Behind the Experts: Understanding MoE in Modern LLMs]]></title>
            <link>https://tejalrk2000.medium.com/behind-the-experts-understanding-moe-in-modern-llms-95dc4595c721?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/95dc4595c721</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[mixture-of-experts]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Wed, 19 Aug 2026 11:19:47 GMT</pubDate>
            <atom:updated>2026-08-19T11:19:47.755Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Mixture Of Experts and Why it works?</h3><blockquote>MoE is an AI architecture where a set of specialized sub-networks also called as “experts” sit inside a larger model, and a lightweight gating mechanism decides which experts process each input token. It’s a divide-and-conquer strategy that optimizes for both performance and efficiency. Instead of using the whole model every time, only a small subset activates per token letting a model contain hundreds of billions or trillions of parameters while using a fraction of them for any given input.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*yLHVEvbFz1n3zzWeeTgBcQ.png" /></figure><ul><li><strong>Gating network</strong>: This acts as a manager who decides which expert or sub network is best suited for each part of the problem. The router or the gating mechanism decides which experts should handle each token.</li><li><strong>Experts</strong>: These are smaller sub networks of the model where each is trained to be really good at a specific part of the problem.</li></ul><h3>Why is this useful?</h3><p>Let’s understand with an example. Suppose a model has 100 experts, each containing 10 billion parameters. That’s potentially <strong>1 trillion parameters</strong> in the whole model. But if the router activates only 2 experts for each token you will be using only <strong>20 billion parameters</strong> for processing that token. That is the reason you get huge model capacity and relatively low computation. As LLMs become capable of handling many different tasks such as coding, mathematics, languages, reasoning, vision, and agentic tasks, one set of parameters has to handle everything. This can make the model less efficient and cause different tasks to interfere with each other. Mixture of Experts (MoE) addresses this by dividing the model into multiple specialized experts. A router selects only the most relevant experts for each input. So, instead of making the entire model work on every token, only a small portion of the model is activated when needed.</p><h3>How does Mixture Of Experts work?</h3><h3>1. Training the Experts</h3><p>You can think of MoE training as training a team: <strong><em>Experts learn how to solve problems, while the router learns which experts to call.</em></strong></p><p>First, the individual experts learn how to solve problems. Each expert is a neural network that can gradually become better at handling certain patterns or types of inputs. During training, the expert makes a prediction, compares it with the correct answer, calculates the error, and updates its parameters to reduce that error.</p><p>For example:</p><ul><li>Expert 1 may become good at <strong>coding</strong></li><li>Expert 2 may specialize in <strong>mathematics</strong></li><li>Expert 3 may handle <strong>language-related tasks</strong></li><li>Expert 4 may become better at <strong>reasoning</strong></li></ul><p>In practice, MoE experts are not always explicitly assigned fixed topics. Their specialization often emerges during training based on which inputs the router sends to them.</p><h3><strong>2. Training the Gating Network</strong></h3><blockquote><strong><em>“Which expert should handle this input?”</em></strong></blockquote><p>That’s the job of the <strong>gating network</strong>, also called the <strong>router</strong>.</p><p>For every input token, the router calculates a score for each expert:</p><h4>G(x) = [ G1​(x) , G2​(x) , … , Gn​(x) ]</h4><p>These scores indicate how suitable each expert is for processing the input.</p><blockquote>Input: “Write a Python program to sort a list”</blockquote><blockquote>Expert 1 → 0.05<br>Expert 2 → 0.10<br>Expert 3 → 0.75 ← selected<br>Expert 4 → 0.10</blockquote><p>The router may select the <strong>top 1 or top 2 experts</strong> rather than sending the input to everyone. Over time, the router learns which experts are more useful for different types of inputs.</p><h3><strong>3. Joint Training</strong></h3><p>Eventually, the <strong>experts and the router need to work together</strong>.</p><p>If the final answer is poor(high error) , the training process adjusts the model so that:</p><ul><li>the experts become better at solving the problems, and</li><li>the router becomes better at selecting the appropriate experts.</li></ul><p>This process is repeated many times.</p><h3><strong>Inference Phase in Mixture of Experts (MoE)</strong></h3><p>During inference, the model receives a new input and needs to produce an answer. The key advantage of MoE is that it does not activate all the experts. Instead, it intelligently selects only the experts that are most relevant to the input. If multiple experts are selected, their outputs are combined to produce the final result. The router’s scores can be used as weights, meaning an expert with a higher routing score has more influence on the final output.</p><p>This is what makes MoE computationally efficient: <strong>the model can have many experts and a large number of parameters, while only a small portion is activated for each input.</strong></p><h3><strong>What Happens Inside an MoE Layer?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hphuWbff_zHYS5AOjX1AZg.png" /><figcaption>Inside an MOE layer</figcaption></figure><p>A <strong>Mixture of Experts (MoE) layer</strong> works by replacing some of the traditional dense <strong>Feed-Forward Network (FFN)</strong> layers in a Transformer with multiple smaller expert networks. Instead of sending every token through the same FFN, the MoE layer uses a <strong>router</strong> to decide which expert or experts are best suited to process each token.</p><h3>1. The Token Enters the MoE Layer</h3><p>When a token reaches an MoE layer, it is first passed to the router (gating network). The router examines the token’s representation and calculates a score for each available expert.</p><p>For example: <strong><em>Token: “Python”</em></strong></p><p>Expert 1 → 0.05<br>Expert 2 → 0.10<br>Expert 3 → 0.75 ← (selected)<br>Expert 4 → 0.10</p><p>The scores indicate how relevant each expert is for that particular token.</p><h3>2. The Router Selects the Top Experts</h3><p>The router does not activate every expert. Instead, it selects the <strong>Top-K experts</strong> with the highest scores.</p><p>For example, with <strong>Top-2 routing</strong>, the two highest-scoring experts are selected:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*Vuvniur79HUJbgs7lbYofA.png" /></figure><p>This selective activation is what allows MoE models to have a large number of total parameters without using all of them for every token.</p><h3>3. The Selected Experts Process the Token</h3><p>Each selected expert processes the token independently. Different experts may gradually learn to respond better to different patterns or types of information.</p><p>Importantly, the model does not necessarily tell an expert:</p><blockquote><em>“You are the coding expert.”</em></blockquote><p>Instead, specialization often emerges naturally during training based on the data and routing decisions.</p><h3>4. The Outputs Are Combined</h3><p>After processing the token, the outputs from the selected experts are combined according to their routing weights.</p><p>Expert 3 → 75% contribution<br>Expert 2 → 25% contribution<br>These experts give the Final MoE Output.</p><h3>How Does the Router Decide?</h3><h3><strong>Top-K softmax routing</strong></h3><p>In simple terms, the router calculates scores using the token representation and then converts them into probabilities. It selects only the highest-scoring experts.</p><p>A simplified representation is:</p><h4><strong>G(x) = Softmax ( TopK ( Wg​x ) )</strong></h4><p>Here:</p><ul><li><strong>x</strong> = input token representation</li><li><strong>Wg​</strong> = router’s learned weights</li><li><strong>Top-K</strong> = number of experts selected</li><li><strong>Softmax</strong> = converts scores into normalized probabilities</li></ul><p>Some implementations also add noise during training to encourage better distribution of tokens across experts and prevent the same experts from being selected all the time.</p><h3>Top-K Routing</h3><p>Different MoE architectures can use different routing strategies.</p><p>For example, Mixtral uses Top-2 routing, where each token can be processed by two experts. More recent MoE designs explore approaches such as finer-grained experts, shared experts, and sigmoid-based routing to improve specialization and reduce the tendency for the router to always favor a small number of experts.</p><h3>The 2026 MoE Landscape</h3><p>By 2025–2026, MoE had moved from being primarily a research technique to becoming a major architecture for large-scale open models. Models such as DeepSeek-V3, DeepSeek-V4, Qwen3, Llama 4, Mistral Large 3, and Kimi K2 demonstrate different ways of scaling model capacity while keeping the number of active parameters much smaller than the total parameter count.</p><blockquote><strong>Mixtral → DeepSeek-V3 → Llama 4 → Qwen3 → Mistral Large 3 → Kimi K2 → DeepSeek-V4</strong></blockquote><p>This trend is not simply <strong>“more parameters.” But i</strong>t is —</p><blockquote><strong>More total capacity → fewer active parameters → finer-grained routing → shared experts → longer context → multimodal and agentic workloads.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VYQ-ff_aBVZhoyRPrKQzIA.png" /><figcaption>Where is MOE going?</figcaption></figure><h3><strong>Why Are Shared Experts Important?</strong></h3><p>In an MoE model, <strong>shared experts</strong> act as a common pathway that processes every token, while routed experts are selected dynamically based on the input. The shared experts provide a <strong>stable foundation of general knowledge and representations</strong> that every token can access. At the same time, the routed experts can specialize in particular patterns, tasks, languages, or reasoning styles.</p><p>This creates a balance between <strong>generalization and specialization</strong>:</p><ul><li><strong>Shared experts →</strong> provide common knowledge and consistent representations.</li><li><strong>Routed experts →</strong> specialize in specific types of inputs.</li><li><strong>Together →</strong> they reduce the risk of the model becoming too fragmented across different expert pathways.</li></ul><p>For example, architectures such as <strong>DeepSeek-V3 and Llama 4</strong> incorporate shared experts alongside routed experts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V1uYqo0HkZF87Sd_K88WeQ.png" /></figure><h3><strong>2026 Research Findings</strong></h3><h3>What Are Super Experts?</h3><p>In a Mixture-of-Experts model, we usually think of the experts as a large team of specialists. The router selects different experts depending on the input.</p><blockquote>But recent research has found something interesting: <strong>not all experts are equally important</strong>.</blockquote><p>A very small number of experts appear to play a disproportionately important role in the model. These experts have been called <strong>Super Experts</strong>. The ICLR 2026 study found that removing only <strong>3 out of 6,144 experts</strong> in Qwen3–30B-A3B caused the model to produce repetitive, uninformative responses and significantly degrade its performance, particularly on mathematical reasoning. Below I have attached a recent paper for reference. <a href="http://2026_iclr_paper/">https://proceedings.iclr.cc/paper_files/paper/2026/file/c01b90b643d0e877cff0f34cbe27410f-Paper-Conference.pdf</a></p><p>“Super” Experts are associated with rare but extremely large activation values in the expert’s down_proj output. These large signals can create unusually large activations in the model&#39;s hidden states and appear to be connected to the formation of attention sinks—patterns that help organize how attention is distributed through the Transformer.</p><p>In the Qwen3–30B-A3B example, researchers identified three Super Experts whose removal caused a dramatic degradation in model behavior.</p><h3>Why does this matter?</h3><p>This discovery is particularly important for <strong><em>model compression and pruning</em></strong><em>.</em></p><p>At first glance, MoE seems like a natural candidate for pruning: if a model has thousands of experts, perhaps we can simply remove the experts that appear less important.</p><p>But “Super Experts” show why this can be dangerous.</p><p>Removing the wrong few experts can cause a much larger performance loss than their tiny number would suggest. This means that future MoE compression techniques need to understand which experts are structurally important, rather than simply removing experts based on how frequently they are activated.</p><h3>Super Experts are a tiny number of experts inside an MoE model that have an unusually large influence on the model’s internal behavior. They may be few in number, but removing them can severely damage the model’s performance.</h3><h3>Why Do Super Experts Matter for Deployment?</h3><p>Removing a few critical experts can severely affect model quality, so <strong>pruning and compression</strong> must be selective. Understanding these experts can help design <strong>smarter compression, quantization, and deployment strategies</strong> that reduce cost without sacrificing performance. Thus, compression strategies that are built around Super Expert preservation are now an active research area.</p><h3>What Is MoE Entanglement?</h3><p>MoE entanglement means that the experts selected in one MoE layer can influence which experts are selected in the next layers. In other words, MoE layers don’t work independently. The output produced by one layer becomes part of the input that guides routing in the following layer.</p><p>Let us think of it like a <strong>chain of decisions</strong>:</p><p><strong><em>Layer 1 selects Expert A → its output influences Layer 2 → Layer 2 selects Expert C → its output influences Layer 3 → and so on..</em></strong></p><h3>Inference, deployment, and systems</h3><h4><strong>Why can MoE be efficient at inference?</strong></h4><p>MoE models separate total model capacity from active computation. A model can contain hundreds of billions of parameters while activating only a fraction of them for each token. For example, an MoE model with 671B total parameters may activate around 37B parameters per token. This allows the model to have the capacity of a much larger network without paying the full computational cost of using every parameter for every token. However, MoE does not eliminate the cost of storing the inactive parameters. Real-world inference efficiency also depends on routing overhead, memory bandwidth, communication between GPUs, and hardware configuration.</p><h4><strong>Why Is MoE Still Hard to Deploy?</strong></h4><p>Although MoE reduces the amount of computation per token, it does not reduce the total size of the model. All the experts still need to be stored somewhere, even if only a few are active at a time.</p><p>For example, imagine an MoE model has 100 experts, but the router activates only 2 experts for each token. The other 98 experts are not computing at that moment — but their parameters still have to be stored in GPU memory, CPU memory, or distributed across multiple machines.</p><p>This creates several deployment challenges:</p><ul><li><strong>VRAM:</strong> Large numbers of experts require a lot of memory.</li><li><strong>Bandwidth:</strong> Moving expert weights and activations between GPUs can become expensive.</li><li><strong>Routing:</strong> The system must quickly send each token to the correct expert.</li><li><strong>GPU communication:</strong> Different experts may be located on different GPUs, requiring fast communication.</li><li><strong>Load balancing:</strong> Some experts may receive many more tokens than others, creating bottlenecks.</li><li><strong>Quantization:</strong> Techniques such as FP8 or INT4 can reduce memory requirements while maintaining reasonable quality.</li><li><strong>Expert parallelism:</strong> Experts can be distributed across multiple GPUs so that the entire model does not have to fit on one device.</li></ul><p>Therefore, MoE makes computation sparse, but the model’s memory and infrastructure requirements can still be enormous, making it hard to deploy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95dc4595c721" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can Computers Solve Everything? Exploring the Limits of Computation]]></title>
            <link>https://ai.plainenglish.io/can-computers-solve-everything-exploring-the-limits-of-computation-8aca7d271496?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/8aca7d271496</guid>
            <category><![CDATA[automata]]></category>
            <category><![CDATA[computational-theory]]></category>
            <category><![CDATA[computer-science]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Sun, 29 Mar 2026 18:41:56 GMT</pubDate>
            <atom:updated>2026-04-01T01:04:22.409Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*YgQB5UKsoJ3fa_13WHqgVQ.jpeg" /><figcaption>Theory of Computation</figcaption></figure><blockquote>The real power of computer science lies not in coding, but in knowing what can be computed.</blockquote><p>Many students initially believe that computer science is primarily about coding. However, this perception begins to change when they encounter the Theory of Computation (TOC). A common question that arises is: <em>Why do we need to study this?</em> Compared to subjects like algorithms, data structures, machine learning, and neural networks, TOC may appear less practical at first glance. In reality, computer science extends far beyond writing, compiling, and debugging code. At its core, it is about <strong>problem solving</strong>, and this is where the Theory of Computation plays a fundamental role. Before building applications, it is essential to understand the theoretical principles that govern them. Behind every line of code lies a foundation of logic, proofs, and mathematical reasoning.</p><p>Just like there are different fields of computer science, there are many different theories in this discipline of theory of computation. Here are four theories that I would love to talk about — Language Theory, Automata Theory, Computability Theory, Complexity Theory. Together, these fields seek to answer some of the most fundamental questions in computer science: <em>What is computation? What problems can be solved? And what are the inherent limitations of computation?</em></p><blockquote><strong>Can we actually solve every problem in this world through computing?</strong></blockquote><p>There are problems that computers can never solve no matter how much time we spent developing an algorithm so we study this theory of computation to understand what computers can and cannot do. For many problems, we can design algorithms and even identify the most efficient possible solution, proving that no other algorithm can perform better. However, there are also problems for which no algorithm exists at all, and many others where we still do not know what the optimal solution looks like. Knowing the fundamental capabilities and limits of computers will help us solve problems more efficiently and we can develop a common vocabulary for which we can use to generate new knowledge. Before diving into algorithms and their applications, it is essential to explore the theoretical foundations that define computation itself.</p><p><strong>Language Theory:</strong> This is all about encoding problems as symbols and representations at a level where computers can understand what’s going on and do computations on them. Just to clarify, languages that we are talking about are computational problems not spoken languages.</p><p><strong>Automata Theory:</strong> Automata theory studies abstract machines and the kinds of problems they can solve. One of the most important concepts in this area is the <strong>Turing Machine</strong>, introduced by Alan Turing. It serves as a fundamental model of computation, helping us understand how any computational process can be carried out. Automata theory is not just about theoretical machines like tapes and heads — it provides deep insights into how computation itself works.</p><p><strong>Computability Theory:</strong> Computability theory asks a fundamental question: <em>Can every problem be solved by a computer program?</em> The answer is <strong>no</strong>. This field is concerned with identifying which problems are computable and which are not. Formally, a function <em>f</em> is said to be computable if there exists a Turing machine that can produce the output <em>f(x)</em> for any given input <em>x</em>. This theory helps define the limits of computation.</p><p><strong>Complexity Theory: </strong>While computability tells us <em>whether</em> a problem can be solved, complexity theory tells us <em>how efficiently</em> it can be solved. It studies the resources required — such as time and space — to compute a function. For example, a problem might require constant time, logarithmic time, polynomial time, or exponential time.</p><p>One of the most famous open questions in this field is the <strong>P vs NP problem</strong>.</p><ul><li><strong>P</strong> represents problems that can be solved efficiently (in polynomial time).</li><li><strong>NP</strong> represents problems whose solutions can be verified efficiently.</li></ul><p>The key question is: <em>Are these two classes the same?</em> In other words, if a solution can be verified quickly, can it also be found quickly?</p><blockquote>Lets see an example.<strong><em> </em>What is a 3 colorable problem?</strong></blockquote><blockquote>If someone provides a coloring, it is easy to verify whether it is correct — this can be done efficiently. However, finding such a coloring from scratch is computationally difficult, and no efficient general solution is known.</blockquote><p>The Theory of Computation may seem abstract and distant from practical coding at first, but it forms the very foundation of computer science. It challenges us to think beyond writing programs and pushes us to understand the deeper questions: <em>What can be computed? How efficiently can it be done? And what lies beyond the limits of computation?</em></p><p>By exploring areas such as language theory, automata, computability, and complexity, we gain a clearer picture of both the power and the boundaries of computers. These concepts not only shape how we design algorithms but also influence the way we approach problem solving as a whole.</p><p>As we move forward into more applied domains like algorithms, machine learning, and software development, the insights from the Theory of Computation continue to guide us. It reminds us that every practical solution is built on a strong theoretical foundation. Understanding this foundation ultimately makes us better thinkers, better problem solvers, and better computer scientists.</p><p><em>Thank you, dear readers, for staying till the end! I hope this blog gave you a new perspective. Stay tuned for more exciting concepts and theories.</em></p><p>#theories #computation #computerscience</p><h3>A message from our Founder</h3><p>Hey, <a href="https://linkedin.com/in/sunilsandhu">Sunil</a> here. I wanted to take a moment to thank you for reading until the end and for being a part of this community. Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don’t receive any funding, we do this to support the community.</p><p>If you want to show some love, please take a moment to follow me on <a href="https://linkedin.com/in/sunilsandhu">LinkedIn</a>, <a href="https://tiktok.com/@messyfounder">TikTok</a>, <a href="https://instagram.com/sunilsandhu">Instagram</a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/">weekly newsletter</a>. And before you go, don’t forget to clap and follow the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8aca7d271496" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/can-computers-solve-everything-exploring-the-limits-of-computation-8aca7d271496">Can Computers Solve Everything? Exploring the Limits of Computation</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Google’s Nested Learning Framework Could Redefine AI Architecture.]]></title>
            <link>https://tejalrk2000.medium.com/why-googles-nested-learning-framework-could-redefine-ai-architecture-50b3f365923f?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/50b3f365923f</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[deeplearing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nested-learning]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Fri, 28 Nov 2025 20:04:00 GMT</pubDate>
            <atom:updated>2025-11-28T20:04:00.268Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Hi folks! Today, I would like to throw light on the Google’s new model that I read about recently. It is something that is new to me and I would like to share with my fellow learners too. In the growing world of AI, our focus is scaling up from one AI model to another day-by-day to explore how we can use it for our study. Today is the age of Agentic AI where each one is trying to utilize AI models to make their work easier and faster. As a AIML enthusiast, constantly learning unlearning and relearning in this domain I find myself inquisitive about the model architecture, its parameters and how its improving everyday with lots of features. I use many models in my day-to-day life for teaching, understanding of advanced concepts , achieving deadlines, framing study materials, etc. Have you ever given a thought on how these massive models work behind the scenes? How do they handle trillions of data points? Do they keep learning constantly or do they stop once training is done? When exactly does the “learning” part end, and the “inferencing” part begin ? I will answer these questions through the paper that I read recently. So, I would like to share about the Google’s HOPE model that learns during inference.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MSL6TQNcTI4vK86lq4H-Ng.avif" /><figcaption>Google’s Nested Learning</figcaption></figure><p>Google has recently introduced a new AI architecture called HOPE, designed to address a major challenge in artificial intelligence known as “catastrophic forgetting” where AI models lose previously learned information when updated with new data. HOPE represents an advanced step in continual learning, enabling AI models to retain long-term memory while continuously learning new information without the need for complete retraining. This architecture mimics human brain behavior through its self-modifying system and nested learning framework. This model unlocks true long-term memory in AI and pushes us closer to real continual learning.</p><h4><strong>Catastrophic Forgetting:</strong></h4><p>When the current AI models learn new things, they forget older ones. Humans don’t do that and now, Google Research is answering this. Google’s new method, called Nested Learning, helps fix AI’s biggest memory issue — catastrophic forgetting.</p><h4><strong>Nested Learning:</strong></h4><p>A completely new paradigm that treats a model as a stack of interconnected optimization processes, each running at different speeds — exactly like how the human brain manages information at different frequencies.</p><p>A recent paper, <strong>“Nested Learning: The Illusion of Deep Learning Architectures,”</strong> revisits how AI systems actually learn. It highlights a key idea from neuropsychology: <strong><em>learning and memorization are not the same</em></strong><em> </em>— even though we often treat them as identical in machine learning.</p><p><strong>According to the paper:</strong></p><ul><li><strong>Memorization</strong> is simply a change in the model triggered by an input.<br> Think of it like storing a fact in your brain.</li><li><strong>Learning</strong>, on the other hand, is the <em>process</em> of forming useful and effective memories that help in future tasks.</li></ul><p>The authors argue that human-like learning depends heavily on <strong>associative memory</strong> — our ability to connect different events or experiences. AI models often struggle with this, especially when new information overwrites old knowledge (catastrophic forgetting).</p><p>The proposed <strong>Nested Learning</strong> view treats deep learning models as layers of different “memory speeds,” similar to how the human brain forms both quick and long-term memories. By organizing the model as a set of nested optimization processes, it helps preserve old knowledge while integrating new information more smoothly.</p><h4><strong>Continuum Memory in HOPE :</strong></h4><p>The HOPE architecture introduces a powerful idea called continuum memory in which a model should not have just two types of memory (short-term and long-term), but a <em>continuous range</em> of memory processes operating at different speeds. So in short and simple words, Continuum memory means the model can gradually update different parts of itself depending on how important or stable the new information is. Some parts will update quickly during inference, others update slowly over time, but the core knowledge changes the least.</p><p>Here’s the idea in simple terms:</p><ol><li>The model is split into different “learning layers”</li><li>Each layer updates at its own speed.</li><li>Some parts learn quickly, others learn slowly</li><li>Together, they form a memory system similar to how the human brain works</li></ol><p>Paper Link: <a href="https://openreview.net/pdf?id=nbMeRvNb7A">https://openreview.net/pdf?id=nbMeRvNb7A</a></p><p><em>Can we meet an AI with unlimited memory and will Nested Learning help in this direction? Let me know your thoughts below.</em></p><p>Stay Updated! Happy Learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50b3f365923f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Noise to Image: The Math Behind Generative Adversarial Networks]]></title>
            <link>https://tejalrk2000.medium.com/from-noise-to-image-the-math-behind-generative-adversarial-networks-eee93adfc317?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/eee93adfc317</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[adversarial-training]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Wed, 30 Jul 2025 18:08:08 GMT</pubDate>
            <atom:updated>2025-07-30T18:08:08.855Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZIHUn1EvLv938RB08xsKAw.webp" /><figcaption>Math behind GANs</figcaption></figure><blockquote><em>Hi folks!</em><br>I’m back with a fascinating concept that’s been gaining a lot of attention across various domains — <strong>Generative Adversarial Networks (GANs)</strong>. These powerful models are not only transforming how machines generate data but are also reshaping the way we approach research problems. Recently, I had the opportunity to explore one of the GAN variants in depth, and I was amazed , not just by how it works, but by its potential to push the boundaries of creativity and data generation in AI.</blockquote><p>Before diving into GANs, it’s important to understand the shift they brought to the machine learning landscape.<br>Before GANs came into picture i.e prior 2014, most machine learning models were <strong><em>discriminative — </em></strong>they focused on learning the boundary between different classes and were widely used for classification and regression tasks. These models excel at answering questions like <em>“What is this?”</em> but often fall short when asked <em>“Can you create something like this?”</em><br>That’s where<strong> <em>generative</em></strong> models come into picture. Unlike discriminative models, generative models learn the underlying distribution of data, enabling them to generate entirely new, realistic-looking samples.<br>The introduction of GANs by Ian Goodfellow in June 2014, marked the true beginning of the creative era in deep learning.</p><p>GANs are useful when we want to build a ML model but we have limited amount of data to work with. Therefore, GANs have the ability to generate synthetic realistic data based on prior data. One such fascinating use of GANs is that, they have been used to create hyper-realistic images of people that don’t actually exist !! Beyond this, GANs have been applied in generative design that enable you create cool 3d furniture for your home. Companies like Adobe are using GANs to build next-generation Photoshop tools, Google uses them for text generation, IBM uses them for data augmentation, and platforms like Snapchat have been using GANs to create image filters for quite a while now.</p><p>In this blog, I’ll walk through the math behind how GANs work, because understanding the <em>why</em> behind GANs is quite important. A solid grasp of the underlying mathematics helps break free from the black-box view of deep learning and opens up opportunities to improve or even design new generative models. I also highly recommend reading the original 2014 paper by Ian Goodfellow that introduced GANs to the world <a href="https://arxiv.org/abs/1406.2661">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*Z79toOegQiSFKlN_ntrP3A.png" /><figcaption>Adversarial Training of GANs</figcaption></figure><p>A Generative Adversarial Network (GAN) is a deep learning framework consisting of two competing neural networks: the Generator and the Discriminator, trained in a game-theoretic manner. Generator uses the noise from a normal or Gaussian distribution as input and generates samples. The second model which is known as discriminator receives samples from the generator and the training data. According to game theory, the generator is trained to produce an image that looks like a real image, whereas the discriminator is learning to discriminate perfectly from generated data to actual data.</p><p>The training objective of a Generative Adversarial Network (GAN) is formulated as a two-player minimax game, as shown in below equation. The discriminator D aims to maximize the probability of correctly distinguishing real samples<strong> </strong><em>x</em> from fake samples generated by the generator <em>G(z)</em>, where <em>z</em> is sampled from a latent noise distribution. Conversely, the generator<strong><em> </em></strong><em>G</em> tries to minimize this objective by producing realistic samples that can fool the discriminator into classifying them as real. This adversarial optimization ensures that <em>G</em> learns to approximate the true data distribution while <em>D</em> improves its ability to detect generated samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*xWVgwKzevUKRjzw6o1ssWg.png" /><figcaption>Value function</figcaption></figure><p>Looking at this equation, you’ll notice that it closely resembles the <strong>binary cross-entropy loss function</strong>, which is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*KS-IctFC6Glnp0WKCDXgxw.png" /><figcaption>Binary cross entropy loss</figcaption></figure><ul><li><strong>Y </strong>represents the true label for the i-th sample</li><li><strong>Y-hat </strong>represents the prediction from the discriminator</li><li><strong>n</strong> is the number of samples in the dataset or batch</li></ul><p>The value function is derived from the <strong>binary cross-entropy</strong> loss. The binary cross-entropy objective function provides a measure of how far we are from achieving our goal of correctly classifying real and fake samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*60y1jyJuP9Zej0wCVMNfgQ.png" /><figcaption>Derivation of BCE Loss</figcaption></figure><p>The left part of the BCE loss indicated in green only matters when the ground truth of the data sample is real or 1. Meanwhile only the right part of the loss function matters when the ground truth of the data sample is fake or 0. Looking at the image below when y = 1 , the loss function is close to 0 when the prediction is close to 1 and infinity when the prediction is close to 0 .When y = 0 , the loss function is close to 0 when the prediction is close to 0 and infinity when the prediction is close to 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*QRwAxMHD-MdZGAH4F-79FQ.png" /><figcaption>Prediction and Loss for BCE loss function</figcaption></figure><p>Adding both parts of the binary cross-entropy loss ; the case when the ground truth is 1 (real) and when it is 0 (fake) , gives us the value function used in Generative Adversarial Networks (GANs). Since this value function must be computed over many data points, we use expectations (denoted by E) to represent the average loss over the data distribution.</p><blockquote>In simple terms, the discriminator aims to maximize this loss function by correctly classifying real as real and fake as fake and the generator aims to minimize it by fooling the discriminator into thinking the fake data is real.</blockquote><ol><li><strong>Discriminator D Maximizes V(D,G): </strong>The goal is to distinguish between real and fake data. Makes <em>D(x)≈1</em> for real samples and <em>D(G(z))≈0</em> for fake samples.</li><li><strong>Generator G Minimizes V(D,G): </strong>The goal is to fool the discriminator. Wants <em>D(G(z))≈1</em> , so that generated samples look real.</li></ol><ul><li><strong>Note:</strong> G does not affect <em>D(x),</em> the term for real data.</li></ul><h3>Training Generative Adversarial Networks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*ggQraaFxkfsSJolUxO2Otw.png" /><figcaption>GAN training Algorithm</figcaption></figure><p>This image is straight from the original 2014 GAN paper, the one that started it all! It shows the core algorithm behind how GANs are trained. Don’t worry if it looks a bit math-heavy , I’m going to break it down in plain English so it’s easy to follow.<br> Take a moment to glance through it, and if you’re curious, I highly recommend checking out the full paper too — it’s a classic in AI research!</p><h3>How GANs are trained?</h3><h4>Training the Discriminator:</h4><ul><li>First, we generate some random noise and pass it through the <strong>generator</strong> to produce <strong>fake data</strong>.</li><li>We then take a batch of <strong>real data</strong> from the actual dataset.</li><li>Both the real and fake data are fed into the <strong>discriminator</strong>, which tries to tell them apart.</li><li>The <strong>discriminator’s </strong>weights are updated (using gradient ascent) so it gets better at distinguishing real from fake.</li></ul><h4>Training the Generator:</h4><ul><li>Next, we <strong>freeze the discriminator</strong> and focus on improving the generator.</li><li>New noise is fed to the generator to create fresh fake samples.</li><li>These are passed to the discriminator, but this time, we want the<strong> discriminator to get fooled</strong>!</li><li>The <strong>generator’s weights are updated</strong> (using gradient descent) to produce data that looks more realistic and <strong>can trick the discriminator</strong>.</li></ul><p>This process repeats over and over, making both the generator and discriminator better with each cycle.</p><h3>Architecture of Generator and Discriminator</h3><ul><li>The generator is a simple feedforward neural network with one hidden layer, a ReLU activation function, and a tanh activation function at the output layer to squash the outputs to values between -1 and 1, representing pixel intensity values of the images.</li><li>The discriminator is a simple feedforward neural network with just one hidden layer . The hidden layer used leaky RELU activation functions while the output layer had just one neuron with a sigmoid activation to squash the outputs to probabilities of being fake (0) or real (1).</li></ul><p>At the beginning of training, neither the discriminator nor the generator has any real idea of what they’re doing. The discriminator’s weights and biases are updated, while the generator’s parameters are frozen. Then, the generator’s weights and biases are updated while the discriminator’s parameters are frozen. This loop keeps repeating during adversarial training, and over time, the generator’s data distribution matches that of the training data, meanwhile the discriminator becomes a straight blue line that outputs 0.5 for all data samples. At this point, the generator has learned the data distribution of the training set and can generate data samples that look similar to the real ones.</p><p><em>Thanks for sticking around till the end! 🙌<br>GANs might seem tricky at first, but with a bit of patience, the logic behind their training and architecture becomes quite intuitive. Understanding how the generator and discriminator play this clever game helps unlock the magic behind AI-generated images and beyond. If you’re just getting started with GANs, don’t worry , take your time experimenting and exploring. The best way to learn is by building and tweaking things yourself.</em></p><p><em>Happy learning , and keep creating!</em></p><h4>References</h4><ol><li><a href="https://www.youtube.com/watch?v=Gib_kiXgnvA">The math of GANs Explained</a></li><li><a href="https://www.youtube.com/watch?v=RAa55G-oEuk">Understanding GAN</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eee93adfc317" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prompt to Prediction: The LLM Journey]]></title>
            <link>https://tejalrk2000.medium.com/prompt-to-prediction-the-llm-journey-da5cc88ac6d1?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/da5cc88ac6d1</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Sun, 18 May 2025 14:59:47 GMT</pubDate>
            <atom:updated>2025-05-18T15:05:09.881Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Over the past couple of months, I found myself drawn into the depths of Large Language Models (LLMs), driven by curiosity and a desire to truly understand how they work under the hood. Concepts like decoder-only architectures, KV caching, Quantization, MQA vs GQA, Mixture of Experts (MoE) , etc. initially felt complex but interesting at the same time. In this blog series, I aim to simplify and explain these topics — not just to document what I’ve learned, but to help others who might be on a similar journey. Whether you’re an ML enthusiast, researcher, or practitioner, I hope this series makes LLM inference a little less mysterious and a lot more accessible.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b7snTC7sVVJ6oevhmM_k6Q.jpeg" /></figure><p>Large Language Models are becoming a part of our daily lives helping us with basic writing, generating creative content to answering like a real person. By understanding how they work, we could use them in a better way. Also, understanding their potential and pitfalls will allow us to make informed decisions about how we interact with them.</p><p>Let me first give a quick summary of the commonly used terms in LLMs.</p><ol><li><strong>Model Parameters:</strong> These are the variables learned by the LLM during training. The number of parameters may vary depending on the size and complexity of the model. Say a small LLM also known as SLM which are smaller versions of their LLM counterparts, have less parameters ranging from few millions to few billions as compared to large LLM like OpenAI’s GPT 3 had over 175 billion parameters and the newer ones are going over trillions.</li><li><strong>Pre-Training:</strong> It is a technique used to train a LLM on a massive amount of general purpose text data. The goal of pretraining is to teach the LLM to understand the relationship between the words and how they are used in different contexts.</li><li><strong>Finetuning: </strong>It is a process of taking a pretrained LLM and specializing it for a specific task. By feeding the LLM with task-specific data we finetune its parameters for exceptional performance in our domain.</li><li><strong>Embeddings:</strong> Before an LLM can process text, it needs to convert it into a format which it can understand. We can say that embeddings are like secret maps made by powerful language models which turn words into numerical codes that reveal their hidden relationships and meanings. Embeddings are vectors stored in an index within a vector database. A sentence is broken down into individual units called as tokens. This “token” can be a smaller part of the word or a character or a larger one like a phrase. These tokens are then assigned numerical values called embeddings. Embeddings allow the LLMs to represent the meaning of each token in a high dimensional space.</li><li><strong>Retrieval Augmented Generation(RAG) :</strong> It is a technique used to improve the accuracy and coherence of LLM output. RAG works by retrieving relevant information from knowledge base and then using this information to generate text.</li><li><strong>LLMOps:</strong> LLMOps is the practice of managing and operating LLMs in production environments. This manages the entire lifecycle of LLM from development and deployment to monitoring and maintenance. This includes monitoring the performance of LLMs, retraining them on new data and ensuring that the are used in a safe and responsible manner.</li></ol><p>Hope this terms have given you a better understanding of LLMs and will provide you an efficient way to work with LLMs.</p><p>In this blog, I would focus on the working of Decoder only model. Large Language Models (LLMs) like GPT-3, LLaMA, and Gemini are changing the way we create and interact with text. Behind the scenes, these models use a special type of neural network called a <strong>decoder-only transformer</strong>.</p><h4><strong>Decoder-only transformers</strong></h4><p>Decoder-only transformers are remarkable architectures. A decoder-only transformer is designed for auto regressive generation — predicting the next tokens given the previous ones. The architecture consists of a <strong>stack of identical layers</strong>, each with two major components:</p><ol><li><strong>Masked Self-Attention</strong></li></ol><p>Self Attention is a mechanism that can take static embeddings as input in a sequence and can generate contextual embeddings dynamically in the same sequence.</p><blockquote><strong>But why Masked?</strong></blockquote><p>In decoder-only models, causal masking is applied to prevent a word from looking into the future — it only attends to past and current tokens. For example, predicting a word at position <em>t </em>it should only use information from positions<em> 0 to t-1 .</em></p><p>Follow this mechanism for each token —</p><ul><li>Compute queries (Q), keys (K), and values (V) from input embeddings.</li><li>Compute attention scores using scaled dot-product attention.</li><li>Apply a <strong>mask</strong> to disallow attending to future tokens.</li><li>Use these scores to produce a <strong>context vector</strong> (weighted sum of value vectors).</li><li>Pass through a linear projection to maintain dimensional consistency.</li></ul><p><strong>2</strong>. <strong>Multilayer Perceptron</strong></p><p>MLPs are essentially feed-forward neural networks that process each word’s embedding independently. They can be thought of as the “<em>knowledge banks</em>” of the transformer, incorporating general knowledge learned during training to further refine the embeddings.</p><blockquote><strong>Why MLPs?</strong></blockquote><ul><li>Self-attention helps relate tokens to each other .</li><li>MLPs help process and encode knowledge about a token’s identity and its role, based on what the attention layer has aggregated.</li><li>MLPs operate by projecting each word’s embedding into a higher-dimensional space, applying a non-linear activation function (like ReLU), and then projecting it back down to the original embedding dimension. This process allows the MLP to extract complex features from the embeddings and incorporate information that’s not explicitly captured by the attention mechanism.</li></ul><h4><strong>Positional Encoding : Capturing Word Order</strong></h4><p>While self-attention considers relationships between words, it doesn’t inherently capture their order within the sequence. To address this, decoder-only transformers utilize positional encoding. This involves adding a vector representing the word’s position in the sentence to its embedding. These positional embeddings can be learned during training or generated using predefined functions, such as sine and cosine waves. By incorporating positional information, the model can understand the significance of word order in sentences.</p><h4><strong>Conclusion</strong></h4><p>Today, we gave a quick start to our LLM journey with some commonly used terms in LLMs and understood decoder only architecture. This architecture is constantly evolving, undergoing modifications to further enhance its capabilities. As we continue to push the boundaries of language models, understanding the principles behind decoder-only transformers remains crucial for unlocking their full potential. So, in the next part of this series, we’ll explore the remaining concepts with some sort of implementation.</p><p>Happy Learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da5cc88ac6d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLMs vs Transformers]]></title>
            <link>https://tejalrk2000.medium.com/llms-vs-transformers-0571e00d3b06?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/0571e00d3b06</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Sun, 03 Nov 2024 16:57:32 GMT</pubDate>
            <atom:updated>2024-11-03T16:57:32.953Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*UEJtKKazIkkLLP_BPfiocA.jpeg" /></figure><blockquote>Hello Learners! I recently read one paper titled “Attention is All You Need” for my ongoing thesis so thought of sharing my learning here. Lets throw some light on LLMs and Transformers.</blockquote><p>LLMs and transformers are both open-source AI models built on deep learning.</p><p>Transformers are the BUILDING BLOCKS of modern-day LLM.</p><p>They were introduced in 2017 paper “Attention is All You Need” to solve problems of sequential processing in RNNs and LSTMs. <br>They outperformed RNNs in Accuracy and Performance by using “Parallel Computing” and “Self-Attention”. <br>Transformer architecture consists primarily of 2 modules (and Attention Mechanism within these modules)<br><strong>Encoder</strong>:</p><ul><li>Think of the encoder as a reader. It takes the input text (like a sentence) and breaks it down to understand its meaning.</li><li>It does this in layers, where each layer helps the model pay attention to different parts of the sentence. This means it can focus on important words and see how they relate to each other, no matter where they are in the sentence.</li><li><strong>Decoder</strong>:</li><li>The decoder is like a storyteller. It takes what the encoder has understood and uses that information to generate a new sentence.</li><li>For example, in machine translation, encoder would create a representation of source language input, and decoder would generate translation in target language.</li></ul><p>Attention Mechanism -</p><p>The attention mechanism is a key part of both the encoder and decoder. It helps the model decide which words to focus on at any given time, so it can understand context and relationships better. Selecting a small amount of important info from large data and focusing on these important pieces while ignoring the majority of unimportant info.</p><p>Self-Attention — the CORE idea behind transformers <br>A variant of attention mechanism which reduces reliance on external info and excels at capturing internal correlations within data or features. It has 3 steps:<br>1. query vector — word being attended to<br>2. key vectors — all the words in sentence. <br>3. value vectors — store info associated with each word.<br>Attention weights are computed by taking dot product between query and key vectors, followed by softmax operation to obtain distribution over words.<br>Steps for building transformer model:</p><ol><li>Input encoding<br>Convert raw text into numerical representations called embeddings (high-dimensional vectors, often pre-trained using models like Word2Vec or GloVe to encapsulate semantics of words)</li><li>Attention mechanism<br>Calculate ‘attention scores’ for different words, indicating their relevance in the sentence context.</li><li>Add positional encoding to input embeddings<br>Based on specific sine and cosine func, these encodings provide crucial word order info for model to understand syntax and semantic relationships.</li><li>Feed-forward neural network<br>Pass each word through a F-F NN within Transformer block. <br>Contrary to simple NN, it typically includes two linear transformations with a ReLU activation in between and processes the output of attention mechanism.<br>Feed-forward layers help model learn complex relationship between inputs &amp; outputs.</li><li>Finish and repeat<br>The one mentioned is “one” transformer block. <br>Stack multiple transformer blocks to enhance model’s learning capacity. With each layer, the model captures more complex relationships and patterns.</li></ol><p>Difference with LLMs?<br>- LLMs are broad models trained on large amounts of data to develop a general understanding of language. <br>Transformers are specific with narrower focus like translation, summarization, or ques-answering.<br>- LLMs can be built on top of transformers for deeper nuances.</p><p>Open-source transformers like BERT, GPT-3, Text-to-Text Transfer Transformer(T5) are already used in AI apps.</p><p>To learn LLMs, learn Transformers!<br>Happy Learning and Happy Diwali to my learners!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0571e00d3b06" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Redefining Small Language Models: Meet the Phi-3.5 Family]]></title>
            <link>https://tejalrk2000.medium.com/redefining-small-language-models-meet-the-phi-3-5-family-45f806fe4152?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/45f806fe4152</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[genai]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Sun, 01 Sep 2024 17:49:20 GMT</pubDate>
            <atom:updated>2024-09-01T17:49:20.070Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Microsoft Unveils Phi-3.5: Powerful AI Models Punch Above Their Weight</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*6z7BZZHm6rBodZFyjW7g5g.jpeg" /></figure><p>In the ever-evolving landscape of artificial intelligence, small language models (SLMs) are redefining what’s possible, balancing capability with efficiency. Microsoft’s Phi model collection has consistently pushed the boundaries of what SLMs can achieve, and with the introduction of the Phi-3.5 series, we are entering a new era of AI innovation.</p><p>This latest members of the Phi family include : Phi-3.5-mini, Phi-3.5-vision, and the groundbreaking Phi-3.5-MoE (Mixture-of-Experts).</p><ol><li><strong>Phi-3.5-MoE (Mixture-of-Experts)</strong></li></ol><p>The Phi-3.5-MoE model, with its Mixture-of-Experts architecture, is designed to handle complex tasks with high efficiency. It comprises 16 experts, each containing 3.8B parameters. This model has 42B parameters, trained on 5 Trillion Tokens on 512 H100s. The model is crushing it in reasoning tasks,beating out Llama 3.1 and even giving GPT-4o a run for its money. This MoE model outperforms a similarly sized dense model in terms of quality and performance. It supports over 20 languages. Like its Phi-3 counterparts, the MoE model employs a robust safety post-training strategy, using a mix of open-source and proprietary synthetic instruction and preference datasets. This post-training process combines Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), utilizing both human-labeled and synthetic datasets. Phi-3.5-MoE with only 6.6B active parameters achieves a similar level of language understanding and math as much larger models. Moreover, the model outperforms bigger models in reasoning capability.</p><p>2<strong>. Phi-3.5-mini</strong></p><p>Phi-3.5-mini enhances multi-lingual support with a 128K context length .This model has 4B parameters, trained on 3.4 Trillion Tokens on 512 H100s. This light weight model supports a massive 128K Token context. Phi-3.5-mini model has undergone further pre-training using multi-lingual synthetic and high-quality filtered data. This was followed by a series of post-training steps which included Supervised Fine-Tuning (SFT), Proximal Policy Optimization (PPO), and Direct Preference Optimization (DPO). These processes utilized a combination of human-labeled, synthetic, and translated datasets. The model used additional continual pre-training and post-training data leading to substantial gains on multi-lingual, multi-turn conversation quality, and reasoning capability. Phi-3.5-mini, with a 128K context length support, excels in tasks like summarizing long documents or meeting transcripts, long document-based QA, and information retrieval.</p><p>3.<strong> Phi-3.5-Vision</strong></p><p>Phi-3.5-vision improves multi-frame image understanding and reasoning, boosting performance on single-image benchmarks and developed based on invaluable customer feedback. This model has 4.2B parameters, trained on 500B vision + text tokens on 256 A100s. This model empowers detailed image comparison, multi-image summarization/storytelling, and video summarization, offering a wide array of applications across various scenarios. This model even outperforms OpenAIs GPT-4o on some benchmarks.</p><p>The best part is that they are all open source and available on hugging face. Thus, the latest Phi-3.5 release will align well with goals of optimizing generative AI models for inferencing on OpenVINO CPU, providing improved performance, comprehensive operator support, and advanced features that enhance the capabilities of our AI applications. The Phi-3.5 model family provides cost-effective, high-capability options for the open-source community, pushing the boundaries of small language models and generative AI.</p><p>Thank you for taking the time to read my blog! I hope you found the insights on the Phi-3.5 models as exciting as I do. Stay tuned for more updates and deep dives into the latest in AI and technology. Follow me and connect on my <a href="https://www.linkedin.com/in/tejal-khade-a379381b0/">LinkedIn</a> handle.</p><p>#GenAI #opensource</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45f806fe4152" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Life of Data Scientist: Before and After ChatGPT]]></title>
            <link>https://tejalrk2000.medium.com/life-of-data-scientist-before-and-after-chatgpt-3411bb7fba86?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/3411bb7fba86</guid>
            <category><![CDATA[analytics-vidhya]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Tue, 08 Aug 2023 17:37:12 GMT</pubDate>
            <atom:updated>2023-08-08T18:26:48.576Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*_7_l4We5Zknm2RIC-d3HCA.jpeg" /></figure><p>The era before ChatGPT and the groundbreaking changes after its integration has transformed the journey of a data enthusiast. Let us understand the impact of ChatGPT from both ends!</p><p>As we all know, this model built by Open AI is a state-of-the-art, world-class language model. Language model uses various statistical and probabilistic techniques to determine the probability of a given sequence of words occurring in a sentence. Language models analyze context of text data to provide a base for their word predictions. It is the reason that machines can understand qualitative information.</p><p>Now, the recently held DataHackSummit2023 at Analytics Vidhya focused on various topics like “Building Scalable Machine Learning models”, “Mastering MLOps- The Art of Productionalizing ML Models”, “Unleashing LLM’s: Training, Finetuning and Evaluating”, “Applied Machine Learning with Generative AI”, “Solving realworld problems using Reinforcement Learning”, “AI for Business Leaders”, “NLP using Generative Models” and also an insightful talk on “Life of Data Scientist: Before and After ChatGPT”. I would like to ellaborate on this topic and share some valuable points that would benefit every data enthusiast.</p><p>The collective insights provide an invaluable examination of the evolution of data science in the era of advanced language model like ChatGPT.</p><ol><li>Before ChatGPT, crafting effective problem statement was very crucial job and it also demanded extensive domain knowledge. Whereas, today the effective findings with the help of AI assistant have become so easy and timesaving , thereby making the problem formulation effortless. It’s simply making long answer short with accurate summary but it will be helpful to those who have prior knowledge and not to newbies.</li><li>Before ChatGPT, coding, documentation and summary generation consumed more time and effort. Now, embracing GenAI, we accelerate these tasks, and increase our level of thinking using valuable resources. We can focus more on problem solving and invest more time in designing effective solutions. Every Data Scientist,Data Analyst and Programmer can be more productive with ChatGPT. ChatGPT can assist you with so many tasksright from generating code snippets using Python, R, C++ as well as machine learning training script for data scientists.</li><li>Before ChatGPT, coding and quering were an important part of hiring process. Now with ChatGPT, forward thinking HR’s would seek problem solving skills, algorithmic skills and the ability to ask insightful questions. Coding can be learned slowly with daily practice using the language model, but the essence of expertise lies in fundamental understanding.</li><li>Data Interpretation and Communication was a crucial task that required prior knowledge of complex analyses, considering business objectives before ChatGPT but now the AI models can generate insights and analysis, that help data scientists in interpreting and communicating the results effectively to stakeholders.</li></ol><p>AI language models like ChatGPT have the potential to augment and support the work of data scientists, but they are likely to replace them partially. Data scientists bring a unique combination of domain expertise, critical thinking, creativity, and ethical considerations, making them indispensable in data science. According to me, AI models like Chat GPT can be a valuable tool for data scientists, but they cannot replace the important role that data scientists play in various industries.</p><blockquote>I really feel Technology will always be an enabler, not a replacer!</blockquote><p>Hope you find this article insightful. Thankyou.</p><p>Keep Learning, Keep Growing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3411bb7fba86" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Web Scraping using BeautifulSoup]]></title>
            <link>https://tejalrk2000.medium.com/web-scraping-using-beautifulsoup-edb05d593216?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/edb05d593216</guid>
            <category><![CDATA[data-collection]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[web-scraping]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Mon, 17 Jul 2023 13:39:02 GMT</pubDate>
            <atom:updated>2023-07-23T06:06:39.952Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t64gpilIVCG76c8dc9qWfg.jpeg" /><figcaption>Web Scraping with BeautifulSoup</figcaption></figure><p>Websites contain huge amount of invaluable data may it be stock prices, customer reviews and feedbacks, product details, sales reports, company contacts, technical blogs, sports statistics and many more you name it. When you want to use this data, you will copy paste the information manually into a new document or you use it in whatever format the website uses. Different websites may have data that you cannot copy and paste. Web scraping can help you extract any kind of data that you want. Once you copy and paste some data, how will you convert or save it in a format of your choice? This is where Web Scraping comes into picture!!</p><p>When you extract web data with the help of web scraping , you would be able to save the data in a format such as CSV. Now you would be able to retrieve, analyze and use the data as you want. So web scraping simplifies the process of extracting data, speeds it up by automating it and creates easy access to the scrapped data by providing it in a CSV format. In simple terms, web scraping saves the trouble of manually downloading or copying any data and automates the whole process.</p><p><strong>What is Web Scraping?</strong></p><p>Data is the new differentiator. Initially, Data scraping refers to a technique in which a computer program extracts data from output generated from another program as data is said to be the core of market research and business strategies. Thus its application can be seen in Web Scraping. Web Scraping is a process of automating the extraction of data in an efficient manner. With the help of web scraping, you can extract data from any website, no matter how large is the data, on your computer. This information is collected and then exported into a format that is more useful for the user may it be a spreadsheet or an API.</p><p>Whether you want to commence with a new project or churn out a new strategy for an existing business, you need to access and analyze a vast amount of data. This is where web scraping is useful. Although web scraping can be done manually, in most cases, automated tools are preferred when scraping web data as they can be less costly and work at a faster rate.</p><p><strong>Web Scraping Applications</strong></p><ol><li><strong><em>Data Science — </em></strong>Real-Time Analytics, Predictive Analysis, Natural Language Processing, Machine Learning Training Models.</li><li><strong><em>Retail and Manufacturing — </em></strong>Competitor Price Monitoring, Fetching Images and Product Descriptions, Overview of feedbacks and reviews, Monitoring Consumer Sentiments.</li><li><strong><em>Financial Research — </em></strong>Aggregated News Articles, Market Data Aggregation, Expenditure Statistics, Extracting Financial Statement.</li><li><strong><em>Product, Marketing and Sales — </em></strong>Data-driven Marketing, Content Marketing, Lead Generation, Competitive Analysis, SEO Monitoring, Reputation Monitoring.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ItNzuZxGSjrEG558" /></figure><p>Now let us learn how to extract data from websites. Data is of two types i.e private and public data. When you extract data it can be from government company or private owned company. <em>Web Scraping is illegal if you try to scrap the data from private companies as their data is secured and inaccessible publicly.</em> Thus, it is important to understand whether the data you are trying to fetch is public or private. Unlike private data, public data can be accessed by everyone from different websites such as NGO’s , government sites, Ecommerce websites, etc.</p><blockquote>There are many libraries used for Web Scraping. Most widely used libraries are BeautifulSoup and Selenium.</blockquote><p><strong>BeautifulSoup </strong>is a library for parsing HTML and XML documents. BeautifulSoup is more limited and is mainly used for parsing and extracting data. BeautifulSoup is more user-friendly than Selenium. It has a simple API and is easy to understand for beginners.</p><p><strong>Selenium</strong> has more functionality since it can automate browser actions such as clicking buttons, filling out forms and navigating between pages. In simple words, it is used for automating the task. Selenium, on the other hand, can be more complex to set up and use as it requires knowledge of programming concepts such as web drivers and browser automation.</p><p>Although, Selenium features more complex capabilities whereas BeautifulSoup is relatively simplistic, the choice between using these two scraping technologies will likely reflect the scope of your project.</p><p>I have used flipkart website for scraping data.</p><p><strong>Importing the libraries:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Egv0gWE8kawqScNBJbIrUQ.png" /><figcaption>importing library</figcaption></figure><p><strong>Loading data from flipkart website:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*GXb4xKGV14YfmRBZHTKoww.png" /><figcaption>getting data</figcaption></figure><p>The data generated is huge and unstructured. Thus we can derive any information from it. Now our aim is to get data of containers by excluding other objects and its details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DW53VReXDmvDTwL0mPzoEQ.png" /><figcaption>required data containers</figcaption></figure><p>To scrap data of these containers, we have to get rid of unnecessary details from the website data. Now we will extract the code for these containers only. Follow these steps to scrap data. Right click on the website → inspect → select container → copy classname.</p><p>Every container is defined by its classname. There are 24 products on one page with same class name. We will extract required data from <em>webdata(contains entire source code of the webpage) </em>by creating <em>soup</em> <em>object</em>. <em>webdata</em> is present in <strong>byte format</strong> so we converted into<strong> soup</strong> type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RAMK66sPxkZ87JbE8R91pg.png" /><figcaption>datatype conversion</figcaption></figure><p>You can check the type of webdata by using <em>type(soup(webdata)). </em>You can see that datatype of webdata is BeautifulSoup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*vALLtsDxAgqJXD3_BxygGA.png" /><figcaption>checking datatype</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*y31vY7i-zUehADlgnEvLzg.png" /><figcaption>directory of soup</figcaption></figure><p>Mention classname using findAll() function of soup directory. Each page contains 24 laptops with 24 separate containers. Type the following command to getdata from 1st page on the website.</p><p><em>soupdata.findAll(‘div’,{‘class’:’_2kHMtA’})</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*ssRZAdEAd8NYNkzBKZcuow.png" /><figcaption>getdata using classname</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*2W-z6rt09xtRo-C6JZHW4g.png" /><figcaption>type &amp; length of soupdata</figcaption></figure><p>In BeautifulSoup library , type of soup data is ResultSet containing 24 results.</p><p><strong>Checking the directory of ResultSet:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*BcjC2U-EOxtmDdynNVOtWQ.png" /><figcaption>directory of ResultSet</figcaption></figure><p><strong>Getting data of 1st Container:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*KSJJpA6pxtLmRbJYcKzBKw.png" /><figcaption>data of first container</figcaption></figure><p>Now let us collect data of 24 laptops(records in one page) and extract its important features into columns.</p><p><strong>Features</strong> → Product, Current_Price, MRP, Stars, Rating, Reviews, Processor, Ram_Size,Image_URL.</p><p>We run a for loop to access all the names of 24 laptops which are displayed on first page. Here, we see that all names are in ResultSet form of soup type. Thus, it cannot support text().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61shsa-v4_jjTuRAVDeomA.png" /><figcaption>24 products list</figcaption></figure><p>To extract text(names of laptops) we have to convert given list into tag. bs4 Tags support text. Thus we can get only names as our output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*oHPME5ooj8B5YM6U1Q-DyA.png" /><figcaption>datatype of product</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*phS4pWfG2OSsMcYXMLY52g.png" /><figcaption>Getting Names of Laptops</figcaption></figure><p>To check type of this data apply type() function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*zR6oUQg1PPFw3moORKkMng.png" /><figcaption>datatype of output</figcaption></figure><p>To get the name of laptops only we apply split() function. Now data is split into list separated by commas. To exclude the specifications, we apply indexing to split() function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WiOCTkfsdi6bsKO1T3fWkg.png" /><figcaption>split() function for lists</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iQRueXMDzAK2rsHicnQzyw.png" /><figcaption>stars data</figcaption></figure><p>Here, some products do not have stars and thus it would return NoneType error for some products. Thus we have added an exception handling for such cases. Now we got the details of stars on the product.</p><p><strong>Rating and Reviews</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*mA9-76dENURk46Xiu2lIZg.png" /><figcaption>rating and review</figcaption></figure><p>Here we use regular expression to scrap out the ratings and reviews of the products. Products with no ratings and reviews will be display as 0 0 .</p><p><strong>Current Price and MRP details:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*xphSLSd2EJyVRLPcztyo2Q.png" /><figcaption>Current Price and MRP</figcaption></figure><p><strong>Image URL</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*rhuQHhRFtSnF8YWsgig38w.png" /><figcaption>ImageUrl</figcaption></figure><p><strong>Write data to file:</strong></p><p>When we want to write data to file such as .csv , .xls we use file handling functions. You can see here, 3 different file functions are used to write data to a .csv file. Once you run it, Laptops_info.csv is created in your directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fIXxDWJde5cillNKyTv10w.png" /><figcaption>write data to file</figcaption></figure><p>Now open your file and you can see all the columns and rows are extracted into .csv file . I have taken 9 columns from website and now the data is well segregated into an excel file that can be used for analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Ggaqi2H9QXvhMuDX5g97w.png" /><figcaption>EXCEL DATA</figcaption></figure><p>In this way we scraped out the data using BeautifulSoup Library. Thus, Web Scraping is an efficient way to grab a great deal of information for analysis, processing, or presentation. Web Scraping enhances decision making since it offers insighful knowledge and precise data. Many bussiness require online information across the industry that benefits the web scraping market, creating new jobs and growth potential. Lastly, I would add — be cautious while collecting sensitive or confidentia data as some national and foreign regulations secure some data types.</p><p>Thankyou!</p><p>Do read out and try yourself by using any website data. Link to code is <a href="https://github.com/TejalKhade28/Web-Scraping">here.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=edb05d593216" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dashboard Designing with PowerBI-Part II]]></title>
            <link>https://tejalrk2000.medium.com/dashboard-designing-with-powerbi-part-ii-cf10f17e60a1?source=rss-bba66688a986------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf10f17e60a1</guid>
            <category><![CDATA[power-bi-desktop]]></category>
            <category><![CDATA[dashboard-design]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[power-bi-step-by-step]]></category>
            <dc:creator><![CDATA[Tejal Khade]]></dc:creator>
            <pubDate>Sun, 25 Jun 2023 12:39:04 GMT</pubDate>
            <atom:updated>2023-06-25T12:39:04.151Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Microsoft says, “Power BI, is a business analytics solution that lets you visualize your data and share insights across your organization, or embed them in your app or website.”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1L5Ebu8nrHa2jwI5IhMVOQ.jpeg" /><figcaption>Design from Unsplash</figcaption></figure><p>Microsoft Power BI is a suite of business intelligence (BI), reporting, and data visualization products and services for individuals and teams. Power BI Desktop is a free, self-service data analysis and report authoring tool that you install on a Windows computer. Power BI is a new cloud-based Business Intelligence service provided by Microsoft and derived from its years of experience in relational databases like Access, SQL server etc. It is a business intelligence platform that allows businesses to clean and completely transform data into meaningful data. It thoroughly analyzes data and shares powerful insights.</p><p><strong>Why Power BI is best BI Platform currently in the market?</strong></p><ol><li><strong>Ease of Use</strong></li><li><strong>Easy to Learn</strong></li><li><strong>Easy to Collaborate</strong></li><li><strong>Cost Effective</strong></li><li><strong>Wide Coverage of Data Sources</strong></li><li><strong>Powerful Tool for — Visualization, Data Shaping, Data Modeling.</strong></li></ol><h4>Creating Power BI Dashboard</h4><blockquote>Power BI Dashboard is a single page visualization to tell a story.</blockquote><h4><strong>Objective of the project: To help an organization to improve employee performance and improve employee retension i.e to reduce attrition by providing insights on the data on the HR Analytics Dashboard.</strong></h4><p>The visualizations on a dashboard are generated from reports and each report is based on one dataset. A single page dashboard is known as a canvas.</p><p>The visualizations you see on the dashboard are called as tiles.</p><p>We have designed the dashboard for an organization to find the attrition within the company and its reasons. We have designed a one page report using the <a href="https://drive.google.com/file/d/1sttnipRBPqbJ2kAadw0O4f2MJRKVw4ta/view?usp=sharing">HR_Analytics</a> dataset as a csv file.</p><p><strong>Title:</strong> This represents what is the dashboard about!</p><ol><li>Go to the Visualizations pane. In the Format area, select the General tab, and then set Title to On to show the title options for the visual.</li></ol><p>2. Next to the Title text, select Conditional formatting (fx). The Title text — Title dialog box appears.</p><p>3. From the dialog box, select the field that you created to use for your title, and then select OK.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*5_mBKGs0Q4ftolng-MsvkA.png" /><figcaption>Dashboard Title</figcaption></figure><p><strong>Slicers:</strong> Slicers are another way of filtering. They’re displayed on the report page, and narrow the portion of the dataset that’s shown in the other report visualizations. Slicers are a great choice when you want to:</p><ul><li>Display commonly used or important filters on the report canvas for easier access.</li><li>Make it easier to see the current filtered state without having to open a drop-down list.</li><li>Filter by columns that are unneeded and hidden in the data tables.</li><li>Create more focused reports by putting slicers next to important visuals.</li></ul><p>I have added a department level slicer at the top. When I select a single department, all other tiles are filtered for that particular department. Like wise we can analyze and gain insights about attrition reasons for each department.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*uO2hcxBgFiLbhOVCKXJH7A.png" /><figcaption>Dashboard Slicer</figcaption></figure><p><strong>KPI Visualization:</strong></p><p>Key Performance Indicators(KPI): KPI’s are the measures that indicate how effectively an organisation can reach it’s goals. KPIs are a great choice:</p><ul><li>To measure progress within the organization. In this dataset, one can answer the question, “What is the attrition rate of the company?”</li><li>To measure distance to a goal. Answers the question, “What is the level of job satisfaction in the company for every employee?”</li></ul><ol><li><strong>Overall Employees</strong></li></ol><p>This KPI tells the number of employees in the company.</p><p>2. <strong>Attrition</strong></p><p>It tells the number of employees who left the company.</p><p>3. <strong>Attrition Rate</strong></p><p>The rate at which employees left the company.</p><p>4. <strong>Average Age</strong></p><p>It tells the average age of the employees who work in the company.</p><p>5. <strong>Average Salary</strong></p><p>It tells the average salary of the employees who work in the company.</p><p>6. <strong>Years at Company</strong></p><p>It tells the average no of years of the employees who work in the company.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IQYDpfcQakayCstaDmUfqw.png" /><figcaption>KPI for Attrition</figcaption></figure><h4>Let us see all the tiles pinned on the dashboard —</h4><p><strong>Attrition by Gender </strong>— Shows the number of males and females who left the job. We are able to see here that 145 males and 84 females have left the job.</p><p><strong>Attrition by Education Field </strong>— It says from which education field, attrition took place mostly? Here the answer is Life Sciences!</p><p><strong>Matrix </strong>— where rows depict Job Role, columns show Job Satisfaction whereas values show the sum of attrition count. We can see that Lab Technician who rated poor job satisfaction has left the job.</p><p><strong>Attrition by Salary Slab</strong> — shows that low salary of employees is one of the reason to leave the job. We can see that count of employees with just 5K salary is maximum.</p><p><strong>Attrition by Job Role</strong> — shows the job role of the employee who left the job. Here 60 Lab Technicians have left the job.</p><p><strong>Attrition by Years At Company</strong> — After how many number of years employees left? The answer to this is just 1 year. 57 of them have left the job just after 1 year.</p><p><strong>Attrition by Age group</strong> — What is the age group of employees who left? Here the answer is 26–35 age group! A total of 111 employees left from this age group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N80XxYkVRiSWFFncjG5Tyg.png" /><figcaption>HR Analytics Dashboard</figcaption></figure><p>Using this dashboard, the organization can easily find out the reasons of attrition. The dashboard can answer all the queries with accurate insights!</p><p>Let us see few questions that could be answered by all my fellow friends!</p><ol><li>How is the attrition rate calculated? State its formula.</li><li>What is the count of males and females who possess the lowest salary?</li><li>What is the attrition rate of employees from the Medical field?</li><li>What is the salary distribution of employees from Medical field?</li><li>What is the maximum Job Satisfaction given by Sales Executive with a salary between 5k to 10k?</li><li>After 5 years how many of them left the job?</li><li>How many Sales Representatives are from 18–25 age group?</li><li>Whis job from HR department has no job satisfaction and why?</li><li>How many employees from R&amp;D department have salary less than the average monthly income?</li><li>Maximum attrition is shown from which department and which age group?</li></ol><p>Similarly, many questions can be answered and thus an organization can successfuly improve the employee performance and employee retension by using the insights gained from this dashboard.</p><p>If you like this blog, do comment and reach out to me on my linkedin handle <a href="https://www.linkedin.com/in/tejal-khade-a379381b0/">here</a> for any queries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf10f17e60a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>