<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mehul Jain on Medium]]></title>
        <description><![CDATA[Stories by Mehul Jain on Medium]]></description>
        <link>https://medium.com/@j13mehul?source=rss-144465630849------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*56vXMR7tw2lpinSfDrCM_g@2x.jpeg</url>
            <title>Stories by Mehul Jain on Medium</title>
            <link>https://medium.com/@j13mehul?source=rss-144465630849------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:23:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@j13mehul/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[7 Patterns To Master for Applied Scientist Interview]]></title>
            <link>https://medium.com/@j13mehul/7-patterns-to-master-for-applied-scientist-interview-a66332254dee?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/a66332254dee</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Fri, 13 Feb 2026 08:42:23 GMT</pubDate>
            <atom:updated>2026-02-13T18:01:28.548Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A practical guide for Data Scientists who want to transition into Applied Science roles</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tACWpv43VxMI2G6v-MVoLQ.png" /><figcaption>AI generated Image</figcaption></figure><p>Let’s first clarify what is the difference between these two roles</p><h3>Data Science vs. Applied Science — What’s the Real Difference?</h3><p>Many candidates assume that Data Science and Applied Science roles are interchangeable, but in practice they are fundamentally different, and this distinction becomes very clear during interviews.</p><p>A Data Scientist is primarily focused on analyzing data to generate insights, building experiments, and producing models or dashboards that guide decision-making. The emphasis is on statistical correctness, exploratory analysis, and deriving patterns from data. Coding requirements are usually moderate, and system thinking often stops at building pipelines that prepare and analyze datasets. Most of the work involves applying established techniques to answer business questions, with ownership centered around experimentation and interpretation of results.</p><p>An Applied Scientist, on the other hand, is expected to build production-grade intelligent systems that directly power user-facing features. The output is not just a model, but a scalable machine learning system integrated into a live product. This requires strong software engineering ability, including writing efficient, maintainable, and production-quality code. Applied Scientists must think beyond modeling to design end-to-end AI architectures that handle data ingestion, training, serving, monitoring, and continuous improvement. Rather than only using existing methods, they often adapt, extend, or innovate on algorithms to meet real-world constraints. Evaluation is multidimensional, involving not just model accuracy but also latency, scalability, cost, and business impact. Ownership does not end with an experiment; it includes deployment, reliability, and measurable impact at scale.</p><p><strong>In short:</strong><br>A Data Scientist answers: <em>“What does the data say?”</em><br>An Applied Scientist answers: <em>“How do we build intelligence that runs for millions of users?”</em></p><p>Companies hire Applied Scientists to <strong>translate ML theory into scalable product features </strong>— search, ranking, recommendation, Copilot intelligence, ads relevance, etc.</p><h3>Structure your preparation</h3><p>If you are preparing for Applied Scientist interviews the same way you prepared for Data Science interviews — by revising algorithms randomly, solving disconnected problems, or memorizing model-specific details — you are likely to feel frustrated. The reason is simple: Applied Scientist interviews are not designed to test whether you know many techniques; they are designed to test whether you can take any machine learning idea and make it work in the real world. Random preparation leads to shallow understanding, while these interviews reward structured thinking and the ability to reason through unfamiliar problems.</p><p>A far more effective approach is to prepare using repeatable patterns rather than isolated questions. Across different teams and problem domains, interview questions tend to follow a small number of recurring themes. The surface problem may look different — one interview might ask about search relevance, another about clustering at scale — but the underlying expectation is the same. Interviewers are consistently evaluating whether you can implement fundamentals from scratch, design systems end to end, scale solutions to massive datasets, optimize for latency and cost, evaluate impact in production, diagnose failures, and maintain models over time with proper safeguards.</p><p>When you recognize these patterns, preparation becomes deliberate instead of reactive. Instead of trying to predict the exact question, you train yourself to handle an entire category of problems. This mirrors the actual job of an Applied Scientist, where you are rarely given textbook scenarios and instead must adapt core principles to new product challenges. By structuring your preparation around these eight patterns, you build the mental framework needed to decompose any applied machine learning problem into clear, practical steps.</p><p>In the sections that follow, I will walk through each of these patterns with representative examples and discuss how strong candidates approach them during interviews. The goal is not just to help you answer specific questions, but to help you develop the systems-level intuition that interviewers are looking for and that the role demands in practice.</p><h3>Pattern 1— Implement Algorithm From Scratch</h3><p><strong>What they give: </strong>TF-IDF, Logistic Regression, Attention, K-Means, etc.</p><p><strong>What they test:</strong></p><ul><li>Mathematical understanding</li><li>Ability to translate math → working code</li><li>Numerical stability considerations</li><li>Memory efficiency</li></ul><p><strong>Preparation Tip: </strong>Rebuild core algorithms without sklearn/pytorch helpers.</p><h4>Question: Implement TF-IDF From Scratch</h4><p><strong>Step 1</strong> — Explain the Math</p><p><strong>Step 2</strong> — Discuss Practical Considerations</p><ul><li>Use <strong>hash maps</strong> for vocabulary</li><li>Store vectors as <strong>sparse matrices</strong></li><li>Precompute document frequencies</li><li>Normalize using L2 norm for cosine similarity</li></ul><p><strong>Step 3</strong> — Implementation Outline</p><p><strong>Step 4 </strong>— Production Insight (This is where candidates fail)</p><ul><li>Incremental IDF update for streaming documents</li><li>Sharding vocabulary across machines</li></ul><h3>Pattern 2— How Would You Scale a given ML Model?”</h3><p><strong>What they give: </strong>A model that works on small data → ask how to make it work for millions/billions.</p><p><strong>What they test:</strong></p><ul><li>Distributed training/inference</li><li>Approximation techniques</li><li>Latency vs accuracy tradeoffs</li><li>Systems thinking</li></ul><p><strong>Typical Follow-ups:</strong></p><ul><li>How do you reduce inference latency?</li><li>How do you shard the model?</li><li>What changes for real-time serving?</li></ul><p><strong>Preparation Tip: </strong>Practice converting academic algorithms into <strong>distributed versions</strong>.</p><h4>Question: You have 1 Billion Rows ,How to Run K-Means?</h4><p>Weak Answer: Sample the data</p><p>Strong Answer (Show Multi-Level Thinking)<br><strong>Step 1 </strong>— Problem With Vanilla K-Means — O(N × K × D × Iterations) — Impossible at billion scale</p><p><strong>Step 2</strong> — Use Hierarchical Reduction</p><ol><li>Mini-Batch K-Means</li><li>Coresets (mathematically compressed dataset)</li><li>PCA / Random Projection to reduce dimension</li><li>Distributed K-Means (Spark / Parameter Server)</li></ol><p><strong>Step 3 </strong>— Real Production Flow</p><p>Raw Data → Feature Hashing → Dimensionality Reduction<br>→ Coreset Construction → Distributed MiniBatch KMeans<br>→ Final Refinement on Centroids Only</p><p><strong>Step 4 </strong>— Why This Works</p><p>You preserve cluster geometry while reducing compute by orders of magnitude.</p><h3>Pattern 3 — Design an End-to-End ML System</h3><p><strong>What they give: </strong>A product scenario like Semantic search, Recommendation system, Fraud detection, Personalization engine</p><p><strong>What they test:</strong></p><ul><li>Architecture design</li><li>Data pipelines</li><li>Feature storage</li><li>Model lifecycle</li><li>Deployment strategy</li></ul><p><strong>Strong answers include:</strong></p><ul><li>Offline + Online separation</li><li>Candidate generation vs ranking</li><li>Monitoring + retraining loop</li></ul><h4>Question: Can you design <strong>semantic search at scale</strong>?</h4><h3>Architecture You Should Propose</h3><pre>User Query<br>   ↓<br>Query Understanding (Transformer Encoder)<br>   ↓<br>Query Embedding<br>   ↓<br>Vector Search (ANN Index: HNSW / FAISS)<br>   ↓<br>Candidate Retrieval<br>   ↓<br>Ranking Model (LTR / Cross Encoder)<br>   ↓<br>Business Rules + Personalization<br>   ↓<br>Final Results</pre><ul><li>ANN instead of brute-force similarity</li><li>Offline embedding generation pipeline</li><li>Real-time query encoding service</li><li>Periodic re-indexing</li><li>Feature store for ranking</li></ul><h3>Pattern 4 — How Would You Evaluate This Model in Production</h3><p><strong>What they give:</strong> A trained model → ask how you validate impact.</p><p><strong>What they test:</strong></p><ul><li>Offline vs online evaluation</li><li>A/B testing design</li><li>Metric alignment with business goals</li><li>Detecting regressions</li></ul><p><strong>Weak candidates talk only about accuracy.</strong><br>Strong candidates discuss <strong>user impact metrics + monitoring.</strong></p><h4>Question: Your new ranking model shows +3% offline NDCG. How do you validate it online?</h4><p>Weak Answer: Survey or sentiment analysis — reactive approach</p><p>Strong Answer Structure — <strong>Offline Metrics Are Not Enough.</strong></p><p>Design an A/B Experiment</p><p>Measure:</p><ul><li>CTR</li><li>Conversion rate</li><li>Revenue/session</li><li>Latency impact</li></ul><p><strong>Guard Against Risk</strong></p><ul><li>Run canary deployment (5% traffic)</li><li>Monitor failure metrics</li><li>Rollback if degradation detected</li></ul><p><strong>Key Insight: </strong>User behavior is the real metric as they are consumers — not offline accuracy.</p><h3>Pattern 5 — What Are the Failure Modes and How Do You Fix Them</h3><p><strong>What they give: </strong>A model that works well but fails in edge cases.</p><p><strong>What they test:</strong></p><ul><li>Debugging ML systems</li><li>Bias/variance diagnosis</li><li>Data quality awareness</li><li>Observability mindset</li></ul><p><strong>Preparation Tip: </strong>Always think, <em>What breaks first when this hits real users?</em></p><h4>Question: Your search relevance suddenly drops for rare queries. How do you debug?</h4><p>Weak Answer: Retraining the model without knowing what is failing</p><p>Strong Answer Structure</p><p><strong>Step 1 — Slice Metrics</strong><br>Evaluate performance by query frequency bucket.</p><p><strong>Step 2 — Identify Cause</strong><br>Rare queries lack training examples → embedding collapse.</p><p><strong>Step 3 — Fix</strong></p><ul><li>Add semantic retrieval (dense vectors)</li><li>Augment training using query rewriting</li><li>Increase long-tail sampling weight</li></ul><p><strong>Step 4 — Add Monitoring</strong><br>Alert when tail-query performance drops.</p><h3>Pattern 6 — How Would You Optimize This for Latency or Cost</h3><p><strong>What they give: </strong>A working but expensive model (often deep learning).</p><p><strong>What they test:</strong></p><ul><li>Model compression (distillation, pruning, quantization)</li><li>Caching strategies</li><li>Approximate search</li><li>Hardware-aware design</li></ul><p>This is critical because production ML is constrained by <strong>SLA and budget</strong>, not just accuracy.</p><h4>Question: Your vision model costs too much to run per request. Optimize it.</h4><p>Weak Answer: Use SLM</p><p>Strong Answer Structure</p><p><strong>Model-Level Optimization</strong></p><ul><li>Knowledge distillation → smaller model</li><li>Pruning redundant layers</li><li>Quantization (FP32 → INT8)</li></ul><p><strong>System-Level Optimization</strong></p><ul><li>Cache embeddings for repeated images</li><li>Use batch inference</li><li>Move preprocessing offline</li></ul><p><strong>Result: </strong>Same accuracy, drastically cheaper inference.</p><h3>Pattern 7 — How Would You Keep This Model Updated Over Time</h3><p><strong>What they give: </strong>A deployed model → ask about long-term maintenance.</p><p><strong>What they test:</strong></p><ul><li>Concept drift handling</li><li>Continuous training pipelines</li><li>Data/version management</li><li>Safe rollout strategies</li></ul><p><strong>Strong answers include:</strong></p><ul><li>Drift detection metrics</li><li>Retraining triggers</li><li>Shadow deployments</li><li>Model registry/versioning</li></ul><h4>Question: How do you keep a fraud detection model accurate as attackers evolve?</h4><p>Weak Answer: Retrain the model periodically</p><p>Strong Answer Structure</p><p><strong>Detect Drift</strong></p><ul><li>Monitor feature distribution (PSI / KL divergence)</li><li>Track prediction confidence changes</li></ul><p><strong>Adapt</strong></p><ul><li>Online hard-negative mining</li><li>Weekly retraining with fresh fraud patterns</li></ul><p><strong>Deploy Safely</strong></p><ul><li>Shadow model comparison</li><li>Gradual rollout</li></ul><p><strong>Key Idea: </strong>Production ML must behave like a continuously learning system.</p><h3>A Simple Way to Remember All Patterns</h3><p>Most Applied Scientist interviews are probing one transformation:</p><blockquote><strong><em>Can you take an ML idea and make it survive reality?</em></strong></blockquote><p>That “reality gap” includes:</p><ul><li>Scale</li><li>Noise</li><li>Latency</li><li>Cost</li><li>Drift</li><li>Users</li><li>Failures</li></ul><h3>The “Answer Framework” to Use in Every Interview</h3><p>Many candidates know the solution but present it poorly. Use this structure consistently:</p><p><strong>Step 1 — Clarify the Problem</strong><br> Restate assumptions, scale, constraints.</p><p><strong>Step 2 — Start With a Simple Baseline</strong><br> Show you understand the fundamental approach first.</p><p><strong>Step 3 — Identify Limitations</strong><br> Explain why the naive solution fails at scale or production.</p><p><strong>Step 4 — Introduce the Real-World Solution</strong><br> Add optimizations, architecture, or modeling improvements.</p><p><strong>Step 5 — Discuss Tradeoffs</strong><br> Accuracy vs latency, cost vs quality, complexity vs maintainability.</p><p><strong>Step 6 — Explain How You Would Validate It</strong><br> Metrics, experiments, monitoring.</p><p>This structure demonstrates maturity and prevents rambling answers.</p><h3>Common Mistakes to Avoid</h3><p>Candidates often unknowingly signal “academic thinking” instead of “applied thinking.”</p><ul><li>Jumping directly to deep learning without considering simpler solutions.</li><li>Ignoring data pipelines and focusing only on models.</li><li>Not discussing latency or scalability.</li><li>Giving library-based answers instead of explaining mechanics.</li><li>Forgetting evaluation and monitoring after deployment.</li></ul><h3>A 4-Month Preparation Roadmap</h3><p>Where do I start?</p><p><strong>Month 1 — Fundamentals Rebuild</strong></p><ul><li>Implement core algorithms from scratch (logistic regression, TF-IDF, clustering).</li><li>Revise probability, optimization intuition, and vector math.</li><li>Focus on understanding derivations rather than memorizing.</li></ul><p><strong>Month 2 — System Thinking</strong></p><ul><li>Design 2–3 ML systems end to end (search, recommendation).</li><li>Practice drawing architectures and explaining tradeoffs.</li><li>Think about offline vs online components.</li></ul><p><strong>Month 3 — Scale and Optimization</strong></p><ul><li>Take known models and ask: how would this run on billions of samples?</li><li>Learn approximate methods (ANN search, mini-batching, sampling).</li><li>Practice latency and cost optimization discussions.</li></ul><p><strong>Month 4 — Production Readiness</strong></p><ul><li>Study evaluation strategies, A/B testing, drift detection, and guardrails.</li><li>Conduct mock interviews focusing on explanation clarity.</li><li>Practice answering in structured storytelling format.</li></ul><p>Preparing for this role is not about becoming better at machine learning models — it is about becoming someone who can turn machine learning into a dependable product capability. Once candidates start thinking in terms of systems, constraints, and lifecycle ownership, they naturally align with what interviewers are trying to assess.</p><p>All the best :)</p><p>Want More…</p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-machine-learning-4965cbdcf8d1">All I needed to crack the Microsoft Interview — Machine Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-deep-learning-c39bad25f089">All I needed to crack the Microsoft Interview — Deep Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-statistics-1f3d3ca8055d">All I needed to crack the Microsoft Interview — Statistics | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-nlp-31f578a86034">All I needed to crack the Microsoft Interview — NLP | by Mehul Jain | Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a66332254dee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-tuning LLMs with Reinforcement Learning]]></title>
            <link>https://medium.com/data-science-at-microsoft/fine-tuning-llms-with-reinforcement-learning-ef84fe42d6a6?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef84fe42d6a6</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[deepseek]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Tue, 08 Jul 2025 07:16:58 GMT</pubDate>
            <atom:updated>2025-11-03T06:20:39.426Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to apply policy gradients like PPO, DPO, OPRO, and GRPO</h4><p>In the fast-evolving world of AI, fine-tuning Large Language Models (LLMs) using Reinforcement Learning (RL) has become an essential method for aligning model outputs with human values, goals, or task-specific requirements. This article explores fine-tuning, dives into the most effective RL-based optimizers for LLMs, and provides practical code examples using the <a href="https://huggingface.co/docs/trl/index">Hugging Face TRL library</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*kwzvRSBB7c0bweM-2GoKNg.png" /><figcaption>Image created by <a href="https://www.bing.com/images/create">Microsoft Bing Image Creator</a>.</figcaption></figure><h3><strong>Fine-tuning in LLMs</strong></h3><p><em>Fine-tuning</em> refers to the process of taking a pretrained language model and continuing its training on a smaller, task-specific dataset. The goal is to specialize the general knowledge of the model to perform better on particular types of tasks, such as question answering, summarization, or instruction-following.</p><p>There are two primary stages in modern LLM alignment:</p><h4>1. Supervised Fine-Tuning (SFT)</h4><p>This is the first stage in aligning an LLM to follow specific instructions. In SFT, a pretrained model is further trained using a dataset of instruction-response pairs, where each input is a prompt (instruction or question) and the output is the ideal response crafted by humans. The objective is to teach the model to generate desirable responses in line with human intentions. SFT uses supervised learning, in which the model is trained to minimize the cross-entropy loss between its predicted tokens and the ground truth human responses. This stage sets a strong behavioral prior for the model before moving to reinforcement learning.</p><ul><li>It requires labeled data in the form of high-quality instruction-response examples.</li><li>The output closely mirrors the style and intent of the training examples.</li><li>It doesn’t require any form of reward model or ranking.</li></ul><pre># Minimal code for fine tuning <br># QLoRA config<br>bnb_config = BitsAndBytesConfig(<br>    load_in_4bit=True,<br>    bnb_4bit_quant_type=&quot;nf4&quot;,<br>    bnb_4bit_compute_dtype=torch_dtype,<br>    bnb_4bit_use_double_quant=True,<br>)<br><br># LoRA config<br>peft_config = LoraConfig(<br>    r=16,<br>    lora_alpha=32,<br>    lora_dropout=0.05,<br>    bias=&quot;none&quot;,<br>    task_type=&quot;CAUSAL_LM&quot;,<br>    target_modules=[&#39;k_proj&#39;, &#39;q_proj&#39;, &#39;v_proj&#39;, &#39;o_proj&#39;]<br>)<br><br># Load tokenizer<br>tokenizer = AutoTokenizer.from_pretrained(base_model)<br><br># Load model<br>model = AutoModelForCausalLM.from_pretrained(<br>    base_model,<br>    quantization_config=bnb_config,<br>    device_map=&#39;auto&#39;<br>    attn_implementation=&#39;flash_attention_2&#39;<br>)<br><br># Dummy SFT dataset<br>dataset = load_dataset(&quot;json&quot;, data_files=&quot;sft_data.json&quot;, split=&quot;train&quot;)  # [{&quot;prompt&quot;: ..., &quot;response&quot;: ...}]<br><br>def format_example(example):<br>    return {<br>        &quot;text&quot;: f&quot;### Prompt:\n{example[&#39;prompt&#39;]}\n\n### Response:\n{example[&#39;response&#39;]}&quot;<br>    }<br><br>dataset = dataset.map(format_example)<br><br># Tokenize<br>def tokenize(ex):<br>    return tokenizer(ex[&quot;text&quot;], truncation=True, padding=&quot;max_length&quot;, max_length=512)<br><br>tokenized_dataset = dataset.map(tokenize)<br><br># Train<br>training_args = TrainingArguments(<br>    output_dir=&quot;./sft_model&quot;,<br>    per_device_train_batch_size=2,<br>    num_train_epochs=3,<br>    logging_steps=10,<br>    save_steps=100<br>)<br><br>trainer = Trainer(<br>    model=model,<br>    args=training_args,<br>    train_dataset=tokenized_dataset,<br>    tokenizer=tokenizer<br>)<br><br>trainer.train()<br>trainer.save_model(&quot;sft_model&quot;)</pre><h4>2. Reinforcement Learning (RL)</h4><p>This stage refines model behavior using reward signals, which quantify the desirability of a model’s output. These signals can come from various sources, such as a reward model trained on human feedback, heuristic rules, or task-specific metrics. In Reinforcement Learning from Human Feedback, the process typically involves:</p><ul><li>Collecting multiple outputs (responses) from a model for a given prompt.</li><li>Having humans rank or rate these outputs based on quality or alignment.</li><li>Training a reward model that predicts these preferences.</li><li>Using this reward model to provide feedback to guide further training of the LLM.</li></ul><p>Imagine that an LLM is trained to answer questions. The goal is to improve its answers using Reinforcement Learning.</p><p>Components of RL:</p><ol><li>State(s): This is the context or input given to the model. Example state: “What is the capital of France?”</li><li>Action (a): This is the response generated by the LLM. Example Actions: <em>a</em>₁: “Paris”, <em>a</em>₂: “London”, <em>a</em>₃: “I don’t know”. The LLM selects one of these based on its policy.</li><li>Reward (<em>r</em>): After generating a response, we give a reward signal based on how good the answer is. Reward examples: “Paris” → +1 (correct answer), “London” → 0, “I don’t know” → -0.1 (uninformative). The reward can be human feedback or come from a heuristic, knowledge base, or simulated environment.</li></ol><p>Before we discuss the various policy optimizers, let’s understand the meaning of a policy gradient.</p><h3><strong>Policy gradient</strong></h3><p>Imagine that you’re learning to play a video game. You try different moves (actions) and based on how much reward you get (points), you update your playing strategy to be better.<strong> </strong>An agent learns in a similar way in how to act in an environment to maximize rewards. A policy tells the agent what action to take in a given situation (state). A policy gradient method directly adjusts the parameters of this policy (neural network) to increase expected rewards.</p><p>Let’s look at a simple problem.</p><p>Suppose our agent plays a very simple game:</p><ul><li>It has one decision to make: Move left (<em>L</em>) or right (<em>R</em>).</li><li>The policy is defined as: π<em>θ</em>(<em>L</em>) = <em>θ</em>, π<em>θ</em>(<em>R</em>) = 1 — <em>θ</em></li></ul><p>Let’s say:</p><ul><li>If it goes left, it gets reward = 1</li><li>If it goes right, it gets reward = 0</li></ul><p>Suppose <em>θ </em>= 0.4, i.e., there is a 40 percent chance of choosing left.</p><p>Let’s sample one trajectory (with one action):</p><ol><li>Action taken: Left</li><li>Reward received: 1</li><li>Policy probability: π<em>θ</em>(<em>L</em>) = 0.4</li><li>Log probability: log(0.4) ≈ −0.916</li><li>Gradient of log with regard to <em>θ</em>: Because π<em>θ</em>(<em>L</em>)=<em>θ</em>, ∇logπ<em>θ</em>(<em>L</em>) = 1/<em>θ</em> = 1/0.4 = 2.5</li><li>Multiply by reward: ∇<em>J</em>(<em>θ</em>) ≈ 2.5 * 1 = 2.5</li></ol><blockquote><strong>Interpretation</strong>: The gradient says: Increase <em>θ</em> (more likely to go left), because left provides good rewards.</blockquote><p>We now update <em>θ</em> using gradient ascent:</p><p><em>θ</em> = <em>θ</em> + <em>α</em>⋅∇<em>J</em>(<em>θ</em>)</p><p>Let learning rate <em>α </em>= 0.1</p><p><em>θ</em> = 0.4 + 0.1 * 2.5 = 0.4 + 0.25 = 0.65</p><p>Now, the agent is 65 percent likely to go left, which was initially 40 percent.</p><p>The policy gradient can be unstable due to high variance in rewards. To fix it, we use advanced tricks that use base model or clipping techniques to limit the parameter updates.</p><p><em>A</em>(<em>s</em>,<em>a</em>) = <em>Q</em>(<em>s</em>,<em>a</em>) − <em>V</em>(<em>s</em>)</p><ul><li><em>Q</em>(<em>s</em>,<em>a</em>): expected total reward <strong>after taking action</strong> a in state <em>s.</em></li><li><em>V</em>(<em>s</em>): expected total reward from <strong>just being in state</strong> <em>s.</em></li></ul><blockquote><strong>Interpretation:</strong></blockquote><blockquote>If <em>A</em>(<em>s</em>,<em>a</em>)&gt;0: Action is better than average → increase probability.</blockquote><blockquote>If <em>A</em>(<em>s</em>,<em>a</em>)&lt;0: Action is worse than average → decrease probability.</blockquote><p><strong>The policy gradient update is:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*3WdFpOM_AWF7859yziaWAw.png" /></figure><p><strong>With advantage, it becomes:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*-TuJGptNNDkSUOB5x6p_CQ.png" /></figure><p><strong>Reward function:</strong> This tells the agent how good or bad its actions are. In RLHF, the reward function is often learned. It assigns higher values to responses that are more preferred by humans. The quality of this function directly affects the fine-tuning outcome, so it’s crucial to have diverse, well-curated human annotations.</p><p><strong>KL Divergence: </strong><a href="https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence">KL Divergence</a> is a measure of the difference between two probability distributions. In this case, we control how much the policy changes by penalizing large KL Divergence between old and new policies.</p><h3><strong>RL optimizers</strong></h3><p>Now we explore four popular RL optimization strategies used in LLM fine-tuning.</p><h4>1. Proximal Policy Optimization (utilized by ChatGTP)</h4><p><a href="https://arxiv.org/pdf/1707.06347">Proximal Policy Optimization</a> (PPO) seeks to improve the stability and efficiency of policy gradient methods. It does this by preventing large, destabilizing updates to the policy. Instead of applying unrestricted updates as in more typical policy gradients, PPO uses a clipping mechanism to restrict how much the new policy can deviate from the old policy. This makes the training process more reliable and avoids catastrophic drops in performance often caused by overly aggressive updates. After generating responses from the policy network, each output <em>y</em> is scored using a reward model <em>R</em>(<em>y</em>) — typically trained on human preference data. These reward values are treated as the environment’s feedback for PPO. Then, the reward is used to compute the <strong>advantage</strong> and <strong>returns</strong> (with the help of the value network).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*76GI2y8IHoPvSjIZgG2GOQ.png" /><figcaption>Image by <a href="https://www.youtube.com/watch?v=TjHH_--7l8g&amp;t=2071s">Serrano Academy</a></figcaption></figure><p><strong>Algorithm steps</strong></p><p>1. The algorithm trains two networks: 1.) a policy network π<em>θ</em>(<em>a</em>∣<em>s</em>) that outputs the probability distribution over actions, and 2.) a value network <em>Vϕ</em>(<em>s</em>) that predicts the expected return (state value) for a given state.</p><p>2. The policy network is used to sample actions; the value network is used to estimate how good a state is (not <em>Q</em>-values but state values), and the reward function predicts the reward value.</p><p><strong>3. Advantage estimation</strong> is computed using:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*UP8sksE0O9viHv-nqx67-A.png" /></figure><p><strong>4. Surrogate loss function</strong> for the policy update is computed as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*P92yXrRSANP7CXZWDh4A0g.png" /></figure><p><strong>5.</strong> <strong>Clipping mechanism</strong> prevents large policy updates by limiting the ratio <em>rt</em>(<em>θ</em>) within a trust region [<em>1</em>−<em>ϵ</em>, <em>1</em>+<em>ϵ</em>].</p><p><strong>6.</strong> The algorithm optimizes the total loss with multiple epochs of mini-batched <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">stochastic gradient descent</a> (SGD) on collected trajectories.</p><p><strong>Pros:</strong></p><ul><li>Stable and robust.</li><li>Easy to implement with clipping.</li></ul><p><strong>Cons:</strong></p><ul><li>Requires a reward model.</li><li>More compute-intensive.</li></ul><p><strong>Key parameters in TRL:</strong></p><ul><li>target_kl: Sets the desired upper bound for KL divergence between the current and reference policy.</li></ul><pre># Code for PPO<br># Load PPO dataset<br># Sample code<br>{<br>  &quot;prompt&quot;: &quot;What is the capital of France?&quot;,<br>  &quot;responses&quot;: [<br>    &quot;Paris is the capital.&quot;,       # rank 1 (best)<br>    &quot;It might be Lyon.&quot;,           # rank 2<br>    &quot;Berlin?&quot;,                     # rank 3<br>    &quot;Could be Madrid.&quot;,            # rank 4<br>    &quot;Not sure, maybe Rome.&quot;        # rank 5<br>  ],<br>  &quot;ranking&quot;: [1, 2, 3, 4, 5]<br>}<br><br># PPO config<br>ppo_config = PPOConfig(<br>    model_name=&quot;sft_model&quot;,<br>    learning_rate=5e-6,<br>    batch_size=2,<br>    mini_batch_size=1,<br>    optimize_cuda_cache=True,<br>    log_with=None,<br>)<br><br># Train<br>ppo_trainer = PPOTrainer(<br>    config=ppo_config,<br>    model=model,<br>    tokenizer=tokenizer,<br>    dataset=dataset<br>)<br><br># get reward value<br>def get_reward(prompt, response):<br>    inputs = tokenizer(prompt + response, return_tensors=&quot;pt&quot;, padding=True, truncation=True).to(&quot;cuda&quot;)<br>    reward = reward_model(**inputs).logits[0].item()<br>    return reward<br><br>for batch in ppo_trainer.dataloader:<br>    query_tensors = tokenizer(batch[&quot;prompt&quot;], return_tensors=&quot;pt&quot;, padding=True, truncation=True).input_ids<br>    response_tensors = []<br>    responses = []<br><br>    for q in query_tensors:<br>        response = ppo_trainer.model.generate(q.unsqueeze(0), max_new_tokens=50)<br>        r = response[:, q.shape[-1]:]<br>        response_tensors.append(r)<br>        responses.append(tokenizer.decode(r[0], skip_special_tokens=True))<br><br>    rewards = [get_reward(p, r) for p, r in zip(batch[&quot;prompt&quot;], responses)]<br>    ppo_trainer.step(query_tensors, response_tensors, rewards)<br><br>ppo_trainer.save_model(&quot;ppo_finetuned_model&quot;)</pre><h4><strong>2. </strong>Direct Preference Optimization (utilized by Llama 2)</h4><p><a href="https://arxiv.org/pdf/2305.18290">Direct Preference Optimization</a> (DPO) is designed to simplify and stabilize the training of language models using human feedback. Unlike PPO, which requires a separately trained reward model to score outputs, DPO learns directly from pairs of outputs — one preferred and one rejected — based on human judgments.</p><p>In DPO, the model is trained to increase the probability of the preferred response while reducing the probability of the rejected one, given the same prompt. This is done by minimizing a loss function derived from a logistic transformation of the log probability difference between the two responses. By doing so, DPO effectively aligns the model’s behavior with human preferences without needing an explicit reward model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*gpn0IOjqqMHCP5L8bvDflw.png" /><figcaption>Image by <a href="https://www.youtube.com/watch?v=k2pD3k1485A&amp;t=1100s">Serrano Academy</a></figcaption></figure><p><strong>Algorithm steps:</strong></p><p>1. The algorithm trains a single network: Only a policy model π<em>θ</em>​ is trained; there is no value or reward model involved.</p><p>2. Input data is in pairs (<em>x</em>, <em>y</em>+, <em>y</em>−) where <em>x</em> is a prompt, <em>y</em>+ is the preferred response, and <em>y</em>− is the rejected one. There pair terminology refers to the comparison between two outputs<em> y</em>+ and <em>y</em>−<strong>.</strong></p><p>3. The objective is to learn a policy π<em>θ</em>​ such that: π<em>θ</em>(<em>y</em>+|<em>x</em>) &gt; π<em>θ</em>(<em>y</em>−|<em>x</em>)</p><p>4. Loss is calculated using a logistic (binary classification) loss:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*lw67K55u0buCq2WGHbXAYA.png" /></figure><p>5. Loss is applied to KL-constrained relative probabilities, i.e., DPO can use the log-ratio between current policy and a reference policy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*VB4tc6R38DFjyxMmkoxHMg.png" /></figure><p>6. No need for explicit reward model or RL step. Directly uses human preferences to update the policy via supervised-style learning.</p><p><strong>Pros:</strong></p><ul><li>No reward model needed.</li><li>Stable and sample efficient.</li></ul><p><strong>Cons:</strong></p><ol><li>Requires quality preference data.</li><li>Offline method only.</li></ol><p><strong>Key parameters in TRL:</strong></p><ul><li>beta: Controls the regularization strength, balancing between preference alignment and staying close to the initial policy</li></ul><pre># Code for DPO<br># Load DPO dataset<br># Sample input<br>{<br>  &quot;prompt&quot;: &quot;What is the capital of France?&quot;,<br>  &quot;chosen&quot;: &quot;The capital of France is Paris.&quot;,<br>  &quot;rejected&quot;: &quot;I think it&#39;s Berlin?&quot;<br>}<br><br># DPO config<br>config = DPOConfig(<br>    model_name_or_path=&quot;sft_model&quot;,<br>    learning_rate=5e-6,<br>    per_device_train_batch_size=2,<br>    max_length=512,<br>    beta=0.1,<br>)<br><br># Train<br>trainer = DPOTrainer(<br>    model=model,<br>    ref_model=None,<br>    args=config,<br>    train_dataset=dpo_data,<br>    tokenizer=tokenizer<br>)<br><br>trainer.train()<br>trainer.save_model(&quot;dpo_finetuned_model&quot;)</pre><h4><strong>3. </strong>Odds Ratio Preference Optimization (ORPO)</h4><p><a href="https://arxiv.org/pdf/2403.07691">Odds Ratio Preference Optimization</a> (ORPO) is a preference-based method that builds on DPO by modifying the objective function to have a stronger separation between preferred and rejected responses. ORPO trains models using pairwise preference data by optimizing the <em>odds ratio</em> between the likelihoods of preferred and rejected outputs. This approach introduces a tighter coupling between preference feedback and the model’s learning objective. ORPO’s optimization objective encourages the model to boost the odds of generating the preferred response over the rejected one in a way that is more aggressive and interpretable than DPO’s logistic loss.</p><p><strong>Algorithm steps:</strong></p><p>1. A policy model π<em>θ</em>​ learns from both instruction-following and preference feedback.</p><p>2. The algorithm combines two objectives: Supervised fine-tuning (SFT) loss on <em>y</em>+ and preference loss from odds ratio of preferred versus rejected responses.</p><p>3. SFT loss (cross-entropy):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/288/1*zkmt6OpIdDMino9bj2yKOQ.png" /></figure><p>4. Preference loss using odds ratio (logit difference):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*BZiQdMOC7Lv1T9GLPrzvLw.png" /></figure><p><em>σ</em> is the sigmoid function</p><p>5. Final loss is a weighted combination:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*DtF3VDX2IUizQDh_Rx2pzg.png" /></figure><p><strong>Pros:</strong></p><ul><li>Direct preference learning without reward modeling.</li><li>Simpler and often more interpretable than DPO.</li><li>Stronger gradients for large preference gaps.</li></ul><p><strong>Cons:</strong></p><ul><li>Susceptible to instability for similar responses.</li><li>Cannot provide qualified reward value like PPO.</li></ul><p><strong>Key parameters:</strong></p><ul><li>alpha: Determines the weighting between supervised fine-tuning loss and preference optimization loss.</li></ul><h4><strong>4. </strong>Generalized Preference Optimization (utilized by DeepSeek)</h4><p><a href="https://arxiv.org/pdf/2402.03300">Generalized Preference Automation</a> (GRPO) builds upon DPO by extending it to handle ranked lists of outputs and arbitrary utility or reward functions. While DPO operates on binary preference pairs (chosen versus rejected), GRPO uses full or partial rankings of responses to derive a target distribution that reflects the relative desirability of all outputs. The model is then trained to match this distribution using techniques like KL divergence minimization. This generalization enables GRPO to learn from more nuanced feedback such as graded relevance, scalar scores, or top-<em>k</em> preferences. It accommodates diverse scenarios in which preferences are not strictly binary but ordered or weighted. GRPO is particularly useful when you have multiple outputs for the same prompt and want the model to prefer some over others in a more fine-grained way than simple pairwise comparison.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*SdSUBQ8SiqT0HgkGXX9frA.png" /><figcaption>Image by <a href="https://www.youtube.com/watch?v=XeUB4h1OO1g&amp;t=439s">Serrano Academy</a></figcaption></figure><p><strong>Algorithm steps:</strong></p><p>1. A policy model π<em>θ</em> ​learns to prefer higher-reward responses within a group of sampled responses for the same prompt.</p><p>2. For each prompt <em>x</em>, sample a group of <em>G</em> responses {<em>y</em>1, <em>y</em>2,…, <em>y</em>G} and compute scalar reward <em>r</em>(<em>yi</em>) for each response.</p><p>3. Advantage estimation is group relative. Normalize rewards across the group to compute advantage. It tries only to improve those responses that are not good.</p><p>4. Loss is a PPO-style clipped surrogate with relative advantages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*Q4VtnwE0y4hXBPyKFXUr_w.png" /></figure><p><strong>Pros:</strong></p><ul><li>Supports multiple ranked outputs.</li><li>Generalizes to various reward formats.</li></ul><p><strong>Cons:</strong></p><ul><li>More complex math.</li><li>Currently under development.</li></ul><p><strong>Key parameters:</strong></p><p>· KL divergence: It penalizes deviation from the reference policy.</p><p>· num_generation: Sets the number of response completions generated per prompt during training.</p><h3><strong>Concluding thoughts</strong></h3><p>Fine-tuning LLMs with reinforcement learning optimizers has significantly improved the alignment and helpfulness of models. Each optimizer has trade-offs, and your choice should be guided by your data format, stability needs, and computational budget.</p><p>However, it’s important to recognize that reinforcement learning from human feedback (RLHF) can introduce biases. These biases may stem from the subjectivity of human annotators, skewed data distributions, or overly restrictive optimization objectives. If not addressed, such biases can lead to models that overfit to narrow definitions of correctness or reinforce existing societal stereotypes.</p><h4><strong>Best practices to reduce bias</strong></h4><ul><li><strong>Diverse and representative feedback:</strong> Involve a wide range of annotators and ensure varied prompts and tasks.</li><li><strong>Regular evaluation:</strong> Continuously test for fairness, inclusivity, and unintended behaviors using diagnostic datasets.</li><li><strong>Preference aggregation:</strong> Use consensus or weighting methods to smooth out individual annotator bias.</li><li><strong>Open-ended objectives:</strong> Avoid overly rigid training signals that penalize creative or contextually valid outputs.</li></ul><p><em>Mehul Jain is on </em><a href="https://www.linkedin.com/in/j13mehul/"><em>LinkedIn</em></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef84fe42d6a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-at-microsoft/fine-tuning-llms-with-reinforcement-learning-ef84fe42d6a6">Fine-tuning LLMs with Reinforcement Learning</a> was originally published in <a href="https://medium.com/data-science-at-microsoft">Data Science + AI at Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Continuous improvement with agentic AI: Conducting a virtual gemba walk]]></title>
            <link>https://medium.com/data-science-at-microsoft/continuous-improvement-with-agentic-ai-conducting-a-virtual-gemba-walk-b39836f16301?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/b39836f16301</guid>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[autogen]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Thu, 17 Apr 2025 07:16:30 GMT</pubDate>
            <atom:updated>2025-05-06T23:08:38.991Z</atom:updated>
            <content:encoded><![CDATA[<p>Leveraging AI-driven multi-agent systems (DeepSeek, AutoGen, Ollama) to revolutionize process optimization, eliminate inefficiencies, and drive data-backed operational excellence</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ZIVNXRgC7LWABIH9AMFIrg.png" /><figcaption>Image generated by <a href="https://www.bing.com/images/create">Microsoft Bing Image Creator</a>.</figcaption></figure><p>In today’s fast-moving business world, companies are constantly seeking ways to improve efficiency, reduce waste, and enhance overall performance. One powerful method for achieving these goals is the gemba walk — a lean management practice where managers visit the actual workplace (the “gemba,” from Japanese) to observe processes, identify inefficiencies, and implement improvements. However, with advancements in agentic AI, businesses can now take this concept further by conducting virtual gemba walks. Using multi-agent AI systems, organizations can automate data collection, analyze workflows, and generate actionable insights for continuous improvement.</p><p>This article introduces the notion of the gemba walk, introduces the concept of a reasoning model, provides a briefing on the DeepSeek-R1 reasoning model specifically, and then walks through the process of how agentic AI can enhance the concept of gemba walks using reasoning models to result in an efficient and high-performing virtual gemba walk, including providing some related implementation code in Python.</p><h3>What is a gemba walk?</h3><p>A gemba walk is a structured approach where leadership teams visit the workplace to:</p><ol><li>Observe real-time operations.</li><li>Engage with employees to understand pain points.</li><li>Identify inefficiencies and waste.</li><li>Implement actionable process improvements.</li></ol><p>Traditional gemba walks require physical presence, time commitment, and subjective decision-making. However, agentic AI can overcome these limitations by automating data collection, conducting virtual observations, and generating AI-driven improvement recommendations.</p><h3>What is a reasoning model?</h3><p>Reasoning models in AI are designed to simulate human-like thinking, allowing systems to analyze complex situations, make decisions, and solve problems. These models help AI draw logical conclusions from both structured and unstructured data — a key feature of agentic AI — making them essential for applications like automated planning, knowledge representation, and multi-agent collaboration. AI reasoning models are especially valuable for multi-step decision-making and contextual understanding. They use structured thinking techniques such as:</p><ul><li><strong>Chain of thought (CoT):</strong> Encourages AI to break down reasoning into clear, logical steps.</li><li><strong>Reinforcement learning (RL):</strong> Helps AI improve decision-making through trial and error.</li><li><strong>Self-reflection: </strong>Enables AI to refine its reasoning by evaluating its past decisions.</li></ul><p>By leveraging these techniques, reasoning models enhance AI’s ability to process information more intelligently and generate deeper insights.</p><h3>A brief about DeepSeek-R1</h3><p>DeepSeek is a family of cutting-edge AI models designed for deep reasoning, complex problem-solving, and contextual understanding. These models excel in tasks requiring multi-step logical inference, mathematical reasoning, and code generation, making them ideal for applications in scientific research, automation, and intelligent decision-making. Here are some details about DeepSeek’s iterations and capabilities to date:</p><h4>1. DeepSeek-R1-Zero: Pure reinforcement learning for reasoning</h4><ul><li><strong>Objective</strong>: Train a reasoning-capable LLM without any supervised fine-tuning (SFT).</li><li><strong>Method</strong>: Applied Group Relative Policy Optimization (GRPO) as the RL framework to improve reasoning performance.</li><li><strong>Key findings</strong>: The model developed reasoning behaviors autonomously, improving significantly on benchmarks. It exhibited challenges such as poor readability and language mixing.</li></ul><h4>2. DeepSeek-R1: Reinforcement learning with a cold start</h4><ul><li><strong>Objective</strong>: Improve reasoning capabilities while addressing readability issues.</li><li><strong>Cold-start data</strong>: Introduced supervised fine-tuning (SFT) with structured chain-of-thought (CoT) examples before RL training.</li><li><strong>Training pipeline</strong>: <br> — Cold start: Collected high-quality CoT data to fine-tune the base model before RL.<br> — Reasoning-oriented RL: Applied RL again to further enhance reasoning skills.<br> — Rejection sampling and supervised fine-tuning: Used RL-trained outputs to create better training data.<br> — Final RL optimization: Trained the model on all scenarios, including general NLP tasks.</li></ul><h4>3. Distillation: Bringing reasoning to smaller models</h4><ul><li><strong>Objective</strong>: Transfer reasoning capabilities to smaller LLMs.</li><li><strong>Method</strong>: Used DeepSeek-R1 as a teacher model to generate 800,000 training samples and fine-tuned smaller models like Qwen and Llama.</li><li><strong>Results</strong>: Distilled models outperformed traditional supervised models, with significant improvements on reasoning benchmarks.</li></ul><h3>How agentic AI can enhance gemba walks using reasoning models</h3><p>Agentic AI can enhance gemba walks using reasoning models by automating data collection, providing data-driven insights, and enabling continuous improvement. Agentic workflows leverage autonomous AI agents to perform reasoning, decision-making, and task execution with minimal human intervention. These workflows follow structured patterns that help agents collaborate, self-improve, and optimize complex tasks like gemba walks, troubleshooting, research, or automation. Some key patterns in agentic workflows include:</p><h4>1. Hierarchical pattern (master-worker model)</h4><ul><li>A master (coordinator) agent delegates subtasks to multiple worker agents.</li><li>Each worker agent specializes in a specific domain or function (e.g., data analysis, documentation, anomaly detection).</li><li>The master agent aggregates and validates the results.</li></ul><h4>2. Recursive reasoning pattern</h4><ul><li>The agent breaks a problem into smaller subproblems.</li><li>It iterates recursively, refining its approach at each step.</li><li>It uses self-evaluation to assess solution quality.</li></ul><h4>3. Chain-of-thought (CoT) workflow pattern</h4><ul><li>Agents explicitly reason step by step, making decisions more transparent.</li><li>Intermediate results are stored and referenced before finalizing a decision.</li></ul><h4>4. Reflexive learning pattern (self-correcting agents)</h4><ul><li>The agent reviews its own responses before finalizing outputs.</li><li>It can use external verification tools or human feedback loops.</li></ul><h4>5. Multi-agent debate and consensus pattern</h4><ul><li>Multiple AI agents with diverse perspectives generate different solutions.</li><li>A moderator agent or voting mechanism selects the best approach.</li></ul><h4>6. Autonomous exploration pattern</h4><ul><li>The agent autonomously gathers data from multiple sources before making decisions.</li><li>It uses context-aware retrieval mechanisms like RAG (retrieval-augmented generation).</li></ul><p>For the gemba walk, we have primarily leveraged a hierarchical multi-agent pattern. Our solution requires multiple specialized agents handling different tasks, such as data collection, analysis, and optimization. A coordinator agent ensures that each agent’s output contributes effectively to the final process improvement plan. One can try a hybrid approach such as a hierarchical multi-agent pattern combined with recursive reasoning and reflexive learning for more complex scenarios.</p><h3>Agent high-level workflow</h3><p><strong>Situation</strong>: I want to optimize our car assembly line process. We are experiencing delays in final assembly, and our production output has dropped by 15 percent in the last three months. I suspect bottlenecks in component delivery, machine downtime, or inefficient worker allocation. Can you help analyze the situation and suggest improvements?</p><p><strong>Step 1:</strong> User input (problem definition) — User provides input:</p><ul><li>Describes the process they want to optimize.</li><li>Highlights known inefficiencies or issues (e.g., delays, cost overruns, quality issues).</li><li>Example: “Our assembly line has a 15 percent drop in efficiency.”</li><li>Possible causes: “Component delivery issues, machine downtime, and worker allocation problems.”</li></ul><p><strong>Step 2: </strong>AI-generated investigation plan — AI plans information gathering:</p><ul><li>Identifies key areas needing investigation (e.g., logistics, production, workforce).</li><li>Generates targeted questions for relevant teams.</li><li>Assigns agents to collect responses.</li><li>Example questions: “What percentage of components are delivered on time? What are the top causes of machine downtime? How is worker productivity tracked?”</li></ul><p><strong>Step 3: </strong>Collecting responses from teams — Teams provide data and insights:</p><ul><li>Logistics, maintenance, production, and HR teams respond.</li><li>Responses are structured into key insights.</li><li>Example response: “85 percent of deliveries are on time; delays are mostly in overseas shipments.”</li></ul><p><strong>Step 4:</strong> AI summarizes and analyzes findings — AI aggregates and processes data:</p><ul><li>Detects patterns and bottlenecks (e.g., delays at welding stations).</li><li>Identifies root causes (e.g., material shortages, untrained staff).</li><li>Creates a structured process flow analysis.</li><li>Example insight: “Machine downtime is responsible for a 10 percent output loss; 60 percent of breakdowns are due to component defects.”</li></ul><p><strong>Step 5:</strong> AI-driven virtual gemba walk — AI conducts virtual walkthrough:</p><ul><li>Simulates real-world process flow based on data.</li><li>Identifies waste, inefficiencies, and gaps in the system.</li><li>Suggests optimizations using lean principles.</li><li>Example observation: “Testing and painting stages overlap, causing rework delays.”</li></ul><p><strong>Step 6:</strong> AI provides recommendations and action plan — AI suggests improvements:</p><ul><li>Proposes data-driven process changes.</li><li>Suggests automation, training, or workflow adjustments.</li><li>Delivers an actionable roadmap for optimization.</li><li>Example recommendations: Improve supply chain tracking to reduce delays. Automate welding inspection to reduce bottlenecks. Cross-train workers to handle peak loads efficiently.</li></ul><p><strong>Step 7:</strong> Implementation and continuous improvement — Track and measure impact:</p><ul><li>Implement recommended changes.</li><li>Monitor improvements in efficiency and cost reduction.</li><li>Conduct ongoing AI-driven gemba walks for continuous refinement.</li><li>Example metrics for impact: Before: 15 percent efficiency loss. After: 5 percent improvement in three months.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*hO6UoyqiEUAfQoZVn-fwAg.png" /><figcaption>Multiagent framework</figcaption></figure><h3>Python implementation</h3><p>In this section, we explore the implementation details of an AI-powered gemba walk system using AutoGen, Ollama, and DeepSeek.</p><pre>import autogen<br><br>config_list = [<br>    {<br>        &quot;model&quot;: &quot;deepseek-r1:1.5b&quot;,<br>        &quot;base_url&quot;: &quot;http://localhost:11434/v1&quot;,<br>        &quot;api_key&quot;: &quot;ollama&quot;,<br>    }<br>]<br><br># Step 1: User Input Agent - Defines the problem/situation<br>user_input_agent = autogen.UserProxyAgent(<br>    name=&quot;User&quot;,<br>    system_message=&quot;Provide a brief description of the situation you want to optimize.&quot;,<br>    code_execution_config={&quot;work_dir&quot;: &quot;coding&quot;, &quot;use_docker&quot;: False}<br>)<br><br># Step 2: Planning Agent - Generates key questions to ask teams<br>planning_agent = autogen.AssistantAgent(<br>    name=&quot;Planner&quot;,<br>    system_message=&quot;Based on the given problem, generate key questions to ask different teams for better understanding.&quot;,<br>    llm_config={&quot;config_list&quot;: config_list},<br>)<br><br># Step 3: Questionnaire Agent - Collects responses from the user<br>questionnaire_agent = autogen.UserProxyAgent(<br>    name=&quot;Questionnaire&quot;,<br>    system_message=&quot;Ask the user to provide answers to the generated questions.&quot;,<br>    code_execution_config={&quot;work_dir&quot;: &quot;coding&quot;, &quot;use_docker&quot;: False}<br>)<br><br># Step 4: Summarization Agent - Synthesizes the collected responses<br>summarization_agent = autogen.AssistantAgent(<br>    name=&quot;Summarizer&quot;,<br>    system_message=&quot;Analyze the responses and summarize the key points to better understand the situation.&quot;,<br>    llm_config={&quot;config_list&quot;: config_list},<br>)<br><br># Step 5: Gemba Walk Agent - Conducts a virtual Gemba Walk analysis<br>gemba_walk_agent = autogen.AssistantAgent(<br>    name=&quot;GembaWalker&quot;,<br>    system_message=&quot;Using the summarized insights, conduct a virtual Gemba Walk to identify inefficiencies and process waste.&quot;,<br>    llm_config={&quot;config_list&quot;: config_list},<br>)<br><br># Step 6: Recommendation Agent - Suggests process improvements<br>recommendation_agent = autogen.AssistantAgent(<br>    name=&quot;Recommender&quot;,<br>    system_message=&quot;Based on the Gemba Walk findings, provide a set of process improvement recommendations.&quot;,<br>    llm_config={&quot;config_list&quot;: config_list},<br>)<br><br># Initialize GroupChat<br>group_chat = autogen.GroupChat(<br>    agents=[<br>        user_input_agent, planning_agent, questionnaire_agent,<br>        summarization_agent, gemba_walk_agent, recommendation_agent<br>    ],<br>    messages=[]<br>)<br><br># Define the manager to orchestrate communication<br>manager = autogen.GroupChatManager(groupchat=group_chat)<br><br># Step 1: Get user input<br>user_input = user_input_agent.initiate_chat(manager)<br><br># Step 2: Generate relevant questions<br>planning_agent.initiate_chat(manager, message=user_input)<br><br># Step 3: Get user responses to questions<br>questionnaire_agent.initiate_chat(manager)<br><br># Step 4: Summarize and analyze responses<br>summarization_agent.initiate_chat(manager)<br><br># Step 5: Conduct the virtual Gemba Walk<br>gemba_walk_agent.initiate_chat(manager)<br><br># Step 6: Provide recommendations for process improvements<br>recommendation_agent.initiate_chat(manager)</pre><h3>Metrics to track</h3><p>To justify the investment in an AI-powered gemba walk system, it’s essential to quantify its impact using both qualitative and quantitative metrics:</p><ul><li><strong>Process improvement efficiency:</strong> Measures how well the system identifies and reduces waste using lean methodology. Compare process cycle times and waste reduction percentages before and after AI implementation.</li><li><strong>Accuracy of insights:</strong> Assesses the validity of AI-generated insights compared to those identified by human experts.</li><li><strong>Adoption rate:</strong> Measures the percentage of teams regularly using the AI-powered gemba walk. A higher adoption rate indicates greater trust and usability.</li><li><strong>Business cost savings:</strong> Evaluates the financial benefits from improved efficiency and reduced waste, translating AI-driven improvements into measurable cost reductions.</li><li><strong>Reduction in process bottlenecks:</strong> Analyzes how effectively the AI system detects and resolves workflow bottlenecks, leading to smoother operations.</li><li><strong>Decision-making speed:</strong> Measures how quickly process improvement suggestions are implemented based on AI recommendations.</li></ul><h3>Limitations and drawbacks</h3><p>While an agentic AI-powered gemba walk offers automation and efficiency, it also comes with some challenges that businesses should consider:</p><p><strong>1. Lack of physical presence and real-world context</strong></p><ul><li>Traditional gemba walks rely on direct human observation to capture details like worker emotions, safety hazards, and environmental factors.</li><li>AI agents can analyze data, but they can’t fully replicate human senses — things like noise levels, workspace cleanliness, or physical obstacles may go unnoticed.</li></ul><p><strong>2. Dependence on data quality and team responses</strong></p><ul><li>The accuracy of AI insights depends on high-quality data. If the information collected is incomplete, biased, or inaccurate, the AI’s analysis may be misleading.</li><li>AI may miss implicit issues — problems that aren’t explicitly mentioned but are noticeable to a human observer.</li></ul><p><strong>3. Potential resistance from employees</strong></p><ul><li>Some employees may be hesitant to share honest feedback if they fear that AI-driven monitoring could lead to increased surveillance or concerns about job security.</li><li>AI-generated recommendations might not always be welcomed by teams who trust human judgment and leadership over machine-driven suggestions.</li></ul><h3>Potential next steps</h3><p>The AI-powered virtual gemba walk provides a structured and efficient approach to process optimization by automating data collection, analysis, and decision-making. Leveraging AutoGen, Ollama, and DeepSeek, we can build intelligent multi-agent systems that conduct gemba walks faster, smarter, and at scale. However, to further enhance gemba walk capabilities and impact, we can explore various scenarios, and so I have listed a few next steps:</p><h4>Integrate historical data for predictive analytics</h4><p><strong>Why:</strong></p><ul><li>Identifying trends and predicting future inefficiencies can help in proactive decision-making.</li></ul><p><strong>How:</strong></p><ul><li>Utilize time-series forecasting models (e.g., LSTMs, ARIMA) to detect patterns.</li><li>Implement anomaly detection using unsupervised learning for early warnings.</li><li>Incorporate historical machine downtime logs to anticipate failures.</li></ul><h4>Connect with Microsoft Teams APIs for automated feedback collection</h4><p><strong>Why:</strong></p><ul><li>Automating feedback loops from frontline workers enhances real-time data collection.</li></ul><p><strong>How:</strong></p><ul><li>Use the Microsoft Teams API to create a chatbot that collects responses asynchronously.</li><li>Auto-trigger surveys and feedback forms after each gemba walk.</li><li>Integrate sentiment analysis to gauge employee feedback.</li></ul><h4>Incorporate knowledge graphs for contextual AI reasoning</h4><p><strong>Why:</strong></p><ul><li>Understanding complex relationships between processes enhances AI-driven insights.</li></ul><p><strong>How:</strong></p><ul><li>Use Neo4j or RDF-based knowledge graphs to store process flows.</li><li>Enable AI to contextually reason about inefficiencies using past learnings.</li><li>Improve root cause analysis by linking issues across departments.</li></ul><h3>Summary</h3><p>In this article, we’ve explored how leveraging AI-driven multi-agent systems can revolutionize process optimization by conducting virtual gemba walks. Traditional gemba walks involve managers visiting the actual workplace to observe processes and identify inefficiencies. However, with advancements in agentic AI, businesses can automate data collection, analyze workflows, and generate actionable insights for continuous improvement. As we’ve seen throughout this article, the use of frameworks like AutoGen and reasoning models such as DeepSeek to build intelligent multi-agent systems that enhance decision-making and drive operational excellence.</p><p><strong><em>Mehul Jain</em></strong><em> is on </em><a href="https://www.linkedin.com/in/j13mehul/"><em>LinkedIn</em></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b39836f16301" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-at-microsoft/continuous-improvement-with-agentic-ai-conducting-a-virtual-gemba-walk-b39836f16301">Continuous improvement with agentic AI: Conducting a virtual gemba walk</a> was originally published in <a href="https://medium.com/data-science-at-microsoft">Data Science + AI at Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Fine-Tuning to Fusion: The Role of Agentic AI in Model Merging]]></title>
            <link>https://medium.com/@j13mehul/from-fine-tuning-to-fusion-the-role-of-agentic-ai-in-model-merging-f695a4252447?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/f695a4252447</guid>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[model-merging]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Sat, 08 Feb 2025 13:46:58 GMT</pubDate>
            <atom:updated>2025-02-08T13:46:58.749Z</atom:updated>
            <content:encoded><![CDATA[<p>After RAG, fine-tuning has never been the go-to method for improving models, but what if we could go beyond fine-tuning? This blog explores how Agentic AI can leverage model merging — a technique that blends multiple LLMs into a single, more powerful system. Let’s break down all this technical jargon and see how it can make AI systems smarter, more efficient, and more adaptable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XS6_TYcUDl1oJ6ZQ3BPMcw.png" /><figcaption>AI Generated Image from ChatGPT</figcaption></figure><p>Imagine an LLM that only understands technical details and another LLM that is good at conversation. Should we retrain or fine-tune?</p><p>If we fine-tune LLM 1, it will start forgetting the technical details, and vice versa. We need a method that can keep the models separate while still combining their capabilities to generate the best response.</p><p>Now, consider another scenario:</p><p>Even if we opt for fine-tuning, one LLM is fine-tuned on classic literature, screenplays, and narrative structures to craft engaging plots, while another model is fine-tuned on encyclopedic data and factual databases. The result would be a creative writer that also maintains fact-checking accuracy.</p><p>One <a href="https://arxiv.org/pdf/2406.07188"><em>Paper</em></a><em> leveraged </em>model merging approach to improve the self-critique capabilities of LLMs and defend against jailbreak attacks. The key idea is to merge an external critic model with the original LLM, strengthening its ability to identify and rewrite unsafe responses without requiring human-labeled data.</p><p>The author introduces <strong>three self-critique defense strategies</strong>:</p><ol><li><strong>Response Rewriting (RR): </strong>The LLM generates an initial response → critiques its own response → rewrites it to remove harmful elements.</li><li><strong>Response Rewriting with External Critic (RR-extcrit):</strong> Instead of self-critique, an external critic model evaluates the response and suggests safer revisions.</li><li><strong>Response Rewriting with Merging (RR-merge): </strong>Merges the base LLM and the critic model into a single model using linear weight interpolation, effectively integrating self-critique capabilities directly into the LLM. The merged model retains the strengths of both models without the computational overhead of running two models separately.</li></ol><h3>What is Model Merging</h3><p>Model merging, also called model fusion, is the process of combining the parameters of multiple models into a single, unified model. Unlike ensemble methods that combine predictions from multiple models at inference, model merging combines models at the parameter level and results in a single final model.</p><pre># Basic Example <br>import torch<br>from transformers import AutoModelForCausalLM, AutoTokenizer<br><br>def load_model(model_name: str):<br>    &quot;&quot;&quot;<br>    Load a model and its tokenizer from the Hugging Face hub.<br>    &quot;&quot;&quot;<br>    model = AutoModelForCausalLM.from_pretrained(model_name)<br>    tokenizer = AutoTokenizer.from_pretrained(model_name)<br>    return model, tokenizer<br><br>def merge_models(model1, model2, alpha: float = 0.5):<br>    &quot;&quot;&quot;<br>    Merge two models by taking a weighted average of their parameters.<br>    &quot;&quot;&quot;<br>    # Ensure both models have the same architecture<br>    state_dict1 = model1.state_dict()<br>    state_dict2 = model2.state_dict()<br>    <br>    # Create a new state dict for the merged model<br>    merged_state_dict = {}<br>    <br>    for key in state_dict1.keys():<br>        if key in state_dict2:<br>            # Merge the parameters using weighted average<br>            merged_state_dict[key] = alpha * state_dict1[key] + (1 - alpha) * state_dict2[key]<br>        else:<br>            raise ValueError(f&quot;Key {key} found in model1 but not in model2.&quot;)<br>    <br>    # Load a fresh instance of the model and load the merged state dict<br>    merged_model = type(model1).from_pretrained(None, config=model1.config)<br>    merged_model.load_state_dict(merged_state_dict)<br>    return merged_model<br><br>def main():<br>    model_name = &quot;gpt2&quot;  # using GPT-2 as an example<br>    alpha = 0.6          # weight for the first model; adjust as desired<br>    <br>    # Load two models (they could be fine-tuned differently or come from different checkpoints)<br>    model1, tokenizer = load_model(model_name)<br>    model2, _ = load_model(model_name)<br>    <br>    # Merge the models<br>    merged_model = merge_models(model1, model2, alpha=alpha)<br>    <br>    # Example usage: generate text with the merged model<br>    prompt = &quot;Once upon a time&quot;<br>    inputs = tokenizer(prompt, return_tensors=&quot;pt&quot;)<br>    <br>    # Generate text (adjust parameters as needed)<br>    output_sequences = merged_model.generate(<br>        input_ids=inputs.input_ids,<br>        max_length=50,<br>        temperature=0.7,<br>        num_return_sequences=1<br>    )<br>    <br>    generated_text = tokenizer.decode(output_sequences[0], skip_special_tokens=True)<br>    print(&quot;Generated text from merged model:\n&quot;, generated_text)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><h3><strong>Merging Methods</strong></h3><p>Model merging involves combining the parameters of multiple trained models into a single model, aiming to enhance performance, robustness, or efficiency while reducing computational overhead.</p><p>Below are some basis model merging techniques:</p><ol><li><strong>Basic Weight Averaging</strong>: Directly averages the weights of two or more models. Merging two models trained on the same dataset</li><li><strong>Weighted Model Averaging</strong>: Assigns different importance to models during merging. Merging models with different performance levels</li><li><strong>Linear Mode Connectivity (LMC) Merging</strong>: Finds a path in weight space where two models can be merged with minimal loss in performance. Merging models trained on different datasets</li><li><strong>Model Soups: </strong>Selectively combines weights from multiple models based on performance. Combining multiple fine-tuned models</li><li><strong>Subspace Merging: </strong>Projects model weights into a lower-dimensional space before merging. Efficient memory merging</li></ol><p>Below are some advanced model merging techniques:</p><h3>1. SLERP (Spherical Linear Interpolation)</h3><p>SLERP is a <strong>nonlinear interpolation method</strong> used to merge model weights while preserving geometric properties in the parameter space. Unlike linear interpolation, which averages weights in a straight line, SLERP finds a <strong>smooth curved path</strong> on a hypersphere.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*-DheQjAFr-2xgRvEMYFlKA.png" /></figure><p>where <strong>ω</strong> is the angle between the two weight vectors <strong>θ1​</strong> and <strong>θ2​</strong> in high-dimensional space.</p><p><strong>Pros:</strong><br>- More stable than simple linear interpolation.<br>- Reduces <strong>catastrophic forgetting</strong>.</p><p><strong>Cons:</strong><br>- Requires angle computation, making it computationally expensive.<br>- Not always compatible with all architectures.</p><h3>2. TIES (Task-Interference Elimination Strategies) Merging</h3><p>TIES is designed to <strong>resolve interference</strong> when merging models trained on <strong>different tasks</strong>. It tries to align weights such that the merged model retains the strengths of each individual model without losing performance in any task.</p><ul><li><strong>Step 1:</strong> Identify conflicting parameters between models (e.g., neurons that behave differently across tasks).</li><li><strong>Step 2:</strong> Adjust or realign weights to reduce interference.</li><li><strong>Step 3:</strong> Merge the models <strong>only in compatible parameter regions</strong> while preserving distinct task knowledge.</li></ul><p><strong>Pros:</strong><br>- Retains individual model strengths.<br>- Reduces performance loss due to conflicting weights.</p><p><strong>Cons</strong>:<br>- More computationally intensive than direct merging.<br>- Requires<strong> task similarity analysi</strong>s before merging.</p><h3>3. DARE (Divergence-Aware Residual Ensemble) Merging</h3><p>DARE tries to <strong>minimize the negative effects of merging models that have diverged significantly</strong> during training. It does this by identifying <strong>residual differences</strong> between models and adjusting for them before merging.</p><ul><li><strong>Step 1:</strong> Compare the weight distributions of two models.</li><li><strong>Step 2:</strong> Find the differences in parameter updates between models.</li><li><strong>Step 3:</strong> Instead of naive averaging, add a residual correction term to balance conflicting updates.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*iIOxtRVzLRkeqXYd53B_kA.png" /></figure><p>where <strong>α </strong>is a scaling factor determined by divergence measures.</p><p><strong>Pros:</strong><br>- More effective for <strong>combining highly divergent models</strong>.<br>- Balances knowledge retention while avoiding model collapse.</p><p><strong>Cons</strong>:<br>- Requires extra computation to calculate divergence metrics.<br>- Not always necessary for models that are already closely aligned.</p><h3>4. Passthrough Merge Method — Most Imp.</h3><p>Unlike standard merging, <strong>Passthrough Merge</strong> does not combine models at the weight level. Instead, it dynamically selects which model handles a given query during inference. <a href="https://huggingface.co/alpindale/goliath-120b">Goliath-120B</a> is one such model which is built by using 2 Llama models and <a href="https://github.com/arcee-ai/mergekit">Mergekit</a>.</p><ul><li><strong>Step 1:</strong> Determines which model is best suited for a given input.</li><li><strong>Step 2:</strong> The agent forwards the query to the selected model without altering the weights of any model.</li><li><strong>Step 3:</strong> If multiple models are relevant, their outputs are combined at the response level rather than merging parameters.</li></ul><p><strong>Pros:</strong><br>- Avoids destructive interference between models.<br>- Works well in <strong>real-time AI assistants</strong> and multi-domain AI systems.</p><p><strong>Cons</strong>:<br>- Doesn’t actually merge models — just routes queries dynamically.<br>- Requires a<strong> selection mechanis</strong>m (e.g., classifier or routing algorithm).</p><h3>When and When Not to Choose Model Merging Technique</h3><p>Model merging is a powerful technique, but it is not always the best choice. Whether to use it depends on factors like computational resources, model compatibility, and task-specific requirements.</p><h4>When to Choose?</h4><ul><li>When You Want to Combine Strengths of Multiple Models</li><li>When You Need a Unified, Single-Pass Model</li><li>When You Want to Improve Generalization</li><li>When You Want to Adapt a Model Without Full Retraining</li><li>When Task Interference is Manageable</li></ul><h4>When NOT to Choose?</h4><ul><li>When Models Have Highly Divergent Architectures</li><li>When Task Interference is Too High</li><li>When You Need Real-Time, Low-Latency Responses</li><li>When Merging Increases Computational Cost Instead of Reducing It</li><li>When Safety and Robustness are a Priority</li></ul><p>In the context of model merging, <strong>Agentic AI </strong>can play a crucial role in improving the efficiency, adaptability, and effectiveness of the merging process.</p><h3>A brief about Agentic AI —</h3><p>Agentic AI refers to AI systems that exhibit autonomy, decision-making, and self-improvement, enabling them to function independently in complex, dynamic environments. Unlike traditional AI models that require predefined instructions, Agentic AI can set goals, plan actions, learn from feedback, and adapt to new situations without human intervention.</p><h4>What Makes AI “Agentic”?</h4><ol><li><strong>Autonomy</strong> — The AI system can operate without constant human supervision.</li><li><strong>Goal-Oriented Planning</strong> — It defines its own sub-goals and plans actions to achieve them.</li><li><strong>Self-Improvement</strong> — It continuously learns from past experiences and adjusts its strategies.</li><li><strong>Decision-Making in Uncertainty</strong> — It can evaluate trade-offs and make decisions even with incomplete data.</li><li><strong>Adaptability</strong> — The system can dynamically change its behavior based on real-world feedback.</li></ol><h4>Core Components of Agentic AI —</h4><ol><li><strong>Perception Module</strong> — Captures raw data (text, images, audio) and extracts meaningful information.</li><li><strong>Memory &amp; Knowledge Base</strong> — Stores past interactions and knowledge to improve long-term learning.</li><li><strong>Tools, Planner &amp; Goal Management</strong> — Breaks down complex tasks into smaller, executable steps.</li><li><strong>Decision-Making Module</strong> — Evaluates different strategies using multi-objective optimization such as Markov Decision Processes</li><li><strong>Action Execution</strong> — Interacts with external systems (APIs, databases) to implement decisions.</li><li><strong>Feedback &amp; Self-Improvement</strong> — Uses reinforcement learning or self-distillation to fine-tune its own responses.</li></ol><h3>How Agentic AI Can Help Model Merging</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*yTlOhjwhHj40dJTiOows1w.png" /><figcaption>AI Generated Image from ChatGPT</figcaption></figure><h3>1. Intelligent Model Selection</h3><p>It is the ability of Agentic AI to autonomously analyze, compare, and decide which models should be merged for optimal performance. Instead of naively averaging all models, Agentic AI can strategically choose the best ones based on multiple factors, such as task relevance, knowledge diversity, and performance metrics.</p><ul><li><strong>Task Similarity Analysis: </strong>Before merging, Agentic AI needs to determine whether the models have complementary knowledge or if they are too different to be merged effectively. If Model A is fine-tuned for financial forecasting and Model B is trained for medical diagnostics, merging them without adaptation might degrade performance.</li><li><strong>Performance-Based Selection: </strong>Agentic AI can evaluate how well different models perform on relevant tasks before deciding which ones to merge. If an LLM is being merged for <strong>customer support</strong>, Agentic AI might test candidate models on: Response accuracy, Emotional intelligence, Handling difficult customers.</li><li><strong>Knowledge Redundancy &amp; Complementarity Check: </strong>Agentic AI ensures that merging models actually adds <strong>useful new knowledge </strong>instead of just duplicating existing capabilities. Two LLMs fine-tuned on news articles might contain highly overlapping knowledge. Merging them would not significantly improve performance. However, merging a legal expert model with a general knowledge LLM creates a more powerful model with specialized legal understanding.</li></ul><h3>2. Real-Time Model Routing</h3><p>Real-time model routing is a dynamic merging approach where an Agentic AI system decides which model (or combination of models) should handle a specific query at runtime, instead of permanently merging their weights. Instead of creating a single static model from multiple sources, the system intelligently selects and routes requests to the most relevant model based on input characteristics. This is Passthrough Merging, as models are not physically merged but routed dynamically based on their expertise.</p><ul><li><strong>Query Analysis</strong>: When a user submits a query, the Agentic AI Master Agent first analyzes it.</li><li><strong>Model Selection Strategy: </strong>Once the AI understands the query, it selects one or more models using one of the following strategies:</li></ul><p><strong>-&gt; Single Model Selection</strong> (if one model is the best fit)</p><ul><li>If the query is about Python coding, the system selects a Code-Specialized LLM.</li><li>If the query asks for a poem, it selects a Creative Writing LLM.</li></ul><p><strong>-&gt; Hybrid Routing</strong> (if multiple models contribute)</p><ul><li>For a query like <em>“Explain quantum computing in simple terms,”</em> the AI might:</li><li>Use a Technical Model to generate an accurate explanation.</li><li>Use a Conversational Model to simplify the response for clarity.</li></ul><p><strong>-&gt; Weighted Response Blending</strong> (if both models contribute equally)</p><ul><li>The AI merges the outputs of two models by giving each response a weight and generating a final reply.</li><li><strong>Execution &amp; Response Generation</strong>: The selected models generate responses, and if necessary AI filters and refines responses to align with safety guidelines. The system ranks multiple outputs to ensure the highest-quality response is shown.</li></ul><h3>3. Continuous Improvement &amp; Feedback Learning</h3><p>One of the biggest challenges in AI model merging is ensuring that the merged model continues to improve over time rather than becoming static. Agentic AI enables a system to evolve autonomously, leveraging feedback loops and self-learning techniques to enhance performance without manual retraining.</p><p>Agentic AI uses a feedback-driven approach where it:</p><ul><li><strong>Monitors Responses &amp; User Interactions : </strong>Every interaction with the AI model is analyzed to determine whether the response was accurate, relevant, and useful.</li><li><strong>Identifies Weaknesses in the Merged Model: </strong>If certain task-specific performance drops, Agentic AI detects which part of the merged model is responsible.</li><li><strong>Dynamically Adjusts Model Weighting: </strong>If feedback suggests that one component (e.g., language fluency) is improving while another (e.g., factual accuracy) is degrading, Agentic AI can re-weight the contribution of different models in the merged system.</li></ul><h4>Let&#39;s breakdown a simple workflow —</h4><ol><li><strong>Query Ingestion and Preprocessing: </strong>The master agent receives a query. Information such as user history, location, time, and previous interactions is gathered to provide context.</li><li><strong>Intent and Context Analysis: </strong>A lightweight classifier analyzes the query to determine its intent. For example, it might decide whether the query is technical, emotionally charged, or general. The system outputs a confidence score for each identified intent (e.g., 60% technical, 30% empathetic, 10% general).</li><li><strong>Mapping to Specialized Models: </strong>Each intent is associated with one or more specialized models. For instance: Routes to a Technical Troubleshooting Model or Medical Diagnostic Model (depending on the domain) or routes to a General Conversational Model.</li><li><strong>Dynamic Merging: </strong>If the classifier yields a strong confidence for a single intent, the query is forwarded to that specialized model. If the intent is mixed or borderline (e.g., both technical and Conversational aspects are high), the master agent calculates a weighted average to merge the outputs of the relevant models.</li><li><strong>Final Response Selection: </strong>If multiple model outputs are produced (for instance, when running parallel inferences for validation), the master agent aggregates or selects the best response based on predefined criteria (like confidence levels or ranking).</li><li><strong>Feedback Loop: </strong>User feedback can be recorded to improve future model selection decisions.</li></ol><h3>Conclusion</h3><p>Model merging powered by Agentic AI represents a significant leap beyond fine-tuning, enabling AI systems to become more adaptive, efficient, and self-improving. By intelligently combining multiple models, adjusting weights dynamically, and continuously learning from feedback, AI can achieve higher accuracy, robustness, and versatility across different tasks.</p><p>However, despite its promise, model merging is still in the experimental stage and comes with several challenges:</p><ul><li><strong>High Computational Costs:</strong> Merging models — especially using advanced techniques like <strong>TIES, SLERP, or DARE</strong> — requires significant GPU/TPU resources, making it expensive to deploy at scale.</li><li><strong>Increased System Complexity:</strong> Implementing dynamic routing and adaptive weighting adds engineering overhead, requiring sophisticated decision-making frameworks.</li><li><strong>Latency Issues:</strong> While model merging reduces the need to run multiple models in parallel, real-time adjustments and dynamic inference add processing delays, which may not be suitable for latency-sensitive applications like chatbots or autonomous systems.</li></ul><h4>References —</h4><ol><li><a href="https://arxiv.org/pdf/2408.07666">https://arxiv.org/pdf/2408.07666</a></li><li><a href="https://arxiv.org/pdf/2406.07188">https://arxiv.org/pdf/2406.07188</a></li><li><a href="https://github.com/arcee-ai/mergekit">https://github.com/arcee-ai/mergekit</a></li></ol><p>Thanks for spending your time on this blog. I am open to suggestions and improvement. Please let me know if I missed any details in this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f695a4252447" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2024 was mostly about RAG]]></title>
            <link>https://medium.com/@j13mehul/2024-was-mostly-about-rag-c744bd0a2654?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/c744bd0a2654</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[rag-system]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[llmops]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 13:58:33 GMT</pubDate>
            <atom:updated>2024-12-06T15:13:27.376Z</atom:updated>
            <content:encoded><![CDATA[<p>In the age of information abundance, producing accurate, context-aware responses has become the key challenge for generative AI models. This is where <a href="https://medium.com/@j13mehul/rag-part-1-from-naive-to-advanced-cb40674a7738"><em>Retrieval-Augmented Generation (RAG)</em></a> steps in, combining the strength of large language models with retrieval mechanisms to create systems that are more relevant, responsive, and grounded in real-world knowledge.</p><p>Unlike traditional language models, which rely solely on pre-trained knowledge, RAG models dynamically retrieve and incorporate external information from databases, documents, or APIs in real time. By doing so, they not only generate more factually accurate responses but also keep up to date with the latest data — a critical advantage in rapidly evolving domains like medicine, technology, and customer support.</p><p>The methods behind <a href="https://medium.com/@j13mehul/rag-part-1-from-naive-to-advanced-cb40674a7738"><em>RAG</em></a> are diverse, reflecting a range of goals from improving retrieval relevance to optimizing generation coherence. Some approaches emphasize enhancing retrieval to find more contextually aligned information, while others fine-tune the generation process to handle retrieved content more effectively. Whether aiming for scalability, speed, accuracy, or knowledge depth, each RAG approach brings unique strengths to the table.</p><p>In this blog, I explored 12 distinct RAG approaches of 2024, examining how each one works, its novelty, and how it addresses specific challenges in retrieval-augmented generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*sas2oGc_XlmVh4g5ZRMIiA.png" /></figure><h3><strong>1.</strong> <strong>Corrective RAG</strong></h3><p>Jan 2024</p><h4><strong>Summary</strong></h4><p>Corrective Retrieval Augmented Generation (CRAG) enhances the text generation capabilities of large language models (LLMs) by improving the relevance and accuracy of retrieved information. Unlike traditional Retrieval-Augmented Generation (RAG) methods, which may introduce outdated or irrelevant content, CRAG utilizes a retrieval evaluator to refine or discard document sources before they inform the model’s response. Additionally, CRAG leverages large-scale web searches to supplement static data sources and employs a decompose-then-recompose algorithm to ensure only essential knowledge is retained. This process allows CRAG to produce higher-quality outputs in both short and long-form generation tasks, supported by consistent accuracy improvements in experimental evaluations.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Related work</strong>: Self-RAG uses a critic model for selective retrieval, NLI models filter irrelevant contexts, SAIL incorporates retrieved documents before instructions, and Toolformer is pre-trained for API calls like Wikipedia. These emphasize the importance of strategically retrieving knowledge, especially in long-form generation.</blockquote><blockquote><strong>Retrieval Evaluator</strong>: CRAG’s evaluator assesses document quality before use in generation, initialized with T5-large and fine-tuned. It is lightweight (0.77 billion parameters), significantly smaller than most LLMs.</blockquote><blockquote><strong>Evaluation Datasets</strong>: CRAG was tested on PopQA (short-form), Biography (long-form), PubHealth (true/false), and Arc-Challenge (multiple choice), using accuracy and FactScore metrics.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*tc6F2g_GbhacNgWnWBKEzw.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2401.15884"><strong>Corrective RAG</strong></a></figcaption></figure><h3>2. <strong>RAPTOR</strong></h3><p>Jan 2024</p><h4><strong>Summary</strong></h4><p>RAPTOR (Recursive and Progressive Tree Of Retrievals) introduces an innovative hierarchical retrieval system designed to process long documents and answer complex questions efficiently. By recursively embedding, clustering, and summarizing chunks of text into a tree structure, RAPTOR enables multi-level abstraction and flexible querying. Its collapsed tree approach consistently outperforms alternatives, offering superior granularity for diverse questions. RAPTOR demonstrates significant improvements in question-answering and long-document tasks, as validated through experiments on datasets like NarrativeQA, QASPER, and QuALITY.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Related work</strong>: Progression from traditional term-based techniques (e.g., TF-IDF, BM25) to LLM bases approaches like<strong>: </strong>Fusion-in-Decoder (FiD): Uses DPR and BM25 to retrieve and process passages independently. RETRO: Employs chunkwise retrieval with cross-chunk attention. Dense and Hybrid Hierarchical Retrieval (DHR/HHR): Enhance accuracy by combining sparse and dense retrieval at document and passage levels.</blockquote><blockquote><strong>Introduces two querying strategies: <em>Tree Traversal</em>,</strong> sequentially traverses layers to select relevant nodes whereas <em>Collapsed Tree</em>, Searches across all layers simultaneously for optimal granularity.</blockquote><blockquote><strong>Algorithm used: </strong>Used UMAP for dimensionality reduction, and GMM for clustering by adjusting the n_neighbors parameter to balance local and global structure preservation during clustering. Employs a two-step clustering process to identify global themes and refine them into specific details.</blockquote><blockquote><strong>Evaluation: </strong>Tested on QASPER with the collapsed tree method outperforming tree traversal due to its flexibility in retrieving appropriately granular information. Assessed performance on NarrativeQA, QASPER, and QuALITY datasets using BLEU, ROUGE, and METEOR metrics.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*hl59eAYn6E16HCk4zz9deg.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2401.18059v1"><strong>RAPTOR</strong></a></figcaption></figure><h3><strong>3.</strong> <strong>Adaptive RAG</strong></h3><p>Mar 2024</p><h4>Summary</h4><p>Adaptive RAG<strong> </strong>designed to optimize the use of retrieval-augmented large language models (LLMs) by dynamically selecting the best approach for handling queries based on their complexity. Unlike existing methods that either inefficiently address simple queries or fall short on complex, multi-step ones, this framework employs a classifier (a smaller LLM) to determine the query complexity and adapt the retrieval strategy accordingly. Queries are categorized into three levels: non-retrieval-based for simple queries, single-hop for moderately complex queries, and multi-hop for intricate, multi-step ones.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Related work</strong>: Decompose multi-hop queries into simpler steps or apply repetitive retrieval cycles (e.g., Chain-of-Thought reasoning with document retrieval). Previous approaches also used frequency-based heuristics or fixed iterations of retrieval and ranking, often leading to inefficiencies</blockquote><blockquote><strong>Adaptive QA Framework Design: </strong>Dynamically selects the optimal retrieval strategy for LLMs based on query complexity. The classifier classifies the query to ‘A’ for simplest non-retrieval-based approach, ‘B’ to queries in single-hop query and ‘C’ to queries in multi-hop query:</blockquote><blockquote><strong>Classifier: </strong>T5-Large model trained to classify query complexity. Labels generated using model predictions and dataset properties.</blockquote><blockquote><strong>Datasets: </strong>For<strong> </strong>Single-hop QA, SQuAD v1.1, Natural Questions, TriviaQA and for Multi-hop QA MuSiQue, HotpotQA, 2WikiMultiHopQA.</blockquote><blockquote><strong>Evaluation Metrics: </strong>Measured <strong>Effectiveness </strong>using F1, Exact Match (EM), and Accuracy and <strong>Efficiency</strong> by retrieval-and-generate steps and average time per query compared to single-step methods.</blockquote><blockquote><strong>LLM Models Utilized: </strong>FLAN-T5 (XL: 3B parameters, XXL: 11B parameters), GPT-3.5 model for answer generation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*r06qLwJTWyX1hqQPdEJvyQ.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2403.14403"><strong>Adaptive RAG</strong></a></figcaption></figure><h3><strong>4.</strong> <strong>Graph RAG</strong></h3><p>Apr 2024</p><h4>Summary</h4><p>While RAG is useful for answering localized queries, it falls short when comprehensive understanding across large datasets is needed. To address this, the paper introduces Graph RAG, a query-focused summarization (QFS) approach leveraging an LLM-derived knowledge graph. By structuring documents into entity-based communities and generating summaries for these communities, Graph RAG enables global summarization. Its two-stage process — constructing a knowledge graph and pre-generating community summaries — ensures a balance of comprehensiveness and token efficiency. This method outperforms traditional RAG approaches, providing diverse and detailed answers to complex queries.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Query-Focused Summarization (QFS)</strong>: Generates natural language summaries instead of extracting text snippets. Implements a map-reduce approach by creating independent community-level summaries are aggregated into a cohesive global answer.</blockquote><blockquote><strong>Graph RAG Workflow</strong>: Utilizes <strong>LLM-derived knowledge graphs</strong> to structure documents into communities through the <strong>Leiden algorithm</strong>. Community summaries are generated using <em>Leaf-Level Communities</em> which Prioritizes key nodes, edges, and covariates and <em>Higher-Level Communities</em> which Ranks and condenses sub-community summaries to fit within token limits.</blockquote><blockquote><strong>Global Answer Generation</strong>: The process involves preparing community summaries by dividing them into token-sized chunks to distribute information evenly. Intermediate answers are generated for each chunk, scored for helpfulness (0–100), and low-scoring answers are filtered out. The highest-scoring answers are combined within a token limit to form a global answer for the user.</blockquote><blockquote><strong>Evaluation</strong>: The evaluation used two datasets — podcast transcripts featuring conversations with tech leaders and a news articles benchmark spanning diverse categories. Using an activity-centered approach, 125 high-level test questions per dataset were generated to assess global sensemaking tasks. Four metrics — comprehensiveness, diversity, empowerment, and directness — were used to evaluate the answers, focusing on detail, perspective, clarity, and user understanding.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*aTFqkT4DTvXaUzUHWG7RbA.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2404.16130"><strong>Graph RAG</strong></a></figcaption></figure><h3><strong>5.</strong> <strong>GNN-RAG</strong></h3><p>May 2024</p><h4><strong>Summary</strong></h4><p>Knowledge Graphs (KGs) excel at storing structured information in a way that captures the rich, interwoven relationships between entities. This study introduces GNN-RAG, an innovative approach that combines the language understanding of LLMs with the reasoning strength of Graph Neural Networks (GNNs) for Knowledge Graph Question Answering (KGQA). GNN-RAG employs a GNN to identify and reason over subgraphs, pinpointing answer candidates by leveraging shortest paths between question entities and potential answers within the graph. These paths are then verbalized and processed through an LLM to refine the answers. This dual-strength framework achieves state-of-the-art performance on major KGQA benchmarks, demonstrating exceptional accuracy, particularly for complex, multi-step reasoning tasks.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Related work: </strong>KGQA methods are categorized into Semantic Parsing (SP) and Information Retrieval (IR). SP methods convert questions into logical queries like SPARQL but require extensive annotation and may produce errors. IR methods use weak supervision with question-answer pairs, retrieving KG subgraphs for reasoning. GNN-RAG combine Graph Neural Networks with Retrieval-Augmented Generation for better KGQA performance.</blockquote><blockquote><strong>Reasoning Paths with LLMs</strong>: GNNs classify nodes as answers vs. non-answers, optimizing for multi-hop question answering. Extracts and verbalizes reasoning paths as “question entity → relation → target entity” for LLM input. LLMs, fine-tuned with prompt tuning, use reasoning paths to generate final answers. Beam-search decoding is used to generate k diverse sets of reasoning paths for better answer coverage</blockquote><blockquote><strong>Retrieval Capabilities:</strong> Deep GNNs (L=3) excel in multi-hop reasoning by leveraging complex graph structures and increasing answer coverage while reducing input tokens. Shallow GNNs are more suited for single-hop questions requiring accurate question-relation matching.</blockquote><blockquote><strong>Evaluation</strong>: GNN-RAG outperforms or matches leading models, including GPT-4 fine-tuned on 7B parameters, on benchmarks like WebQSP and CWQ, especially excelling on multi-hop and multi-entity questions with high F1 scores.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*jTKj_vqDG0sSRdcHLbfANA.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2405.20139"><strong>GNN-RAG</strong></a></figcaption></figure><h3><strong>6.</strong> <strong>Multi-Head RAG</strong></h3><p>Jun 2024</p><h4>Summary</h4><p>MRAG (Multi-aspect Retrieval-Augmented Generation) introduces a novel approach to enhancing retrieval in RAG pipelines by leveraging the activations from multi-head attention layers of decoder models. Unlike traditional methods that rely on single embeddings, MRAG creates multiple embeddings from attention heads, capturing diverse aspects of the data without increasing space requirements. This allows for better multi-aspect understanding and retrieval accuracy. Comprehensive evaluations demonstrate MRAG’s superiority over baseline methods, such as Standard RAG, with significant improvements in retrieval success for both exact matches and category matches. Real-world applications, including legal document synthesis and chemical accident analysis, validate its effectiveness across diverse workloads. Furthermore, MRAG achieves these gains without additional computational costs, making it both scalable and efficient. The methodology includes a flexible pipeline for data preparation and query execution, robust evaluation metrics, and compatibility with synthetic and real-world datasets. With up to a 20% performance boost over traditional methods, MRAG sets a new benchmark in retrieval-augmented generation.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Evaluation: </strong>Compared Standard RAG which uses a single embedding vector from the last decoder layer and Split RAG which divides the last decoder activations into separate spaces, applying voting to aggregate results.</blockquote><blockquote><strong>Embedding Models:</strong> Two embedding models from the MTEB leaderboard were tested the SFR-Embedding-Model and the e5-mistral-7b-instruct, both based on the Mistral 7B architecture with 32 decoder blocks and 32 attention heads per multi-head attention.</blockquote><blockquote><strong>Multi-Aspect: </strong>To evaluate MRAG’s performance on multi-aspect queries, a framework was developed using Wikipedia-based datasets spanning 25 categories, queries combining information from multiple categories, and novel metrics. Metrics like Retrieval Success Ratio and Category Retrieval Success Ratio assess MRAG’s ability to retrieve relevant documents or category-aligned ones.</blockquote><blockquote><strong>Performance: </strong>On<strong> </strong>exact document matches, MRAG improves retrieval success ratio by over 10% on average compared to Standard RAG. On category matches, Offers a 25% performance boost, highlighting better multi-aspectual understanding.</blockquote><blockquote><strong>Fusion MRAG:</strong> Combines MRAG with Fusion RAG for LLM-enhanced retrieval, achieving 10–30% accuracy gains over Standard RAG.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*uF22c1JXIYDw_O49dzW4sg.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2406.05085v1"><strong>Multi-Head RAG</strong></a></figcaption></figure><h3><strong>7.</strong> <strong>Speculative RAG</strong></h3><p>Jul 2024</p><h4>Summary</h4><p>Speculative RAG introduces a groundbreaking approach to Retrieval-Augmented Generation (RAG) by integrating a two-step process with specialized models for drafting and verification. At its core, the framework clusters retrieved documents based on content similarity, sampling one document from each cluster to create diverse and non-redundant subsets. These subsets are then processed by a smaller, specialized model — dubbed the RAG Drafter — to generate draft answers with detailed rationales in parallel. In the second step, a larger, generalist model — the RAG Verifier — validates these drafts by assessing their rationale and reliability. This efficient delegation allows the system to bypass the exhaustive review of repetitive documents, significantly reducing latency while improving the quality of responses. By combining content diversity, parallel processing, and focused verification, Speculative RAG achieves superior performance compared to traditional RAG systems. It sets a new standard for delivering faster, more accurate, and contextually rich answers, making it an innovative tool for advancing AI-driven information retrieval and generation.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Algorithm Highlights</strong>: Cluster (K-means clustering &amp; instruction aware embedding Model) retrieved documents into subsets representing different perspectives. Generate answer drafts and rationales for each subset using the specialist model (MDrafter — Mistral7Bv0.1). Verify drafts with the generalist model (MVerifier — Mistral7Bv0.1/Mixtral8x7Bv0.1) based on self-consistency and self-reflection scores. Select and integrate the best draft into the final answer.</blockquote><blockquote><strong>Performance Gains</strong>: Outperforms baselines like standard RAG, Self-Reflective RAG, and Corrective RAG on benchmarks such as TriviaQA, MuSiQue, PubHealth, and ARC-Challenge. Parallel draft generation reduces latency by up to 51.25% while enhancing the accuracy by ~13%.</blockquote><blockquote><strong>Evaluation Scores</strong>: Self-consistency score measures alignment of the draft with the question and Self-reflection score evaluates the draft’s reliability via self-assessment prompts.</blockquote><blockquote><strong>Limitations</strong>: Requires additional training for the specialist drafter model, adding some complexity.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*Q_TPTIqUzkQ_YmpIUrqmlA.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2407.08223"><strong>Speculative RAG</strong></a></figcaption></figure><h3><strong>8.</strong> <strong>Efficient RAG</strong></h3><p>Aug 2024</p><h4><strong>Summary</strong></h4><p>EfficientRAG introduces a novel approach to multi-hop question answering by addressing the challenges retrieval-augmented generation (RAG) methods face in handling complex, multi-step queries. It is a plug-and-play retriever designed to optimize multi-hop question answering by iteratively refining queries without frequent LLM calls. It employs two core modules — Labeler &amp; Tagger and Filter — to classify tokens, annotate relevant chunks, and generate subsequent queries. These iterative refinements enrich the retrieved information, improve response accuracy, and significantly reduce latency compared to traditional methods. The system uses synthetic data generated by LLMs for training and fine-tuning its components, resulting in high retrieval performance across multiple datasets. Experimental results highlight EfficientRAG’s adaptability, superior recall, and enhanced efficiency in both in-domain and out-of-domain settings.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Synthetic Data Construction:</strong> Uses LLMs to decompose multi-hop questions into single-hop sub-questions. Annotates important tokens in chunks via Spacy. Generates negative samples by retrieving similar but irrelevant chunks.</blockquote><blockquote><strong>Training</strong>: Employs DeBERTa-v3-large for both Labeler and Filter tasks. Fine-tunes these models for token labeling and chunk filtering, projecting embeddings into binary classifications.</blockquote><blockquote><strong>Experiments: </strong>Demonstrated high accuracy on HotpotQA and 2WikiMQA, outperforming baselines with significant accuracy improvements (+6 points on 2WikiMQA).</blockquote><blockquote><strong>Performance: </strong>High Recall@K on multi-hop datasets (81.84% for HotpotQA, 84.08% for 2WikiMQA) with minimal chunks retrieved. Reduced iterations and improved latency by 60%-80% over traditional RAG methods while maintaining GPU efficiency.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*uAU5RiAoDLmY7gHU5V3XBw.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2408.04259"><strong>Efficient RAG</strong></a></figcaption></figure><h3><strong>9.</strong> <strong>Memo RAG</strong></h3><p>Sep 2024</p><h4><strong>Summary</strong></h4><p>MemoRAG is an innovative Retrieval-Augmented Generation (RAG) paradigm designed to overcome the limitations of traditional RAG systems in handling ambiguous queries and unstructured or distributed knowledge. By introducing a memory module and a dual-system architecture, MemoRAG creates a global, compact memory of the database to generate task-specific clues. These clues guide retrievers to locate relevant information, enabling effective handling of long contexts and complex queries. Experiments on various benchmarks, including the ULTRADOMAIN dataset, highlight MemoRAG’s superior performance across diverse domains and tasks like QA, summarization, and personalized recommendations.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Memory Model</strong>: Employs light, long-range language models (LLMs) like memorag-qwen2–7b-inst and memorag-mistral-7b-inst based on Qwen2–7B-Instruct and Mistral-7B-Instruct-v0.2, respectively.</blockquote><blockquote><strong>Global Memory Formation</strong>: Encodes input sequence X into a compact global memory representation Xm using token compression ratios (ranging from 2 to 16) to process up to 2M tokens (e.g., 128K × 16 tokens for memorag-qwen2–7b-inst).</blockquote><blockquote><strong>Task-Specific Clues</strong>: Memory-generated clues outline expected answers and guide retrieval methods such as sparse retrieval, dense retrieval (default), or reranking.</blockquote><blockquote><strong>Pre-Training</strong>: Utilizes RedPajama dataset for learning memory formation from randomly sampled long contexts.</blockquote><blockquote><strong>Supervised Fine-Tuning (SFT)</strong>: Trained on task-specific datasets to optimize clue generation.</blockquote><blockquote><strong>ULTRADOMAIN Dataset</strong>: 428 English college textbooks covering 18 domains (e.g., philosophy, mathematics, natural sciences). Contexts span up to 1M tokens.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*Kktu5Ru6AbzLoT7upDY4LA.png" /><figcaption>Source: <strong>1.</strong> <a href="https://arxiv.org/pdf/2409.05591v2"><strong>Memo RAG</strong></a></figcaption></figure><h4><strong>10.</strong> <a href="https://arxiv.org/pdf/2410.02719"><strong>Uncertainty RAG</strong></a></h4><p>Oct 2024</p><h4>Summary</h4><p>UncertaintyRAG leverages a Signal-to-Noise Ratio (SNR)-based span uncertainty technique to enhance model calibration and retrieval performance in long-context RAG tasks. By addressing the semantic inconsistencies introduced by random chunking and proposing an efficient unsupervised learning framework, UncertaintyRAG achieves state-of-the-art performance with minimal data. The method integrates smoothly with LLMs, requiring no fine-tuning, making it both scalable and robust under distribution shifts.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Core Idea: </strong>Introduces a Signal-to-Noise Ratio (SNR)-based span uncertainty technique to estimate similarity between text chunks.</blockquote><blockquote><strong>Unsupervised Learning: </strong>Trains a retrieval model that outperforms open-source embedding models like BGE-M3 and MiniLM in long-context RAG tasks.</blockquote><blockquote><strong>Data Sampling &amp; Scaling: </strong>HotpotQA, MultiFieldQA, Qasper, NarrativeQA, QMSum (Total: 178,755 chunks).</blockquote><blockquote><strong>Anchor Sample Scaling: </strong>Chunks clustered using KNN into k clusters. c chunks selected from each cluster, scaling to 50×c chunks total.</blockquote><blockquote><strong>Positive/Negative Scaling: </strong>For each anchor chi, M chunks are scored with BM25, selecting top m (positive) and bottom m (negative) chunks. Generates triplets (chi,chi+,chi−) for contrastive training.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*OgouLz9RQpEIocpJzye14Q.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2410.02719"><strong>Uncertainty RAG</strong></a></figcaption></figure><h3><strong>11.</strong> <strong>Probing-RAG</strong></h3><p>Oct 2024</p><h4><strong>Summary</strong></h4><p>Probing-RAG incorporates a lightweight <strong>prober model</strong> into the RAG pipeline to evaluate the necessity of additional retrieval during answer generation. By leveraging intermediate hidden states of the LLM and training the prober with labeled examples generated through <strong>Chain-of-Thought (CoT)</strong> prompting, Probing-RAG achieves superior performance compared to existing adaptive retrieval methods while minimizing redundant retrieval calls. Its key contributions include effective retrieval decision-making, reduced computational overhead, and robust performance even with limited training data.</p><h4><strong>Key Points</strong></h4><blockquote><strong>Concept: </strong>Decision using a <strong>prober model</strong> trained on <strong>LLM hidden states</strong>.</blockquote><blockquote><strong>Prober Design: </strong>Implemented as a feed-forward neural network with one hidden layer and an output layer for binary classification. Integrated with Gemma-2B, an 18-layer open-source LLM. Operates on hidden states at even-numbered layers, starting from the 6th layer. Input to prober consists of normalized hidden states (Tlk′′​) of tokens corresponding to the rationale (r) and answer (a^). Prober training minimizes cross-entropy loss during classification.</blockquote><blockquote><strong>Training Dataset:</strong> Constructed using three open-domain QA datasets — HotpotQA, NaturalQA and TriviaQA</blockquote><blockquote><strong>Labeling Methodology:</strong> Based on Chain-of-Thought (CoT) prompting, labels assigned as — y=1: Retrieval unnecessary (correct a^) and y=0: Retrieval necessary (incorrect a^).</blockquote><blockquote><strong>Evaluation and Efficiency: </strong>Probing-RAG achieves a <strong>6.59%</strong> improvement over no-retrieval and an <strong>8.35%</strong> improvement over single-step approaches. Probing-RAG performs 1.17x retrieval calls, significantly fewer than DRAGIN (6.83x) and others like Adaptive-RAG (1.54x) and FLARE (2.67x).</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*s3hOH3p4Wxoj9idwM45Wnw.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2410.13339"><strong>Probing-RAG</strong></a></figcaption></figure><h3><strong>12.</strong> <strong>CORAG</strong></h3><p>Nov 2024</p><h4><strong>Summary</strong></h4><p>Retrieval-Augmented Generation (RAG) systems face three critical challenges: overlooking inter-chunk correlations, non-monotonic utility of chunks, and the unique handling needs of diverse query types. To address these, CORAG introduces a Monte Carlo Tree Search (MCTS)-based policy framework to optimize chunk combinations under cost constraints. CORAG employs dynamic configurations tailored to queries using a contrastive learning-based configuration agent. Its architecture achieves substantial improvements in retrieval relevance, efficiency, and scalability, as demonstrated on datasets like WikiPassage QA and MARCO</p><h4><strong>Key Points</strong></h4><blockquote><strong>Monte Carlo Tree Search (MCTS): </strong>Core algorithm for sequential optimization of chunk combinations.</blockquote><blockquote><strong>Cost Constraints: </strong>Integrated into the utility function to ensure efficiency under token usage limits.</blockquote><blockquote><strong>Datasets and Evaluation Metrics: </strong>WikiPassageQA and MS MARCO were used and ROUGE scores for retrieval relevance evaluation<strong>.</strong></blockquote><blockquote><strong>Policy Tree: </strong>Root node is an empty state; child nodes represent chunk combinations. Combines benefit (via similarity scores from LLM) and cost (token usage). Utilizes the Upper Confidence Bound (UCB) algorithm to balance exploration (search count) and exploitation (node value).</blockquote><blockquote><strong>Configuration Inference: </strong>BGE-M3 generates 1024-dimensional embeddings for query representation. Learns MCTS parameters (iterations, cost coefficient, exploration coefficient) using a Siamese neural network with three fully connected layers.</blockquote><blockquote><strong>Optimized via a joint loss function combining: </strong>Classification loss (cross-entropy) for reranker prediction. Regression loss (mean squared error) for MCTS parameter tuning. Contrastive loss for embedding differentiation.</blockquote><blockquote><strong>Performance Results: </strong>CORAG achieved a 25% improvement in ROUGE scores over NaiveRAG and RAPTOR.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Fw8QyVWD_xtpUd3CRpFWoA.png" /><figcaption>Source: <a href="https://arxiv.org/pdf/2411.00744"><strong>CORAG</strong></a></figcaption></figure><h3><strong>Conclusion</strong></h3><p>As we look back on the advancements and breakthroughs of the current year, it’s clear that we are on the cusp of even greater transformations in technology, innovation, and industry practices. 2024 has proven to be a pivotal year, laying the groundwork for the future of automation, AI, and digital transformation. With each stride forward, businesses and individuals alike are experiencing unprecedented shifts in how work is conducted, how solutions are crafted, and how the world interacts with technology.</p><p>As we move into 2025, we can expect the next wave of innovations to center around even more refined AI models, with an emphasis on ethical considerations, increased automation across various sectors, and smarter, more intuitive tools for both business and personal use. One area of focus will likely be AI-driven decision-making Agents that can better understand and predict human needs (Hyper personalization) while maintaining a balance with privacy and security concerns. The integration of AI into daily workflows, especially through assistive AI like digital twin, will likely be at the forefront of many industries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c744bd0a2654" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoding signals from unstructured text: The AI way]]></title>
            <link>https://medium.com/data-science-at-microsoft/decoding-signals-from-unstructured-text-the-ai-way-cc2242c95a83?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc2242c95a83</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[small-language-model]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Tue, 03 Dec 2024 08:18:12 GMT</pubDate>
            <atom:updated>2024-12-03T08:18:12.498Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By </em><a href="https://www.linkedin.com/in/aparanagupta/"><em>Aparana Gupta</em></a><em> and </em><a href="https://www.linkedin.com/in/j13mehul/"><em>Mehul Jain</em></a></p><p>In today’s fast-paced business environment, data is the cornerstone of progress and innovation. By leveraging advanced analytics, artificial intelligence, and emerging technologies, businesses can enhance decision-making, optimize operations, uncover opportunities, and transform challenges into successes. A strategic data approach involves understanding and processing various data types — structured, unstructured, and semi-structured — to derive meaningful insights. This article presents a case study on processing unstructured text data from various sources to generate distilled intuitive and actionable insights.</p><p>Moreover, it underscores the myriad challenges encountered during manual data processing, including bias, privacy concerns, and inefficiencies, while also emphasizing the numerous benefits of using technology to significantly reduce processing time and improve strategic decision-making.</p><p>The case study constituted an opportunity to design a state-of-the-art AI solution that extracts signals from unstructured text, gleans actionable insights, and distills summaries highlighting key themes, topics, and sentiments augmented with and distilled through an insight navigation tree. Our novel approach encompasses a six-step process to present condensed, precise, and meaningful actionable insights incrementally while also preserving their essence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZuzxMe64d8v4rPWjmDJilQ.png" /><figcaption><em>Figure 1: Six-step process for decoding signals from unstructured text.</em></figcaption></figure><p>We describe each component in turn below.</p><h3><strong>1. Data collection and preprocessing</strong></h3><p>As we embarked on the process of batch processing to source and ingest unstructured text datasets, we explored a multitude of techniques to handle and transform the raw data into a structured format suitable for analysis. This was crucial for ensuring that the data was clean, coherent, and ready for further processing.</p><p>Our batch processing setup involved the use of Small Language Models (SLMs) like <a href="https://huggingface.co/microsoft/Phi-3.5-mini-instruct">Phi 3.5</a> to handle challenges such as ambiguous sentences, emoji conversion, and slang.</p><h4><strong>Ambiguous sentences</strong></h4><p><strong>Purpose</strong>: We leveraged SLMs to refine the text, remove ambiguity, and enhance coherence and overall quality through rewriting, ensuring the text was well structured and easier to understand for further analysis. This approach was beneficial for text containing reviews and feedback, where the original content might have been cluttered with extraneous information.</p><p><strong>Example</strong>:</p><ul><li><strong>Before</strong>: “Hi! The product I bought from <a href="https://example.com/">https://example.com</a> last week is amazing! But the delivery was late.”</li><li><strong>After</strong>: “Product bought last week was amazing but delivery late.”</li></ul><h4><strong>Emoji conversion</strong></h4><p><strong>Purpose</strong>: Feedback and review texts often contained emojis that conveyed nuanced sentiments but were inconsistently interpreted by analysis tools. Converting emojis to textual descriptions helped sentiment analysis capture the intended sentiment more effectively, providing clearer context. This ensured accurate sentiment capture and analysis, which traditional text analysis tools might have missed.</p><p><strong>Example:</strong></p><ul><li><strong>Before:</strong> “The product is 😍”</li><li><strong>After:</strong> “The product is amazing.”</li></ul><h4><strong>Handling slang</strong></h4><p><strong>Purpose:</strong> Reviews often contained slang, abbreviations, or incomplete sentences, which could hinder comprehension. Adding context to prompts during rewriting helped translate slang into standard language and completed fragmented thoughts, improving text quality and coherence. Slang and incomplete sentences could obscure meaning, so translating and clarifying them ensured the text was clear and easy to understand.</p><p><strong>Example:</strong></p><ul><li><strong>Before:</strong> “The product is lit. But… delivery was late.”</li><li><strong>After:</strong> “The product is excellent, but the delivery was delayed.”</li></ul><h3><strong>2. Clustering</strong></h3><p>Faced with vast unstructured datasets, we turned to the realm of unsupervised Machine Learning, specifically clustering, to transform our chaotic text into coherent, manageable clusters. This strategic move allowed us to unearth common themes and topics, sharpening our analysis and making it more comprehensible.</p><p>In document summarization, clustering grouped similar sections to streamline key information extraction. For topic modeling, it revealed intricate hierarchical structures within extensive text corpora. In customer feedback analysis, clustering improved efficiency by grouping reviews to identify common themes and sentiments. Additionally, it supported spam filtering, social network analysis, content-based profiling, and data reduction, making it essential for extracting insights and enhancing decision-making.</p><p>Despite our confidence in <a href="https://en.wikipedia.org/wiki/Hierarchical_clustering">clustering techniques</a>, we meticulously explored the best-suited method for our needs among various options — DBSCAN, hierarchical, and agglomerative, among others.</p><p>After thorough analysis, we embraced agglomerative clustering, which aligned perfectly with our need to examine the nested structure of data. This method offered several advantages over other techniques, namely <em>k</em>-means and DBSCAN:</p><ul><li><strong>Pre-determination of number of clusters:</strong> Unlike <em>k</em>-means, agglomerative clustering’s flexible approach allowed us to avoid defining the number of clusters in advance.</li><li><strong>Handled different cluster shapes and sizes:</strong> Agglomerative clustering managed clusters of various shapes and sizes better than <em>k</em>-means, which assumed spherical clusters of similar sizes.</li><li><strong>Robust to noise and outliers:</strong> While DBSCAN was also effective at handling noise and clusters, agglomerative clustering’s additional benefits around handling noise and outliers made it our preferred choice.</li><li><strong>Scalability:</strong> Although agglomerative clustering was computationally intensive compared to DBSCAN, we optimized it with single and complete linkage to handle our large datasets.</li><li><strong>Parameter sensitivity:</strong> DBSCAN required careful selection of parameters — such as eps and minPts — which was challenging for us. Agglomerative clustering did not need such parameters and was easier to implement.</li><li><strong>Handled varying densities:</strong> DBSCAN struggled with clusters when their densities varied. Agglomerative clustering merged clusters based on similarity rather than density, enabling us to form clusters based on word or topic similarity.</li><li><strong>Deterministic approach:</strong> Unlike DBSCAN, agglomerative clustering was deterministic, providing a stable model for a given set of data.</li></ul><h3><strong>3. MapReduce summarization</strong></h3><p>After clustering the unstructured text into coherent groups, we utilized the <a href="https://python.langchain.com/docs/tutorials/summarization/#map-reduce">MapReduce</a> framework to process these clusters in parallel, creating summaries that were both meaningful and representative of the text.</p><ul><li><strong>Map step:</strong> The text was segmented into smaller chunks or sub-documents and processed in parallel, accelerating the summarization process. Each chunk was then individually summarized using a language model.</li><li><strong>Reduce step:</strong> The individual summaries produced in the map step was then merged into a single, cohesive summary. This final summary encapsulated the key points from the entire document.</li></ul><p>We found the MapReduce summarization technique highly effective for summarizing large corpora of unstructured text by summarizing chunks individually (map) and then combining them (reduce), especially when sub-documents didn’t rely heavily on preceding context. This method was both scalable and efficient for large volumes of text. While we also explored the Refine Method and Stuff Chain, each presented unique advantages and limitations.</p><ul><li><strong>Refine method:</strong> The Refine method simplified summarization by iteratively updating the summary with each new document, making it suitable for continuous data streams.</li><li><strong>Stuff Chain method:</strong> The Stuff Chain method processed large documents by splitting them into chunks, summarizing each separately, and then combining them. While efficient for handling large files, it was constrained by the language model’s context window.</li></ul><p>We share here our findings when we compared these alternative techniques to MapReduce:</p><h4><strong>MapReduce versus Refine method</strong></h4><ul><li><strong>Scalability: </strong>MapReduce was more scalable as it processed data in parallel, whereas the Refine method was more sequential.</li><li><strong>Complexity: </strong>The Refine method was simpler and easier to implement but did not handle large datasets as efficiently as MapReduce.</li><li><strong>Functionality: </strong>MapReduce offered more robust functionality for large-scale summarization tasks, while the Refine method was better suited for continuous data streams.</li></ul><h4><strong>MapReduce versus Stuff Chain</strong></h4><ul><li><strong>Scalability: </strong>MapReduce was more scalable and could handle larger volumes of text, while Stuff Chain was limited by the language model’s context window.</li><li><strong>Efficiency: </strong>MapReduce processed data in parallel, making it more efficient for large datasets, whereas Stuff Chain struggled with very large corpora.</li><li><strong>Flexibility: </strong>MapReduce was more flexible in handling various data structures and sizes, while Stuff Chain was more straightforward but less adaptable to complex data.</li></ul><h3><strong>4. Attributes identification</strong></h3><p>After sourcing and pre-processing the unstructured text data, we employed agglomerative clustering and MapReduce summarization techniques to organize the data into meaningful clusters. The next crucial step was the identification of key attributes within these clusters. This involved extracting prominent topics from each cluster, which provided a high-level overview of the subject matter in each group. This process facilitated easy labeling and a deeper understanding of the data.</p><p>To achieve this, we experimented with various topic modeling techniques, including the <a href="https://maartengr.github.io/BERTopic/index.html">BERTopic</a> library. BERTopic leveraged transformer embeddings to capture the semantic meaning of the text, resulting in more coherent and contextually relevant topics. Here’s a brief overview of the process:</p><p><strong>Embedding the text:</strong> We used the SentenceTransformer model to generate embeddings that captured the semantic nuances of the text.</p><pre>from sentence_transformers import SentenceTransformer<br><br># Load a pre-trained SentenceTransformer model<br>model = SentenceTransformer(&#39;all-MiniLM-L6-v2&#39;)<br><br># Encode your text data to get embeddings<br>embeddings = model.encode(your_texts, show_progress_bar=True)</pre><p><strong>Topic modeling with BERTopic:</strong> Using the embeddings, we applied the BERTopic model to identify topics within the text.</p><pre>from bertopic import BERTopic<br><br># Initialize BERTopic model<br>topic_model = BERTopic()<br><br># Fit the model on your text data and extract topics<br>topics, _ = topic_model.fit_transform(your_texts)</pre><p><strong>Analyzing the topics</strong>: We extracted detailed information about the identified topics and visualized them for better understanding.</p><pre># Retrieve topic information<br>topic_info = topic_model.get_topic_info()<br>print(topic_info)<br><br># Retrieve top words for a specific topic (e.g., topic 0)<br>top_words = topic_model.get_topic(0)<br>print(top_words)<br><br># Visualize the topics<br>fig = topic_model.visualize_topics()</pre><p>While BERTopic was a powerful technique for topic modeling, we also explored the following other techniques: <a href="https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation">Latent Dirichlet Allocation</a> (LDA), <a href="https://en.wikipedia.org/wiki/Latent_semantic_analysis">Latent Semantic Analysis</a> (LSA), <a href="https://en.wikipedia.org/wiki/Non-negative_matrix_factorization">Non-Negative Matrix Factorization</a> (NMF), and <a href="https://en.wikipedia.org/wiki/Word2vec">Top2vec</a>. Additionally, leveraging Large Language Models (LLMs) or Small Language Models (SLMs) for prompting yielded even better results. These models excelled in understanding context, nuances, and the relationships among topics, providing a more comprehensive analysis of the data. Both LLMs and SLMs were effective for topic modeling, with LLMs offering a broader and more contextually rich understanding, while SLMs provided efficiency and speed for specific use cases.</p><h3><strong>5. Generating prime synopsis</strong></h3><p>To maximize the value derived from unstructured text data, we generated a “golden summary” or prime synopsis. This was essentially a summary of summaries, providing a concise yet comprehensive overview of all the clusters combined. This ensured that no crucial information was overlooked.</p><p>To achieve this, we employed advanced techniques such as the <a href="https://github.com/richawo/chain-of-density">Chain of Density</a> (CoD) and <a href="https://www.reddit.com/r/PromptEngineering/comments/1bh610y/chainofverification_cove_explained/?rdt=61851">Chain of Verification</a> (CoVe) prompt engineering methods. The CoD technique enhanced the quality of AI-generated summaries by gradually increasing the density of information without significantly expanding the summary’s length. By systematically adding critical details while maintaining readability, CoD significantly improved the quality and relevance of the summaries. On the other hand, the CoVe technique enhanced the reliability of responses generated by Large Language Models (LLMs) by systematically verifying their outputs. By breaking down complex queries into simpler verification tasks, CoVe enhanced the overall quality of generated content across various applications and built trust in AI-generated content.</p><h3><strong>6. Sentiment tagging</strong></h3><p>In addition to summarization, <a href="https://en.wikipedia.org/wiki/Sentiment_analysis">sentiment tagging</a> was performed to extract the sentiments (positive, negative, neutral) associated with each text entry. This allowed us to identify not only the topics being discussed but also how people felt about them. For instance, sentiment analysis revealed which topics elicited positive feedback and which were sources of dissatisfaction.</p><p>To achieve accurate sentiment tagging, we used various approaches to identify positive, negative, and neutral texts. One of the key challenges in sentiment analysis was dealing with paradoxical sentiments and emojis. Paradoxical sentiments were difficult to classify as positive, negative, or neutral.</p><p>To address this, we used the <a href="https://en.wikipedia.org/wiki/Prompt_engineering#Chain-of-thought">Chain-of-Thought</a> (COT) reasoning technique. This technique enhanced LLMs’ performance in scenarios with conflicting sentiments by encouraging the model to articulate its reasoning process step by step. This allowed the model to weigh different emotional cues before arriving at a conclusion. For example, when analyzing feedback with mixed feelings about a service, COT helped clarify whether the overall sentiment leaned positive or negative based on the arguments presented.</p><h3><strong>Custom evaluation metrics</strong></h3><p>After developing the text summarization and sentiment analysis process, it was crucial to thoroughly evaluate its effectiveness to ensure that the generated summaries and insights were both precise and actionable without losing their essence. We adopted the approach we describe as follows for this critical evaluation.</p><h4><strong>Creating the prime synopsis and score card</strong></h4><p>To comprehensively measure the performance of the summarization process, we leveraged traditional metrics like <a href="https://en.wikipedia.org/wiki/ROUGE_(metric)">ROUGE</a> and <a href="https://en.wikipedia.org/wiki/BLEU">BLEU</a> scores, creating two distinct and insightful views:</p><ul><li><strong>Prime synopsis</strong>: This high-level summary was meticulously derived from the summaries of individual text clusters, distilling the entire text corpus into one cohesive and comprehensive overview. It served as a benchmark for assessing the quality and completeness of the generated summaries.</li><li><strong>Score card</strong>: The score card captured key topics from the text data and mapped each one to its associated sentiment (positive, negative, or neutral). This structured view of sentiment across different topics provided a clear and nuanced picture of the data’s emotional landscape. By visualizing the sentiment distribution, we could swiftly identify areas of concern and satisfaction, enabling more informed and strategic decision-making.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tpIxEv2spc3-jzSd-3tk2w.png" /><figcaption><em>Figure 2: Score card Illustration.</em></figcaption></figure><p>To evaluate the consistency and completeness of generated summaries and topics, we used the prime synopsis and score card. We ensured that the golden summary aligned with key topics and sentiments in the score card, leveraging LLMs for consistency checks. Topic coverage was another crucial metric, assessing how well key topics were represented in the golden summary. By using these metrics, we refined our text summarization tool, enhancing the quality of outputs. These evaluation methods were vital for business stakeholders, ensuring reliable and comprehensive insights from text datasets, which were essential for informed decision-making.</p><p>For visualization, we used an intuitive and interactive framework that organized insights, making it easy to explore and navigate through the key findings and texts. In this article we refer to it as Distilled Insight Navigation Tree (DINT).</p><h3><strong>Use cases and applicability</strong></h3><p>The synergy of text summarization and sentiment analysis has the potential to deliver remarkable efficacy across a multitude of use cases, especially ones involving analyzing vast amounts of text data. We share here a tested use case where we believe that this solution provides substantial insights and value:</p><h4><strong>Customer product reviews</strong></h4><p><strong>Use case</strong>: We harnessed a dataset comprising e-commerce product reviews to evaluate the prowess of our summarization and sentiment analysis tool.</p><p><strong>Outcome</strong>: The summaries and scorecards provided quick and actionable insights into product performance, highlighting key themes like product quality, shipping experience, and customer service. This tool helped businesses identify customer priorities and urgent issues. For example, in tablet reviews, it effectively synthesized feedback to reveal critical insights into product features and user experiences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*pkIVCym3lYmD2GeRK0-VoA.png" /><figcaption><em>Figure 3: DINT for customer product reviews.</em></figcaption></figure><p>In the same vein, it can also be leveraged to glean insights from customer support, survey, feedback text and similar other use cases.</p><h3><strong>Learnings and challenges</strong></h3><p>Throughout our journey to create an effective text summarization tool, we faced several challenges and learned important lessons. Handling noisy data, such as typos, slang, and irrelevant information, required robust preprocessing techniques to ensure the algorithms focused on the core meaning. Balancing summary quality and completeness was another key challenge, which we addressed by leveraging the golden summary approach and the score card to systematically evaluate and refine the summaries. Additionally, we encountered consistency issues with LLMs, where relevant topics sometimes failed to appear in the final summaries or sentiments were misclassified. To improve consistency and reliability, we employed custom evaluation metrics like topic coverage and consistency checks, iteratively fine-tuning the prompts and evaluation techniques to enhance the accuracy and relevance of the summaries.</p><h3><strong>Conclusion</strong></h3><p>Automated text summarization and sentiment analysis offers transformative solutions for businesses overwhelmed with unstructured text data, enabling them to swiftly convert complex information into clear, actionable insights. These AI-infused techniques enhance efficiency by reducing the time required to process large datasets, allowing businesses to act on insights quickly. By identifying key themes and sentiments, organizations can make more informed decisions, addressing customer pain points and refining internal processes. Despite challenges like noisy data and model inconsistencies, meticulous preprocessing, prompt refinement, and iterative evaluation can lead to high-quality summaries. Ultimately, these solutions can be used to empower leaders to make smarter, data-driven decisions with enhanced clarity and agility, helping businesses stay ahead in managing vast volumes of text data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc2242c95a83" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-at-microsoft/decoding-signals-from-unstructured-text-the-ai-way-cc2242c95a83">Decoding signals from unstructured text: The AI way</a> was originally published in <a href="https://medium.com/data-science-at-microsoft">Data Science + AI at Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All I needed to crack the Microsoft Interview — NLP]]></title>
            <link>https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-nlp-31f578a86034?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/31f578a86034</guid>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Tue, 01 Oct 2024 08:34:16 GMT</pubDate>
            <atom:updated>2025-01-28T10:55:12.076Z</atom:updated>
            <content:encoded><![CDATA[<h3>All I needed to crack the Microsoft Interview — NLP</h3><p><em>For me, these notes were invaluable for a quick review before any data science interview. I hope you find them helpful too.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgMDq8ldjG0FPWr6QhBUhQ.jpeg" /><figcaption>Image by Microsoft Copilot</figcaption></figure><p>In this blog, I’ve organized the content to help you grasp four key aspects of any topic that an interviewer might bring up:</p><ol><li>Understand why the topic was introduced in the first place.</li><li>Discover what problems it addresses compared to earlier methods.</li><li>Explore the challenges that still remain in this approach.</li><li>Working steps of the topic.</li></ol><p>Lets being —</p><h3>1. Basics of NLP</h3><h4>Stemming</h4><p>Stemming was introduced to reduce inflected or derived words to their base form, allowing different variations of a word to be treated as the same in text processing tasks like search engines and indexing.</p><p>It improved efficiency in text retrieval by reducing the dimensionality of the vocabulary, making it easier to match similar terms without needing to handle every variation explicitly.</p><p>Stemming can be too aggressive, sometimes resulting in non-real words or losing the actual meaning, which may lead to incorrect matches in certain contexts.</p><h4>Lemmatization</h4><p>Lemmatization was introduced to address the limitations of stemming by converting words to their meaningful base or dictionary form (lemma) while preserving grammatical context.</p><p>It improved accuracy by retaining the proper meaning of words and producing valid dictionary entries, unlike stemming which might create non-words.</p><p>Lemmatization requires more computational resources and language-specific knowledge, making it slower and more complex to implement compared to stemming.</p><h4>Tokenization</h4><p><strong>There are majorly 2 types of Subword Tokenization</strong></p><h4>WordPiece</h4><p>WordPiece was introduced to efficiently handle rare words and out-of-vocabulary issues in language models by breaking words into subword units, allowing for a more flexible representation of language.</p><p>It improved the model’s vocabulary efficiency by enabling the use of a smaller vocabulary while still capturing a wide range of words and their meanings, leading to better performance in various NLP tasks.</p><p>WordPiece can still struggle with the granularity of tokenization, potentially resulting in overly fragmented representations of common words, which may hinder the understanding of certain contexts in natural language processing.</p><h4>Byte Pair Encoding</h4><p>Byte Pair Encoding (BPE) was introduced as a data compression technique to iteratively replace the most frequent pair of bytes in a sequence with a single, unused byte, optimizing tokenization in NLP by efficiently encoding words.</p><p>It improved handling of rare and out-of-vocabulary words by creating subword units, allowing models to learn representations for common prefixes and suffixes, thus enhancing vocabulary flexibility and coverage.</p><p>BPE can lead to suboptimal tokenization, where meaningful linguistic structures may be split, making it challenging for models to capture the full semantic context of words, particularly in morphologically rich languages.</p><h3>2. Featurization</h3><h4>Tf-IDF</h4><p>TF-IDF (Term Frequency-Inverse Document Frequency) was introduced to enhance the representation of text data by combining term frequency with the importance of terms across a corpus, allowing for a more nuanced understanding of word significance in documents.</p><p>It improved upon simple term frequency by reducing the weight of common words and highlighting rare, meaningful terms, making it particularly effective for information retrieval and document similarity tasks.</p><p>TF-IDF can struggle with capturing semantic relationships between words and may not perform well with short documents or when terms are used in different contexts, which can lead to limitations in understanding nuanced meanings.</p><h4>Continuous Bag of Words</h4><p>Continuous Bag of Words (CBOW) was introduced as part of the Word2Vec framework to predict target words based on their surrounding context words, facilitating the learning of word embeddings that capture semantic relationships in a continuous vector space.</p><p>CBOW improved upon traditional methods by providing a more efficient way to generate dense vector representations of words, enabling the capture of context and meaning through a simpler and faster training process compared to earlier models like Latent Semantic Analysis (LSA).</p><p>CBOW can struggle with rare words and may not effectively capture complex linguistic structures, as it averages the context words without considering their order, potentially losing nuanced meaning present in specific contexts.</p><p><strong>Steps</strong></p><ul><li>Determine the size of the context window (e.g., 2 words on either side of the target word) to identify which words will be used as input for predicting the target word.</li><li>Create an embedding layer with a vocabulary size and a specified dimensionality for word vectors. This layer will map each word to a continuous vector representation.</li><li>For each word in the context, retrieve its corresponding word vector from the embedding layer, resulting in a set of vectors representing the context words.</li><li>Calculate the average (or sum) of the vectors of the contextual words to form a single aggregated vector that represents the overall context.</li><li>Feed the aggregated context vector into a dense (fully connected) layer. The layer applies a linear transformation followed by a softmax activation function to generate probabilities for each word in the vocabulary.</li><li>Calculate the categorical cross-entropy loss between the predicted probabilities from the output layer and the actual target word (one-hot encoded). This loss guides the training process to optimize the model’s weights.</li></ul><h4>Skip gram</h4><p>Skip-gram was introduced as part of the Word2Vec framework to learn word embeddings by predicting surrounding context words based on a given target word, emphasizing the relationships between words in different contexts.</p><p>It improved upon methods like CBOW by allowing the model to focus on capturing the relationships of words in a more fine-grained manner, leading to better representations for rare words and enhancing the quality of embeddings by leveraging context information.</p><p>Skip-gram can be computationally intensive, especially with large vocabularies, and may also generate embeddings that capture noise in the data if not properly regularized, which can affect the quality and interpretability of the resulting word vectors.</p><ul><li>Determine the size of the context window (e.g., 2 words on either side of the target word) to identify which context words will be predicted based on the given target word.</li><li>Create an embedding layer with a vocabulary size and a specified dimensionality for word vectors. This layer maps each word to a continuous vector representation.</li><li>For each target word, retrieve its corresponding word vector from the embedding layer. This vector serves as the input for the model.</li><li>Feed the target word vector into a dense (fully connected) layer, which applies a linear transformation followed by a softmax activation function. This generates probabilities for all words in the vocabulary.</li><li>The output layer provides probabilities for the context words within the defined window, predicting multiple context words for each input target word.</li><li>Instead of updating the weights for all words in the vocabulary, randomly select a few negative samples (words that are not in the context) for each training instance. This reduces computational complexity.</li><li>Treat each context word and its negative samples as binary classification problems, where the goal is to maximize the probability of predicting context words and minimize the probability of negative samples.</li><li>Calculate the binary cross-entropy loss for both the predicted context words and the negative samples. This loss guides the training process to optimize the model’s weights more efficiently.</li></ul><h4>GloVe</h4><p>GloVe (Global Vectors for Word Representation) was introduced to create word embeddings by leveraging global word co-occurrence statistics from a corpus, allowing the model to capture semantic relationships between words based on their context in a more comprehensive manner.</p><p>It improved upon previous models like Word2Vec by integrating both local context (as in Skip-gram) and global statistical information, resulting in embeddings that reflect word meaning more accurately and robustly in various linguistic contexts.</p><p>GloVe can struggle with rare words and may require substantial memory and computational resources to handle large corpora, leading to potential difficulties in scalability and effectiveness when dealing with highly dynamic or evolving language.</p><ul><li>Construct a co-occurrence matrix X where each entry Xij represents the number of times word iii appears in the context of word j across the entire corpus. This captures global statistical information about word usage.</li><li>Apply a weighting function to the co-occurrence counts to downweight the influence of frequent co-occurrences.</li><li>GloVe model learns word embeddings by minimizing its objective function</li><li>Use an optimization algorithm (like stochastic gradient descent) to minimize the objective function, adjusting the word vectors and biases. This trains the model to capture the relationships between words based on their co-occurrence statistics.</li></ul><h4>FastText</h4><p>FastText was introduced to enhance word representation by considering subword information (character n-grams), enabling the model to generate better embeddings for rare words and morphologically rich languages.</p><p>It improved upon traditional word embedding models like Word2Vec by allowing the embeddings to capture nuanced meanings and similarities between words, even when they have not appeared in the training data, thus enhancing the model’s robustness and generalization capabilities.</p><p>FastText can be computationally intensive, especially during training, and its reliance on subword information may lead to embeddings that are less effective for capturing broader contextual meanings or syntactic nuances when used in downstream tasks.</p><p><strong>Steps</strong></p><ul><li>For each word in the vocabulary, generate character n-grams (subwords) to capture morphological features. For example, for the word “apple” with n=3, the n-grams would include “app,” “ppl,” “ple,” along with special tokens like “&lt;ap” and “le&gt;.”</li><li>Create an embedding layer that maps both words and their character n-grams to continuous vector representations. This allows the model to learn from both whole words and subword structures.</li><li>When training, for each input word, retrieve its corresponding word vector and the vectors for all its n-grams from the embedding layer. For example, the input word “apple” would be represented by its embedding and the embeddings of its character n-grams.</li><li>Combine the word vector and its n-gram vectors by averaging or summing them to create a single vector representation for the input word. This aggregated vector captures both the semantic meaning and morphological structure of the word.</li><li>Feed the aggregated vector into a dense (fully connected) layer that applies a linear transformation followed by a softmax activation function. This generates probabilities for each word in the vocabulary.</li><li>Calculate the categorical cross-entropy loss between the predicted probabilities and the actual target words. This loss is used to update the model weights during training.</li></ul><h3>3. Types of Embedding</h3><h4>Sentence Embeddings</h4><p>Sentence embeddings were introduced to provide a fixed-size vector representation for entire sentences, enabling models to capture semantic meaning and relationships between sentences rather than just individual words.</p><p>They improved upon word-level embeddings by allowing for better comparisons and analysis of sentences in tasks like semantic similarity, text classification, and information retrieval, enhancing performance on a variety of NLP applications.</p><p>Sentence embeddings can struggle with capturing the nuances of context and ambiguity in language, leading to potential loss of important information, and may also require extensive training data to achieve high-quality representations for diverse linguistic structures.</p><h4>ELMo Embedding</h4><p>ELMo (Embeddings from Language Models) was introduced to provide deep contextualized word representations by capturing the meaning of words based on their context in sentences, enhancing the understanding of language nuances and semantic relationships.</p><p>It improved upon static word embeddings by generating different embeddings for the same word depending on its usage in different contexts, allowing for better performance in downstream tasks like sentiment analysis, named entity recognition, and machine translation.</p><p>ELMo can be computationally intensive, requiring significant resources for training and inference, and may also struggle with real-time applications due to its architecture, which is not as efficient as models like BERT and GPT for certain tasks.</p><h3>4. Models</h3><h4>RNNs, LSTMs, and GRUs</h4><p>RNNs, LSTMs, and GRUs were introduced to address the challenges of processing sequential data, allowing models to maintain memory of previous inputs and capture temporal dependencies in tasks such as language modeling and machine translation.</p><p>They improved upon traditional feedforward neural networks by enabling the modeling of sequences of variable lengths and preserving context over time, with LSTMs and GRUs specifically designed to mitigate issues like vanishing gradients, leading to better performance in handling long-range dependencies.</p><p>Despite their advancements, RNNs, LSTMs, and GRUs can still be computationally expensive and slow to train, especially on long sequences, and they may struggle with very long-range dependencies or require significant tuning to achieve optimal performance in complex NLP tasks.</p><h4>Sequence to sequence model</h4><p>Sequence-to-sequence (Seq2Seq) models were introduced to handle tasks where the input and output are both sequences, such as machine translation, by using an encoder-decoder architecture that can efficiently process variable-length sequences.</p><p>Seq2Seq models improved upon traditional models by enabling end-to-end training, allowing them to learn complex mappings between input and output sequences without the need for predefined alignments, which enhanced performance on tasks like translation and summarization.</p><p>Seq2Seq models can struggle with long-range dependencies due to limitations in capturing contextual information, leading to issues like information loss during encoding, and they may require attention mechanisms to improve performance on longer sequences.</p><p><strong>Steps</strong></p><ul><li>Begin with an input sequence. Each word in the sequence is typically converted into a continuous vector representation using an embedding layer.</li><li>Pass the input sequence through an encoder, often implemented as an RNN, LSTM, or GRU. The encoder processes the input sequentially, maintaining a hidden state that captures information about the input sequence.</li><li>Once the entire input sequence is processed, the final hidden state of the encoder is used as a context vector. This vector summarizes the relevant information from the input sequence, encapsulating the context needed for the decoder.</li><li>Initialize the decoder with the context vector. The decoder, also an RNN, LSTM, or GRU, generates the output sequence one word at a time, conditioned on the context vector.</li><li>If using attention, each time the decoder generates a word, it computes attention weights over the encoder’s hidden states, allowing it to focus on specific parts of the input sequence. This results in a weighted context vector for each time step, improving performance.</li><li>The decoder outputs a probability distribution over the vocabulary for the next word at each time step, using a softmax activation function. This continues until a designated end-of-sequence token is produced.</li><li>Calculate the categorical cross-entropy loss between the predicted probabilities and the actual target words. Use backpropagation to update the model weights, minimizing the loss over multiple training iterations.</li></ul><h4>Transformer</h4><p>Transformer model was introduced to overcome the limitations of recurrent neural networks (RNNs) in handling long-range dependencies, providing a more efficient and scalable architecture for sequence processing through parallelization and self-attention mechanisms.</p><p>It improved the ability to capture complex relationships between words in a sequence by using self-attention to weigh the significance of each word relative to others, leading to better performance on tasks like machine translation and natural language understanding.</p><p>Transformers can be computationally expensive and require significant memory resources, particularly for large models, making them challenging to train and deploy effectively in resource-constrained environments or with very large datasets.</p><p><strong>Steps</strong></p><ul><li>The input sequence is tokenized and converted into embeddings. Positional encodings are added to these embeddings to retain information about the order of words in the sequence since the Transformer has no inherent sequence order.</li><li>Each token in the input sequence attends to all other tokens using the self-attention mechanism. This involves:</li><li><strong>Calculating Query, Key, and Value Vectors</strong>: For each token, compute the query, key, and value vectors by multiplying the embedding by weight matrices.</li><li><strong>Attention Scores</strong>: Compute attention scores by taking the dot product of query and key vectors, followed by a softmax operation to get attention weights.</li><li><strong>Weighted Sum</strong>: Multiply the value vectors by the attention weights to obtain a context-aware representation for each token.</li><li>After the attention mechanism, each token representation is passed through a position-wise feed-forward neural network (FFN) consisting of two linear transformations with a ReLU activation in between. This processes each token independently.</li><li>Layer normalization and residual connections are applied after each multi-head attention and feed-forward layer, helping to stabilize training and improve convergence.</li><li>The Transformer architecture consists of several identical layers (often 6 or more), each containing the multi-head attention and feed-forward components. This stacking enables the model to learn complex representations.</li><li>For the decoder, a similar architecture is followed, with additional masking to prevent attending to future tokens in the output sequence. The final output is passed through a linear layer with softmax activation to generate probabilities for the next word in the sequence.</li></ul><h4>BERT</h4><p>BERT (Bidirectional Encoder Representations from Transformers) was introduced to improve natural language understanding by enabling deep bidirectional context in word embeddings, allowing models to consider both left and right context during training.</p><p>It improved upon traditional models by utilizing masked language modeling and next sentence prediction, leading to better contextual representations of words, significantly enhancing performance on a wide range of NLP tasks such as question answering and sentiment analysis.</p><p>BERT can be resource-intensive, requiring substantial computational power for training and inference, and it may struggle with real-time applications due to its architecture, which may not be optimized for very long sequences or extremely large datasets.</p><p>Steps</p><ul><li>BERT takes tokenized input sequences and converts them into embeddings. Each token’s embedding is the sum of:</li></ul><p><strong>A. Token Embeddings</strong>: Unique embeddings for each word or subword.</p><p><strong>B. Segment Embeddings</strong>: Distinguish between different sentences in a pair (e.g., A and B in question answering).</p><p><strong>C. Positional Embeddings</strong>: Encodes the position of each token in the sequence to maintain order.</p><ul><li>BERT is built entirely on the Transformer encoder architecture, consisting of multiple stacked layers (often 12 or more). Each layer contains:</li><li><strong>Multi-Head Self-Attention</strong>: This mechanism allows each token to attend to all other tokens in the input sequence, capturing contextual relationships.</li><li><strong>Feed-Forward Neural Networks (FFN)</strong>: Each attention output is passed through a position-wise FFN, applying non-linear transformations to enhance learning capacity.</li><li><strong>Layer Normalization and Residual Connections</strong>: These techniques are applied after each attention and FFN sub-layer to stabilize training and facilitate gradient flow.</li><li>Unlike traditional language models that process text in a unidirectional manner, BERT utilizes a bidirectional approach. This means each token is informed by its context both to the left and right, leading to a deeper understanding of the sentence structure and meaning.</li></ul><p>BERT is pre-trained on a large corpus using two primary tasks:</p><ul><li><strong>Masked Language Model (MLM)</strong>: Randomly masks a percentage of tokens in the input and trains the model to predict these masked tokens based on their context.</li><li><strong>Next Sentence Prediction (NSP)</strong>: Trains the model to predict whether a given sentence follows another sentence, aiding in understanding sentence relationships.</li><li>After pre-training, BERT can be fine-tuned on specific downstream tasks (e.g., sentiment analysis, question answering) by adding task-specific output layers. This involves minimal task-specific architecture changes, allowing BERT to leverage its pre-trained knowledge.</li><li>For classification tasks, the final layer may include a softmax function to generate probabilities over classes. For tasks like question answering, it may predict the start and end positions of the answer span in the text.</li></ul><h4><strong>GPT</strong></h4><p>GPT (Generative Pre-trained Transformer) was introduced to create a model capable of generating coherent and contextually relevant text by leveraging unsupervised pre-training on large text corpora followed by fine-tuning on specific tasks.</p><p>It improved upon earlier generative models by utilizing a unidirectional transformer architecture, allowing for the generation of text in a more fluid and natural manner, resulting in superior performance in text completion, conversation generation, and creative writing tasks.</p><p>GPT can generate plausible but factually incorrect information and may exhibit biases present in the training data, leading to challenges in reliability and ethical considerations when deploying the model in real-world applications.</p><p><strong>Steps</strong></p><ul><li>GPT takes tokenized input sequences, where each token is converted into an embedding. Positional encodings are added to the token embeddings to retain information about the order of tokens in the sequence, as GPT processes tokens in a sequential manner. GPT is built on the Transformer decoder architecture.</li></ul><p>Each layer consists of:</p><ul><li><strong>Masked Multi-Head Self-Attention</strong>: This mechanism allows the model to attend to all previous tokens in the sequence while masking future tokens, ensuring that the prediction for a token depends only on its preceding context.</li><li><strong>Feed-Forward Neural Networks (FFN)</strong>: After the attention mechanism, the output is passed through a position-wise FFN that applies non-linear transformations to enhance the model’s capacity to learn complex representations.</li><li><strong>Layer Normalization and Residual Connections</strong>: These are used after each attention and FFN sub-layer to stabilize training and improve convergence.</li><li>In the self-attention mechanism, a <strong>causal attention mask </strong>is applied to prevent attending to future tokens. This is critical for autoregressive modeling, where predictions are made one token at a time.</li><li>GPT is pre-trained using a simple but effective objective: <strong>Next Token Prediction</strong>. The model learns to predict the next token in a sequence given all previous tokens, training on vast amounts of text data. This approach allows it to capture grammar, facts, and some level of reasoning.</li><li>After pre-training, GPT can be fine-tuned on specific tasks by modifying the output layer. For example, for text classification, a classification head is added, while for text generation tasks, the pre-trained model can be used directly.</li><li>During text generation, the model predicts the next token based on the previous tokens, generating text in an autoregressive manner until a specified length or an end-of-sequence token is produced.</li></ul><h3>5. Miscellaneous</h3><h4>Tagging schemes in NER</h4><h4>1. BIO (Beginning, Inside, Outside)</h4><p>Each token is tagged as B (beginning of an entity), I (inside an entity), or O(outside any entity).</p><h4>2. BILOU (Beginning, Inside, Last, Outside, Unit)</h4><p>An extension of BIO that adds a L (last token of an entity) and U (a single token entity). This allows the model to distinguish between multi-token and single-token entities.</p><h4>3. IOB (Inside, Outside, Beginning)</h4><p>Similar to BIO, but it emphasizes that the first token of an entity is marked as B for &quot;Beginning&quot; and the rest are tagged as I.</p><h4>4. BIOES (Beginning, Inside, Outside, End, Single)</h4><p>Another variation of the BIO scheme that includes E (end of an entity) and S (single-token entity). This format helps specify the start and end of entities more clearly.</p><h4>Search Models</h4><ol><li>Greedy Search — Greedy search generates the next word by selecting the one with the highest probability at each time step, without considering future words.</li><li>Random Sampling — Instead of always choosing the most probable word, this method samples from the probability distribution of the next word.</li><li>Top-k Sampling — This method samples from the top k most probable words rather than the entire distribution, ensuring that only the highest-scoring options are considered.</li><li>Top-p Sampling — Instead of using a fixed k, nucleus sampling selects from the smallest set of words whose cumulative probability exceeds a threshold p. This means that the number of candidates can vary.</li><li>Coverage Mechanism — This technique focuses on ensuring that all parts of the input sequence are adequately represented in the output, particularly useful in tasks like machine translation.</li></ol><p>All the best :)</p><p>Want More…<br><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-machine-learning-4965cbdcf8d1">All I needed to crack the Microsoft Interview — Machine Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-deep-learning-c39bad25f089">All I needed to crack the Microsoft Interview — Deep Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-statistics-1f3d3ca8055d">All I needed to crack the Microsoft Interview — Statistics | by Mehul Jain | Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31f578a86034" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All I needed to crack the Microsoft Interview — Statistics]]></title>
            <link>https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-statistics-1f3d3ca8055d?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f3d3ca8055d</guid>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Tue, 01 Oct 2024 08:27:51 GMT</pubDate>
            <atom:updated>2025-01-28T10:53:30.513Z</atom:updated>
            <content:encoded><![CDATA[<h3>All I needed to crack the Microsoft Interview — Statistics</h3><p><em>For me, these notes were invaluable for a quick review before any data science interview. I hope you find them helpful too.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*152p4eKrV8d5M5qNqZDK8A.jpeg" /><figcaption>Image by Microsoft Copilot</figcaption></figure><h3><strong>Probability Concepts</strong></h3><h4>1. Conditional Probability</h4><h4>Definition:</h4><p><strong>Conditional probability</strong> refers to the probability of an event occurring given that another event has already occurred. Mathematically, it is denoted as P(A∣B), which is the probability of event A occurring given that event B has occurred.</p><h4>Assumptions:</h4><ol><li>P(B) must be greater than 0, meaning event B has a non-zero probability.</li><li>The events are dependent, meaning the occurrence of B affects the probability of A.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Spam filtering</strong>: In email systems, the conditional probability that an email is spam given that certain keywords (like “free money”) appear can be used to determine the likelihood of the email being spam.</li></ul><h4>2. Joint Probability</h4><h4>Definition:</h4><p><strong>Joint probability</strong> is the probability of two events occurring simultaneously. It is denoted as P(A∩B), which represents the probability that both event A and event B occur.</p><h4>Assumptions:</h4><ol><li>Events A and B can either be dependent or independent.</li><li>For independent events, P(A∩B)=P(A)⋅P(B), meaning the occurrence of A does not affect the occurrence of B.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Weather prediction</strong>: Suppose you want to find the probability that it will rain and that it will also be windy on a particular day. The joint probability can be used to calculate the chance of both events happening at the same time.</li></ul><h4>3. Bayes’ Theorem</h4><h4>Definition:</h4><p><strong>Bayes’ Theorem</strong> is a formula that describes the probability of an event based on prior knowledge of related events. It is used to update the probability estimate for an event as more information becomes available.</p><h4>Assumptions:</h4><ol><li>Requires prior probabilities (prior belief about the event).</li><li>Assumes that the likelihood of events can be computed or estimated.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Medical diagnostics</strong>: If a doctor wants to know the probability that a patient has a particular disease given a positive test result, Bayes’ Theorem helps incorporate the probability of the disease (prior knowledge) and the accuracy of the test (likelihood). For example, if a person tests positive for a rare disease, Bayes’ Theorem can help the doctor weigh the true likelihood of the disease, considering the test’s sensitivity and the disease’s base rate in the population.</li></ul><h4>4. Naive Bayes</h4><h4>Definition:</h4><p><strong>Naive Bayes</strong> is a family of simple probabilistic classifiers based on applying <strong>Bayes’ Theorem</strong> with the assumption of independence between features. In other words, it assumes that the presence (or absence) of a particular feature in a class is independent of the presence (or absence) of any other feature, hence the term “naive.”</p><p>There are different variants of Naive Bayes classifiers depending on the distribution of data (e.g., Gaussian Naive Bayes, Multinomial Naive Bayes).</p><p>Laplace smoothing adjusts the probability estimates to account for the possibility that unseen events might occur. It adds a small constant (usually 1) to each count in the frequency table, ensuring that no probability is ever zero.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/215/1*B1ZrFvuvB_G2NrMjRR62fQ.png" /></figure><p>Where:</p><ul><li>Nx,C​ is the count of occurrences of feature xi​ in class C.</li><li>NC is the total count of occurrences of class C.</li><li>α is the smoothing parameter (typically set to 1 for Laplace smoothing).</li><li>V is the number of unique feature values (vocabulary size).</li></ul><h4>Assumptions:</h4><ol><li>The features are conditionally independent of each other given the class label.</li><li>All features contribute equally and independently to the outcome.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Email spam detection</strong>: Naive Bayes is widely used in spam filtering systems. It calculates the probability of an email being spam or not based on the words it contains. Each word in the email is treated as an independent feature contributing to the overall likelihood that the email is spam.</li></ul><h4>5. Marginal Probability</h4><h4>Definition:</h4><p><strong>Marginal probability</strong> refers to the probability of a single event occurring, without regard to the outcome of any other event. It is essentially the probability of one event happening in a larger probability distribution involving multiple random variables. Marginal probability is obtained by summing (or integrating, in the case of continuous variables) over the joint probabilities of the event with all possible outcomes of the other events.</p><p>For a discrete case, if A and B are two events, the marginal probability of event A, denoted as P(A), is given by summing over all joint probabilities involving A and B: P(A)=∑P(A∩B)</p><p>For continuous variables, you would integrate the joint probability over the other variables.</p><h4>Assumptions:</h4><ol><li>Marginal probability assumes that you are working within a joint probability distribution that involves multiple random variables.</li><li>The events need to be properly defined, with clearly identified joint probabilities to sum over.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Insurance risk modeling</strong>: An insurance company might want to know the marginal probability of a car accident occurring based on all recorded factors (age, location, type of vehicle, etc.). Even though there are many factors at play (joint probabilities), the company might focus only on the marginal probability of an accident, irrespective of other variables.</li></ul><h3><strong>2. Distributions</strong></h3><h4>1. Normal Distribution</h4><h4>Definition:</h4><p>The <strong>Normal distribution</strong>, also known as the Gaussian distribution, is a continuous probability distribution that is symmetric around its mean. The distribution has a bell-shaped curve, where most of the observations cluster around the central peak, and the probabilities for values taper off symmetrically on either side of the mean.</p><h4>Assumptions:</h4><ol><li>The data is continuous.</li><li>The distribution is symmetric around the mean.</li><li>The mean, median, and mode of the data are all equal.</li><li>About 68% of the data falls within one standard deviation from the mean, 95% within two standard deviations, and 99.7% within three standard deviations (Empirical Rule).</li></ol><h4>Real-world Application:</h4><ul><li><strong>Stock market analysis</strong>: Stock returns often follow a normal distribution over time. Analysts use normal distribution properties to calculate probabilities of stock prices falling within a certain range based on past performance.</li></ul><h4>2. Bernoulli Distribution</h4><h4>Definition:</h4><p>The <strong>Bernoulli distribution</strong> is a discrete probability distribution of a random variable which can take on one of two possible outcomes: 1 (success) with probability p and 0 (failure) with probability 1-p. It’s the simplest distribution, representing the outcome of a single binary trial.</p><h4>Assumptions:</h4><ol><li>There are only two possible outcomes (success or failure).</li><li>The probability of success p remains constant across all trials.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Click-through rate (CTR) in online ads</strong>: For a specific ad campaign, each time an ad is shown to a user, there are two possible outcomes — either the user clicks (success) or they don’t (failure). The Bernoulli distribution can model the probability of a click.</li></ul><h4>3. Binomial Distribution</h4><h4>Definition:</h4><p>The <strong>Binomial distribution</strong> is a discrete probability distribution that describes the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. If there are n trials and the probability of success in each trial is p, then the Binomial distribution gives the probability of achieving exactly k successes.</p><h4>Assumptions:</h4><ol><li>Each trial is independent.</li><li>Each trial has two possible outcomes (success or failure).</li><li>The probability of success p is the same for all trials.</li><li>The number of trials n is fixed.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Manufacturing quality control</strong>: A factory produces light bulbs, and the probability that a light bulb is defective is p. If the factory tests 10 bulbs, the binomial distribution can help model the probability that exactly 2 bulbs are defective.</li></ul><h4>4. Poisson Distribution</h4><h4>Definition:</h4><p>The <strong>Poisson distribution</strong> is a discrete probability distribution that expresses the probability of a given number of events occurring within a fixed interval of time or space, provided that the events occur with a known constant rate and are independent of the time since the last event.</p><h4>Assumptions:</h4><ol><li>Events occur independently of each other.</li><li>The average rate (mean) of occurrence is constant.</li><li>Two events cannot occur at exactly the same time.</li><li>The probability of an event occurring in a given time interval is proportional to the length of the interval.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Customer support call frequency</strong>: A company tracks the number of support calls they receive per hour. Using a Poisson distribution, they can model the probability of receiving exactly 5 calls in an hour, given the average rate of 5 calls per hour.</li></ul><h3><strong>3. Statistical Tests</strong></h3><h4>1. Z-Test</h4><h4>Definition:</h4><p>A <strong>Z-test</strong> is a statistical test used to determine whether there is a significant difference between the means of two datasets, or between the mean of a sample and a population mean. It is applied when the sample size is large (typically &gt; 30) and the population standard deviation is known.</p><h4>Assumptions:</h4><ol><li>The data is normally distributed.</li><li>The sample size is large (n &gt; 30) or the population variance is known.</li><li>The sample observations are independent.</li><li>The variance of the population is known (in case of population comparison).</li></ol><h4>Real-world Application:</h4><ul><li><strong>A/B testing</strong>: In website optimization, you can use a Z-test to check whether a new webpage design (B) performs better than the original design (A) in terms of user engagement.</li></ul><h4>2. T-Test</h4><h4>Definition:</h4><p>A <strong>T-test</strong> is used to compare the means of two groups, particularly when the sample size is small and/or the population variance is unknown. There are two main types:</p><ul><li><strong>Independent T-test</strong>: Compares means of two independent groups.</li><li><strong>Paired T-test</strong>: Compares means of the same group at different times.</li></ul><h4>Assumptions:</h4><ol><li>The data follows a normal distribution.</li><li>The variance of the two groups is roughly equal (for independent t-tests).</li><li>The observations are independent of each other.</li><li>Sample sizes are small (typically n &lt; 30).</li></ol><h4>Real-world Application:</h4><ul><li><strong>Product performance comparison</strong>: Suppose a new feature is rolled out to 10 random users and compared against 10 users who didn’t get the feature. The T-test can help determine whether the feature significantly improves user experience.</li></ul><h4>3. F-Test</h4><h4>Definition:</h4><p>An <strong>F-test</strong> is used to compare the variances of two populations. It checks whether the variances of two datasets are significantly different. This test is typically applied in the context of ANOVA or to test model fit.</p><h4>Assumptions:</h4><ol><li>The data in both groups is normally distributed.</li><li>The data samples are independent.</li><li>The samples are drawn from populations with equal variances (in specific cases like ANOVA).</li></ol><h4>Real-world Application:</h4><ul><li><strong>Quality control in manufacturing</strong>: Suppose you want to compare the variance in the length of parts produced by two different machines. An F-test can be used to determine if one machine consistently produces parts with more variation than the other.</li></ul><h4>4. Chi-Square Test</h4><h4>Definition:</h4><p>The <strong>Chi-Square test</strong> is a non-parametric test used to determine whether there is a significant association between categorical variables. There are two types:</p><ul><li><strong>Chi-square goodness-of-fit test</strong>: Checks how well observed data matches expected data.</li><li><strong>Chi-square test of independence</strong>: Checks if two categorical variables are related.</li></ul><h4>Assumptions:</h4><ol><li>The sample data is drawn randomly from the population.</li><li>The variables are categorical.</li><li>The expected frequency count for each category should be at least 5.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Customer segmentation</strong>: A chi-square test can help determine if there’s a relationship between gender (male/female) and the type of product (electronics, fashion, etc.) they purchase frequently in an online store.</li></ul><h4>5. ANOVA (Analysis of Variance)</h4><h4>Definition:</h4><p><strong>ANOVA</strong> is a statistical test used to compare the means of three or more groups to see if at least one group is significantly different from the others. It works by analyzing the variance within groups and between groups.</p><h4>Assumptions:</h4><ol><li>The populations from which the samples are drawn are normally distributed.</li><li>Homogeneity of variances (the variance within each group should be roughly equal).</li><li>The observations are independent of each other.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Marketing campaign effectiveness</strong>: Suppose a company runs three different marketing campaigns and wants to test if there is a significant difference in sales outcomes among the three campaigns. ANOVA can determine whether the performance of one campaign is significantly different from the others.</li></ul><h3><strong>4. Functions</strong></h3><h4>1. Probability Density Function (PDF)</h4><p>A <strong>Probability Density Function (PDF)</strong> describes the probability distribution of a continuous random variable. Unlike probabilities for discrete variables, which can be directly obtained, for continuous variables, the PDF gives a <em>density</em> of probabilities rather than a probability for each exact value.</p><h4>Assumptions:</h4><ul><li>The variable is continuous, meaning it can take any value in a given range.</li></ul><h4>Real-world Application:</h4><ul><li><strong>Height of individuals</strong>: In a population, height is a continuous variable. A PDF can model the probability density of people’s heights, showing that most heights cluster around the mean, with fewer people at extreme values.</li></ul><h3>2. Probability Mass Function (PMF)</h3><h4>Definition:</h4><p>A <strong>Probability Mass Function (PMF)</strong> is used for discrete random variables and gives the probability that a discrete random variable is exactly equal to some value.</p><h4>Assumptions:</h4><ul><li>The variable is discrete, meaning it can take specific, separate values (e.g., 0, 1, 2, etc.).</li></ul><h4>Real-world Application:</h4><ul><li><strong>Number of customers arriving at a store</strong>: The number of customers arriving at a store in a day is a discrete variable. The PMF can describe the probability of exactly 5, 10, or 15 customers arriving, for example.</li></ul><h3>3. Cumulative Distribution Function (CDF)</h3><h4>Definition:</h4><p>The <strong>Cumulative Distribution Function (CDF)</strong> is a function that gives the probability that a random variable (either continuous or discrete) takes a value less than or equal to a certain value.</p><p>The CDF is always non-decreasing and ranges from 0 to 1.</p><h4>Assumptions:</h4><ul><li>Can be used for both discrete and continuous random variables.</li><li>The CDF at x=∞ equals 1.</li></ul><h4>Real-world Application:</h4><ul><li><strong>Income distribution</strong>: The CDF can be used to calculate the proportion of a population that earns less than or equal to a certain income threshold. It helps in understanding the cumulative proportion of people falling below certain income levels.</li></ul><h3>Comparison of PDF, PMF, and CDF:</h3><ul><li><strong>PDF</strong> is for <strong>continuous</strong> random variables and gives the probability density over a range.</li><li><strong>PMF</strong> is for <strong>discrete</strong> random variables and gives the exact probability for specific outcomes.</li><li><strong>CDF</strong> applies to both and gives the cumulative probability up to a certain value.</li></ul><h3><strong>5. Estimation and Inference</strong></h3><h4>1. Point Estimate vs Confidence Interval</h4><h4>Point Estimate:</h4><ul><li>A <strong>point estimate</strong> is a single value (or “best guess”) calculated from sample data used to estimate a population parameter. For example, if you want to estimate the population mean μ, you could use the sample mean as a point estimate of μ.</li><li><strong>Example</strong>: If you survey 100 people to estimate the average income in a city and the sample average income is $50,000, then $50,000 is your point estimate of the city’s average income.</li></ul><h4>Confidence Interval:</h4><ul><li>A <strong>confidence interval (CI)</strong> gives a range of values within which the true population parameter is likely to lie, along with a confidence level that quantifies the certainty of that range. Common confidence levels include 90%, 95%, or 99%. A 95% confidence interval means that if you were to take many samples, 95% of the intervals calculated from these samples would contain the true population parameter.</li><li>The general form for a confidence interval is: Point Estimate ± Margin of Error.</li><li>where the margin of error is influenced by the standard deviation and the confidence level.</li><li><strong>Example</strong>: If you estimate the average income with a 95% confidence interval of $48,000 to $52,000, you are 95% confident that the true average income lies in this range.</li></ul><p><strong>Point estimate</strong> provides a single value, while <strong>confidence intervals</strong> provide a range of plausible values for the population parameter.</p><h4>2. Hypothesis Testing</h4><h4>Definition:</h4><p><strong>Hypothesis testing</strong> is a statistical method used to make inferences about population parameters based on sample data. It involves comparing a null hypothesis (H0) against an alternative hypothesis (H1 or Ha).</p><p>The basic steps are:</p><p><strong>Formulate hypotheses</strong>:</p><ul><li><strong>Null hypothesis (H0)</strong>: This is the assumption that there is no effect or no difference (status quo).</li><li><strong>Alternative hypothesis (H1)</strong>: This is what you want to prove (e.g., there is a difference).</li></ul><p><strong>Collect data</strong> and choose a significance level (α), typically 0.05 or 0.01, which represents the probability of rejecting the null hypothesis when it is actually true.</p><p><strong>Perform the test</strong> (Z-test, T-test, etc.) and calculate a test statistic based on the sample data.</p><p><strong>Make a decision</strong>: Based on the test statistic and the ppp-value, either reject or fail to reject the null hypothesis.</p><h4>Example:</h4><ul><li>Testing whether a new drug is more effective than a placebo.</li><li>H0: The drug is no more effective than the placebo.</li><li>H1: The drug is more effective than the placebo.</li></ul><h4>3. Type I and Type II Errors</h4><h4>Type I Error (False Positive):</h4><ul><li>A <strong>Type I error</strong> occurs when the null hypothesis H0​ is true, but you incorrectly reject it. This is also known as a <strong>false positive</strong>. The probability of making a Type I error is denoted by α, which is also the significance level.</li><li><strong>Example</strong>: Concluding that a drug is effective when it actually isn’t.</li></ul><h4>Type II Error (False Negative):</h4><ul><li>A <strong>Type II error</strong> occurs when the null hypothesis H0​ is false, but you fail to reject it. This is also known as a <strong>false negative</strong>. The probability of making a Type II error is denoted by β, and the <strong>power</strong> of the test is 1−β, which is the probability of correctly rejecting a false null hypothesis.</li><li><strong>Example</strong>: Concluding that a drug is not effective when it actually is.</li></ul><p>A <strong>Type I error</strong> rejects a true null hypothesis, while a <strong>Type II error</strong> fails to reject a false null hypothesis.</p><h4>4. P-value</h4><h4>Definition:</h4><p>The <strong>p-value</strong> is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is true. It is a measure of the strength of evidence against the null hypothesis.</p><ul><li>A <strong>small p-value</strong> (typically p&lt;0.05) indicates strong evidence against the null hypothesis, so you reject H0​.</li><li>A <strong>large p-value</strong> (typically p&gt;0.05) suggests weak evidence against the null hypothesis, so you fail to reject H0​.</li></ul><h4>Example:</h4><ul><li>In a clinical trial, if the p-value is 0.03, there is a 3% chance of observing results as extreme as those in your study if the null hypothesis (that the drug has no effect) is true. Since this is lower than 0.05, you would reject the null hypothesis and conclude that the drug likely has an effect.</li></ul><p>The <strong>p-value</strong> helps you decide whether to reject or fail to reject the null hypothesis but does not tell you the size of the effect or its practical significance.</p><h4>5. Maximum Likelihood Estimation (MLE)</h4><h4>Definition:</h4><p><strong>Maximum Likelihood Estimation (MLE)</strong> is a method used to estimate the parameters of a statistical model. MLE selects the parameter values that maximize the likelihood function, which measures how likely the observed data is, given the parameters of the model.</p><p>Given a set of observations X={x1,x2,…,xn}, and a probability distribution parameterized by θ, the likelihood function is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/210/1*bn-n476RdImE7t9zQVJCKQ.png" /></figure><p>The goal is to find θ, the parameter value that maximizes the likelihood function, i.e.,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/205/1*n_AsB6BwieUufu5d_fr-QQ.png" /></figure><p>In practice, we often maximize the log-likelihood function log⁡L(θ;X), as it simplifies the calculations.</p><h4>Assumptions:</h4><ol><li>The data are drawn from a probability distribution that depends on one or more unknown parameters.</li><li>The observations are <strong>independent and identically distributed</strong> (IID).</li></ol><h4>Real-world Application:</h4><ul><li><strong>Logistic Regression</strong>: In logistic regression, MLE is used to estimate the coefficients (parameters) of the logistic model, maximizing the likelihood of observing the given binary outcomes based on input features.</li></ul><h3><strong>6. Correlation and Relationship Measures</strong></h3><h4>1. Correlation and Covariance</h4><h4>Covariance:</h4><ul><li><strong>Covariance</strong> measures the degree to which two random variables change together. If both variables increase or decrease simultaneously, the covariance is positive. If one increases while the other decreases, the covariance is negative.</li><li><strong>Positive covariance</strong>: Indicates that the two variables tend to move in the same direction.</li><li><strong>Negative covariance</strong>: Indicates that the two variables tend to move in opposite directions.</li></ul><h4>Correlation:</h4><ul><li><strong>Correlation</strong> standardizes the measure of covariance, giving a value between -1 and 1 that expresses how strongly two variables are related. The most common correlation measure is the <strong>Pearson correlation coefficient</strong> ρ, which is the covariance of the two variables divided by the product of their standard deviations.</li></ul><h4>Assumptions:</h4><ol><li>For Pearson correlation, both variables should be linearly related.</li><li>The data should be measured on at least an interval scale (not categorical).</li></ol><h4>Real-world Application:</h4><ul><li><strong>Stock returns analysis</strong>: Correlation and covariance are used to study how the returns of different stocks move together. Investors might look at the correlation between two assets to build a diversified portfolio.</li></ul><h4>2. Spearman’s Rank Correlation</h4><h4>Definition:</h4><p><strong>Spearman’s Rank Correlation Coefficient</strong> is a non-parametric measure of the strength and direction of the monotonic relationship between two variables. Unlike Pearson correlation, which assesses linear relationships, Spearman’s correlation evaluates whether one variable consistently increases or decreases as the other variable does, regardless of whether the <strong>relationship is linear</strong>.</p><p>The Spearman correlation ρs​ is calculated by converting the original data into ranks and then applying the Pearson correlation formula to these ranks.</p><h4>Assumptions:</h4><ol><li>The data should have an ordinal or continuous scale.</li><li>The relationship between the variables should be monotonic, meaning one variable consistently increases or decreases as the other does.</li></ol><h4>Real-world Application:</h4><ul><li><strong>Educational rankings</strong>: Spearman’s rank correlation can be used to compare students’ rankings in two different exams. It helps assess whether the students who performed well in one test also performed well in another, regardless of the exact scores.</li></ul><p>All the best :)</p><p>Want More…<br><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-nlp-31f578a86034">All I needed to crack the Microsoft Interview — NLP | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-deep-learning-c39bad25f089">All I needed to crack the Microsoft Interview — Deep Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-machine-learning-4965cbdcf8d1">All I needed to crack the Microsoft Interview — Machine Learning | by Mehul Jain | Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f3d3ca8055d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All I needed to crack the Microsoft Interview — Deep Learning]]></title>
            <link>https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-deep-learning-c39bad25f089?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/c39bad25f089</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[microsoft]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Thu, 22 Aug 2024 10:37:00 GMT</pubDate>
            <atom:updated>2025-06-28T10:05:09.587Z</atom:updated>
            <content:encoded><![CDATA[<h3>All I needed to crack the Microsoft Interview — Deep Learning</h3><p><em>For me, these notes were invaluable for a quick review before any data science interview. I hope you find them helpful too.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_6WLAnlQMseCgH7Fkl9Uhw.jpeg" /><figcaption>Image by Microsoft Copilot</figcaption></figure><h3>Purpose of the Bias Term in Neural Networks</h3><p>The bias term in neural networks plays a crucial role in enabling the model to learn and represent complex patterns.</p><ul><li>Without a bias term, the activation function is forced to pass through the origin, which can limit the model’s ability to fit the data. By adding a bias, the function can better align with the data distribution.</li><li>Without a bias term, If input X is Zero, the Z will become zero and it will propagate zero values throughout the network. This can lead to zero outputs for certain activation functions (e.g., ReLU), which might hinder the network’s ability to learn and propagate useful information.</li><li>With High Bias Value, the model may become less sensitive to the variations in the input data, leading to poor performance on both training and validation datasets. The activation function is shifted significantly, which can lead to outputs that are consistently higher or lower, regardless of the input values.</li><li>With too Low Bias Value, there is a risk that the model might overfit the training data, especially if the network has many parameters.</li></ul><h3>Forward Propagation</h3><ul><li><strong>Definition</strong>: The process by which input data is passed through the network layer by layer to produce an output.</li><li><strong>Activation Functions</strong>: Non-linear functions like ReLU, Sigmoid, or Tanh applied to introduce non-linearity.</li><li><strong>Key Concept</strong>: Each layer’s output becomes the next layer’s input, leading to the final output.</li><li><strong>Objective</strong>: To compute the predicted value based on the current set of weights and biases.</li><li><strong>No Learning</strong>: During forward propagation, the network doesn’t learn; it simply processes the input data.</li></ul><p><strong>Flow</strong>:</p><ul><li><strong>Input Layer</strong>: Receives the input features.</li><li><strong>Hidden Layers</strong>: Each neuron computes a weighted sum of inputs, applies an activation function, and passes the output to the next layer.</li><li><strong>Output Layer</strong>: Produces the final prediction.</li></ul><h3>Backward Propagation</h3><ul><li><strong>Definition</strong>: The process of updating the network’s weights based on the error of the output.</li></ul><p><strong>Error Calculation</strong>:</p><ul><li><strong>Loss Function</strong>: Calculates the difference between the predicted output and the actual output (e.g., Mean Squared Error, Cross-Entropy Loss).</li><li><strong>Gradient Calculation</strong>: Determines how much each weight contributes to the total error.</li></ul><p><strong>Gradient Descent</strong>:</p><ul><li><strong>Optimization Algorithm</strong>: Adjusts the weights to minimize the loss by moving in the direction of the negative gradient.</li><li><strong>Learning Rate</strong>: Controls the size of the steps taken to update the weights.</li><li><strong>Chain Rule</strong>: Used to compute gradients layer by layer from the output back to the input.</li><li><strong>Weight Update</strong>: After computing gradients, weights and biases are updated in the opposite direction of the gradient to reduce the error.</li><li><strong>Objective</strong>: Minimize the loss function to improve the model’s accuracy.</li><li><strong>Iterative Process</strong>: Repeated many times (epochs) to train the network.</li></ul><h3>Activation function</h3><p><strong>Purpose</strong>: Introduce non-linearity into the network, allowing it to learn complex patterns and functions.</p><p><strong>ReLU (Rectified Linear Unit)</strong>:</p><ul><li><strong>Formula</strong>: f(x)=max⁡(0,x)</li><li><strong>Advantages</strong>: Simple, efficient, mitigates vanishing gradient problem. Data is zero centered</li><li><strong>Disadvantages</strong>: Can cause “dying ReLUs” where neurons get stuck during training.</li><li>Derivative Range: {0,1}</li><li>Derivative is 1 for positive inputs and 0 for non-positive inputs, leading to the “dying ReLU” problem.</li></ul><p><strong>Sigmoid</strong>:</p><ul><li><strong>Formula</strong>: f(x)=1/(1+e^−x)</li><li><strong>Advantages</strong>: Maps output to a range of (0, 1), useful for binary classification.</li><li><strong>Disadvantages</strong>: Prone to vanishing gradients, slow convergence. Data is not zero centered hence it take more computational time to reach global minima</li><li>Derivative Range: (0,0.25]</li><li>The derivative is small, leading to potential vanishing gradient problems in deep networks.</li></ul><p><strong>Tanh</strong>:</p><ul><li><strong>Formula</strong>: f(x)=tanh⁡(x)</li><li><strong>Advantages</strong>: Outputs range from (-1, 1), centered around zero, faster convergence than Sigmoid.</li><li><strong>Disadvantages</strong>: Still susceptible to vanishing gradients.</li><li>Derivative Range: (0,1]</li><li><strong>Key Note</strong>: The derivative is larger than Sigmoid, reducing the vanishing gradient problem, but still not ideal for very deep networks.</li></ul><p><strong>Leaky ReLU</strong>:</p><ul><li><strong>Formula</strong>: f(x)=max⁡(0.01x,x)</li><li><strong>Advantages</strong>: Addresses the “dying ReLU” problem by allowing a small gradient for negative inputs.</li><li><strong>Disadvantages</strong>: More complex than ReLU.</li><li>Derivative Range: {α,1} where α is a small constant (e.g., 0.01)</li><li>The small positive slope α for negative inputs helps prevent neurons from “dying.”</li></ul><p><strong>Softmax</strong></p><ul><li><strong>Formula</strong>: f(xi)=e^xi/∑e^xj</li><li><strong>Advantages</strong>: Converts logits into probabilities that sum to 1, ideal for multi-class classification.</li><li><strong>Disadvantages</strong>: Computationally expensive for large number of classes.</li><li>Derivative Range: (0,1)(0, 1)(0,1)</li><li>The derivative involves all output components, leading to interaction between the outputs during backpropagation.</li></ul><p><strong>ELU (Exponential Linear Unit)</strong>:</p><ul><li><strong>Formula</strong>: f(x)={ x if x&gt;0, α(ex−1) if x≤0}</li><li><strong>Advantages</strong>: Helps mitigate the vanishing gradient problem. Outputs negative values (controlled by parameter α), which can make learning faster and more robust.</li><li><strong>Disadvantages</strong>: More computationally expensive than ReLU. Requires tuning of the α parameter.</li><li>Derivative Range: (α,1] where α is a positive constant (often 1)</li><li>The gradient for negative inputs is controlled by α, preventing vanishing gradients.</li></ul><p><strong>Swish</strong>:</p><ul><li><strong>Formula</strong>: f(x)=x⋅Sigmoid(x)</li><li><strong>Advantages</strong>: Smooth, non-monotonic function that can improve performance over ReLU and other activation functions. Allows small negative values, which can help in avoiding the “dying neuron” problem.</li><li><strong>Disadvantages</strong>: More computationally intensive than ReLU.</li><li>Derivative Range: (0,1]</li><li>Smooth derivative that can handle both positive and negative inputs effectively, leading to better performance in some cases.</li></ul><p><strong>Softplus</strong>:</p><ul><li><strong>Formula</strong>: f(x)=ln⁡(1+e^x)</li><li><strong>Advantages</strong>: Smooth approximation of ReLU that never outputs zero, allowing for positive gradients. Reduces the issue of “dying ReLU” as it is always active.</li><li><strong>Disadvantages</strong>: Slower than ReLU, especially for large negative values.</li><li>Derivative Range: (0,1)</li><li>The derivative is always positive, avoiding the zero-gradient issue, but can be very small for large negative inputs, leading to slow learning.</li></ul><p><strong>Selection Criteria</strong>:</p><ul><li><strong>Problem Type</strong>: Use Sigmoid for binary classification, Softmax for multi-class, ReLU/Leaky ReLU for hidden layers.</li><li><strong>Network Depth</strong>: For deep networks, ReLU and its variants are preferred due to their ability to avoid vanishing gradients.</li><li><strong>Output Range</strong>: Choose based on whether you need outputs between (0, 1), (-1, 1), or unbounded.</li><li><strong>Use ELU or Swish</strong> for hidden layers in deep networks to maintain the benefits of non-linearity while preventing vanishing gradients.</li><li><strong>Choose Softplus</strong> as an alternative to ReLU when a smoother function is desired.</li><li><strong>Use Softmax</strong> for the final output layer in multi-class classification tasks</li></ul><h3>Exploding Gradient Problem</h3><h4><strong>Definition</strong>:</h4><p>The exploding gradient problem occurs when gradients during backpropagation become excessively large, leading to very large updates to the network weights. This can cause the model to become unstable, with weights growing exponentially, and can result in the loss function oscillating or diverging instead of converging.</p><h4><strong>Causes</strong>:</h4><ul><li><strong>Deep Networks</strong>: The problem is more common in very deep neural networks where the gradient is multiplied repeatedly through many layers, causing it to grow exponentially.</li><li><strong>Large Weights</strong>: When the weights in the network are large, the gradient can become disproportionately large during backpropagation.</li><li><strong>Initialization Issues</strong>: Poor initialization of weights can exacerbate the problem, especially if weights are too large.</li></ul><h4>Symptoms:</h4><ul><li><strong>Loss Divergence</strong>: The loss function fails to converge and may increase rapidly.</li><li><strong>Unstable Training</strong>: The training process becomes unstable, with erratic updates and poor learning.</li><li><strong>Numerical Overflow</strong>: Gradients or weights can become so large that they cause numerical overflow, resulting in NaNs (Not a Number) in calculations.</li></ul><h4>Mitigation Strategies:</h4><p><strong>Gradient Clipping</strong>:</p><ul><li>Cap the gradients during backpropagation to a maximum value to prevent them from becoming too large.</li><li>Set a threshold max_norm and if the gradient exceeds this norm, scale it down.</li><li>Commonly used in recurrent neural networks (RNNs) and very deep feedforward networks.</li></ul><p><strong>Proper Weight Initialization</strong>:</p><ul><li>Initialize weights using methods that prevent gradients from exploding, such as:</li><li><strong>Xavier Initialization</strong>: Suitable for layers with sigmoid or tanh activations.</li><li><strong>He Initialization</strong>: Preferred for layers with ReLU or variants.</li><li>Keep the initial weights small enough to prevent large gradient values during the early stages of training.</li></ul><p><strong>Lower Learning Rate</strong>:</p><ul><li>Reduce the learning rate to slow down the weight updates, minimizing the risk of large gradients causing instability.</li><li>Smaller updates help maintain control over the gradient magnitude and ensure smoother convergence.</li></ul><p><strong>Batch Normalization</strong>:</p><ul><li>Normalize the inputs of each layer to maintain consistent and controlled activations, thereby reducing the risk of gradients exploding.</li><li>Also helps in accelerating training and improving convergence.</li></ul><p><strong>Use of Regularization</strong>:</p><ul><li><strong>L2 Regularization</strong>: Adds a penalty to the loss function proportional to the square of the weights, preventing weights from growing too large and indirectly controlling gradient size.</li><li><strong>Dropout</strong>: Regularization technique that randomly drops neurons during training, which can help in preventing any single path through the network from dominating and contributing to gradient explosion.</li></ul><p><strong>Gradient Norm Penalty</strong>:</p><ul><li>Explicitly add a term to the loss function that penalizes large gradient norms.</li><li>Forces the network to learn in a way that avoids large gradients, stabilizing training.</li></ul><h3>Vanishing Gradient Problem</h3><h4><strong>Definition</strong>:</h4><ul><li>The vanishing gradient problem occurs when gradients become exceedingly small during backpropagation, especially in deep neural networks. This causes the network to learn very slowly or even stop learning altogether, as the weights are updated minimally or not at all.</li></ul><h4><strong>Causes</strong>:</h4><ul><li><strong>Deep Networks</strong>: The problem is prevalent in deep networks where the gradient is multiplied by the derivative of the activation function across many layers, leading to an exponentially small gradient.</li><li><strong>Sigmoid and Tanh Activations</strong>: These functions have derivatives in the range of (0,0.25](0, 0.25](0,0.25] and (0,1](0, 1](0,1] respectively, which can diminish rapidly as they propagate backward through layers.</li><li><strong>Initialization Issues</strong>: Poor initialization of weights, especially if they are too small, can exacerbate the vanishing gradient problem.</li></ul><h4>Symptoms:</h4><ul><li><strong>Slow Convergence</strong>: The model takes an excessively long time to converge, as the gradient updates are too small.</li><li><strong>Stagnation</strong>: The training loss plateaus and the model stops improving despite further training.</li><li><strong>Layers Learning Unequally</strong>: The earlier layers in the network learn very slowly compared to the later layers.</li></ul><h3>Mitigation Strategies:</h3><p><strong>Use of ReLU and Its Variants</strong>:</p><ul><li>ReLU (Rectified Linear Unit) and its variants like Leaky ReLU, ELU, and Swish have derivatives of 1 for positive inputs, which help maintain stronger gradients.</li><li>Prevents the gradient from vanishing as it propagates through deep networks.</li><li>Commonly used in deep neural networks to maintain effective learning.</li></ul><p><strong>Proper Weight Initialization</strong>:</p><ul><li><strong>Xavier Initialization</strong>: Used when activations like Sigmoid or Tanh are employed. This method helps maintain the variance of activations and gradients across layers.</li><li><strong>He Initialization</strong>: Preferred for ReLU and its variants, this method ensures that the variance of gradients remains stable across layers, preventing them from shrinking too much.</li><li>Keeps the gradients from shrinking or exploding at the start of training.</li></ul><p><strong>Batch Normalization</strong>:</p><ul><li>Normalizes the input to each layer to maintain a stable distribution of activations throughout training.</li><li>Helps prevent the gradient from vanishing by ensuring that each layer receives inputs of similar scale.</li><li>Can also act as a regularizer, reducing the need for other regularization methods.</li></ul><p><strong>Residual Connections (Skip Connections)</strong></p><ul><li>Introduce connections that skip one or more layers, allowing gradients to flow directly through these connections during backpropagation.</li><li>Used in architectures like ResNet, where the output of a layer is added to the output of a previous layer.</li><li>Helps maintain gradient strength even in very deep networks by providing alternate gradient paths.</li></ul><p><strong>LSTM and GRU in RNNs</strong>:</p><ul><li>For recurrent neural networks (RNNs), using LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Units) architectures can mitigate the vanishing gradient problem.</li><li>These architectures are designed to maintain a more stable gradient over time, making them suitable for learning long-term dependencies.</li></ul><p><strong>Use of Activation Functions with Non-Zero Gradients</strong>:</p><ul><li>Opt for activation functions that do not saturate (i.e., where the derivative is never zero) in large portions of their input range.</li><li>ReLU, Leaky ReLU, ELU, and Swish, which have non-zero gradients for a wider range of inputs compared to Sigmoid or Tanh</li></ul><h3>Dropout</h3><p>Dropout is a regularization technique used to prevent overfitting in neural networks by randomly dropping units (neurons) during training.</p><p><strong>How It Works</strong>:</p><ul><li><strong>Random Neuron Dropping</strong>: During each training iteration, a random subset of neurons is temporarily removed from the network, including their incoming and outgoing connections.</li><li><strong>Dropout Rate</strong>: Specifies the percentage of neurons to drop, typically between 20–50%.</li><li><strong>During Training</strong>: Neurons are randomly dropped, forcing the network to learn more robust features that are not dependent on any specific neurons.</li><li><strong>During Inference</strong>: No neurons are dropped; instead, the weights are scaled down by the dropout rate to account for the absence of dropout during inference.</li></ul><h3>Regularization Techniques in Deep Learning</h3><p>Regularization techniques help to prevent overfitting by adding constraints or penalties to the model.</p><h4>L1 Regularization (Lasso)</h4><ul><li>Adds a penalty proportional to the absolute value of the weights to the loss function.</li><li>Encourages sparsity by driving some weights to zero, effectively performing feature selection.</li><li>Useful when you expect only a small number of features to be relevant.</li></ul><h4>L2 Regularization (Ridge)</h4><ul><li>Adds a penalty proportional to the square of the weights to the loss function.</li><li>Discourages large weights, leading to a more evenly distributed set of smaller weights, which can improve generalization.</li><li>Commonly used when all features are expected to contribute, but need to be controlled.</li></ul><h4>Elastic Net Regularization</h4><ul><li>Combines L1 and L2 regularization by adding both absolute and squared weight penalties to the loss function.</li><li>Provides a balance between the sparsity of L1 and the smoothness of L2, useful in cases where neither purely L1 nor L2 is optimal.</li><li>Effective for high-dimensional data where feature selection and model complexity control are both desired.</li></ul><h4>Dropout</h4><ul><li>Randomly drops a subset of neurons during each training iteration.</li><li>Prevents overfitting by ensuring that the model does not rely too heavily on any particular neurons or paths, improving generalization.</li><li>Commonly applied in fully connected layers of deep networks, especially in tasks with a large amount of training data.</li></ul><h4>Data Augmentation</h4><ul><li>Increases the diversity of the training data by applying random transformations (e.g., rotations, flips, zooms) to the input data.</li><li>Improves model generalization by exposing the model to a wider variety of input scenarios, making it more robust to variations.</li><li>Widely used in computer vision tasks to enhance limited datasets.</li></ul><h4>Early Stopping</h4><ul><li>Stops training when the model’s performance on a validation set starts to degrade, preventing overfitting.</li><li>Ensures the model does not overfit to the training data by stopping training at the optimal point.</li><li>Effective when a validation set is available to monitor performance during training.</li></ul><h4>Batch Normalization</h4><ul><li>Normalizes the inputs of each layer to have zero mean and unit variance.</li><li>Stabilizes and accelerates training by reducing internal covariate shift, acting as a regularizer by adding noise to the network.</li><li>Commonly used in deep networks, especially those with many layers.</li></ul><h4>Layer Normalization</h4><ul><li>Normalizes the inputs across the features of a layer, rather than across the batch.</li><li>Provides consistent normalization for each layer, independent of the batch size, improving convergence and stability.</li><li>Particularly effective in recurrent neural networks (RNNs) and transformer models where the batch size might vary.</li></ul><h4>Weight Decay</h4><ul><li>A form of L2 regularization applied during gradient descent, where the weights are scaled down after each update.</li><li>Similar to L2 regularization, it prevents weights from growing too large, leading to better generalization.</li><li>Often used in conjunction with other optimization techniques like SGD.</li></ul><h4>Max-Norm Regularization</h4><ul><li>Constrains the norm of the weight vector to be below a certain threshold.</li><li>Prevents weights from growing too large, which can help in controlling the model capacity.</li><li>Less common, but useful in conjunction with dropout.</li></ul><h3>Types of Optimizers in Deep Learning</h3><p>Optimizers are algorithms used to minimize the loss function by adjusting the model’s weights.</p><h4>Stochastic Gradient Descent (SGD)</h4><ul><li>Updates weights based on the gradient of the loss function with respect to each mini-batch of data.</li><li><strong>Pros</strong>: Simple and effective; works well with large datasets.</li><li><strong>Cons</strong>: Can be slow to converge; sensitive to learning rate settings.</li></ul><h4>Momentum</h4><ul><li>Enhances SGD by adding a fraction of the previous update to the current update to accelerate convergence.</li><li><strong>Pros</strong>: Helps to navigate along the relevant direction and smoothens out the updates.</li><li><strong>Cons</strong>: May still be sensitive to learning rates and may not perform well on very noisy gradients.</li></ul><h4>Nesterov Accelerated Gradient (NAG)</h4><ul><li>A variant of Momentum that applies the gradient update before calculating the new gradient, providing a more accurate correction.</li><li><strong>Pros</strong>: Provides faster convergence by correcting the trajectory before making the update.</li><li><strong>Cons</strong>: Computationally more complex than standard Momentum.</li></ul><h4>Adagrad (Adaptive Gradient Algorithm)</h4><ul><li>Adapts the learning rate for each parameter based on its past gradients.</li><li><strong>Pros</strong>: Automatically adjusts the learning rate based on parameter-specific learning rates.</li><li><strong>Cons</strong>: Can lead to overly aggressive learning rate decay and may not converge well on non-convex problems.</li></ul><h4>RMSprop (Root Mean Square Propagation)</h4><ul><li>Modifies Adagrad to include a moving average of squared gradients, preventing the learning rate from decreasing too quickly.</li><li><strong>Pros</strong>: Addresses Adagrad’s aggressive learning rate decay and works well in practice.</li><li><strong>Cons</strong>: Requires tuning of additional hyperparameters such as ρ\rhoρ.</li></ul><h4>Adam (Adaptive Moment Estimation)</h4><ul><li>Combines the benefits of Momentum and RMSprop by maintaining a moving average of both the gradients and their squared values.</li><li><strong>Pros</strong>: Combines the advantages of Momentum and adaptive learning rates, robust to various hyperparameter settings.</li><li><strong>Cons</strong>: More computationally intensive and may require fine-tuning.</li></ul><h4>AdaDelta</h4><ul><li>An extension of Adagrad that seeks to reduce the sensitivity to initial learning rates by using a moving window of gradient updates.</li><li><strong>Pros</strong>: Avoids the rapid decay of the learning rate seen in Adagrad.</li><li><strong>Cons</strong>: Complexity in implementation and tuning.</li></ul><h4>FTRL (Follow The Regularized Leader)</h4><ul><li>Optimizes a regularized loss function, suitable for large-scale and sparse datasets.</li><li><strong>Pros</strong>: Effective for large-scale machine learning problems, particularly with sparse data.</li><li><strong>Cons</strong>: Requires careful tuning and is less commonly used than other optimizers.</li></ul><h3>Types of Loss Functions in Deep Learning</h3><p>Loss functions measure the difference between the predicted and actual values and are essential for training models.</p><h4>1. Mean Squared Error (MSE)</h4><ul><li>Measures the average squared difference between predicted values and true values.</li><li><strong>When to Choose</strong>: When you want to measure the average squared difference between predicted and actual values. It’s ideal for problems where large errors are more significant and should be penalized more heavily.</li></ul><h4>2. Mean Absolute Error (MAE)</h4><ul><li>Measures the average absolute difference between predicted values and true values.</li><li><strong>When to Choose</strong>: When you want to measure the average absolute difference between predictions and true values. It’s less sensitive to outliers compared to MSE.</li></ul><h4>3. Huber Loss</h4><ul><li>Combines MSE and MAE to be less sensitive to outliers while maintaining smoothness.</li><li><strong>When to Choose</strong>: When you need a combination of MSE and MAE, offering robustness to outliers while maintaining smoothness. Useful when you want to balance sensitivity to outliers and smooth error gradients.</li></ul><h4>4. Binary Cross-Entropy Loss (Log Loss)</h4><ul><li>Measures the performance of a classification model where each class is binary.</li><li><strong>When to Choose</strong>: When dealing with binary classification problems where the output represents the probability of a class. It penalizes incorrect classifications based on how confident the model was.</li></ul><h4>5. Categorical Cross-Entropy Loss</h4><ul><li>Measures the performance of a classification model where classes are mutually exclusive.</li><li><strong>When to Choose</strong>: For problems where the output belongs to one of several classes. The loss function handles multiple classes and requires the output to be a probability distribution over the classes.</li></ul><h3>Convolutional Neural Networks (CNNs)</h3><p>CNNs are a class of deep neural networks specifically designed for processing structured grid data, such as images.</p><p><strong>Key Components</strong>:</p><ul><li><strong>Convolutional Layers</strong>: Apply convolutional filters to the input data to capture spatial hierarchies. Each filter detects specific features like edges, textures, or patterns.</li><li><strong>Activation Functions</strong>: Typically use ReLU (Rectified Linear Unit) to introduce non-linearity and enhance feature learning.</li><li><strong>Pooling Layers</strong>: Reduce the spatial dimensions of the feature maps (e.g., max pooling, average pooling) to reduce computational load and control overfitting.</li><li><strong>Fully Connected Layers</strong>: After flattening the pooled feature maps, these layers perform classification based on the learned features.</li><li><strong>Normalization Layers</strong>: Often include batch normalization to stabilize and accelerate training.</li></ul><p><strong>Advantages</strong>:</p><ul><li><strong>Spatial Hierarchies</strong>: Effectively captures spatial hierarchies and local patterns.</li><li><strong>Parameter Sharing</strong>: Convolutional filters are shared across the input space, reducing the number of parameters and improving computational efficiency.</li><li><strong>Translation Invariance</strong>: Able to detect features irrespective of their position in the input data.</li></ul><h4>Recurrent Neural Networks (RNNs)</h4><p>RNNs are a class of neural networks designed for processing sequential data by maintaining a hidden state that captures information about previous elements in the sequence. This makes RNNs well-suited for tasks where context and order are important.</p><p><strong>Key Components</strong>:</p><ul><li><strong>Recurrent Connections</strong>: Neurons have connections that loop back to themselves, allowing the network to maintain a memory of previous inputs.</li><li><strong>Hidden State</strong>: Carries information from previous time steps to influence the current output.</li><li><strong>Activation Functions</strong>: Often use functions like tanh or ReLU in hidden layers; output layers may use sigmoid or softmax functions depending on the task.</li></ul><p><strong>Advantages</strong>:</p><ul><li><strong>Sequential Data Handling</strong>: Capable of handling sequences and maintaining context over time.</li><li><strong>Memory</strong>: Can capture temporal dependencies and patterns in sequential data.</li></ul><h3>LSTM</h3><p><a href="https://colah.github.io/posts/2015-08-Understanding-LSTMs/">Link</a></p><p>LSTM networks are a type of Recurrent Neural Network (RNN) designed to address the limitations of standard RNNs, particularly the vanishing gradient problem, which can make learning long-term dependencies challenging.</p><h4>Components of LSTM</h4><p><strong>Cell State (Memory)</strong></p><ul><li>The cell state carries information across time steps in the sequence. It acts as a memory that can store and propagate relevant information over long sequences.</li><li>It is updated through gating mechanisms that control what information is added, kept, or removed.</li></ul><p><strong>Gates</strong> control the flow of information into and out of the cell state. Each gate has its own function and is implemented using a combination of sigmoid and tanh activation functions.</p><ul><li><strong>Forget Gate: </strong>Determines what information from the cell state should be discarded.</li><li><strong>Input Gate: </strong>Decides what new information should be added to the cell state.</li><li><strong>Output Gate: </strong>Determines the output of the LSTM cell based on the updated cell state.</li></ul><h3>Autoencoder</h3><p>Autoencoders are a type of neural network used for unsupervised learning tasks. They are designed to learn efficient representations of input data by encoding it into a lower-dimensional space and then reconstructing it back to the original space.</p><h4>Components of Autoencoders</h4><p><strong>Encoder</strong></p><ul><li><strong>Purpose</strong>: The encoder maps the input data to a lower-dimensional latent space (or bottleneck).</li><li><strong>Architecture</strong>: Typically consists of one or more layers of neural networks (e.g., dense layers, convolutional layers) that progressively reduce the dimensionality of the input.</li><li><strong>Output</strong>: Encoded representation (latent vector) of the input data.</li></ul><p><strong>Latent Space (Bottleneck)</strong></p><ul><li><strong>Purpose</strong>: A compressed representation of the input data where the important features are retained.</li><li>The dimensionality of the latent space is usually smaller than the input space, capturing essential features while discarding redundant information.</li></ul><p><strong>Decoder</strong></p><ul><li><strong>Purpose</strong>: The decoder reconstructs the original input data from the latent representation.</li><li><strong>Architecture</strong>: Consists of layers that progressively increase the dimensionality of the encoded data back to the original input space.</li><li><strong>Output</strong>: Reconstructed data that should ideally be as close as possible to the original input.</li></ul><p>All the best :)</p><p>Want More…<br><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-nlp-31f578a86034">All I needed to crack the Microsoft Interview — NLP | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-machine-learning-4965cbdcf8d1">All I needed to crack the Microsoft Interview — Machine Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-statistics-1f3d3ca8055d">All I needed to crack the Microsoft Interview — Statistics | by Mehul Jain | Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c39bad25f089" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All I needed to crack the Microsoft Interview — Machine Learning]]></title>
            <link>https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-machine-learning-4965cbdcf8d1?source=rss-144465630849------2</link>
            <guid isPermaLink="false">https://medium.com/p/4965cbdcf8d1</guid>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mehul Jain]]></dc:creator>
            <pubDate>Thu, 15 Aug 2024 11:00:11 GMT</pubDate>
            <atom:updated>2025-01-28T10:51:36.706Z</atom:updated>
            <content:encoded><![CDATA[<h3>All I needed to crack the Microsoft Interview — Machine Learning</h3><p><em>For me, these notes were invaluable for a quick review before any data science interview. I hope you find them helpful too.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qaVh8JVhD-jO-uh3mjqlug.jpeg" /><figcaption>Image by Microsoft Copilot</figcaption></figure><h3>Naive Bayes Classifier</h3><ul><li>Assumption: Features should be independent. This is a very Naïve assumption hence the name naïve</li><li>It is Based on Bayes theorem</li><li>Naive Bayes is a linear classifier but on a logarithmic scale</li><li>Feature Likelihood Estimation for categorical features, use relative frequencies (counts) of each feature value within each class.</li><li>Feature Likelihood Estimation for continuous features, use probability density functions like Gaussian distribution to model the data within each class. It is important to remove the outliers, one outlier will affect the mean and variance and consequently, all the likelihood probabilities will be affected.</li><li>Time and space complexity — O(n*d*c) n-categories, d- dimensions and c-classes.</li><li>Naive Bayes classifier is well-suited for text classification tasks, spam detection, sentiment analysis, and other classification problems with a large number of features</li><li>Naïve Bayes can handle missing data — Laplace smoothing is a technique used to avoid zero probabilities when a feature value has not been seen during training. It is applied to prevent division by zero in the likelihood estimation.</li><li>Categorical → In test data, if we have got a new category that was not there in the train data. Consequently, there will no likelihood probabilities available for this category and will result in zero’s. To avoid this zero probability issue we do Laplace smoothing.</li><li>Converges faster, Well with Categorical features, Robust to outliers (It skips missing value probability) but Correlation affect performance.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*lWzUkREsBnmKrJQLxKhMgg.png" /></figure><h3><strong>Linear Regression</strong></h3><ul><li>Expectation of error E(e) = 0, equally likely that the error is +ve or -ve</li><li>Can be solved suing close form — multiplying the metrics and taking its inverse is difficult — O(D³)</li><li>Even if solving gradient = 0 (close form) is feasible, gradient decent can be more efficient</li><li>Gradient descent relies on choosing step size and convergence criteria</li><li>High leverage points — which are far from the other observations</li><li>Influential points — which will influence the coefficients</li><li>Asymmetric cost function — if the cost of over estimating and underestimating is not same</li><li>If independent features are correlated it may affect performance and Sensitive to missing value and outlier</li><li>In L2, if we remove all the feature with low Coeff, it will not help as 2 highly correlated features will reduce the Coeff as they are summing up the Y value</li><li>In L1 since |w| is not differentiable at zero, we use sub gradients — Coordinated decent</li><li>Coordinated decent is used in L1 — It keeps the one weight constant — update the other and vice versa — it reaches to min like a steps in the contour. It don’t not require learning rate</li><li>Elastic net handles both the issues of L1 and L2</li><li>In L1 and L2, slope is reduced — we are penalizing if the slop is steep</li><li><strong>Debiasing Lasso — </strong>Lasso skinks coefficient related to Least Square solution which will result in more bias, less variance</li></ul><p>Can reduce the bias as follow-</p><ul><li>Run Lasso to select feature</li><li>Run least squares regression with only selected features</li><li>L1 issue is it picks random variable if 2 or more feature correlated. Ridge provides a stable predictions.</li></ul><p>Assumption:</p><ul><li>Linear relationship — check using plots</li><li>Homoscedasticity — non-constant variance in the error terms — check using plots</li><li>Multicollinearity — independent variables show moderate to high correlation — check using VIF</li><li>Normality — check using QQ plot</li><li>Autocorrelation — correlation in error terms — check using Durbin — Watson (DW)</li></ul><p>Important Params —</p><ul><li>fit_intercept — if we want to the intercept</li><li>Normalize — if fit_intercept is true, then Normalize the features</li><li>Positive — if we want all Coeff to be +ve</li></ul><h3><strong>Logistic Regression</strong></h3><ul><li>Likelihood l(w): Measure quality of fit for model with coefficient w</li><li>Maximum Likelihood estimator — product of all probability — goal is to make it as large as possible</li><li>It used Gradient ascend</li><li>In logistic, if we have huge weights the sigmoid slope would be huge, and for very example it will give us very high probability (for each class) which might not be true</li><li>Assumption: Linear relationship independent features and log odds</li><li>Multicollinearity can be handled using L1 and L2 without dropping the feature</li><li>Why do we use Logistic Regression rather than Linear Regression — Linear will change the line if any new outlier is added — we need to change the threshold to predict the correct values. Also, Linear model can also predicts values greater than 1 and smaller than 0</li><li>Log Loss (also called Binary Cross-Entropy). It penalizes misclassifications and is designed to minimize the difference between the predicted probabilities and the true class labels.</li><li>If there are significant dependencies or overlaps between classes, multinomial classification might be more appropriate.</li><li>Prone to overfitting and Sensitive to missing value and outlier</li></ul><p>Important Params —</p><ul><li>Penalty — L1 or L2</li><li>C- inverse of regularization strength</li><li>Class weight — balance or not</li><li>Max iter — no. of iteration for the solver to converge</li><li>L1 ratio — if 0 then L2, 1 then L1 if in between then Elasticnet</li></ul><h3><strong>Decision Trees</strong></h3><ul><li>Entropy — Measure the purity of split with respect to a node only — range [0(pure),1(impure)]</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/0*Wvt7xtGXw8_Xuy48" /></figure><ul><li>Information gain — Parent Entropy — weighted avg of Child Entropy</li><li>Gini — Gini is half of Entropy, this is computationally better then Entropy (Doesn’t have log in formula)</li><li>For Numeric features — Sorting all the values, Consider a threshold(row value) and split the data into 2 parts</li><li>For continues Y variable — Variance is used — Minimum variation in the nodes after the split</li><li>For pruning — Total cost = classification error + no of leaf nodes</li><li>Regularization — pruning, early stopping — if the error is not reducing even with further split or don’t split if the leaf have only x nodes</li><li>Can handle to missing value and outlier</li><li>Overfitting, Low Bias and High Variance</li><li>Not suitable for large dataset</li></ul><p>Importanct Params —</p><ul><li>Criteria — Gini, entropy, log loss</li><li>Splitter — Best, random</li><li>Max depth</li><li>Min sample split — min no of sample required to split the node</li><li>Min sample leaf — min samples required to be a leaf node</li><li>Max features</li><li>Min impurity decrease — Node will split if this split reduce a impurity</li><li>Class weight — balance or not</li><li>CCP_alpha — this is used to regularize the number of leaves</li></ul><h3><strong>Random forest (Bagging)</strong></h3><ul><li>Decision Tree — Low Bias and High Variance</li><li>Random Forest — Low Bias and Low Variance</li><li>Bootstrap aggregation</li><li>Row sampling with Replacement — some rows will be repeated across samples</li><li>Calculates feature importance by measuring how much the accuracy of the model decreases when the values of a particular feature are randomly permuted. Features that cause a significant decrease in accuracy are considered more important.</li><li>Less Overfitting</li><li>Biased with features having many categories</li></ul><p><strong>How OOB Error Estimate Works:</strong></p><ul><li>When constructing each tree in a Random Forest, a random subset of the data is sampled with replacement. This results in some data points being left out, forming the out-of-bag samples.</li><li>For each data point, it is not used in the construction of some trees due to the bootstrapping process. The out-of-bag samples for a given tree are used to evaluate the model’s performance on unseen data.</li><li>For each out-of-bag sample, predictions are collected from the trees that did not use that sample during training. The predictions are then aggregated, usually by a majority vote for classification or averaging for regression.</li><li>The aggregated predictions for the out-of-bag samples are compared with the true labels, and the error (classification error or mean squared error for regression) is computed. This error estimate is the out-of-bag error.</li></ul><p>Important Params —</p><ul><li>N estimator — number of trees</li><li>Same as decision tree</li><li>Bootstrap — Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree</li><li>Oob_score — Whether to use out-of-bag samples to estimate the generalization score</li><li>max_samples — If bootstrap is True, the number of samples to draw from X to train each base estimator</li></ul><h3><strong>Adaboost (boosting)</strong></h3><p>Working of Adaboost —</p><ul><li>Step 1: weight = 1/n</li><li>Step 2: Stumps are created (Decision trees with depth =1) and select the best stump using entropy</li><li>Step 3: Update the weights — Increase the weight for incorrect records and decrease for others</li><li>Step 4: New data set will be created based on probability buckets then train a new Stump using these records (wide range prob bucket will be picked multiple times)</li><li>Step 5: Use the sign of weighted average technique for final results — where weight is the sum of all the mistakes</li></ul><h3><strong>GBM (Boosting)</strong></h3><p>Working of GBM</p><ul><li>Step 1: Base model — Avg of all values, or find min Y using derivative</li><li>Step 2: Compute Residual Error (Actual — Base value)</li><li>Step 3: Create a tree where Y var is Residual</li><li>Step 4: Update the value using learning rate.</li><li>The “gradient” in Gradient Boosting refers to the negative gradient of the loss function with respect to the model’s predictions. It guides the algorithm to focus on the instances where the model performs poorly and attempts to minimize the error in those areas.</li><li>Calculates feature importance based on the total reduction in the loss function achieved by a particular feature when it is used for splitting in the trees. Features that result in higher reductions are considered more important.</li><li>Handle missing data by considering missing values as a separate category during tree construction.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*-oJrojFLTeXLfM2pNy2gVg.png" /></figure><h3><strong>XGBoost</strong></h3><p>Working of XGB</p><ul><li>Step1: Create Binary classification only</li><li>Step2: Compute similarity weight for all nodes (how similar the node elements are)</li><li>Step3: Information gain to understand which feature to select for splitting</li><li>Step4: Update the residual using sigmoid function(for classification)</li><li>Learns the best direction to send missing values based on the overall loss reduction during the split.</li><li>Can handle multi-class classification using the “multi:softmax”</li></ul><h3>SVM</h3><ul><li>Aim is to maximize the marginal distance between the marginal plans</li><li>The marginal plan should pass from a <strong>point</strong> which is closest to hyperplane. Points are called Support vector</li><li>Aim of Kernal: Transform Low dimension to high dimension</li><li>SVM Kernal — Polynomial, Sigmoid radial basis function</li><li>Regularization — How many error points X Sum of distance of those error points from plan</li><li>Cost function in SVM is the hinge loss function, which penalizes misclassifications. It aims to maximize the margin between the classes while minimizing the classification errors.</li><li>Hinge loss(y, f(x)) = max(0, 1 — y * f(x))</li><li>If this is not greater than 1 then there is a misclassification</li><li>More effective in high dimensional space because that data is space and it is able to get good support vectors</li><li>Over fitting is less — To avoid overfitting, we use Soft margins</li><li>More training time, Difficult to choose good kernel, Difficult to tune, Sensitive to missing value and outlier</li></ul><p>Important Params —</p><ul><li>C — Regularization parament, it is inversely proportional to Regularization strength</li><li>Kernal — Linear, ploy, rbf, sigmoid</li><li>Gamma — Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. Higher values of gamma allow the model to fit the training data more precisely, potentially leading to overfitting.</li><li>Coeff — Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.</li><li>Shrinkage — can speed up the training, might provide inaccurate results</li><li>Probability — when we need probability estimate</li></ul><h3><strong>K-Nearest Neighbor</strong></h3><ul><li>Calculate the distance from the nearest points and select top k</li><li>How many nearest neighbors belongs to category A and B</li><li>Impacted by imbalance data and outliers</li></ul><p>Working of KNN</p><ul><li>Step 1: Calculate Distances: Calculate the distance between the test_instance and all instances in the train_data.</li><li>Step 2: Sort Distances: Sort the distances in ascending order to find the k nearest neighbors.</li><li>Step 3: Select Neighbors: Choose the k instances with the smallest distances as the nearest neighbors.</li><li>Step 4: Count Votes: Count the occurrences of each class label among the k neighbors.</li><li>Step 5: Select Class Label: Choose the class label with the highest number of votes as the predicted class for the test_instance.</li><li>Weighted Knn to get smoother curve — it will use Kernel — Y is weighted average</li><li>If the kernel have smaller bandwidth the curve will not be smooth</li><li>KD tree for search for any data point it will reduce the time complexity to O(log N)</li><li>It can provide a very good fit as the data increase but it will take time for one query</li></ul><p>Important Params —</p><ul><li>N neighbor — no of neighbor</li><li>Weights — Uniform, distance — weight points by the inverse of their distance</li><li>Algo — ball tree, KD tree, brute force</li><li>P — 1 is Manhattan distance, 2 is Euclidian distance</li></ul><h3><strong>K-Means Clustering</strong></h3><p>Working of K-means -</p><ul><li>Initialize the K centroids</li><li>Separate the groups using centroids and group the points</li><li>Take the mean of that group and iterate</li><li>Kmeans works as <strong>coordinate descent</strong></li><li>It will only converge to local optimum because of its initialization</li><li>Kmeans++ uses Smart initialization -&gt; Choose first cluster center uniformly at random from the data points. For each obs x, compute distance to nearest cluster center. Choose new cluster from amongst the data points with probability of x being chosen proportional to d(x)²</li></ul><p>How to select K value:</p><ul><li>Elbow method -&gt; with in cluster sum of square wrt centroid</li><li>Silhouette method -&gt;</li><li>For each data point i in the dataset:</li><li>a(i): The average distance of i to all other data points within the same cluster.</li><li>b(i): The average distance of i to all data points in the nearest neighboring cluster (the cluster that is not i’s cluster).</li><li>The Silhouette score for data point i is then given by: (b(i) — a(i)) / max(a(i), b(i))</li><li>A score close to +1 indicates that the data point is well-clustered and lies far away from neighboring clusters, indicating good separation.</li><li>A score close to 0 indicates that the data point is close to the boundary between two clusters.</li><li>A score close to -1 indicates that the data point may have been assigned to the wrong cluster and would be better suited in another cluster.</li><li>Assumptions -&gt;</li><li>Clusters are spherical and have approximately equal variance.</li><li>Clusters have roughly the same number of data points.</li><li>Impacted by outliers</li></ul><p>Important Params —</p><ul><li>N cluster</li><li>Init — K-means++</li><li>n_init — The number of times the algorithm will be run with different centroid seeds. The final results will be the best output in terms of inertia</li></ul><h3><strong>Hierarchical Clustering</strong></h3><ul><li>It used Dendrogram — X axis — points, Y axis — Distance between points</li><li>Hack to select the number of clusters — Cut the Longest vertical line where none of the horizontal lines are passings</li><li>Agglomerative: It starts with each data point as a separate cluster and merges the closest clusters iteratively until all data points are in one cluster.</li><li>Divisive: It starts with all data points in a single cluster and recursively splits clusters until each data point forms a separate cluster.</li><li>Time complexity O(N²logN)</li><li>Hidden Markov clustering — when data depends on time. Probability depends on the pervious cluster assignment</li></ul><h3><strong>DBSCAN </strong>Clustering</h3><ul><li>It create clusters around core points and border points and leave Noise points</li><li><strong>Core points:</strong> Core points have at least minPts neighboring points within ε distance and form the central points of clusters.</li><li><strong>Border points:</strong> Border points have fewer than minPts neighboring points within ε distance but are reachable from any core points. They belong to a cluster but are not the core points themselves.</li><li><strong>Noise points:</strong> Noise points have fewer than minPts neighboring points within ε distance and are not reachable from any core point. They are treated as outliers.</li><li>ε can be found using K distance graph</li><li>K dist — Y = distance btw 2 points , X = data points sorted by distance</li><li>Can handle outliers</li><li>DBSCAN can handle clusters of different shapes and sizes</li><li>Difficulty in handling datasets with varying density or density drops</li></ul><h3><strong>GMM Clustering</strong></h3><ul><li>Probabilistic model used for clustering data by assuming that the data points are generated from a mixture of multiple Gaussian distributions.</li><li>GMM clustering works by estimating the parameters of the Gaussian distributions that best represent the data.</li><li>1 D Gaussian: specified by mean and variance</li><li>2 D Gaussian: specified by means and covariance metrics</li><li>Computationally heavy — we restrict the parameters in metrics — we keep only diagonal params. Ellipse will have only one orientation</li><li>Updates the means and covariance matrices of the Gaussians until convergence.</li></ul><p>Key components of GMM clustering are:</p><ul><li>Gaussian distributions: Each cluster is represented by a Gaussian distribution with a mean and covariance matrix.</li><li>Mixing coefficients: Represent the proportion of data points in each cluster.</li><li>Covariance matrices: Capture the spread and orientation of the data within each cluster.</li><li>Cluster weight — how much prevalent the cluster is</li><li><strong>Expectation step (E-step), </strong>where the probabilities of data points belonging to each cluster are computed. We compute responsibility of cluster taking an observation given cluster params</li><li><strong>Maximization step (M-step),</strong> we will calculate the MLE of Gaussians params using soft assignments</li><li>EM is a coordinate ascent — it will locally converge</li><li>No of clusters in GMM — Bayesian Information Criterion (BIC) or the Akaike Information Criterion (AIC). Both are based on the concept of information theory and aim to strike a balance between model fit and model complexity.</li><li>Can handle outliers</li></ul><h3><strong>LDA</strong></h3><ul><li>Bayesian model</li><li>Mix membership modelling</li><li>Used for topic modeling</li><li>LDA assumes that each document is a mixture of multiple topics, and each topic is a distribution over words in the vocabulary</li></ul><p>For each document:</p><ul><li>Randomly select a distribution of topics.</li><li>For each word in the document:</li><li>Randomly select a topic from the distribution of topics.</li><li>Randomly select a word from the distribution of words associated with that topic.</li><li><strong>Gibbs Sampling:</strong> is an iterative sampling method used for approximate inference in probabilistic models like LDA.</li><li>It allows us to sample from complex probability distributions by iteratively sampling from conditional distributions</li></ul><h3><strong>Linear discrimination analysis</strong></h3><ul><li>LDA is like PCA, but it focuses on Maximizing the seperatibility among know categories</li><li>It creates a new axis and projects the data onto this new axis in a way to maximize the separation of the two categories</li><li>Max the dist between Mean 1 and mean 2 and min the variation (scatter) within each category</li><li>If there are more than 2 cat than, we calculate the distance btn the centroid and mean of a calls</li></ul><h3><strong>PCA</strong></h3><ul><li>Step1: Centre the data (standardization)</li><li>Step2: Calculate covariance metrics (correlation) — if we multiple any vector with the covariance metrics, it will push a vector closer to eigen vector(Vector with greatest variance)</li><li>Step3: Calculate Eigen value and Eigen vector of metrics (If we multiply Eigen vector by Matrix the direction will be same or opposite)</li><li>Eigen vector represents the direction of maximum variance in the data</li><li>Eigen values represent the amount of variance explained</li><li>Step4: Order the eigen vector to find first PC</li><li>PCs are the eigenvectors of the covariance matrix, and the corresponding eigenvalues represent the variance along each PC</li><li>Impacted by outliers</li><li>t-SNE is a non-linear technique, while PCA is a linear technique.</li></ul><h3>Forecasting</h3><ul><li><strong>Moving Average (MA)</strong>: Uses the average of ERROR observations</li><li><strong>Exponential Smoothing (ES</strong>): Assigns exponentially decreasing weights to past observations.</li><li><strong>Autoregressive (AR)</strong>: Predicts future values based on past observations and a linear relationship</li></ul><p><strong>Stationarity time series</strong>:</p><ul><li>Constant Mean: The mean (average) of the series remains constant and does not exhibit any significant trend or shift over time.</li><li>Constant Variance: The variance (or standard deviation) of the series remains constant over time, indicating consistent dispersion of data points.</li><li>Constant Autocovariance: The autocovariance between two observations at different time points remains constant regardless of the time shift.</li></ul><p><strong>Components of ARIMA:</strong></p><ul><li>Autoregression (AR): Autoregression involves predicting future values based on previous observations in the time series. The AR component models the relationship between an observation and a number of lagged observations (its own past values).</li><li>Differencing (I for Integrated): Differencing involves transforming a time series to achieve stationarity. It subtracts the current value from a previous value to remove trends or seasonality.</li><li>Moving Average (MA): The MA component models the relationship between the current observation and a residual error from a moving average model applied to lagged observations.</li></ul><p>ACF helps identify MA(q), PACF helps identify AR(p), and differencing (d) is assessed for stationarity.</p><p><strong>Autocorrelation Function (ACF):</strong></p><ul><li>Definition: ACF measures the correlation between a time series and its lagged values at different time lags.</li></ul><p><strong>Partial Autocorrelation Function (PACF):</strong></p><ul><li>Definition: PACF measures the correlation between a time series and its lagged values after removing the effects of intermediate lagged observations.</li></ul><h3><strong>Information Criteria:</strong></h3><ul><li>Akaike Information Criterion (AIC):</li><li>Measures the goodness of fit of a model, balancing the model’s complexity and its ability to minimize information loss.</li><li>Lower AIC values indicate a better trade-off between complexity and goodness of fit.</li><li>Bayesian Information Criterion (BIC):</li><li>Similar to AIC but penalizes models with more parameters more severely.</li><li>Helps in model selection by favoring simpler models if they provide similar goodness of fit.</li></ul><h3><strong>Metrics</strong></h3><p><strong>MSE:</strong></p><ul><li>It penalizes even small errors by squaring them, which essentially leads to an overestimation of how bad the model is.</li><li>Due to the squaring factor, it’s fundamentally more prone to outliers than other metrics.</li></ul><p><strong>MAE :</strong></p><ul><li>It’s more robust towards outliers than MAE, since it doesn’t exaggerate errors.</li><li>It gives us a measure of how far the predictions were from the actual output but not the direction</li></ul><p><strong>RMSE</strong>: It handles the penalization of smaller errors done by MSE by square rooting it.</p><p><strong>Precision</strong>: Focuses on the proportion of true positive predictions among all positive predictions. It is useful when the cost of false positives is high (e.g., in medical diagnosis).</p><p><strong>Recall </strong>(Sensitivity or True Positive Rate): Measures the proportion of true positives correctly identified among all actual positive instances. It is valuable when the cost of false negatives is high (e.g., in fraud detection).</p><p><strong>Precision recall curve</strong> is for selecting threshold</p><p><strong>AOC ROC</strong> — Y = Sensitivity, X = 1- Specificity</p><p><strong>Sensitivity</strong> — proportion of the positive class got correctly classified</p><p><strong>1-Specificity</strong> — proportion of the negative class got correctly classified.</p><p>For better understanding, I would highly recommend to write your own code from scratch without using libraries such as SKlearn. You can follow this amazing <a href="https://github.com/patrickloeber/MLfromscratch"><em>Github repo</em></a></p><p>All the best :)</p><p>Want More…<br><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-nlp-31f578a86034">All I needed to crack the Microsoft Interview — NLP | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-deep-learning-c39bad25f089">All I needed to crack the Microsoft Interview — Deep Learning | by Mehul Jain | Medium</a></p><p><a href="https://medium.com/@j13mehul/what-i-revised-before-my-microsoft-interview-statistics-1f3d3ca8055d">All I needed to crack the Microsoft Interview — Statistics | by Mehul Jain | Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4965cbdcf8d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>