<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mareddy Sushma on Medium]]></title>
        <description><![CDATA[Stories by Mareddy Sushma on Medium]]></description>
        <link>https://medium.com/@sushma.mareddy?source=rss-e9ca8a4420b7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*3NfD-m5yA81BpXkmYgpoPw.jpeg</url>
            <title>Stories by Mareddy Sushma on Medium</title>
            <link>https://medium.com/@sushma.mareddy?source=rss-e9ca8a4420b7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 10:34:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sushma.mareddy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Image Segmentation]]></title>
            <link>https://medium.com/@sushma.mareddy/image-segmentation-e4744d818486?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e4744d818486</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Thu, 20 Feb 2025 21:10:38 GMT</pubDate>
            <atom:updated>2025-02-20T21:10:38.587Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Image segmentation is one of the most fascinating areas in computer vision. It allows machines to understand images at a deeper level, just like how we see and interpret the world around us. But how does it actually work? In this article, we’ll break down segmentation from the basics to advanced techniques, including CNN and Transformer-based models, in the simplest way possible!</em></p><h3>What is Image Segmentation?</h3><p>Imagine you’re looking at a picture of a dog in a park. You instantly recognize the dog, the grass, the sky, and the trees. Your brain automatically separates these objects and understands their relationship to each other.</p><p>Computers, however, don’t see images the way we do—they see numbers (pixel values). To help them make sense of images, we use <strong>image segmentation</strong>. Segmentation is the process of breaking down an image into meaningful parts so that a computer can analyze and understand it.</p><h3>Types of Image Segmentation</h3><p>Depending on the level of detail needed, there are three main types of segmentation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*e5cCz_q87RgMn--k.jpg" /></figure><h4>1. Semantic Segmentation</h4><p>This labels every pixel in an image with a class name, but it does not distinguish between different objects of the same type.</p><p><strong>Example:</strong> If there are 3 cats in an image, semantic segmentation labels all of them as “cat” without differentiating them.</p><p><strong>Use Cases:</strong></p><ul><li>Self-driving cars (to identify roads, pedestrians, cars, etc.)</li><li>Medical imaging (detecting tumors in scans)</li></ul><h4>2. Instance Segmentation</h4><p>This not only classifies objects but also distinguishes between individual instances of the same object.</p><p><strong>Example:</strong> If an image has 3 cats, instance segmentation identifies them as &quot;Cat 1&quot;, &quot;Cat 2&quot;, and &quot;Cat 3&quot; separately.</p><p><strong>Use Cases:</strong></p><ul><li>Object tracking in videos</li><li>Counting objects (like cars in traffic monitoring)</li></ul><h4>3. Panoptic Segmentation</h4><p>A mix of both <strong>semantic</strong> and <strong>instance segmentation</strong>. It labels objects <strong>and</strong> tells them apart.</p><p><strong>Example:</strong> It identifies that there are multiple cats in an image and also separates background elements like the couch, floor, and table.</p><p><strong>Use Cases:</strong></p><ul><li>Scene understanding in augmented reality (AR)</li><li>Robotics (robots understanding environments better)</li></ul><h3>Traditional (Old School) Segmentation Methods</h3><p>Before deep learning, segmentation relied on <strong>rule-based</strong> methods. These techniques still work for simple tasks but struggle with complex images.</p><h4>Thresholding</h4><p>This method divides an image based on pixel brightness. If a pixel is <strong>brighter</strong> than a certain value, it’s considered <strong>&quot;object&quot;</strong>, otherwise, it’s <strong>&quot;background&quot;</strong>.</p><p><strong>Example:</strong> Black-and-white document scanning.</p><p><strong>Limitations:</strong> Doesn’t work well when lighting changes.</p><h4>Edge Detection</h4><p>Finds object edges using mathematical filters like <strong>Canny Edge Detector</strong> or <strong>Sobel Filter</strong>.</p><p><strong>Example:</strong> Used in barcode scanning.</p><p><strong>Limitations:</strong> Doesn’t understand objects, only their boundaries.</p><h4>Region-Based Segmentation</h4><p>Groups similar pixels together based on color, texture, or intensity.</p><p><strong>Example:</strong> <strong>Watershed Algorithm</strong> (often used in medical imaging).</p><p><strong>Limitations:</strong> Can over-segment images, leading to unnecessary details.</p><h3>Deep Learning-Based Segmentation (Modern Approaches)</h3><p>Deep learning has <strong>revolutionized segmentation</strong> with two major architectures:</p><ol><li><strong>CNN-Based Models</strong> (Convolutional Neural Networks)</li><li><strong>Transformer-Based Models</strong> (Inspired by NLP Transformers)</li></ol><p>Let’s explore both in detail!</p><h3>CNN-Based Segmentation Models</h3><p>CNNs are <strong>good at recognizing patterns</strong> in images and are widely used for segmentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/0*V8UHtjwtb99OCnGJ" /></figure><h4>Fully Convolutional Networks (FCN) – The First Deep Learning Segmentation Model</h4><p><strong>How it works:</strong> Removes <strong>fully connected layers</strong> in traditional CNNs and replaces them with <strong>convolutional layers</strong> to make predictions on every pixel.</p><p><strong>Strengths:</strong></p><ul><li>First model to achieve <strong>pixel-wise classification</strong>.</li><li>Can be used for <strong>semantic segmentation</strong>.</li></ul><p><strong>Limitations:</strong></p><ul><li>Can’t capture <strong>fine details</strong>.</li></ul><h4>U-Net – Best for Medical Images</h4><p><strong>How it works:</strong></p><ul><li>It’s <strong>U-shaped</strong> and has <strong>skip connections</strong> that help recover lost details.</li><li>Initially designed for <strong>biomedical image segmentation</strong> (e.g., detecting tumors).</li></ul><p><strong>Strengths:</strong></p><ul><li>Works well with <strong>small datasets</strong>.</li><li>Preserves <strong>fine details</strong> (thanks to skip connections).</li></ul><p><strong>Limitations:</strong></p><ul><li>Not the best for <strong>real-time applications</strong> due to its complexity.</li></ul><h4>DeepLab (v1, v2, v3, v3+) – Google’s Improvement</h4><p><strong>How it works:</strong></p><ul><li>Uses <strong>Atrous (Dilated) Convolutions</strong> to capture <strong>long-range dependencies</strong>.</li><li>Includes <strong>Conditional Random Fields (CRFs)</strong> for sharper object boundaries.</li></ul><p><strong>Strengths:</strong></p><ul><li><strong>DeepLab v3+</strong> is one of the <strong>best</strong> CNN-based segmentation models.</li><li>Works well for <strong>complex scenes</strong>.</li></ul><p><strong>Limitations:</strong></p><ul><li>Computationally expensive.</li></ul><h3>Transformer-Based Segmentation Models</h3><p>Recently, <strong>Vision Transformers (ViTs)</strong> have outperformed CNNs in some segmentation tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7RetvYtnSa8cksCD.png" /></figure><h4>Vision Transformer (ViT) – Inspired by NLP</h4><p><strong>How it works:</strong></p><ul><li>Breaks an image into <strong>patches</strong> and processes them like <strong>text tokens</strong>.</li><li>Uses <strong>self-attention</strong> to capture <strong>long-range relationships</strong> between pixels.</li></ul><p><strong>Strengths:</strong></p><ul><li>Learns <strong>global</strong> information better than CNNs.</li></ul><p><strong>Limitations:</strong></p><ul><li>Needs <strong>a lot of data</strong> to work well.</li></ul><h4>SegFormer – A Lightweight Transformer Model</h4><p><strong>How it works:</strong></p><ul><li><strong>Combines CNN &amp; Transformer</strong> features for efficiency.</li><li>Works even with <strong>small datasets</strong>.</li></ul><p><strong>Strengths:</strong></p><ul><li><strong>Fast and lightweight</strong> (great for edge devices).</li></ul><p><strong>Limitations:</strong></p><ul><li>Less interpretability compared to ViT.</li></ul><h4>SAM (Segment Anything Model by Meta)</h4><p><strong>How it works:</strong></p><ul><li>Can segment <strong>any object in any image</strong> <strong>without fine-tuning</strong>!</li><li>Uses a massive dataset to generalize segmentation tasks.</li></ul><p><strong>Strengths:</strong></p><ul><li>Works in <strong>zero-shot learning</strong> mode.</li><li>Can be used for <strong>real-time applications</strong>.</li></ul><p><strong>Limitations:</strong></p><ul><li><strong>Over-generalization</strong>—sometimes segments objects incorrectly.</li></ul><h3>Final Thoughts</h3><p>Segmentation is a <strong>powerful tool</strong> in computer vision, from helping doctors detect diseases to enabling self-driving cars. While CNNs have been the <strong>traditional choice</strong>, <strong>transformers</strong> are pushing the boundaries even further.</p><p>As AI research evolves, <strong>newer and better segmentation models</strong> will keep emerging.</p><p>Let me know if you need to understand any model in detail, I’ll surely write another article to explain! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4744d818486" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mixture of Experts (MoE) — A Simple Explanation]]></title>
            <link>https://medium.com/@sushma.mareddy/mixture-of-experts-moe-a-simple-explanation-8f7f07ea1439?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f7f07ea1439</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Thu, 20 Feb 2025 02:35:16 GMT</pubDate>
            <atom:updated>2025-02-20T02:35:16.822Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mixture of Experts (MoE) — A Simple Explanation</h3><p>Imagine you are in a <strong>big classroom</strong> where students have different strengths. Some are great at <strong>math</strong>, others at <strong>writing</strong>, and some are <strong>science wizards</strong>. Now, if you have a <strong>math problem</strong>, would you ask the best writer for help? No! You would ask the student who is best at math.</p><p>This is exactly how <strong>Mixture of Experts (MoE)</strong> works in artificial intelligence (AI). Instead of using <strong>one single model</strong> for everything, MoE <strong>divides the work</strong> among multiple smaller models called <strong>experts</strong>, and each one specializes in solving a different part of the problem. Let’s break it down step by step!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4ijMK5FYrxmzLHsv.png" /></figure><h3>1. The Core Idea: Divide and Conquer</h3><p>Instead of using <strong>one large neural network</strong>, MoE uses <strong>many smaller networks</strong> (called <strong>experts</strong>) that <strong>work together</strong>. But here’s the trick:</p><ul><li>Not <strong>all experts</strong> work on every problem.</li><li>A special <strong>“gate”</strong> decides <strong>which experts</strong> are best for the job.</li><li>Only the <strong>most relevant experts</strong> are activated for each input.</li></ul><p>This is like having a <strong>team of tutors</strong> where:</p><ul><li>A <strong>math tutor</strong> helps with math questions.</li><li>A <strong>history tutor</strong> helps with history.</li><li>A <strong>science tutor</strong> helps with science.</li></ul><p>Rather than asking <strong>everyone</strong> for help, you ask the right <strong>specialist</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/0*dCR6Ula57R9TFaiB.png" /></figure><h3>2. How MoE Works (Step by Step)</h3><p>MoE follows <strong>three main steps</strong>:</p><h4>Step 1: Input Comes In</h4><ul><li>You give a <strong>question</strong> (input) to the model.</li><li>Example: “What is 2+2?”</li></ul><h4>Step 2: The Gating Network Picks Experts</h4><ul><li>A <strong>small network</strong> (called the <strong>gating network</strong>) looks at the question and decides which experts are best.</li><li>Example: Since “2+2” is a math question, it picks the <strong>math expert</strong>.</li></ul><h4>Step 3: Experts Answer the Question</h4><ul><li>The <strong>chosen experts</strong> work on the problem.</li><li>Example: The math expert answers, “2+2 = 4.”</li><li>The final <strong>output</strong> is the combined answer.</li></ul><h4>Bonus: Learning Which Expert to Pick</h4><ul><li>The <strong>gating network</strong> gets better over time by learning which experts give the best answers.</li><li>If it picks the wrong expert, it <strong>adjusts itself</strong> to make a better choice next time.</li></ul><h3>3. Why Use MoE? (Advantages)</h3><p>MoE has <strong>several advantages</strong> over using a single big model:</p><h4>1. Efficiency: Uses Less Computing Power</h4><ul><li>Instead of activating <strong>all</strong> the experts, MoE <strong>only activates a few</strong>, saving time and power.</li></ul><h4>2. Scalability: Can Handle Large Models</h4><ul><li>AI models are getting <strong>huge</strong> (like GPT-4 or DeepSeek-V3). MoE helps manage <strong>billions of parameters</strong> efficiently.</li></ul><h4>3. Specialization: Experts Get Really Good at Their Job</h4><ul><li>Each expert can <strong>focus on what they do best</strong>, leading to <strong>better accuracy</strong>.</li></ul><h4>4. Better Performance with Fewer Parameters</h4><ul><li>MoE can <strong>outperform</strong> traditional models by using <strong>smaller, specialized networks</strong>.</li></ul><h3>4. MoE in Transformers &amp; Large Language Models (LLMs)</h3><p>Many big AI models like <strong>GPT-4, DeepSeek-V3, and Switch Transformers</strong> use MoE to improve efficiency. Here’s how:</p><ol><li><strong>Instead of a giant transformer, they use MoE layers.</strong></li><li><strong>Only a few experts are activated at a time</strong> to process each input.</li><li>This makes them <strong>faster</strong> and <strong>cheaper</strong> to run while maintaining high accuracy.</li></ol><h4>Example: Google’s Switch Transformer</h4><ul><li>Google’s <strong>Switch Transformer</strong> (an MoE model) is <strong>7x more efficient</strong> than traditional models while keeping the same quality.</li></ul><h3>5. Mathematical Understanding of MoE</h3><p>The MoE model consists of:</p><ol><li><strong>Experts</strong> → E1,E2,…,En(small neural networks)</li><li><strong>Gating Function</strong> → G(x) (decides which experts to use)</li><li><strong>Final Output</strong> → A weighted sum of selected experts’ outputs.</li></ol><p>Mathematically, for an input x, the output is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*qGHNENu16YQpQPYIH_VTsA.png" /></figure><p>where:</p><ul><li>Gi(x) is the weight assigned by the gate to expert Ei</li><li>Ei(x) is the output from expert i.</li></ul><p><strong>Only a few experts are chosen (sparse activation), making the model more efficient.</strong></p><h3>6. Challenges in MoE</h3><p>While MoE is powerful, it has some <strong>challenges</strong>:</p><h4>Load Balancing Issues</h4><ul><li>Some experts may get <strong>too much work</strong> while others get <strong>too little</strong>.</li><li><em>Solution:</em> <strong>Better gating mechanisms</strong> to balance the load.</li></ul><h4>Harder to Train</h4><ul><li>The gating network must learn to <strong>select the best experts</strong>.</li><li><em>Solution:</em> Techniques like <strong>Softmax and Top-k Selection</strong> improve training.</li></ul><h4>Memory and Communication Overhead</h4><ul><li>Using many experts means more <strong>memory and data transfer</strong>.</li><li><em>Solution: </em><strong>Efficient model parallelism</strong> helps distribute work properly.</li></ul><h3>7. Future of MoE</h3><p>Mixture of Experts is shaping the <strong>future of AI</strong>. With <strong>efficient training</strong> and <strong>smarter gating networks</strong>, it is expected to power <strong>next-generation AI models</strong>.</p><p>Big tech companies like <strong>Google, OpenAI, and DeepSeek</strong> are using MoE for:</p><ul><li><strong>Super-efficient LLMs</strong></li><li><strong>Autonomous AI agents</strong></li><li><strong>Multimodal learning (Text + Vision + Audio)</strong></li></ul><h3>Final Thoughts: Why should we care?</h3><p>MoE makes AI models:</p><ul><li><strong>Faster</strong></li><li><strong>Smarter</strong></li><li><strong>More efficient</strong></li><li><strong>Scalable to billions of parameters</strong></li></ul><p>And guess what? The future <strong>GPT models</strong> will likely rely on <strong>Mixture of Experts</strong> to <strong>run better on less hardware</strong>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f7f07ea1439" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vision Transformers (ViT): A Deep Dive into the Future of Computer Vision]]></title>
            <link>https://medium.com/@sushma.mareddy/vision-transformers-vit-a-deep-dive-into-the-future-of-computer-vision-cab358cb855c?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/cab358cb855c</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Mon, 17 Feb 2025 03:28:33 GMT</pubDate>
            <atom:updated>2025-02-17T03:28:33.098Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How Transformers Are Redefining Image Understanding, One Patch at a Time</em></p><h3>Introduction: The CNN Era and Its Limits</h3><p>Imagine teaching a child to recognize a cat. You might start by pointing out its ears, whiskers, and tail. Convolutional Neural Networks (CNNs) work similarly — they scan images locally, focusing on edges, textures, and patterns. For years, CNNs dominated computer vision, powering everything from facial recognition to self-driving cars.</p><p>But here’s the problem: <strong>CNNs struggle to see the bigger picture</strong>. They’re great at spotting local features but often miss long-range relationships. For example, a CNN might detect a car’s wheels and windows but fail to understand how these parts relate to the whole vehicle.</p><p>Enter <strong>Vision Transformers (ViT)</strong> — a revolutionary approach inspired by language models like BERT and GPT. ViTs replace CNNs’ “local vision” with <strong>global attention</strong>, enabling machines to understand images holistically. Let’s unpack how this works.</p><h3>1. From Pixels to Patches: How ViT Processes Images</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1l4GGmOQBCNUSRjBzvK04g.png" /></figure><h4>The Big Idea: Treat Images Like Sentences</h4><p>In Natural Language Processing (NLP), Transformers process words in a sentence by analyzing their relationships. ViT borrows this concept but treats <strong>image patches</strong> as “visual words.”</p><p><strong>Step 1: Split the Image into Patches</strong></p><ul><li>An image is divided into fixed-size patches (e.g., 16x16 pixels).</li><li>A 224x224 image → 196 patches (14x14 grid).</li><li><em>Analogy</em>: Imagine cutting a photo into puzzle pieces. Each piece is a patch.</li></ul><p><strong>Step 2: Flatten and Embed Patches</strong></p><ul><li>Each patch is flattened into a 1D vector (e.g., 16x16x3 pixels → 768-dimensional vector).</li><li>A trainable linear projection (embedding layer) maps these vectors into a latent space.</li><li><em>Why?</em> This converts raw pixels into a format the Transformer can process.</li></ul><p><strong>Step 3: Add Positional Embeddings</strong></p><ul><li>Unlike CNNs, Transformers lack inherent spatial awareness.</li><li><strong>Positional embeddings</strong> are added to each patch to retain location information.</li><li><em>Example</em>: A patch in the top-left corner gets a unique positional code.</li></ul><p><strong>Step 4: The [CLS] Token</strong></p><ul><li>A special “classification token” ([CLS]) is prepended to the patch sequence.</li><li>This token aggregates global information for the final prediction (like BERT’s [CLS] token).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*etFnGzmu0ze3rxdV" /></figure><h3>2. The Transformer Encoder: Where the Magic Happens</h3><h4>Inside the Encoder Layer</h4><p>The Transformer Encoder consists of alternating <strong>Multi-Head Self-Attention (MHSA)</strong> and <strong>Feed-Forward Networks (FFN)</strong>. Let’s break it down:</p><p><strong>Multi-Head Self-Attention (MHSA)</strong></p><ol><li><strong>Query, Key, Value Vectors</strong>: Each patch is projected into three vectors.</li><li><strong>Attention Scores</strong>: Calculate how much each patch “focuses” on others.</li></ol><ul><li>Softmax(Q·Kᵀ / √d) → Attention weights.</li></ul><p>3<strong>. Weighted Sum</strong>: Combine value vectors using attention weights.</p><p>4.<strong> Multi-Head</strong>: Repeat this process across multiple “heads” to capture diverse relationships.</p><p><em>Why It Matters</em>:</p><ul><li>A patch of a dog’s ear can attend to its tail, eyes, and paws, building a holistic understanding.</li></ul><p><strong>Layer Normalization &amp; Skip Connections</strong></p><ul><li>Normalizes inputs and adds residual connections to stabilize training.</li></ul><p><strong>Feed-Forward Network (FFN)</strong></p><ul><li>A simple two-layer MLP applied to each patch independently.</li><li>Adds non-linearity and transforms features.</li></ul><h3>3. ViT vs. CNN: A Head-to-Head Comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*o-hDS4YX0uMvX9N2" /></figure><p><em>Key Takeaway</em>:</p><ul><li>ViTs excel at <strong>global reasoning</strong> but need massive data to shine.</li><li>CNNs are still better for tasks requiring <strong>local feature extraction</strong> (e.g., edge detection).</li></ul><h3>4. Training ViT: Why Data is King</h3><h4>The JFT-300M Effect</h4><p>ViT’s breakthrough performance came from pre-training on <strong>JFT-300M</strong>, a proprietary dataset with 300 million images. Lessons learned:</p><ol><li><strong>Scale Matters</strong>: ViT outperforms CNNs only when trained on large datasets.</li><li><strong>Hybrid Models</strong>: Initial layers using CNNs + ViT encoders can boost data efficiency.</li></ol><h4>Fine-Tuning for Specific Tasks</h4><ul><li>Pre-trained ViTs can be adapted to tasks like segmentation or detection by replacing the MLP head.</li><li>Example: Add a U-Net-style decoder for segmentation.</li></ul><h3>5. Code Walkthrough: Build a ViT from Scratch in PyTorch</h3><pre>import torch  <br>import torch.nn as nn  <br>from einops import rearrange <br>class VisionTransformer(nn.Module):  <br>    def __init__(self, image_size=224, patch_size=16, num_classes=1000):  <br>        super().__init__()  <br>        num_patches = (image_size // patch_size) ** 2  <br>        patch_dim = 3 * patch_size ** 2  <br>        <br>        # Patch Embedding Layer  <br>        self.patch_embed = nn.Linear(patch_dim, 768)  <br>        self.cls_token = nn.Parameter(torch.randn(1, 1, 768))  <br>        self.pos_embed = nn.Parameter(torch.randn(1, num_patches + 1, 768))  <br>        <br>        # Transformer Encoder  <br>        self.transformer = nn.TransformerEncoder(  <br>            nn.TransformerEncoderLayer(d_model=768, nhead=12),  <br>            num_layers=12  <br>        )  <br>        <br>        # Classification Head  <br>        self.mlp_head = nn.Sequential(  <br>            nn.LayerNorm(768),  <br>            nn.Linear(768, num_classes)  <br>        )  <br>    def forward(self, x):  <br>        # Split into patches  <br>        x = rearrange(x, &#39;b c (h p1) (w p2) -&gt; b (h w) (p1 p2 c)&#39;, p1=16, p2=16)  <br>        x = self.patch_embed(x)  <br>        <br>        # Add [CLS] token and position embeddings  <br>        cls_token = self.cls_token.expand(x.shape[0], -1, -1)  <br>        x = torch.cat([cls_token, x], dim=1)  <br>        x += self.pos_embed  <br>        <br>        # Transformer Encoder  <br>        x = self.transformer(x)  <br>        <br>        # Extract [CLS] token for classification  <br>        cls_output = x[:, 0]  <br>        return self.mlp_head(cls_output) </pre><p><strong>Explanation</strong>:</p><ul><li>patch_embed: Converts patches into embeddings.</li><li>cls_token: Learns a global representation.</li><li>pos_embed: Injects spatial information.</li><li>transformer: Processes embeddings with self-attention.</li></ul><h3>6. The Future: Where Are ViTs Headed?</h3><h4>Efficient ViTs</h4><ul><li><strong>Swin Transformer</strong>: Uses shifted windows to reduce computational complexity.</li><li><strong>MobileViT</strong>: Optimized for mobile devices with hybrid CNN-Transformer designs.</li></ul><h4>Beyond Classification</h4><ul><li><strong>Detection</strong>: DETR (Detection Transformer) replaces handcrafted anchors with learnable queries.</li><li><strong>Generation</strong>: ViT-GAN uses transformers for high-fidelity image synthesis.</li></ul><h4>Industry Adoption</h4><ul><li><strong>Healthcare</strong>: Analyzing X-rays and MRI scans with global context.</li><li><strong>Autonomous Vehicles</strong>: Understanding road scenes with long-range dependencies.</li></ul><h3>Conclusion: The Coexistence of ViTs and CNNs</h3><p>ViTs aren’t here to replace CNNs — they’re expanding the toolkit. While ViTs dominate tasks requiring <strong>global context</strong> (e.g., scene understanding), CNNs remain unbeatable for <strong>efficiency-focused applications</strong> (e.g., real-time video processing).</p><p>The future lies in <strong>hybrid architectures</strong> like CoAtNet (CNN + Transformer) and <strong>efficient attention mechanisms</strong> that combine the best of both worlds.</p><p><strong>Your Turn!</strong></p><ul><li><em>Debate</em>: Do you think ViTs will eventually replace CNNs, or will they coexist?</li><li><em>Experiment</em>: Try fine-tuning a ViT on a custom dataset (e.g., Kaggle’s Cats vs. Dogs).</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cab358cb855c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chain-of-Thought Prompting: Unlocking Complex Reasoning with Language Models]]></title>
            <link>https://medium.com/@sushma.mareddy/chain-of-thought-prompting-unlocking-complex-reasoning-with-language-models-355bece4efe5?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/355bece4efe5</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Sun, 06 Oct 2024 19:17:44 GMT</pubDate>
            <atom:updated>2024-10-06T19:17:44.675Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Chain-of-Thought Prompting (CoT)</strong> is an innovative technique in the realm of natural language processing (NLP) that enhances the reasoning ability of language models. Introduced by Wei et al. (2022), this approach encourages models to break down complex problems into intermediate steps, much like how we might solve a math problem by first identifying known information and then building step-by-step toward the solution.</p><p>By utilizing intermediate reasoning steps, Chain-of-Thought (CoT) prompting makes it possible for models to solve more intricate problems that require logical deduction rather than a simple retrieval of information. One particularly powerful aspect of this technique is its combination with few-shot prompting, wherein the model is given examples of reasoning before being asked to apply it to new problems. Let’s explore this concept further, breaking it down with clear examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GKDhVxKslKn_PMBq.png" /></figure><h3>What is Chain-of-Thought Prompting?</h3><p>Imagine you’re solving a puzzle. If you try to guess the solution outright, you might miss something important. Instead, you break it down: what do you know already, what steps can help you solve it, and how can you combine those steps to find the answer? This is exactly what CoT prompting encourages language models to do.</p><p>In standard prompting, you give a model a task, and it provides an answer. In CoT prompting, the model generates a series of reasoning steps before reaching the answer. This is especially useful for problems that involve logic, arithmetic, or tasks where multiple facts need to be combined to arrive at the solution.</p><p>Let’s look at an example to illustrate this better:</p><h3>Example 1: Number Problem</h3><p><strong>Task:</strong> Find out if the sum of the odd numbers in the list is even or odd.</p><p><strong>Prompt:</strong><br><em>The odd numbers in this list add up to an even number: 2, 5, 9, 6, 7, 12, 13.</em></p><p><strong>Without Chain-of-Thought Prompting:</strong><br>The model might output something like “True” or “False” without explaining its reasoning.</p><p><strong>With Chain-of-Thought Prompting:</strong><br>Let’s break down the steps:</p><ul><li>First, identify the odd numbers in the list: 5, 9, 7, 13.</li><li>Now, sum the odd numbers: 5 + 9 + 7 + 13 = 34.</li><li>Check if the sum is even or odd: 34 is even.</li></ul><p><strong>Answer:</strong> True.</p><p>By guiding the model to think in steps, CoT prompting makes sure that it evaluates the task more logically, often leading to better accuracy in complex tasks.</p><h3>How CoT Works with Few-Shot Prompting</h3><p>Few-shot prompting is when you show the model a few examples of how to solve a task before asking it to solve a new one. In the context of CoT, you provide not only the final answer but also the reasoning steps leading to the answer. This helps the model recognize the pattern of reasoning and apply it to new examples.</p><h3>Example 2: Simple Arithmetic Story</h3><p><strong>Task:</strong> You went to the market with $50. You bought a toy for $15, a book for $10, and an ice cream for $5. Then you found $20 on the ground. How much money do you have now?</p><p><strong>Without Chain-of-Thought Prompting:</strong><br>The model might guess “$40” based on its limited understanding of the task.</p><p><strong>With Chain-of-Thought Prompting:</strong><br>Prompt the model to follow a logical reasoning process:</p><ul><li>Start with $50.</li><li>Subtract the cost of the toy: $50 — $15 = $35.</li><li>Subtract the cost of the book: $35 — $10 = $25.</li><li>Subtract the cost of the ice cream: $25 — $5 = $20.</li><li>Add the $20 you found: $20 + $20 = $40.</li></ul><p><strong>Answer:</strong> You have $40.</p><p>By encouraging the model to think step by step, we ensure that each action is accounted for, leading to a more accurate answer.</p><h3>Zero-Shot Chain-of-Thought (Zero-Shot CoT)</h3><p>Sometimes, you might not have examples to show the model, but you still want it to reason step by step. This is where Zero-Shot CoT comes into play. Kojima et al. (2022) introduced a simple yet powerful idea: by adding a phrase like “Let’s think step by step” to the prompt, the model is nudged to perform reasoning before providing an answer — even without prior examples.</p><h3>Example 3: Logical Puzzle</h3><p><strong>Task:</strong> Sarah has 8 candies. She gives 3 candies to John and 2 candies to Alice. How many candies does Sarah have now?</p><p><strong>Without Zero-Shot CoT:</strong><br>The model might rush and guess an incorrect number, like “6.”</p><p><strong>With Zero-Shot CoT Prompting:</strong><br>Let’s guide the model through reasoning:</p><p><strong>Prompt:</strong> <em>Sarah has 8 candies. She gives 3 candies to John and 2 candies to Alice. How many candies does Sarah have now? Let’s think step by step.</em></p><p><strong>Model’s Output:</strong></p><ul><li>Start with 8 candies.</li><li>Subtract 3 candies given to John: 8–3 = 5.</li><li>Subtract 2 candies given to Alice: 5–2 = 3.</li></ul><p><strong>Answer:</strong> Sarah has 3 candies.</p><p>This technique is particularly useful when you have no examples to offer and the task requires logical progression rather than a simple lookup.</p><h3>Automatic Chain-of-Thought (Auto-CoT)</h3><p>Manually crafting reasoning examples for CoT prompting can be labor-intensive, and there’s always the risk of including errors in the reasoning chains. This is where <strong>Automatic Chain-of-Thought (Auto-CoT)</strong> comes in, as introduced by Zhang et al. (2022).</p><p>Auto-CoT automates the process of generating reasoning chains by clustering similar questions and then using a Zero-Shot CoT approach to create reasoning chains for each cluster. The model generates reasoning chains on its own, using heuristics like the length of the question or the number of reasoning steps. This helps scale CoT to larger datasets without needing manual intervention, though some oversight is still needed to correct any mistakes that arise.</p><h3>Example 4: Grouping Tasks for Auto-CoT</h3><p>Let’s say you have a dataset of math word problems. Auto-CoT would first group similar problems (e.g., addition-based problems, subtraction-based problems) into clusters. It then selects one representative question from each cluster and generates a reasoning chain for it using Zero-Shot CoT. This reasoning chain is then applied across other problems in the cluster, improving efficiency while maintaining diversity in the prompts.</p><p>For instance:</p><p><strong>Cluster:</strong> Addition-based problems<br><strong>Representative Task:</strong> <em>You have 12 apples and buy 7 more. How many apples do you have in total?</em></p><ul><li>Step 1: Identify initial quantity (12 apples).</li><li>Step 2: Add the new apples (12 + 7).</li><li>Answer: 19 apples.</li></ul><p>Once this reasoning chain is generated, it can be applied to similar addition problems in the dataset.</p><h3>Final Thoughts: Why CoT is a Game-Changer</h3><p>Chain-of-Thought prompting is transforming how large language models tackle complex problems. By encouraging reasoning through intermediate steps, CoT prompting allows models to handle more intricate tasks than they could with traditional prompting techniques. Whether through few-shot examples, zero-shot prompting, or the automatic generation of reasoning chains, CoT opens up new possibilities for applying language models to fields like logic, mathematics, and more nuanced question-answering.</p><p>As these models continue to evolve, CoT prompting will likely play an essential role in advancing their reasoning capabilities. Whether you’re working with a few examples or none at all, encouraging a model to think step by step can dramatically improve its performance on tasks that require more than just recalling facts.</p><p>This concludes our exploration of Chain-of-Thought Prompting. Stay tuned for more exciting advances in AI reasoning and prompt engineering!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=355bece4efe5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Few-Shot Prompting-Making AI Smarter with Examples]]></title>
            <link>https://medium.com/@sushma.mareddy/few-shot-prompting-b7b6d0712f93?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7b6d0712f93</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Sun, 21 Jul 2024 02:31:59 GMT</pubDate>
            <atom:updated>2024-07-21T02:37:41.342Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Few-Shot Prompting?</h3><p>Imagine you have a friend who’s good at understanding things but sometimes needs a little help with new tasks. If you show them a few examples of what you want, they quickly get the hang of it and can do it on their own. This is what few-shot prompting is all about for large language models (LLMs) like GPT-4.</p><h4>Why Few-Shot Prompting?</h4><p>While these models are pretty smart and can often figure things out without any examples (zero-shot prompting), sometimes they need a bit more guidance. By giving them a few examples, we help them understand better and perform tasks more accurately. It’s like showing your friend how to solve a math problem a couple of times before they can do it themselves.</p><blockquote>A Simple Example</blockquote><p>Let’s say we want the model to recognize different animal sounds. First, we give it a few examples:</p><p><em>Prompt:</em><br>“Identify the animal based on its sound. <br>Example 1: <br>Sound: Woof <br>Animal: Dog <br>Example 2: <br>Sound: Meow <br>Animal: Cat <br>Example 3: <br>Sound: Quack <br>Animal: Duck <br>Sound: Moo <br>Animal:”</p><p><em>Output:</em><br>“Cow”</p><p>By showing the model a few examples, it understands what we’re asking better and can identify new animal sounds more accurately.</p><h4>Why is it Called “Few-Shot”?</h4><p>Think of it like learning to ride a bike. If you practice a few times with someone guiding you, you get the hang of it much faster. In the same way, few-shot prompting gives the model a few practice shots before it has to do the task on its own.</p><h4>How Does It Work?</h4><p>Few-shot prompting leverages the model’s ability to learn from patterns. When we provide examples, the model identifies the patterns and applies them to new inputs. This helps improve its performance, especially for tasks it finds tricky.</p><blockquote>Example 1: Making Up Stories</blockquote><p>Suppose we want the model to create a short story about different objects. Here’s how we can do it:</p><p><em>Prompt:</em><br>“Write a short story about the following objects: <br>Example 1: <br>Objects: apple, tree, sun <br>Story: Once upon a time, an apple grew on a tree under the bright sun. <br>Example 2: <br>Objects: car, road, moon <br>Story: A car zoomed down the road as the moon shone brightly in the sky. <br>Objects: boat, river, stars <br>Story:”</p><p><em>Output:</em><br>“A boat sailed down the river while the stars twinkled above.”</p><blockquote>Example 2: Describing Weather</blockquote><p>We can also use few-shot prompting to help the model describe different weather conditions. Here’s how:</p><p><em>Prompt: </em><br>“Describe the weather in one sentence. <br>Example 1: <br>Weather: sunny, warm <br>Description: The sun is shining brightly, and it feels warm outside. <br>Example 2: <br>Weather: rainy, cold <br>Description: The rain is pouring down, and it feels cold outside. <br>Weather: snowy, windy <br>Description:”</p><p><em>Output:</em><br>“Snow is falling gently, and the wind is blowing softly.”</p><p>By showing the model how to describe similar weather conditions, it can figure out new ones more easily.</p><h4>When Few-Shot Prompting Doesn’t Work?</h4><p>Few-shot prompting works well for many tasks, but it’s not perfect. For very complex tasks, the model might still struggle. Let’s look at an example where few-shot prompting falls short:</p><blockquote>Example: Complex Reasoning with Puzzles</blockquote><p>Suppose we ask the model to solve a puzzle involving colors and shapes. This requires some detailed reasoning.</p><p><em>Prompt:</em><br>“Identify the pattern in the following sequences: <br>Example 1: <br>Sequence: red circle, blue square, red circle, blue square <br>Pattern: alternates between red circle and blue square <br>Example 2: <br>Sequence: green triangle, green triangle, blue triangle, blue triangle <br>Pattern: two green triangles followed by two blue triangles <br>Sequence: red square, red square, blue circle, blue circle <br>Pattern:”</p><p><em>Output:</em><br>“alternates between red square and blue circle”</p><p>The correct pattern is two red squares followed by two blue circles. This shows that even with several examples, the model might not get complex reasoning tasks right.</p><h4>Why Few-Shot Prompting Needs a Boost</h4><p>Sometimes, even a few examples aren’t enough for the model to get it right. In these cases, we can try giving more examples (like 5-shot or 10-shot) or using other advanced techniques. One such technique is called chain-of-thought prompting, where we break down the problem into smaller steps.</p><h4>Conclusion</h4><p>Few-shot prompting is like giving a few helpful examples to a smart friend. It makes the model better at understanding and performing tasks. Whether it’s recognizing animal sounds, making up stories, or describing weather conditions, showing a few examples can make a big difference. And when things get really tough, we can always try new techniques to help the model do its best.</p><p>So next time you use a language model, remember how a few examples can go a long way in making it even smarter and more helpful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7b6d0712f93" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Zero-Shot Prompting]]></title>
            <link>https://medium.com/@sushma.mareddy/understanding-zero-shot-prompting-3d7226a8dfb6?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d7226a8dfb6</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Sun, 21 Jul 2024 02:13:45 GMT</pubDate>
            <atom:updated>2024-07-21T02:17:45.126Z</atom:updated>
            <content:encoded><![CDATA[<h4>How AI Can Answer Questions Without Any Practice</h4><h4>What is Zero-Shot Prompting?</h4><p>Imagine you have a super-smart friend who knows a lot about everything. You can ask this friend any question, and they’ll try to answer it, even if they’ve never seen a similar question before. This is a bit like what zero-shot prompting is for large language models (LLMs) like GPT-4.</p><h4>The Magic of Large Language Models</h4><p>Large language models (LLMs) are trained on huge amounts of data, which means they’ve read lots of books, articles, and websites. This helps them understand language really well. When we talk about “zero-shot prompting,” we’re asking these models to do something without giving them any examples or hints beforehand.</p><h4>A Simple Example</h4><p>Let’s say you want to know if a sentence sounds happy, sad, or neutral. You could ask the model:</p><p><em>Prompt:</em><br>“Classify the text into neutral, negative, or positive. <br>Text: I spilled my ice cream. <br>Sentiment:”</p><p><em>Output:</em><br>“Negative”</p><p>Here, you didn’t give the model any examples of sentences with their sentiments. You just asked it to classify the sentiment of the sentence. The model uses its understanding of language to figure out the answer. This is zero-shot prompting!</p><h4>Why is it Called “Zero-Shot”?</h4><p>Think of it like playing basketball. If you’ve never practiced a particular shot before but still try to make it, that’s a zero-shot attempt. You’re going for it without any practice. In the same way, when we use zero-shot prompting, we’re asking the model to perform a task without showing it examples first.</p><h4>How Do Models Get So Smart?</h4><p>These models, like GPT-4, are trained using a process called instruction tuning. This means they learn to follow instructions better by practicing on lots of different tasks described in words. Researchers found that this makes the models better at understanding what we want them to do, even if we don’t give them examples.</p><h4>Improving with Human Feedback</h4><p>To make these models even better, researchers use something called RLHF, which stands for Reinforcement Learning from Human Feedback. Imagine your smart friend gets tips from a teacher on how to give better answers. This feedback helps the model learn what kind of answers people like the most.</p><h4>When Zero-Shot Doesn’t Work</h4><p>Sometimes, zero-shot prompting might not give you the best answer. In those cases, you can use few-shot prompting. This means you show the model a few examples of the task you want it to do. For example:</p><p><em>Prompt:</em><br>“Classify the text into neutral, negative, or positive. <br>Example 1: <br>Text: I got a new puppy today! <br>Sentiment: Positive <br>Example 2: <br>Text: I lost my favorite toy. <br>Sentiment: Negative <br>Text: I spilled my ice cream. <br>Sentiment:”</p><p><em>Output:</em><br>“Negative”</p><p>Here, by giving a few examples, the model can understand better what you mean and give more accurate answers.</p><h4>Why is Zero-Shot Prompting Cool?</h4><p>Zero-shot prompting is exciting because it shows just how powerful and flexible these models are. They can understand and perform new tasks just by reading the instructions, without needing examples every time. This makes them really useful for a wide range of applications, from chatbots to language translation to helping with homework!</p><h4>Conclusion</h4><p>Zero-shot prompting is like having a super-smart friend who can help you with all sorts of tasks, even if they’ve never seen them before. Thanks to instruction tuning and feedback from humans, these models are getting better and better at understanding and following instructions. And when zero-shot prompting isn’t enough, a few examples can make all the difference.</p><p>So, next time you use a language model, remember the magic of zero-shot prompting and how it helps make these models so incredibly useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d7226a8dfb6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring the Magical World of BERT: The Super Language Model]]></title>
            <link>https://medium.com/@sushma.mareddy/exploring-the-magical-world-of-bert-the-super-language-model-18662d6601e4?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/18662d6601e4</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Mon, 10 Jun 2024 00:30:03 GMT</pubDate>
            <atom:updated>2024-06-10T00:30:03.796Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you ever wished you could talk to your computer and have it understand you perfectly? Well, that’s what BERT, a super cool language model, can do! BERT is like a super-smart friend who can help us with all kinds of language tasks, from answering questions to translating between languages, and even writing stories!</p><h4>What is BERT?</h4><p>BERT stands for “Bidirectional Encoder Representations from Transformers.” That’s a mouthful, right? Let’s break it down:</p><ul><li>Bidirectional means that BERT can look at a sentence from left to right and right to left, just like you would read a book.</li><li>Encoder means that BERT takes in some text and turns it into a special code that computers can understand.</li><li>Representations refers to the way BERT represents the meaning of words and sentences.</li><li>Transformers are a special type of machine learning model that BERT is built on.</li></ul><p>So, in simple terms, BERT is a super-smart language model that can understand and represent the meaning of words and sentences in a way that computers can work with.</p><h4>BERT’s Architecture: The Building Blocks</h4><p>BERT is made up of different building blocks, just like a house is made of bricks and wood. The main building blocks of BERT are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G7x9nPM-YwX7i7xTR435Yw.png" /></figure><p>1. <em>Embeddings:</em> These are like tiny codes that represent each word in a sentence. They help BERT understand the meaning of words.</p><p>2. <em>Encoder Layers</em>: These are layers of neurons (like little brain cells) that process the embeddings and learn about the relationships between words in a sentence.</p><p>3. <em>Attention Mechanism</em>: This is like a special spotlight that helps BERT focus on the most important parts of a sentence when trying to understand it.</p><h4>Pre-training: BERT’s Magical Learning Process</h4><p>Before BERT can start helping us with language tasks, it needs to go through a special training process called pre-training. This is like going to school and learning everything you can about language and how people use it.</p><p>During pre-training, BERT is shown millions and millions of sentences from books, websites, and other sources. It then plays two fun games to learn about language:</p><ol><li><em>Masked Language Modeling:</em> In this game, some words in a sentence are hidden (or “masked”), and BERT has to guess what those words are based on the surrounding context. It’s like a fill-in-the-blank puzzle!</li></ol><p>Example:</p><ul><li>Original Sentence: “I love cats.”</li><li>Masked Sentence: “I [MASK] cats.”</li><li>BERT’s Guess: “love”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R_femtoSuJ3Pp12B6wqvFQ.png" /></figure><p><em>2. Next Sentence Prediction:</em> In this game, BERT is shown two sentences and has to figure out if the second sentence logically follows the first one. It’s like a true-or-false quiz about how sentences are connected.</p><p>Example:</p><ul><li>Sentence A: “I love cats.”</li><li>Sentence B: “They are very playful.”</li><li>BERT’s Task: Determine if Sentence B follows Sentence A (Yes or No)</li></ul><p>By playing these games over and over again, BERT learns a ton about how language works and how to understand and represent the meaning of words and sentences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MQcZ-pEtFmkl27L_LeEc5g.png" /></figure><h4>Fine-tuning: Teaching BERT New Tricks</h4><p>After pre-training, BERT is ready to learn new tasks! This process is called fine-tuning, and it’s like teaching BERT a new skill or trick.</p><p>Let’s say we want BERT to help us with question answering. We would give it a bunch of questions and their answers, and BERT would learn to recognize patterns in the questions and find the relevant information in the text to provide the correct answer.</p><p>It’s like teaching a dog a new trick: first, you show the dog what you want it to do (like sit or roll over), and then you give it lots of practice and rewards until it learns the trick perfectly.</p><p>With fine-tuning, BERT can learn all sorts of language tasks, like summarization, translation, text generation, and more!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*HCgtKlLxkjdwo5Piv4GW1w.png" /><figcaption>The whole picture if we use BERT as a pretrained model in a downstream supervised task ( let say question answering task )</figcaption></figure><p><em>Example of Fine-tuning for Spam Classification :</em></p><p>Let’s say we want BERT to help us with spam classification. We would train it with a dataset of emails labeled as either “spam” or “not spam.” BERT would learn to recognize the patterns and characteristics of spam emails.</p><p>Example:</p><ul><li>Email: “Congratulations! You’ve won a free vacation. Click here to claim your prize.”</li><li>BERT’s Output: “Spam”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5wF957gw_oaDSEb1QeEVpA.png" /></figure><h4>Putting BERT to Work: Real-World Applications</h4><p>BERT is like a super-smart language assistant that can help us with all kinds of tasks. Here are a few examples of how BERT is being used in the real world:</p><p><em>- Search Engines:</em> BERT helps search engines like Google understand the intent behind our queries and provide more relevant results.<br><em>- Virtual Assistants</em>: Virtual assistants like Alexa and Siri use BERT to better understand and respond to our voice commands and questions.<br>-<em> Content Creation:</em> Writers and content creators can use BERT to help them generate high-quality, engaging content on various topics.<br>- <em>Chatbots:</em> BERT powers chatbots that can have more natural and contextual conversations with users.<br>- <em>Language Learning</em>: BERT can help language learners by providing better translations, explanations, and practice exercises.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*KGQrNrxk9rCWbAjmPutxHg.png" /></figure><p>The possibilities are endless, and as BERT continues to learn and grow, it will become even more helpful and capable in assisting us with all sorts of language-related tasks.</p><p>In a world where language is so important for communication and understanding, BERT is like a magical tool that can help us bridge the gap between humans and machines. So, the next time you talk to a computer or use a language-based application, remember that BERT might just be the super-smart friend behind the scenes, helping to make it all possible!</p><h4>References :</h4><ol><li>BERT Paper — <a href="https://arxiv.org/pdf/1810.04805">https://arxiv.org/pdf/1810.04805</a></li><li>Bert Explained — <a href="https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270">https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270</a></li><li>Inside Bert — <a href="https://towardsdatascience.com/deconstructing-bert-part-2-visualizing-the-inner-workings-of-attention-60a16d86b5c1">https://towardsdatascience.com/deconstructing-bert-part-2-visualizing-the-inner-workings-of-attention-60a16d86b5c1</a></li><li>Deconstructing Bert — <a href="https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77">https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77</a></li><li>BERT in question answering — <a href="https://medium.com/saarthi-ai/build-a-smart-question-answering-system-with-fine-tuned-bert-b586e4cfa5f5">https://medium.com/saarthi-ai/build-a-smart-question-answering-system-with-fine-tuned-bert-b586e4cfa5f5</a></li><li>Fine Tuning BERT — <a href="https://d2l.ai/chapter_natural-language-processing-applications/finetuning-bert.html">https://d2l.ai/chapter_natural-language-processing-applications/finetuning-bert.html</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18662d6601e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Transformer Model: A Simplified Explanation]]></title>
            <link>https://medium.com/@sushma.mareddy/the-transformer-model-a-simplified-explanation-936ec4f7e40d?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/936ec4f7e40d</guid>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Fri, 07 Jun 2024 15:29:29 GMT</pubDate>
            <atom:updated>2024-06-07T15:42:06.324Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>The Transformer model revolutionized neural machine translation by using attention mechanisms to improve both performance and training speed. Unlike traditional models, Transformers are highly parallelizable, making them more efficient and scalable.</blockquote><blockquote>Let’s break down the key components and concepts of the Transformer model step-by-step.</blockquote><p>At a high level, the Transformer model consists of two main parts:<br><strong>1. Encoder:</strong> Processes the input sentence.<br><strong>2. Decoder:</strong> Generates the output sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*FZL7q2uBqUe2jsdVofSgRQ.png" /><figcaption>Encoder and Decoder</figcaption></figure><p>These components are connected and work together to translate a sentence from one language to another.</p><h4>Encoders and Decoders</h4><p>Both the encoder and decoder are made up of multiple identical layers. The original Transformer paper suggests using six layers for both encoders and decoders, but this can be adjusted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*17nXkmMYONGSbkb_irP4dg.png" /></figure><p><strong>Encoder Layers</strong>:</p><ul><li><strong>Self-Attention Mechanism</strong>: Allows the model to focus on different words in the input sentence when processing a specific word.</li><li><strong>Feed-Forward Neural Network</strong>: Processes the output of the self-attention mechanism independently for each word.</li></ul><p><strong>Decoder Layers</strong>:</p><ul><li><strong>Self-Attention Mechanism</strong>: Similar to the encoder but only attends to previous positions in the sequence to prevent seeing future words.</li><li><strong>Encoder-Decoder Attention</strong>: Helps the decoder focus on relevant parts of the input sentence.</li><li><strong>Feed-Forward Neural Network</strong>: Processes the output of the attention mechanisms</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*akLLP4Z-ASvKajNeWE8QXg.png" /></figure><h4><strong>Self-Attention Mechanism</strong></h4><p>Self-attention allows the model to weigh the importance of different words in a sentence. For example, in the sentence “The puppy didn’t chew the slipper because it was tired,” self-attention helps the model understand that “it” refers to “puppy” and not “slipper.”</p><p><strong>Steps in Self-Attention:</strong></p><ol><li><strong>Create Query, Key, and Value Vectors:</strong> For each word, create three vectors by multiplying the word embedding by learned weight matrices.</li><li><strong>Calculate Attention Scores:</strong> Compute the dot product of the Query vector of one word with the Key vectors of all words to get attention scores.</li><li><strong>Normalize Scores:</strong> Use softmax to convert the scores into probabilities.</li><li><strong>Weighted Sum of Values:</strong> Multiply each Value vector by the corresponding softmax score and sum them up to get the final attention output.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*91mGjYTTf6F8s69kLCxaXA.png" /></figure><p>Example :</p><p>sentence (X) = Hello world</p><p>The sentence we want to translate is first tokenized into words or subwords. For example, this sentence “Hello world” is split into [“Hello”, “world”].</p><p><em>Step 1 :</em> Each word in the tokenized sentence is converted into a dense vector of fixed size, say 512 dimensions. These vectors are called embeddings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*Gt5svGnjWA399KXoIqAxEw.png" /><figcaption>Embeddings for words “Hello” and “World”</figcaption></figure><p><em>Step 2 :</em> Let the three matrices that we trained during the training process be: W_Q, W_K, W_V</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*ebplmzRDu0v2NnAS70Zc6w.png" /><figcaption>These three matrices are trained during the training process and are used to generate Query, Key and Value vectors.</figcaption></figure><p>For each word “Hello” and “world” , three vectors (Query, Key, and Value) are created by multiplying the word embedding by learned weight matrices(W_Q, W_K, W_V). These new vectors are smaller in dimension than the embedding vector. Their dimensionality is 64.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*gQqLaQj7p6FyNk12hR7NjA.png" /><figcaption>Query , Key and value vectors for word “Hello”</figcaption></figure><p>Similarly We get Query, key, value vectors for X2 (q2,k2,v2)</p><p><em>Step 3 :</em> We calculate the <em>Attention score</em> for each word by taking the dot product of the query vector and the corresponding key vector. For instance, when processing the self-attention for the word at position #1, the initial score is derived from the dot product of q1 and k1. The subsequent score is obtained from the dot product of q1 and k2.</p><p>For word X1,<br>Attention Score with itself = <strong>q1 x k1</strong> <br>Attention Score with X2 = <strong>q1 x k2</strong></p><p>Similarly the Attention Score for X2 is also calculated the same way</p><p>Attention Score with itself = <strong>q2 x k2</strong><br>Attention Score with X1 = <strong>q2 x k1</strong></p><p><em>Step 4:</em> Next, we proceed by dividing the scores by 8, which is the square root of the dimension of the key vectors mentioned in the paper (64). This division aims to ensure more consistent gradient behavior. While alternate values could be considered, 8 is the default choice. Following this, we apply a <em>softmax operation</em> to the divided scores. Softmax serves to normalize the scores, ensuring they are all positive and collectively sum up to 1.</p><p>For X1 :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*fvxZPnmgb9JXagHD7OpEOg.png" /></figure><p>For X2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*pK6V0NRz7PezGbabONclPQ.png" /></figure><p><em>Step 5 :</em> We proceed by multiplying each value vector by its corresponding softmax score, preparing them for summation. This step is guided by the intuition to preserve the values associated with the words we aim to emphasize, while attenuating the influence of irrelevant words.</p><p>For X1 :</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*WrPKxrmM6waI_ujqM5U1DA.png" /></figure><p>For X2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*4OhcFlAEcS-V33LteJzJSQ.png" /></figure><p><em>Step 6 :</em> We then sum up the weighted value vectors, resulting in the output of the self-attention layer at this position.</p><p>For X1 :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*MgWCpG6d7280G0EqUdzp4Q.png" /><figcaption>Output of self-attention layer for word “Hello”</figcaption></figure><p>For X2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*0RRtosZSuwz2HCoJhtzcdg.png" /><figcaption>Output of self-attention layer for word “World”</figcaption></figure><p><strong>Self-Attention Calculations in a Nutshell :</strong></p><p>The initial step involves calculating the Query, Key, and Value matrices. This is achieved by organizing our embeddings into a matrix X and then multiplying it by the corresponding trained weight matrices (WQ, WK, WV) and at the end calculating the Z.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*uY9Bxtdzmfg1vgVlTxkmiQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*VJhuZoNvPJr5m9_olKZwAA.png" /></figure><h4><strong>Feed-Forward Neural Network</strong></h4><p>The output from the self-attention mechanism is processed by a feed-forward neural network independently for each word to further refine the representations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*4_D0t9aytbxylGc2v_a5NQ.png" /></figure><h4><strong>Multi-Headed Attention</strong></h4><p>Instead of having a single set of Query, Key, and Value vectors, multi-headed attention uses multiple sets, allowing the model to focus on different parts of the sentence simultaneously. Each set of attention heads operates in its own subspace, providing diverse perspectives.</p><ol><li>Input Sentence</li></ol><p>Example : “Hello World”</p><p>2. Embed each word</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/242/1*JYZoQPJYfk0Fqmbr3hDVBQ.png" /></figure><p>3. We have multiple weights for 8 Attention heads, We then generate 8 sets of Query, Key, and Value vectors. Each set for each Attention head.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HUEL17CuGLAyRdzUcY-Ilw.png" /><figcaption>Multiply X with weight matrices to generate Q,K,V matrices for each Attention head</figcaption></figure><p>4. Calculate Attention using the generated Q,K,V matrices</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*obKXz3Xyb18t7dMZAWx9cQ.png" /></figure><p>5. Concatenate All the Attention heads and Multiply with weight Matrix W⁰ to produce output of the layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rk-o99VpHBRsjKxiLicfJg.png" /></figure><h4><strong>Positional Encoding</strong></h4><p>Transformers don’t inherently understand the order of words. Positional encoding is added to the word embeddings to give the model a sense of word order. This is done by adding a unique positional vector to each word’s embedding.</p><ol><li>Convert each word to a 512-dimensional vector.</li><li>Calculate positional encodings for each position.</li><li>Add the positional vector to each word’s embedding</li><li>These combined vectors are then fed into the encoder part of the Transformer model.</li></ol><h4>Residual Connections and Layer Normalization</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*Z5YY46YoKA-v6PikLA3FtQ.png" /></figure><ol><li><strong>Residual Connections:</strong></li></ol><p>Imagine you’re building a really tall tower with LEGO blocks. Sometimes, it gets tricky to balance the tower, and it might fall over if you’re not careful. Residual connections are like adding a support beam that connects each level of the tower to the ground. This support beam helps keep the tower stable, even as it gets taller.</p><p>Residual connections, also known as skip connections, help in the effective training of deep neural networks by allowing gradients to flow through the network more directly, which addresses the problem of vanishing gradients. They enable the model to learn the identity function more easily, thus preserving the information through layers.</p><p><em>How It Works</em><strong>:</strong> A residual connection adds the input of a layer to its output. Mathematically, if F(x) is the function learned by the layer, the output with a residual connection is:</p><p>Output= F(x)+x</p><p><em>Example:</em></p><p>Consider an input x to a layer in the Transformer:</p><ul><li>The layer computes F(x) (for instance, the self-attention mechanism or the feed-forward network).</li><li>The output with the residual connection is F(x)+x.</li></ul><p>This simple addition ensures that the original input information is directly available at the next layer, facilitating better gradient flow.</p><p><strong>2. Layer Normalization:</strong></p><p>Now, think about baking cookies. When you mix the dough, you want all the ingredients to be evenly spread out, so every cookie tastes just right. Layer normalization is like mixing the dough really well, making sure that every part of the mixture is balanced.</p><p>Layer normalization standardizes the outputs of a layer across the features for each training example. This normalization helps stabilize and accelerate the training process by ensuring that the inputs to the next layer have a consistent scale and distribution.</p><p><em>How It Works:</em> Layer normalization normalizes the inputs to a layer based on the mean and variance for each training example independently</p><p>In the Transformer model, these two mechanisms are combined in a specific way to ensure stability and efficiency. This combination is applied after each sub-layer (e.g., self-attention or feed-forward network) within the encoder and decoder layers.</p><p><strong><em>Steps:</em></strong><br><em>1.Sub-Layer Operation:</em><br>- Compute the output of the sub-layer, F(x)</p><p><em>2. Add Residual Connection:</em><br>- Add the input x to the sub-layer output: F(x)+x</p><p><em>3. Apply Layer Normalization:</em><br>- Normalize the sum using layer normalization: LayerNorm(F(x)+x)</p><p>Example:</p><p>Consider a self-attention sub-layer with input <em>x</em>:</p><ol><li>Compute self-attention: <em>SelfAttention(x)</em></li><li>Add residual connection: <em>x+SelfAttention(x)</em></li><li>Apply layer normalization: <em>LayerNorm(x+SelfAttention(x))</em></li></ol><h4>Decoder</h4><p>Think of the Decoder as a magical translator machine. It’s the part of the Transformer model that takes the processed information (from the Encoder) and turns it into a new sentence in a different language. In our example, it’s translating “Hello world” into “Bonjour le monde”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*w7740ZgaY2dzLC-jNmgO-w.png" /></figure><p><strong>Step 1: Preparing to Translate</strong></p><ul><li><em>Input Sentence:</em> The Decoder starts with the sentence it needs to translate, which in this case is “Hello world”.</li><li><em>Word Embeddings</em>: Just like the Encoder, the Decoder first converts each word into a special numerical representation (like turning words into secret codes).</li></ul><p><strong>Step 2: Self-Attention Mechanism</strong></p><ol><li><em>Look at Previous Words</em>: When translating, the Decoder looks at the words it has already translated to decide what word should come next. This is done using a Self-Attention Mechanism, but it only looks at the previous words (not the future ones) to avoid cheating.</li><li><em>Example</em>: If we already translated “Hello” to “Bonjour”, the Decoder will focus on “Bonjour” to figure out the next word.</li></ol><p><strong>Step 3: Encoder-Decoder Attention</strong></p><ol><li><em>Focus on Important Words</em>: The output of the top Encoder is transformed into a set of attention vectors, called Key (K) and Value (V) vectors. These vectors help the Decoder focus on the most important parts of the input sentence.</li><li><em>Example</em>: The Decoder might focus more on “Hello” when deciding how to start the translation. It uses the K and V vectors from the Encoder to understand which parts of “Hello world” are most important.</li></ol><p><strong>Step 4: Feed-Forward Neural Network</strong></p><ol><li><em>Process the Information</em>: The output from the attention mechanisms is then passed through a Feed-Forward Neural Network. This step is like mixing all the ingredients together in a bowl to make sure everything is combined well.</li></ol><p><strong>Step 5: Generate the Translation</strong></p><ol><li><em>Word by Word:</em> The Decoder generates the translated sentence one word at a time. The output of each step is fed to the bottom of the Decoder in the next time step, and the Decoders bubble up their decoding results just like the Encoders did.</li><li><em>Special Symbol:</em> The process repeats until a special symbol is reached, indicating the Decoder has completed its output.</li><li><em>Example:</em> It starts with “Bonjour” and then adds “le” and finally “monde” to complete “Bonjour le monde”.</li></ol><h4>Linear and Softmax Layer</h4><p>Imagine you’re building a robot that helps translate sentences from English to French. After the robot processes the English sentence and gets all the clues about what it means, it still needs to decide exactly which French words to use. This is where the Linear and Softmax layers come in. Let’s break down how these layers help the robot make the final decision.</p><p><strong>Linear Layer: Making Sense of Information</strong><br>Think of the Linear Layer as a special tool that helps the robot organize its thoughts. After the Decoder has done all its hard work, the Linear Layer takes the information and prepares it for the final step.</p><p>The Linear Layer works like a simple math tool that combines all the clues the robot has gathered. It takes all the hidden clues (which are still in secret code form) and transforms them into numbers that represent how confident the robot is about each possible word it might choose next.</p><p><strong>Softmax Layer: Picking the Best Word</strong><br>After the Linear Layer has done its job, the robot needs to make a final decision. The Softmax Layer helps the robot choose the most likely word from all the possible words.</p><p>The Softmax Layer takes the numbers from the Linear Layer and turns them into probabilities that add up to 1. This makes it easy to see which word is the most likely choice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*IhQqnDlge4Y_cgtApdVLfA.png" /></figure><h4>Detailed Steps with Example :</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*eCfDW1qivgpFzCsmPTk_CA.png" /></figure><p><strong>1. Input Sentence</strong><br><em>Input:</em> “Hello world”<br><em>Words:</em> [“Hello”, “world”]</p><p><strong>2. Word Embeddings</strong><br>Each word is converted into a vector.<br>Example: “Hello” -&gt; [0.1, 0.2, 0.3], “world” -&gt; [0.4, 0.5, 0.6]</p><p><strong>3. Positional Encoding</strong><br>Adds positional information to embeddings.<br>Example: “Hello” (position 1) -&gt; [0.1, 0.3, 0.5], “world” (position 2) -&gt; [0.4, 0.7, 0.9]</p><p><strong>4. Encoder</strong><br><em>Self-Attention</em>: Each word focuses on all other words.<br><em>Feed-Forward Network:</em> Processes attention output.<br><em>Normalization and Residuals:</em> Stabilize training.</p><p><strong>5. Generating Attention Vectors K and V</strong><br>Transforms encoder output into Key (K) and Value (V) vectors for the decoder.</p><p><strong>6. Decoder</strong><br><em>Self-Attention: </em>Focuses on previously generated words.<br><em>Encoder-Decoder Attention: </em>Uses K and V to focus on relevant input parts.<br><em>Feed-Forward Network:</em> Processes combined outputs.<br><em>Normalization and Residuals: </em>Ensures stable training.</p><p><strong>7. Generating Output Words</strong><br>Generates words one at a time until the sequence ends.<br>Example: Generates “Bonjour”, then “le”, then “monde”.</p><p><strong>8. Linear and Softmax Layers</strong><br><em>Linear Layer:</em> Converts decoder output to match vocabulary size.<br><em>Softmax Layer:</em> Converts to probability distribution, selecting the most likely next word.</p><p><strong>Final Output</strong><br><em>Input:</em> “Hello world”<br><em>Output:</em> “Bonjour le monde”</p><p>The Transformer model is revolutionary because it’s highly efficient, scalable, and accurate. It can process words in parallel, making it faster than previous models. This technology is not only used in language translation but also in various other applications like summarizing text, generating text, and even understanding images.</p><p>By breaking down complex sentences and focusing on important parts using attention mechanisms, the Transformer model ensures that translations and other language tasks are done with high accuracy. Its ability to handle context and relationships within the text makes it a critical tool in natural language processing.</p><p>In essence, the Transformer model acts like a super-smart robot that can understand and translate languages efficiently, making our interactions with technology smoother and more intuitive.</p><h4>References:</h4><ol><li><a href="https://arxiv.org/abs/1706.03762">Attention All you need paper</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=936ec4f7e40d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginner Guide to start machine learning.]]></title>
            <link>https://medium.com/@sushma.mareddy/beginner-guide-to-start-machine-learning-b44eafce4dbe?source=rss-e9ca8a4420b7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b44eafce4dbe</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mareddy Sushma]]></dc:creator>
            <pubDate>Mon, 06 Jul 2020 16:06:35 GMT</pubDate>
            <atom:updated>2020-07-06T16:29:12.549Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are a complete starter to machine learning, then this Article helps you start your machine learning journey. This is a rough roadmap you can follow on your way to becoming an insanely talented Machine Learning Engineer. Of course, you can always modify the steps according to your needs to reach your desired end-goal!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sKfeZ7Ln3qsrsvPyU4ObAQ.jpeg" /><figcaption>ML Engineer Path</figcaption></figure><p><strong>Machine Learning</strong> involves studying computer algorithms and statistical models for a specific task using patterns and inference instead of explicit instructions. And there is no doubt that Machine Learning is an insanely popular career choice today. According to Indeed, Machine Learning Engineer Is The Best Job of 2020 with a <strong>344%</strong> growth and an average base salary of <strong>$146,085</strong> per year.</p><p>You don’t need a fancy PhD in math. You don’t need to be the world’s best programmer. And you certainly don’t need to pay $16,000 for an expensive “Bootcamp.”</p><p>Whether your goal is to become a data scientist, use ML algorithms as a developer, or add cutting-edge skills to your business analysis toolbox, you can pick up applied machine learning skills much faster than you might think.</p><p>Excited about what machine learning can achieve? Let’s look at a learning path to make you a machine learning expert.</p><p><strong>STEP-1: Basics of Programming</strong></p><p>There are multiple languages which provide machine learning capabilities. Also, there is development work happening at a rapid pace across several languages. Currently “R” and “Python” are the most commonly used languages. Before entering into the world of ML, I would recommend you to choose one of these two languages (R or python) which can help to focus on machine learning.</p><p>Python is currently the most popular language for ML. In fact, there are many Python libraries that are specifically useful for Artificial Intelligence and Machine Learning such as <a href="https://keras.io/">Keras</a>, <a href="https://www.tensorflow.org/">TensorFlow</a>, <a href="https://scikit-learn.org/stable/">Scikit-learn</a>, etc.</p><p>So if you want to learn ML, it’s best if you learn Python</p><p><strong>STEP-2: Linear Algebra and Multivariate Calculus</strong></p><p>Both Linear Algebra and Multivariate Calculus are important in Machine Learning. However, the extent to which you need them depends on your role as a data scientist. If you are more focused on application heavy machine learning, then you will not be that heavily focused on maths as there are many common libraries available. But if you want to focus on R&amp;D in Machine Learning, then mastery of Linear Algebra and Multivariate Calculus is very important as you will have to implement many ML algorithms from scratch.</p><p>Here are few Courses for Linear Algebra and Calculus:</p><ol><li><a href="https://www.youtube.com/watch?v=fNk_zzaMoSs&amp;list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab">The Essence of linear algebra</a></li><li><a href="https://www.youtube.com/watch?v=WUvTyaaNkzM&amp;list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr">The Essence of Multivariate Calculus</a></li><li><a href="https://www.coursera.org/specializations/mathematics-machine-learning">Mathematics for MachineLearning</a></li></ol><p><strong>STEP:3 Statistics</strong></p><p>Before Starting ML, it&#39;s obvious and very important to have good knowledge of Statistics. It is good to have an understanding of the descriptive and inferential statistics before you start serious machine learning development. We need statistics to help transform observations into information and to answer questions about samples of observations.</p><blockquote>“ Statistics is the art of making numerical conjectures about puzzling questions. The methods were developed over several hundred years by people who were looking for answers to their questions.”</blockquote><p><a href="https://en.wikipedia.org/wiki/Descriptive_statistics">Descriptive statistics</a> refer to methods for summarizing raw observations into information that we can understand and share.</p><p><a href="https://en.wikipedia.org/wiki/Statistical_inference">Inferential statistics</a> is a fancy name for methods that aid in quantifying properties of the domain or population from a smaller set of obtained observations called a sample</p><p>Here are a few books for statistics:</p><ul><li><a href="https://amzn.to/2InAS0T">Applied Predictive Modeling</a>, 2013</li><li><a href="https://amzn.to/2Gvhkqz">An Introduction to Statistical Learning with Applications in R</a>, 2013.</li><li><a href="https://amzn.to/2GIN9jc">Programming Collective Intelligence: Building Smart Web 2.0 Applications</a>, 2007.</li><li><a href="https://amzn.to/2pUA0tU">Statistics</a>, Fourth Edition, 2007.</li><li><a href="https://amzn.to/2H224Tp">All of Statistics: A Concise Course in Statistical Inference</a>, 2004.</li><li><a href="https://amzn.to/2Gv0A2V">Statistics in Plain English</a>, Third Edition, 2010.</li></ul><p>Statistics is the most important prerequisite before starting Machine learning. Don&#39;t worry I will write another article briefly describing Statistics.</p><p><strong>STEP:4 Data Exploration / Cleaning / Preparation</strong></p><p>The most time-consuming part in ML is actually <strong>data collection, integration, cleaning, and preprocessing.</strong> So make sure to practice with this because you need high-quality data but large amounts of data are often dirty. So this is where most of your time will go!!!. What differentiates a good machine learning professional from an average one is the quality of feature engineering and data cleaning which happens on the original data.</p><p>Start learning a few important libraries in Python:</p><p>▪ NumPy — mainly useful for its N-dimensional array objects</p><p>▪ pandas — Python data analysis library, including structures such as data frames</p><p>▪ matplotlib — 2D plotting library producing publication-quality figures</p><p>▪ scikit-learn — the machine learning algorithms used for data analysis and data mining tasks.</p><blockquote>“Torture the data, and it will confess to anything.” — By Ronald Coase</blockquote><p><strong>STEP:5 Introduction to Machine Learning</strong></p><p>Having all the Prerequisites you are ready to start Machine learning. Machine learning focuses on the development of computer programs that can access data and use it to learn for themselves.</p><p>Machine Learning can also be defined as the process of solving a practical problem by 1)gathering a dataset 2)algorithmically building a statistical model based on that dataset. That statistical model is assumed to be used somehow to solve the practical problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*k1_72tiZB3gmqvi9k2IuBg.png" /><figcaption>ML Types (algorithms)</figcaption></figure><p>Start learning each algorithm, understand the math behind every algorithm. Learn various models and practice on real datasets<strong>.</strong> This will help you in creating your intuition around which types of models are appropriate in different situations. Along with these steps, it is equally important to understand how to interpret the results obtained by using different models. This is easier to do if you understand various tuning parameters and regularization methods applied to different models.</p><p>For a broad introduction to Machine Learning, Stanford’s <a href="https://www.coursera.org/learn/machine-learning">Machine Learning Course</a> by Andrew Ng is quite popular. It focuses on machine learning, data mining, and statistical pattern recognition with explanation videos are very helpful in clearing up the theory and core concepts behind ML</p><p>There is actually a life Cycle of Machine Learning you need to follow when solving real-world problems. Every step in it necessary and important to learn and implement as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*u_cmwR4gbrNDSqtDPqtwtQ.png" /><figcaption>ML life-cycle</figcaption></figure><p><strong>STEP:6 Take part in Competitions</strong></p><p>After you have understood the basics of Machine Learning, you can move on to the crazy part!!! Competitions! These will basically make you even more proficient in ML by combining your mostly theoretical knowledge with practical implementation.</p><p><a href="https://www.kaggle.com/"><strong>Kaggle</strong></a> is a place as what we want a more active, engaged and competitive platform. Data scientists are passionate about their rank and model performance. Go, dive into one of the live competitions currently running on Kaggle and give all that you have learnt a try! Good luck!</p><p>Try Learning from each competition you take part in, continue your learning. Read the kernels of various people mostly the kaggle rankers, you will get to learn a lot of stuff about data preparation/preprocessing, feature transformations and many more. It&#39;s a place where you constantly keep learning and implementing new things.</p><p><strong>STEP:6 Continue Learning</strong></p><p>There are a lot more things you need to explore and learn in machine learning, Its a never-ending process. Try out learning advanced algorithms, step into deep learning, Build a lot of projects and deploy them.</p><p>Once you are comfortable with basic ML algorithms, get into advanced level algorithms. Get started with BigData, Know Hadoop and its ecosystem and spark to understand and analyse data that is stored, processed and manipulated using Hadoop. Then, later start reading research papers and try implementing them.</p><p>Try out building end-to-end Machine Learning models and start writing blogs as well because communication is also very important and is a part of ML life-cycle, You need to be able to communicate the results either by preparing ppts, writing blogs, etc.</p><p>Congratulations!!! You are well on your way to becoming a full-fledged Machine Learning Engineer and you can continue enhancing your skills by working on more and more challenges and eventually creating more and more creative and difficult Machine Learning projects.</p><blockquote>“Learning is a never-ending process . When you stop learning, you stop improving”</blockquote><p><strong>Best Books:</strong></p><ol><li><a href="https://www.amazon.com/Machine-Learning-Absolute-Beginners-Introduction-ebook/dp/B07335JNW1/">“Machine Learning For Absolute Beginners: A Plain English Introduction (Second Edition)” by Oliver Theobald</a></li><li><a href="https://www.amazon.com/Machine-Learning-Dummies-John-Mueller-ebook/dp/B01FL4BGOI/">“Machine Learning For Dummies” by John Paul Mueller and Luca Massaron</a></li><li><a href="https://www.amazon.com/Fundamentals-Machine-Learning-Predictive-Analytics/dp/0262029448/">“Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies” by John D. Kelleher, Brian Mac Namee, and Aoife D’Arcy</a></li><li><a href="https://www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0123748569">“Data Mining: Practical Machine Learning Tools and Techniques” by Ian H. Witten, Eibe Frank, and Mark A. Hall</a></li><li><a href="https://geni.us/u6I0eVb">The Hundred-Page Machine Learning Book</a></li></ol><p>Hope you enjoyed this learning path on machine learning. If you have any suggestions to improve this learning path, please feel free to share them through the mail.</p><p>All The Best!!!!!!😇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b44eafce4dbe" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>