<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Muhammadsufian on Medium]]></title>
        <description><![CDATA[Stories by Muhammadsufian on Medium]]></description>
        <link>https://medium.com/@muhammadsufian602?source=rss-6cf16ace5a71------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*6XYz5muNx6yDXXEQ</url>
            <title>Stories by Muhammadsufian on Medium</title>
            <link>https://medium.com/@muhammadsufian602?source=rss-6cf16ace5a71------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 22:19:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@muhammadsufian602/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Generative Adversarial Networks (GANs) Explained from Scratch: A Complete Beginner’s Guide]]></title>
            <link>https://medium.com/@muhammadsufian602/generative-adversarial-networks-gans-explained-from-scratch-a-complete-beginners-guide-e3b7d6c4bd32?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3b7d6c4bd32</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[gans]]></category>
            <category><![CDATA[generative-adversarial]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sat, 01 Aug 2026 12:43:37 GMT</pubDate>
            <atom:updated>2026-08-01T12:43:37.190Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine asking an artist to create a realistic painting without ever showing them how to paint. Instead of giving direct instructions, another person simply reviews every painting and says, <em>“This looks real”</em> or <em>“This doesn’t.”</em> After receiving this feedback thousands of times, the artist gradually learns to create paintings that are almost impossible to distinguish from real ones.</p><p>This simple idea forms the foundation of <strong>Generative Adversarial Networks (GANs)</strong>, one of the most fascinating breakthroughs in deep learning. Unlike traditional neural networks that classify or predict existing data, GANs learn to <strong>generate entirely new data</strong> that closely resembles the training data.</p><p>From generating realistic human faces and creating artwork to restoring damaged photographs and producing synthetic medical images, GANs have transformed the field of generative artificial intelligence.</p><p>In this article, we’ll build a strong theoretical understanding of GANs from scratch. We’ll explore how they work, understand the roles of the Generator and the Discriminator, and see how these two neural networks compete to produce increasingly realistic images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*z08zKlxSaBkzCDUk6cGC6g.jpeg" /></figure><h3>Why Do We Need GANs?</h3><p>Before understanding what a GAN is, let’s first understand <strong>why it was introduced</strong>.</p><p>Most traditional deep learning models are <strong>discriminative models</strong>. Their primary objective is to analyze existing data and make predictions.</p><p>For example, a CNN can:</p><ul><li>Classify whether an image contains a cat or a dog.</li><li>Detect objects in an image.</li><li>Recognize handwritten digits.</li><li>Identify diseases from medical scans.</li></ul><p>Notice one common characteristic.</p><p>These models always work with <strong>existing data</strong>. They analyze what already exists but cannot create entirely new samples.</p><p>Suppose we ask a CNN:</p><blockquote>“Generate a completely new human face.”</blockquote><p>It simply cannot perform this task because it was never designed for data generation.</p><p>This limitation led researchers to develop <strong>generative models</strong>, whose objective is fundamentally different.</p><p>Instead of predicting labels, generative models learn the underlying patterns and distribution of the training data so they can create <strong>new samples</strong> that resemble real ones. This is exactly where GANs come into the picture.</p><h3>What is a Generative Adversarial Network (GAN)?</h3><p>A <strong>Generative Adversarial Network (GAN)</strong> is a deep learning model consisting of <strong>two neural networks</strong> that learn by competing against each other.</p><p>These networks are:</p><ul><li><strong>Generator (G)</strong></li><li><strong>Discriminator (D)</strong></li></ul><p>Unlike conventional neural networks that optimize a single objective, GANs rely on a competitive learning process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bdx1tsZwhlBnwL_DwEJD9A.png" /></figure><blockquote>The word <strong>Generative</strong> means the model can generate new data.</blockquote><blockquote>The word <strong>Adversarial</strong> means the two networks have opposite objectives and continuously challenge one another during training.</blockquote><p>Their roles are simple:</p><ul><li>The <strong>Generator</strong> tries to create realistic fake images.</li><li>The <strong>Discriminator</strong> tries to determine whether an image is real or fake.</li></ul><p>Because both networks continuously improve to outperform each other, the quality of the generated images gradually becomes better over time.</p><p>This competition is the key idea behind GANs.</p><h3>Understanding the Adversarial Concept</h3><p>The easiest way to understand GANs is to think of a counterfeiter and a detective.</p><blockquote>Imagine a person whose job is to create counterfeit currency.</blockquote><p>Initially, the fake notes are poorly made, making them easy to identify.</p><p>Now imagine another person whose job is to inspect every note and determine whether it is genuine or counterfeit.</p><p>Every time the counterfeiter fails, they learn from their mistakes and improve the quality of the fake notes.</p><p>At the same time, the inspector also becomes better at identifying increasingly convincing counterfeits.</p><p>As this competition continues:</p><ul><li>The counterfeiter improves.</li><li>The inspector improves.</li><li>Both become more skilled over time.</li></ul><p>GANs follow exactly the same learning strategy.</p><p>The <strong>Generator</strong> acts like the counterfeiter, while the <strong>Discriminator</strong> acts like the inspector. Neither network is perfect at the beginning, but continuous competition allows both to learn and improve.</p><h3>The Two Brains Behind a GAN</h3><p>Every GAN consists of two neural networks working together.</p><p>Although they share the same objective of improving the overall system, their individual goals are completely opposite.</p><blockquote>One network creates.</blockquote><blockquote>The other evaluates.</blockquote><blockquote>One tries to deceive.</blockquote><blockquote>The other tries to detect deception.</blockquote><p>This constant interaction is what makes GANs fundamentally different from most deep learning models.</p><p>Let’s understand each network individually before exploring how they work together.</p><p>In the next section, we’ll dive deep into the <strong>Generator</strong>, understand how it transforms random noise into realistic images, and then explore how the <strong>Discriminator</strong> learns to identify whether an image is real or fake.</p><h3>Understanding the Generator</h3><p>Now that we know a GAN consists of two competing neural networks, let’s begin with the first one — the <strong>Generator</strong>.</p><p>The Generator is the creative part of a GAN. Its responsibility is to create entirely new images that resemble the real images in the training dataset.</p><p>However, an important question arises:</p><blockquote><strong>If the Generator creates images, what does it use as input?</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_Ht72Um3Q6zkS5aGsDKlQ.png" /></figure><p>Unlike CNNs or other image-based neural networks, the Generator does <strong>not</strong> receive an image as input. Instead, it starts with a vector of random numbers known as the <strong>latent vector</strong> or <strong>noise vector</strong>.</p><p>A simplified view is shown below:</p><pre>Random Noise (Latent Vector)<br>            │<br>            ▼<br>       Generator<br>            │<br>            ▼<br>      Generated Image</pre><p>Initially, this random vector contains no meaningful visual information. It is simply a collection of randomly generated numbers.</p><p>For example:</p><pre>[0.42, -1.15, 0.83, 0.19, ...]</pre><p>To us, these numbers appear meaningless. However, during training, the Generator gradually learns how to transform these random values into realistic images.</p><h3>Why Does the Generator Use Random Noise?</h3><p>A common misconception is that the Generator starts with an incomplete image and improves it.</p><p>This is <strong>not</strong> how GANs work.Instead, every generated image begins with completely random noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*fzKoy5k1F89MczSfi_-svg.png" /></figure><blockquote>Why?</blockquote><p>Because the Generator’s objective is <strong>not to modify existing images</strong>. Its objective is to learn the underlying distribution of the training dataset so it can create entirely new samples.</p><p>Think of the random noise as a blank canvas.</p><p>Just as an artist begins with an empty canvas before creating a painting, the Generator begins with random numbers before learning how to generate an image.</p><p>Each different noise vector produces a different output image, allowing the Generator to generate an unlimited variety of images after training.</p><h3>How Does the Generator Learn?</h3><p>At the beginning of training, the Generator knows absolutely nothing.</p><p>When it receives random noise as input, its output looks like random pixels rather than a meaningful image.</p><p>For example:</p><pre>Random Noise<br>      │<br>      ▼<br> Generator<br>      │<br>      ▼<br>Random Pixels</pre><p>Since the Generator has not learned any patterns yet, the generated image looks nothing like the images in the training dataset.</p><blockquote>The Discriminator immediately recognizes it as fake.</blockquote><p>However, this failure is actually part of the learning process.</p><p>After every training iteration, the Generator receives feedback from the Discriminator.</p><p>Using this feedback, it updates its internal weights through backpropagation and gradually learns important visual features such as:</p><ul><li>Shapes</li><li>Edges</li><li>Colors</li><li>Textures</li><li>Object structures</li><li>Fine details</li></ul><p>After many iterations, the Generator starts producing images that look increasingly realistic.</p><p>Its ultimate goal is simple:</p><blockquote><strong>Generate images that are so realistic that the Discriminator believes they are real.</strong></blockquote><h3>Understanding the Generator Architecture</h3><p>The Generator is responsible for transforming a <strong>random noise vector</strong> into a realistic image. It does not create the image in a single step. Instead, the image is generated gradually through multiple layers, where each layer adds more details and increases the image size.</p><p>The complete Generator architecture :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*pqJGDRkT6TlV_fofN3-ADw.jpeg" /></figure><p>Let’s understand the purpose of each step.</p><h3>Step 1: Random Noise (Latent Vector)</h3><p>The Generator begins with a vector of random numbers called the <strong>latent vector</strong> or <strong>noise vector</strong>.</p><p>For example:</p><pre>[0.35, -1.21, 0.87, 0.14, ...]</pre><p>This vector contains no meaningful visual information. It simply acts as the starting point from which the Generator learns to create an image.</p><p>Every different noise vector produces a different generated image.</p><h3>Step 2: Fully Connected Layer</h3><p>The random noise is first passed through a <strong>Fully Connected (Dense) Layer</strong>.</p><p>Its purpose is to transform the one-dimensional noise vector into a larger set of learned features.</p><p>At this stage, the output is still not an image. Instead, it is a collection of numerical features that will later be converted into image feature maps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1zTzidazJt0C5OVa-aQeHQ.png" /></figure><h3>Step 3: Reshaping</h3><p>The output from the Fully Connected Layer is still a one-dimensional vector.</p><p>Since convolution operations require multi-dimensional data, this vector is <strong>reshaped</strong> into a small three-dimensional feature map.</p><p>For example:</p><pre>1D Vector<br>      │<br>      ▼<br>4 × 4 × 1024 Feature Map</pre><blockquote>This small feature map becomes the starting point for image generation.</blockquote><h3>Step 4: ConvTranspose2D</h3><p>The feature maps are then passed through <strong>Transposed Convolution (ConvTranspose2D)</strong> layers.</p><p>Unlike a standard convolution that reduces image size, ConvTranspose2D increases the spatial dimensions.</p><p>For example:</p><pre>4 × 4<br>   │<br>   ▼<br>8 × 8<br>   │<br>   ▼<br>16 × 16<br>   │<br>   ▼<br>32 × 32<br>   │<br>   ▼<br>64 × 64</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPV-LYuWCt1yuUGfVdA2ew.jpeg" /></figure><p>With every ConvTranspose2D layer, the Generator gradually builds a larger and more detailed image.</p><h3>Step 5: Batch Normalization</h3><p>After each ConvTranspose2D layer, <strong>Batch Normalization</strong> is applied.</p><p>Its purpose is to normalize the feature values, making training more stable and helping the Generator learn faster.</p><p>It also reduces sudden changes in feature distributions during training.</p><h3>Step 6: ReLU Activation</h3><p>Next, a <strong>ReLU (Rectified Linear Unit)</strong> activation function is applied.</p><p>ReLU introduces non-linearity into the network, allowing the Generator to learn complex image patterns instead of simple linear relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*MkPy4QD_3Yq-_9ESqbyfuA.jpeg" /></figure><blockquote>Without activation functions, the Generator would not be able to generate realistic images.</blockquote><h3>Step 7: Repeat the Process</h3><p>The sequence:</p><ul><li>ConvTranspose2D</li><li>Batch Normalization</li><li>ReLU</li></ul><p>is repeated several times.</p><p>With every repetition:</p><ul><li>The image becomes larger.</li><li>More visual features are learned.</li><li>Shapes become clearer.</li><li>Textures become richer.</li><li>Fine details gradually appear.</li></ul><p>The Generator slowly transforms random noise into a realistic image.</p><h3>Step 8: Output Layer</h3><p>After the repeated upsampling stages, the final <strong>ConvTranspose2D</strong> layer produces an image with the required dimensions.</p><p>For example:</p><pre>64 × 64 × 3</pre><p>where:</p><ul><li><strong>64 × 64</strong> represents the image size.</li><li><strong>3</strong> represents the RGB color channels.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wA7WHE5Dv4_BbdBev8A_aA.jpeg" /></figure><h3>Step 9: Tanh Activation</h3><p>Finally, a <strong>Tanh</strong> activation function is applied to the output image.</p><p>Tanh converts every pixel value into the range:</p><pre>-1 to +1</pre><p>Most GAN datasets are normalized to this range before training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*dl5BSaUuYBaBgVNJ_CXtFA.png" /></figure><p>Using Tanh ensures that the generated image has the same value range as the real training images, resulting in more stable learning.</p><p>At this point, the Generator produces its final output, the generated image.</p><h3>Understanding the Discriminator</h3><p>While the Generator creates images, the <strong>Discriminator</strong> evaluates them.</p><p>The Discriminator acts as a <strong>binary classifier</strong>, meaning it predicts whether an image belongs to one of two possible classes:</p><ul><li><strong>Real</strong></li><li><strong>Fake</strong></li></ul><p>Unlike the Generator, the Discriminator does not generate any images.</p><p>Instead, it receives an image as input and predicts the probability that the image is real.</p><p>A simplified workflow is:</p><pre>Input Image<br>      │<br>      ▼<br> Discriminator<br>      │<br>      ▼<br>Probability</pre><blockquote>The output is always a value between <strong>0</strong> and <strong>1</strong>.</blockquote><p>For example:</p><ul><li><strong>0.98</strong> → The image is almost certainly real.</li><li><strong>0.85</strong> → Probably real.</li><li><strong>0.60</strong> → Slightly real.</li><li><strong>0.35</strong> → Probably fake.</li><li><strong>0.08</strong> → Almost certainly fake.</li></ul><p>The closer the prediction is to <strong>1</strong>, the more confident the Discriminator is that the image is real.</p><p>The closer it is to <strong>0</strong>, the more confident it is that the image is fake.</p><h3>How Does the Discriminator Learn?</h3><p>The Discriminator is trained using two different types of images.</p><h3>Real Images</h3><p>These images come directly from the training dataset.</p><p>Since they are genuine images, their correct label is:</p><pre>Real = 1</pre><h3>Fake Images</h3><p>These images are generated by the Generator.</p><p>Their correct label is:</p><pre>Fake = 0</pre><blockquote>During training, the Discriminator compares its predictions with the correct labels.</blockquote><p>Whenever it makes a mistake, backpropagation updates its weights so that it can make more accurate predictions in future iterations.</p><p>As training continues, the Discriminator becomes increasingly skilled at distinguishing real images from fake ones.</p><h3>Understanding the Discriminator Architecture</h3><p>The Discriminator performs the opposite task.</p><p>Instead of generating an image, it receives an image and determines whether it is <strong>real</strong> or <strong>fake</strong>.</p><p>Its architecture gradually extracts important image features while reducing the spatial dimensions.</p><p>The complete Discriminator architecture follows this pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*gEzPHBryVjWR79tAk99Vdw.jpeg" /></figure><p>Let’s understand each step.</p><h3>Step 1: Input Image</h3><p>The Discriminator receives 2 images as input.</p><p>The 2 images are:</p><ul><li>A real image from the training dataset.</li><li>A fake image generated by the Generator.</li></ul><p>Its task is to determine which one it is.</p><h3>Step 2: Conv2D</h3><p>The input image passes through <strong>Convolution (Conv2D)</strong> layers.</p><p>These layers extract important visual features from the image.</p><p>Initially, they learn:</p><ul><li>Edges</li><li>Lines</li><li>Corners</li><li>Basic shapes</li></ul><p>As the network becomes deeper, it learns more complex visual structures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RYKkBaPxlC11GkOJb0IpBA.jpeg" /></figure><h3>Step 3: Leaky ReLU</h3><p>After every convolution layer, <strong>Leaky ReLU</strong> is applied.</p><p>Like ReLU, it introduces non-linearity into the network.</p><p>However, unlike ReLU, Leaky ReLU allows a small amount of information to pass even for negative values.</p><p>This helps prevent neurons from becoming inactive during training and improves gradient flow.</p><h3>Step 4: Batch Normalization</h3><p>Next, Batch Normalization is applied.</p><p>It stabilizes the learning process by keeping feature distributions consistent across different training batches.</p><p>This enables faster and more reliable training.</p><h3>Step 5: Repeat the Process</h3><p>The sequence:</p><ul><li>Conv2D</li><li>Leaky ReLU</li><li>Batch Normalization</li></ul><p>is repeated several times.</p><p>With each repetition:</p><ul><li>The image becomes smaller.</li><li>Feature extraction becomes stronger.</li><li>The network focuses on increasingly meaningful visual patterns.</li></ul><h3>Step 6: Flatten</h3><p>After all convolution operations, the extracted feature maps are converted into a one-dimensional vector using the <strong>Flatten</strong> operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vyxLpf33ZcjtbDkbuJg36g.jpeg" /></figure><p>This prepares the data for the final classification stage.</p><h3>Step 7: Fully Connected Layer</h3><p>The flattened feature vector is passed to a <strong>Fully Connected Layer</strong>.</p><p>This layer combines all extracted features and prepares a final prediction.</p><h3>Step 8: Sigmoid Activation</h3><p>Finally, the <strong>Sigmoid</strong> activation function converts the output into a probability between <strong>0</strong> and <strong>1</strong>.</p><p>For example:</p><ul><li><strong>0.97</strong> → Real image.</li><li><strong>0.81</strong> → Probably real.</li><li><strong>0.42</strong> → Probably fake.</li><li><strong>0.06</strong> → Fake image.</li></ul><p>This probability becomes the Discriminator’s final decision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*v019Tx8FSOIOxlJdsr-7CA.png" /></figure><h3>Generator vs. Discriminator</h3><p>Although both are neural networks, their objectives are completely opposite.</p><p>In simple terms:</p><ul><li><strong>Generator = Image Creator</strong></li><li><strong>Discriminator = Image Judge</strong></li></ul><blockquote>One creates, The other evaluates.</blockquote><p>Neither network can accomplish the complete task alone.</p><p>Only by working together do they gradually learn to generate highly realistic images.</p><h3>How Both Networks Compete</h3><p>The learning process in a GAN is often described as a game between two players.</p><p>The Generator’s objective is to fool the Discriminator by creating realistic fake images.</p><p>At the same time, the Discriminator’s objective is to correctly identify whether an image is real or fake.</p><p>Initially:</p><pre>Generator<br>     │<br>Creates Poor Image<br>     │<br>     ▼<br>Discriminator<br>     │<br>Predicts &quot;Fake&quot;</pre><p>Since the Discriminator easily identifies the fake image, the Generator receives a large error and updates its weights.</p><p>After many iterations, the Generator improves.</p><p>Now the process looks like this:</p><pre>Generator<br>     │<br>Creates Better Image<br>     │<br>     ▼<br>Discriminator<br>     │<br>Not Sure</pre><p>Eventually:</p><pre>Generator<br>     │<br>Creates Realistic Image<br>     │<br>     ▼<br>Discriminator<br>     │<br>Predicts &quot;Real&quot;</pre><p>At this stage, the Generator has successfully fooled the Discriminator.</p><p>However, the competition doesn’t stop here.</p><p>The Discriminator now learns from its mistake and becomes better at detecting fake images. This forces the Generator to improve even further.</p><p>This continuous competition creates a powerful learning cycle:</p><pre>Better Generator<br>        │<br>        ▼<br>Stronger Discriminator<br>        │<br>        ▼<br>Better Generator<br>        │<br>        ▼<br>Stronger Discriminator<br>        │<br>        ▼<br>      </pre><p>Both networks continuously challenge each other, and as a result, both become increasingly effective over time.</p><p>This adversarial learning process is the core idea behind Generative Adversarial Networks and the primary reason they are capable of generating highly realistic images.</p><h3>Loss Function and Training Process in GANs</h3><p>Now that we understand the Generator and Discriminator architectures, the next question is:</p><blockquote><strong>How do these two networks learn and improve during training?</strong></blockquote><p>The answer lies in the <strong>loss function</strong>.</p><p>A loss function measures how well a neural network is performing by comparing its prediction with the correct target.</p><ul><li><strong>Small Loss</strong> : The prediction is close to the target.</li><li><strong>Large Loss</strong> : The prediction is far from the target.</li></ul><p>During GAN training, both the Generator and the Discriminator have their own objectives, so they also have their own losses.</p><p>However, before calculating these losses, the <strong>Discriminator is always trained first</strong>, followed by the <strong>Generator</strong>.</p><h3>Why is the Discriminator Updated First?</h3><p>The Generator learns by receiving feedback from the Discriminator.</p><p>If the Discriminator is not trained first, it cannot provide meaningful feedback because it has not yet learned how to distinguish between real and fake images.</p><p>Therefore, every GAN training iteration follows this order:</p><pre>Step 1 → Train the Discriminator<br>Step 2 → Train the Generator</pre><p>This process repeats throughout training.</p><h3>Step 1: Updating the Discriminator</h3><p>The first network to be trained is the <strong>Discriminator</strong>.</p><p>During this step, the Discriminator receives two different types of images:</p><ul><li><strong>Real images</strong> from the training dataset.</li><li><strong>Fake images</strong> generated by the Generator.</li></ul><p>Its objective is straightforward:</p><ul><li>Predict <strong>1</strong> for real images.</li><li>Predict <strong>0</strong> for fake images.</li></ul><p>A simplified workflow is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GP0-cqK6u0bG7C_JTX8aA.jpeg" /></figure><p>The Discriminator makes predictions for both real and fake images.</p><p>It then compares these predictions with the correct labels to calculate its loss.</p><h3>Binary Cross Entropy (BCE) Loss</h3><p>Since the Discriminator performs <strong>binary classification</strong>, there are only two possible classes:</p><ul><li><strong>Real = 1</strong></li><li><strong>Fake = 0</strong></li></ul><p>Because there are only two classes, GANs commonly use <strong>Binary Cross Entropy (BCE) Loss</strong>.</p><p>The BCE loss is calculated using the following formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*kcXvR0qsLSQtwSRkVhX7lQ.png" /></figure><p>Where:</p><ul><li><strong>(y)</strong> = Actual (target) label.</li><li><strong>(p)</strong> = Predicted probability from the Discriminator.</li></ul><p>The BCE loss measures how close the predicted probability is to the correct target.</p><h3>Calculating the Discriminator Loss</h3><p>The Discriminator calculates two separate losses.</p><h3>1. Loss for Real Images</h3><p>Suppose:</p><ul><li>Target = <strong>1</strong></li><li>Prediction = <strong>0.95</strong></li></ul><p>The prediction is very close to the target.</p><p>Therefore:</p><ul><li><strong>Loss is small.</strong></li></ul><h3>2. Loss for Fake Images</h3><p>Suppose:</p><ul><li>Target = <strong>0</strong></li><li>Prediction = <strong>0.08</strong></li></ul><p>Again, the prediction is close to the target.</p><p>Therefore:</p><ul><li><strong>Loss is small.</strong></li></ul><blockquote>The total Discriminator loss is simply the sum of these two losses.</blockquote><pre>Discriminator Loss<br>        │<br>        ├── Real Image Loss<br>        │<br>        └── Fake Image Loss<br>Total Loss = Real Loss + Fake Loss</pre><p>A small total loss indicates that the Discriminator is successfully distinguishing between real and fake images.</p><h3>Backpropagation for the Discriminator</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*BrdgGzKYRKBaxwWpN7a5vg.png" /></figure><p>Once the Discriminator loss has been calculated, the network learns through <strong>backpropagation</strong>.</p><p>The process is straightforward:</p><pre>Prediction<br>      │<br>      ▼<br>Calculate Loss<br>      │<br>      ▼<br>Backpropagation<br>      │<br>      ▼<br>Update Weights</pre><p>Backpropagation computes how much each weight contributed to the error.</p><p>Using this information, the optimizer updates the weights so that the Discriminator makes better predictions during the next training iteration.</p><p>Once the Discriminator has been updated, we move to the Generator.</p><h3>Step 2: Updating the Generator</h3><p>After training the Discriminator, the Generator is trained.</p><p>Unlike the Discriminator, the Generator never sees real images directly.</p><p>Instead, it receives:</p><ul><li>Random noise as input.</li></ul><p>The Generator then produces a fake image and sends it to the Discriminator.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2csTd1qiYpfAm4wn2hi_BQ.jpeg" /></figure><p>Now comes the interesting part.</p><p>Although the image is fake, the Generator <strong>does not want the Discriminator to predict 0</strong>.</p><p>Instead, it wants the Discriminator to believe that the fake image is <strong>real</strong>.Therefore, during Generator training:</p><pre>Target = 1</pre><p>even though the generated image is fake.</p><h3>Calculating the Generator Loss</h3><p>Suppose the Discriminator predicts:</p><pre>Prediction = 0.08</pre><p>This means the Discriminator is almost certain that the generated image is fake.</p><p>Since the Generator wanted a prediction close to <strong>1</strong>, this produces a <strong>large loss</strong>.</p><p>For example:</p><ul><li>Target = <strong>1</strong></li><li>Prediction = <strong>0.08</strong></li></ul><p>Result:</p><ul><li><strong>Large Loss</strong></li></ul><p>A large loss tells the Generator:</p><blockquote>“Your generated image is not realistic enough. Improve it.”</blockquote><p>After updating its weights, the Generator gradually improves.</p><p>The predictions begin to move closer to the desired target.</p><pre>0.08<br>   │<br>   ▼<br>0.24<br>   │<br>   ▼<br>0.47<br>   │<br>   ▼<br>0.71<br>   │<br>   ▼<br>0.89<br>   │<br>   ▼<br>0.97</pre><p>As the prediction approaches <strong>1</strong>, the Generator becomes increasingly successful at fooling the Discriminator.</p><h3>Backpropagation for the Generator</h3><p>Once the Generator loss has been calculated, backpropagation is performed again.</p><p>This time, however, only the <strong>Generator’s weights</strong> are updated.</p><p>The Discriminator is <strong>not updated</strong> during this step.</p><p>The process is:</p><pre>Random Noise<br>      │<br>      ▼<br> Generator<br>      │<br>      ▼<br>Fake Image<br>      │<br>      ▼<br>Discriminator<br>      │<br>      ▼<br>Calculate Generator Loss<br>      │<br>      ▼<br>Backpropagation<br>      │<br>      ▼<br>Update Generator Weights</pre><p>The Generator gradually learns how to produce images that appear increasingly realistic.</p><h3>Complete Training Cycle</h3><p>A single training iteration of a GAN can be summarized as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KXAiRFbpAuqZZZ2fFi6BDA.jpeg" /></figure><p>This process is repeated for thousands of iterations.</p><p>Initially, the Generator produces poor-quality images, making them easy for the Discriminator to identify.</p><p>As training progresses:</p><ul><li>The Generator becomes better at creating realistic images.</li><li>The Discriminator becomes better at identifying fake images.</li></ul><p>This continuous competition forces both networks to improve together, eventually enabling the Generator to produce images that are remarkably similar to real data.</p><h3>Conclusion</h3><p>Generative Adversarial Networks (GANs) introduced a completely new way of training deep learning models. Instead of relying on a single neural network, GANs use two networks — the <strong>Generator</strong> and the <strong>Discriminator</strong> — that compete with each other in an adversarial learning process. This continuous competition enables the Generator to produce increasingly realistic images while the Discriminator becomes better at distinguishing real images from generated ones.</p><p>In this article, we built a strong theoretical foundation by exploring every core concept behind GANs. We began by understanding why GANs were introduced and how they differ from traditional deep learning models. We then examined the roles of the Generator and Discriminator, their architectures, the concept of latent space, the complete training workflow, Binary Cross Entropy (BCE) loss, and how backpropagation allows both networks to improve through iterative learning.</p><p>One of the most important ideas to remember is that <strong>GANs do not learn by being explicitly told how to generate images</strong>. Instead, the Generator gradually learns from the feedback provided by the Discriminator. Over thousands of training iterations, this adversarial process allows the Generator to transform random noise into highly realistic images that closely resemble the training data.</p><p>Although the underlying mathematics and training procedure may initially seem challenging, the overall concept is surprisingly intuitive once you understand the interaction between these two networks. The Generator continuously tries to fool the Discriminator, while the Discriminator continuously learns to detect fake images. This dynamic competition is the key reason why GANs have become one of the most powerful generative models in modern artificial intelligence.</p><p><strong>Thank you for reading! I hope this article helped you build a solid theoretical understanding of Generative Adversarial Networks (GANs). If you found it helpful, consider sharing it with others who are learning deep learning and generative AI. In the next article, we’ll move from theory to practice by implementing a GAN in PyTorch, where we’ll explore every line of code step by step.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3b7d6c4bd32" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Mask R-CNN: Architecture, Dataset, Training, and PyTorch Implementation]]></title>
            <link>https://medium.com/@muhammadsufian602/understanding-mask-r-cnn-architecture-dataset-training-and-pytorch-implementation-255b8f50fe2c?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/255b8f50fe2c</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[mask-rcnn]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sat, 25 Jul 2026 16:35:58 GMT</pubDate>
            <atom:updated>2026-07-25T16:35:58.892Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you’re building an AI system that detects cars in a parking lot. It successfully draws bounding boxes around every vehicle, but it can’t identify the exact shape of each car or determine which pixels belong to it. For applications requiring pixel-level precision, simple object detection isn’t enough. This is where <strong>Mask R-CNN</strong> comes in, combining object detection with instance segmentation to accurately locate and outline every individual object in an image</p><p>Imagine you’re building an AI system for a self-driving car. The car can detect pedestrians, vehicles, bicycles, and traffic signs using an object detection model. It successfully draws bounding boxes around every object in the scene.</p><p>But here’s the problem.</p><p>A bounding box only tells the model <strong>where an object is located</strong>, not <strong>which pixels actually belong to that object</strong>.</p><p>For example, consider a person standing beside a bicycle. An object detector may produce two overlapping bounding boxes, but it cannot precisely identify which pixels belong to the person and which belong to the bicycle. This lack of pixel-level understanding becomes a major limitation in applications where accuracy is critical.</p><p>This is exactly the problem that <strong>Mask R-CNN</strong> was designed to solve.</p><p>Instead of predicting only a class label and a bounding box, Mask R-CNN also generates a <strong>pixel-wise segmentation mask</strong> for every detected object. This enables the model to understand the exact shape and boundaries of each object, making it one of the most powerful models for instance segmentation.</p><p>In this article, we’ll build a solid conceptual understanding of Mask R-CNN before writing a single line of code. By the end, you’ll understand not only how the model works but also why each component exists and how they work together during training and inference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*UPpceG-ajRNnokWxmVkO-g.jpeg" /></figure><h3>Why Object Detection Isn’t Always Enough</h3><p>Traditional object detection models answer two important questions:</p><ul><li><strong>What is the object?</strong></li><li><strong>Where is the object?</strong></li></ul><p>For many applications, these answers are sufficient. However, some tasks require much more detailed information.</p><p>Imagine a medical imaging system trying to identify a tumor. A bounding box only provides an approximate location, while a surgeon needs to know the tumor’s exact shape and boundaries.</p><p>Similarly, in autonomous driving, distinguishing between a pedestrian’s body and the surrounding background can significantly improve safety.</p><p>This is where <strong>instance segmentation</strong> becomes essential.</p><blockquote>Instead of predicting a rectangular box around an object, instance segmentation predicts every pixel belonging to that object.</blockquote><p>As a result, the model knows:</p><ul><li>The object’s class</li><li>Its location</li><li>Its exact shape</li><li>Which pixels belong to it</li><li>Which pixels belong to the background</li></ul><p>This additional level of detail allows AI systems to make far more precise decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b1pNN-Fl79hvvvoiphaqbw.png" /></figure><h3>What is Mask R-CNN?</h3><p>Mask R-CNN is a deep learning model designed for <strong>instance segmentation</strong>.</p><p>Unlike traditional object detection models, it performs three tasks simultaneously:</p><ul><li>Detects objects</li><li>Predicts bounding boxes</li><li>Generates segmentation masks</li></ul><p>Think of it as an upgraded version of Faster R-CNN.</p><p>Faster R-CNN already performs object detection by identifying objects and predicting their bounding boxes. Mask R-CNN extends this architecture by adding another branch responsible for predicting a binary mask for every detected object.</p><p>This additional mask prediction branch transforms the model from an object detector into an instance segmentation model.</p><p>Simply put:</p><blockquote>Faster R-CNN tells you <strong>where</strong> an object is</blockquote><blockquote>Mask R-CNN tells you <strong>where it is and exactly what shape it has.</strong></blockquote><h3>The Evolution of the R-CNN Family</h3><p>Understanding Mask R-CNN becomes much easier when we first understand how it evolved.</p><h3>1. R-CNN</h3><p>The original R-CNN introduced the idea of combining region proposals with Convolutional Neural Networks.</p><p>Its workflow was straightforward:</p><ul><li>Generate candidate object regions.</li><li>Run a CNN separately on every region.</li><li>Classify each region individually.</li></ul><p>Although accurate, this approach was extremely slow because the CNN had to process thousands of candidate regions independently.</p><h3>2. Fast R-CNN</h3><p>Fast R-CNN improved efficiency by processing the entire image only once.</p><p>Instead of applying the CNN to every proposal separately, the model extracted a shared feature map from the image. Region proposals were then mapped onto this feature map.</p><p>This dramatically reduced computation while improving detection accuracy.</p><h3>3. Faster R-CNN</h3><p>Although Fast R-CNN improved speed, it still relied on an external algorithm to generate region proposals.</p><p>Faster R-CNN solved this by introducing the <strong>Region Proposal Network (RPN)</strong>.</p><p>Instead of depending on a separate proposal generation method, the network learned how to generate object proposals during training.</p><p>This made the entire detection pipeline much faster and fully trainable.</p><h3>4. Mask R-CNN</h3><p>Mask R-CNN builds directly upon Faster R-CNN.</p><p>Instead of stopping after object detection, it adds one additional branch that predicts a segmentation mask for every detected object.</p><p>This simple architectural addition enables the model to perform instance segmentation without sacrificing object detection performance.</p><h3>Understanding the Mask R-CNN Architecture</h3><p>At first glance, Mask R-CNN looks complicated because it consists of several interconnected components.</p><p>However, each component has a single, well-defined responsibility.</p><p>The complete workflow can be summarized as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PTe5GAx6-j00MiQeGTmakw.png" /></figure><p>Instead of learning everything at once, let’s understand each block individually.</p><h3>Backbone Network (ResNet)</h3><p>Every computer vision model begins with feature extraction.</p><p>Raw images contain millions of pixel values that have very little meaning on their own. The backbone network transforms these pixels into meaningful visual features.</p><blockquote>Mask R-CNN commonly uses <strong>ResNet</strong> as its backbone.</blockquote><p>As the image passes through ResNet, the network gradually learns increasingly complex patterns.</p><p>The early layers identify simple visual elements such as:</p><ul><li>Edges</li><li>Corners</li><li>Lines</li></ul><p>Intermediate layers learn:</p><ul><li>Shapes</li><li>Textures</li><li>Object parts</li></ul><p>The deeper layers recognize complete objects and semantic information.</p><p>Instead of working directly with pixels, every subsequent component of Mask R-CNN operates on these learned feature maps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qXC3DGi4UhS5_dzB8daGGw.png" /></figure><h3>Feature Pyramid Network (FPN)</h3><p>Objects rarely appear at a single scale.</p><p>Some images contain tiny objects, while others contain very large ones.</p><p>If the model relied on only one feature map, it would struggle to detect objects of varying sizes.</p><p>The <strong>Feature Pyramid Network (FPN)</strong> solves this problem by combining features extracted from multiple depths of the backbone.</p><p>This allows the model to detect:</p><ul><li>Small objects</li><li>Medium-sized objects</li><li>Large objects</li></ul><p>using richer multi-scale representations.</p><p>As a result, Mask R-CNN becomes significantly more robust across different object sizes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4tQczJN05-gwhetRNb5TPw.png" /></figure><h3>Region Proposal Network (RPN)</h3><p>After feature extraction, the model must determine <strong>where objects might exist</strong>.</p><p>Instead of scanning every possible location, the Region Proposal Network efficiently predicts a set of candidate object regions. These candidate regions are called <strong>Region Proposals</strong>.</p><blockquote>The RPN filters out most of the background and focuses only on regions that are likely to contain objects.</blockquote><p>Rather than examining millions of image locations, the rest of the model now processes only a small number of high-quality proposals.</p><p>This greatly improves both speed and efficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c5lpYiRhrTtQWw2plsPTrg.png" /></figure><h3>RoI Align</h3><p>Once region proposals have been generated, the model needs to extract features from each proposed object.</p><blockquote>However, every proposal has a different size.</blockquote><p>Neural networks require inputs with consistent dimensions.</p><p>RoI Align solves this problem by converting each proposal into a fixed-size feature representation while carefully preserving spatial information.</p><p>This is particularly important for segmentation because predicting accurate masks requires precise pixel alignment.</p><p>Without RoI Align, even small localization errors could produce inaccurate segmentation masks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V8dSRHHjYQEYoCyf4S6sdQ.png" /></figure><h3>Classification Head</h3><p>The Classification Head determines <strong>what object is present inside each proposed region</strong>.</p><p>For every region, it predicts the probability of belonging to each known class.</p><p>Examples include:</p><ul><li>Person</li><li>Dog</li><li>Cat</li><li>Car</li><li>Bicycle</li></ul><p>The class with the highest confidence becomes the final prediction.</p><h3>Bounding Box Head</h3><p>While the Region Proposal Network already predicts approximate object locations, these proposals are rarely perfect.</p><p>The Bounding Box Head refines them.</p><p>Instead of generating entirely new boxes, it adjusts the proposal coordinates so that they better match the true object boundaries.</p><p>The result is a much tighter and more accurate bounding box.</p><h3>Mask Head</h3><p>The Mask Head is what makes Mask R-CNN unique.</p><p>Instead of predicting another bounding box, it predicts a <strong>binary segmentation mask</strong> for every detected object.</p><p>Each pixel is classified as either:</p><ul><li>Object</li><li>Background</li></ul><p>This allows the model to recover the exact outline of every detected instance.</p><p>Even when multiple objects overlap, each receives its own independent segmentation mask. This ability to separate individual object instances is why the task is called <strong>instance segmentation</strong> rather than semantic segmentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r62aqlIjdcF9PSE5gnvAFQ.png" /></figure><h3>Understanding the Dataset Structure</h3><p>No deep learning model can learn without properly labeled data.</p><p>Unlike image classification, where each image belongs to a single category, instance segmentation requires much richer annotations because the model must learn:</p><ul><li>What object is present</li><li>Where it is located</li><li>Which pixels belong to that object</li></ul><p>For this reason, Mask R-CNN datasets contain much more information than datasets used for traditional object detection.</p><h3>COCO Dataset Structure</h3><p>The most widely used dataset for Mask R-CNN is the <strong>COCO (Common Objects in Context)</strong> dataset.</p><p>A typical COCO dataset looks like this:</p><pre>dataset/<br>│<br>├── images/<br>│     image1.jpg<br>│     image2.jpg<br>│     image3.jpg<br>│<br>└── annotations.json</pre><blockquote>Notice something important.</blockquote><p>There is <strong>only one annotation file</strong> for the entire dataset.</p><p>Unlike YOLO, COCO does <strong>not</strong> create one annotation file per image.</p><p>Instead, every annotation is stored inside a single JSON file.</p><p>This makes it much easier to manage thousands of images while keeping all annotations in one structured location.</p><h3>What is annotations.json?</h3><p>Think of annotations.json as the <strong>database</strong> of your dataset.</p><p>It stores all the information required to train the model.</p><p>For every object, it typically includes:</p><ul><li>Image ID</li><li>Category ID</li><li>Bounding Box</li><li>Segmentation Polygon</li><li>Object Area</li><li>Object ID</li><li>Crowd Information (if applicable)</li></ul><p>Unlike a simple text file, JSON can store hierarchical and structured data, making it ideal for instance segmentation datasets.</p><h3>Why Doesn’t Mask R-CNN Use TXT Files Like YOLO?</h3><p>This is one of the most common beginner questions.</p><p>YOLO only predicts:</p><ul><li>Class</li><li>Bounding Box</li></ul><p>Therefore, each image only needs a small text file containing a few numbers.</p><p>Mask R-CNN, however, predicts much more.</p><p>Each object requires:</p><ul><li>Class label</li><li>Bounding box</li><li>Segmentation mask</li><li>Polygon coordinates</li><li>Image reference</li><li>Additional metadata</li></ul><p>A plain text file isn’t flexible enough to store this information efficiently. That’s why most Mask R-CNN datasets use JSON instead.</p><h3>What Are Mask Images?</h3><p>Some custom datasets also include a <strong>masks</strong> folder.</p><p>Each mask image has exactly the same dimensions as its corresponding original image. Instead of storing colors, each pixel represents whether it belongs to an object or the background.</p><p>For example:</p><pre>Original Image<br>🐶 🌳</pre><pre>Mask</pre><pre>11110000<br>11110000<br>11110000<br>00000000</pre><p>Where:</p><ul><li><strong>1</strong> represents the object.</li><li><strong>0</strong> represents the background.</li></ul><p>These masks serve as the ground truth during training.</p><p>The Mask Head learns to predict masks that closely match these ground-truth annotations.</p><h3>COCO vs. Custom Dataset</h3><p>Not every Mask R-CNN dataset follows the same structure.</p><h3>COCO Format</h3><pre>dataset/<br>│<br>├── images/<br>└── annotations.json</pre><p>The segmentation information is stored directly inside annotations.json. No separate mask images are required.</p><h3>Custom Dataset</h3><pre>dataset/<br>│<br>├── images/<br>├── masks/<br>└── annotations.json (optional)</pre><p>Here, each image often has a corresponding mask image stored inside the masks folder. Both formats are valid.</p><p>The choice depends on how the dataset was created.</p><h3>Using a Pretrained Model vs. Training Your Own Model</h3><p>Before implementing Mask R-CNN, you need to decide which of these two situations applies to your project.</p><h3>Case 1 Using the Pretrained Model</h3><p>Suppose your application only needs to detect objects that already exist in the COCO dataset, such as:</p><ul><li>Person</li><li>Dog</li><li>Cat</li><li>Car</li><li>Bicycle</li></ul><p>In this case, the pretrained model already knows these classes. No additional training is required.</p><p>You simply load the model, switch it to evaluation mode, and perform inference on new images. This is the fastest and simplest way to use Mask R-CNN.</p><h3>Case 2 Training on Your Own Dataset</h3><p>Now imagine you’re building an agricultural application that detects:</p><ul><li>Apples</li><li>Oranges</li><li>Bananas</li></ul><blockquote>These classes are not the prediction targets of the pretrained model.</blockquote><p>Although the pretrained backbone has already learned useful visual features, its prediction layers are still configured for the original dataset.</p><p>To adapt the model to your own problem, you must train it on your dataset.</p><h3>Why Replace the Prediction Heads?</h3><p>The pretrained Mask R-CNN model is originally designed to predict the classes from the dataset on which it was trained.</p><p>When your project contains different categories, the final prediction layers no longer match your dataset.</p><p>Instead of training the entire network from scratch, we keep the powerful feature extractor and replace only the prediction heads responsible for:</p><ul><li>Classification</li><li>Bounding Box Regression</li><li>Segmentation Masks</li></ul><blockquote>The backbone already knows how to extract useful visual features.</blockquote><p>The new heads simply learn how to classify and segment your custom classes. This process is known as <strong>fine-tuning</strong>.</p><h3>Complete Training Workflow</h3><p>Once your dataset is ready, the overall training pipeline becomes straightforward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BEJwY8UJ6CpF6l4X8u5z8Q.png" /></figure><p>Every successful Mask R-CNN project follows this general workflow.</p><h3>Understanding the Optimizer</h3><p>Training is essentially the process of improving the model after every batch of data.</p><p>After making predictions, the model compares them with the ground truth and computes an error called the <strong>loss</strong>.</p><p>The optimizer’s job is to reduce this error.</p><blockquote>Think of the optimizer as a coach.</blockquote><p>After every mistake, the coach tells the model how to adjust its parameters so it performs better on the next attempt.</p><p>Without an optimizer, the model would never improve because its weights would remain unchanged.</p><h3>Learning Rate</h3><p>The learning rate determines <strong>how much the weights should change after each update</strong>.</p><p>A very small learning rate results in slow but stable learning.</p><blockquote>A very large learning rate causes large jumps, making training unstable.</blockquote><p>Choosing an appropriate learning rate allows the model to improve steadily while avoiding overshooting the optimal solution.</p><h3>Momentum</h3><p>Momentum helps the optimizer maintain a consistent direction during training.</p><p>Imagine pushing a shopping cart. Without momentum, every push starts from rest.</p><p>With momentum, the cart continues moving forward, requiring less effort to maintain its direction.</p><p>Similarly, momentum allows the optimizer to move more smoothly through the optimization landscape, reducing unnecessary zig-zagging and often reaching a better solution more quickly.</p><h3>Python Implementation of Mask R-CNN</h3><p>Now that we understand the theory, let’s implement Mask R-CNN using <strong>PyTorch</strong> and <strong>Torchvision</strong>. The implementation below covers the essential steps for training a custom instance segmentation model.</p><h3>Step 1: Install the Required Libraries</h3><p>Install the required Python packages before starting the project.</p><pre>pip install torch torchvision opencv-python matplotlib numpy</pre><h3>Step 2: Import the Required Libraries</h3><p>Import the libraries needed for model loading, image processing, and visualization.</p><pre>import torch<br>import torchvision<br>import cv2<br>import numpy as np<br>import matplotlib.pyplot as plt<br>from torch.utils.data import DataLoader<br>from torchvision.models.detection import maskrcnn_resnet50_fpn<br>from torchvision.models.detection.faster_rcnn import FastRCNNPredictor<br>from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor</pre><h3>Step 3: Load the Pretrained Mask R-CNN Model</h3><p>Load a pretrained Mask R-CNN model trained on the COCO dataset.</p><pre>model = maskrcnn_resnet50_fpn(weights=&quot;DEFAULT&quot;)</pre><h3>Step 4: Replace the Prediction Heads</h3><p>Replace the default prediction heads if you’re training on a custom dataset.</p><pre>num_classes = 4   # Background + 3 custom classes<br>in_features = model.roi_heads.box_predictor.cls_score.in_features<br>model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)<br>in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels<br>model.roi_heads.mask_predictor = MaskRCNNPredictor(<br>    in_features_mask,<br>    256,<br>    num_classes<br>)</pre><h3>Step 5: Create the Dataset and DataLoader</h3><p>Load your custom dataset using PyTorch’s DataLoader.</p><pre>train_dataset = CustomDataset(&quot;dataset/&quot;)<br>train_loader = DataLoader(<br>    train_dataset,<br>    batch_size=2,<br>    shuffle=True,<br>    collate_fn=lambda x: tuple(zip(*x))<br>)</pre><h3>Step 6: Configure the Optimizer</h3><p>Define the optimizer and learning parameters.</p><pre>optimizer = torch.optim.SGD(<br>    model.parameters(),<br>    lr=0.005,<br>    momentum=0.9,<br>    weight_decay=0.0005<br>)</pre><h3>Step 7: Train the Model</h3><p>Train the model by computing the loss and updating the weights.</p><pre>model.train()<br>for images, targets in train_loader:<br>    loss_dict = model(images, targets)<br>    loss = sum(loss for loss in loss_dict.values())<br>    optimizer.zero_grad()<br>    loss.backward()<br>    optimizer.step()</pre><h3>Step 8: Save the Trained Model</h3><p>Save the trained model for future inference.</p><pre>torch.save(model.state_dict(), &quot;mask_rcnn_model.pth&quot;)</pre><h3>Step 9: Load the Model for Inference</h3><p>Load the saved model and switch it to evaluation mode.</p><pre>model.load_state_dict(torch.load(&quot;mask_rcnn_model.pth&quot;))<br>model.eval()</pre><h3>Step 10: Run Predictions</h3><p>Use the trained model to make predictions on new images.</p><pre>with torch.no_grad():<br>    predictions = model(images)</pre><h3>Step 11: Visualize the Results</h3><p>Draw the predicted masks and bounding boxes on the input image using OpenCV or Matplotlib.</p><pre># Draw predicted masks<br># Draw bounding boxes<br># Display the segmented image</pre><p>This implementation provides the complete workflow for training and using a Mask R-CNN model. In a production project, you would expand the CustomDataset class, add validation, implement evaluation metrics, and include data augmentation to improve the model&#39;s performance.</p><h3>Real-World Applications</h3><p>Mask R-CNN is widely used in areas where pixel-level understanding is essential.</p><p>Some common applications include:</p><ul><li>Medical image analysis</li><li>Autonomous driving</li><li>Agricultural crop monitoring</li><li>Industrial defect detection</li><li>Robotics and automation</li><li>Satellite image analysis</li><li>Video surveillance</li><li>Wildlife monitoring</li></ul><p>Its ability to detect and precisely segment individual objects makes it valuable across many computer vision domains.</p><h3>Advantages</h3><ul><li>Performs object detection and instance segmentation simultaneously.</li><li>Produces accurate pixel-level masks.</li><li>Works well with transfer learning using pretrained backbones.</li><li>Highly flexible for custom datasets.</li><li>Strong performance on complex scenes with multiple objects.</li></ul><h3>Limitations</h3><ul><li>More computationally expensive than one-stage detectors like YOLO.</li><li>Slower inference due to the additional mask prediction branch.</li><li>Requires detailed annotations for training.</li><li>Training can be resource-intensive on large datasets.</li></ul><h3>Conclusion</h3><p>Mask R-CNN represents a significant milestone in the evolution of computer vision, bridging the gap between traditional object detection and pixel-level scene understanding. While earlier models focused primarily on identifying objects and predicting their locations, Mask R-CNN extends this capability by generating precise segmentation masks for every detected instance, enabling a much richer understanding of visual data.</p><p>Throughout this article, we explored the complete journey of Mask R-CNN — from understanding why object detection alone is sometimes insufficient, to examining the evolution of the R-CNN family, breaking down each architectural component, understanding dataset structures, comparing COCO and custom datasets, learning the overall training workflow, and finally implementing the model using PyTorch.</p><p>One of the biggest strengths of Mask R-CNN is its modular design. Each component, whether it’s the Backbone, Feature Pyramid Network (FPN), Region Proposal Network (RPN), RoI Align, Classification Head, Bounding Box Head, or Mask Head, has a specific role that contributes to the final prediction. Once you understand the purpose of each module, the overall architecture becomes much less intimidating.</p><p>Although Mask R-CNN requires more computational resources and detailed annotations than conventional object detection models, the additional pixel-level information it provides makes it invaluable for applications where precision is critical. From autonomous vehicles and medical imaging to agriculture, robotics, satellite imagery, and industrial inspection, Mask R-CNN continues to be one of the most widely adopted instance segmentation models in both research and industry.</p><p>As you continue exploring computer vision, don’t stop at simply using pretrained models. Try fine-tuning Mask R-CNN on your own datasets, experiment with different backbone networks, apply data augmentation techniques, and evaluate your models using appropriate performance metrics. These practical experiments will deepen your understanding and help you build more accurate and robust computer vision applications.</p><p>I hope this article has provided you with a clear and practical understanding of how Mask R-CNN works — from the underlying concepts to the implementation. If you found this guide helpful, consider following me on Medium for more in-depth articles on Artificial Intelligence, Deep Learning, Computer Vision, and Machine Learning, where we break down complex topics into practical, beginner-friendly explanations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=255b8f50fe2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[YOLOv8 Explained from Scratch: The Ultimate Beginner’s Guide to Object Detection (Easy to Learn &…]]></title>
            <link>https://medium.com/@muhammadsufian602/yolov8-explained-from-scratch-the-ultimate-beginners-guide-to-object-detection-easy-to-learn-59c0c655014d?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/59c0c655014d</guid>
            <category><![CDATA[yolov8]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 19:37:30 GMT</pubDate>
            <atom:updated>2026-07-23T19:37:30.873Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>YOLOv8 Explained from Scratch: The Ultimate Beginner’s Guide to Object Detection (Easy to Learn &amp; Remember)</strong></h3><p>Learn what YOLOv8 is, why it was created, how it detects multiple objects in a single image, and how it differs from a traditional CNN — all explained in the simplest way possible.</p><p>Object detection is one of the most exciting applications of deep learning. From self-driving cars and security cameras to face recognition and medical imaging, many real-world AI systems rely on the ability to not only recognize objects but also determine exactly where they are.</p><p>If you’ve already learned <strong>Convolutional Neural Networks (CNNs)</strong>, you’re halfway there. YOLOv8 builds upon many CNN concepts, but instead of simply classifying an image, it takes things a step further by detecting multiple objects and locating them within the image.</p><p>In this guide, we’ll build our understanding step by step, just like we did with CNNs, ResNet, and DenseNet. By the end of this chapter, you’ll understand what YOLOv8 is, why it was developed, how it works at a high level, and how to use a pretrained YOLOv8 model in Python.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Baq_6Vnio6he81I0vPobqg.png" /></figure><blockquote>Let’s begin from the very basics.</blockquote><h3>What Is YOLOv8?</h3><p>Before understanding YOLOv8, let’s quickly revisit what a traditional CNN does.</p><p>Imagine you give a CNN an image containing a cat.🐱</p><p>The CNN predicts: <strong>Cat</strong></p><p>If the image contains a dog instead: 🐶</p><p>The CNN predicts: <strong>Dog</strong></p><p>Simple.</p><p>A CNN is designed for <strong>image classification</strong>, meaning it answers only one question:</p><blockquote><strong>“What is the main object in this image?”</strong></blockquote><p>This works perfectly when an image contains a single dominant object. However, real-world images are rarely that simple.</p><h3>The Limitation of a Traditional CNN</h3><p>Now imagine the following image:</p><p>🚗 👨 🐶 🚲</p><p>Instead of containing just one object, the image has:</p><ul><li>A car</li><li>A person</li><li>A dog</li><li>A bicycle</li></ul><p>A traditional CNN can tell you that one of these objects exists, but it cannot answer questions like:</p><ul><li>Where is the dog?</li><li>Where is the bicycle?</li><li>Where is the person?</li><li>How many objects are present?</li></ul><p>In other words, a CNN understands <strong>what</strong> is in the image, but it doesn’t understand <strong>where</strong> those objects are located.</p><p>This limitation made CNNs unsuitable for many real-world applications.</p><h3>Why Was YOLO Created?</h3><p>Researchers wanted a model that could answer three questions simultaneously:</p><ul><li><strong>What objects are present?</strong></li><li><strong>Where is each object located?</strong></li><li><strong>How many objects are there?</strong></li></ul><p>This task is known as <strong>Object Detection</strong>.</p><p>Unlike image classification, object detection identifies every object in an image and predicts its exact location.</p><p>One of the most successful models developed for this purpose is <strong>YOLO</strong>.</p><h3>What Does YOLO Mean?</h3><p><strong>YOLO</strong> stands for:</p><blockquote><strong>You Only Look Once</strong></blockquote><p>The name reflects the model’s biggest advantage.</p><p>Earlier object detection algorithms analyzed the same image multiple times before making predictions. While this approach could produce accurate results, it was relatively slow.</p><p>YOLO introduced a different idea.</p><p>Instead of repeatedly scanning different parts of the image, it processes the entire image <strong>only once</strong> and predicts everything in a single forward pass.</p><p>Because of this design, YOLO became one of the fastest object detection models while still maintaining excellent accuracy.</p><h3>What Can YOLOv8 Detect?</h3><p>Suppose the following image is given as input:</p><p>🚗 👨 🐶 🚲</p><p>YOLOv8 can detect every object at the same time.</p><p>Its predictions might look something like this:</p><pre>Object         Location<br>Car   `        Bounding Box<br>Person         Bounding Box<br>Dog            Bounding Box<br>Bicycle        Bounding Box</pre><p>Instead of making one prediction for the entire image, YOLO predicts every object individually.</p><p>The impressive part is that all of these predictions are generated in <strong>one forward pass</strong>.</p><h3>YOLOv8 in One Simple Sentence</h3><p>If someone asks you to explain YOLOv8 in one sentence, you can say:</p><blockquote><strong>YOLOv8 is a deep learning model that detects multiple objects in an image and predicts both their class and their location in a single forward pass.</strong></blockquote><p>This single sentence captures the core idea behind YOLO.</p><h3>Input and Output of YOLOv8</h3><p>The input to YOLOv8 is exactly the same as a CNN.</p><p>An image is first converted into pixels through preprocessing and then passed into the neural network.</p><p>The difference lies in the output.</p><p>A CNN produces something like:</p><pre>Cat</pre><p>YOLOv8 produces much more information for every detected object.</p><p>For each object, it predicts:</p><ul><li>The object’s class</li><li>Its location</li><li>A confidence score</li></ul><p>For example:</p><pre>Dog</pre><pre>x = 120<br>y = 90<br>width = 80<br>height = 120</pre><pre>Confidence = 98%</pre><p>This tells us much more than simply saying “Dog.”</p><p>It tells us:</p><ul><li>Which object was detected.</li><li>Where the object is located.</li><li>How confident the model is about the prediction.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cr7YspNlrm31V3UWnPnKFg.jpeg" /></figure><h3>Understanding YOLO’s Predictions</h3><p>Every object detected by YOLO consists of three important pieces of information.</p><h3>1. Object Class</h3><p>The class tells us <strong>what</strong> the object is.</p><p>Examples include:</p><ul><li>Dog</li><li>Cat</li><li>Person</li><li>Car</li><li>Bicycle</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZCtCM4xdIHttFO14qV4LIA.jpeg" /></figure><h3>2. Bounding Box</h3><p>A bounding box tells us <strong>where</strong> the object is located.</p><p>Instead of storing every pixel of an object, YOLO represents its location using four values:</p><ul><li>Center X</li><li>Center Y</li><li>Width</li><li>Height</li></ul><p>These four numbers define a rectangle that surrounds the object.</p><p>This rectangle is called the <strong>Bounding Box</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pNa7ED5u6jTOueOsx0eWRQ.jpeg" /></figure><h3>3. Confidence Score</h3><p>YOLO also predicts a confidence score for every detection.</p><p>For example:</p><pre>Dog</pre><pre>Confidence = 98%</pre><p>The higher the confidence score, the more certain the model is that its prediction is correct.</p><h3>Does YOLOv8 Still Use CNN?</h3><p>A common misconception among beginners is that YOLO is completely different from CNNs.</p><blockquote>That isn’t true.</blockquote><p>YOLOv8 still begins with convolutional layers that extract useful features from an image.</p><p>So everything you’ve learned about CNN feature extraction is still valuable.</p><p>The main difference is that after extracting features, YOLO uses additional layers to detect objects instead of simply classifying the image.</p><p>You can think of YOLO as a powerful extension of CNNs rather than a replacement.</p><h3>The High-Level Workflow of YOLOv8</h3><p>Although we’ll study the internal architecture in detail later, it’s helpful to understand the overall pipeline first.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izj2pz3Qr_8Nlr45-EdbYA.jpeg" /></figure><p><strong>Here’s what happens:</strong></p><ul><li>The input image is preprocessed.</li><li>The CNN backbone extracts useful visual features.</li><li>These features are passed to YOLO’s detection head.</li><li>The detection head predicts the object class, bounding box, and confidence score for every detected object.</li></ul><p>All of this happens in a single forward pass, making YOLO both fast and efficient.</p><h3>CNN vs. YOLO: The Easiest Way to Remember the Difference</h3><p>The simplest way to remember the difference is this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NM6GEdt22QZb0akUo3oX5g.jpeg" /></figure><h3>CNN</h3><pre>Image<br>   ↓<br>Cat</pre><p>A CNN only tells you <strong>what</strong> is in the image.</p><h3>YOLO</h3><pre>Image<br>   ↓</pre><pre>Cat      → Location</pre><pre>Dog      → Location</pre><pre>Bottle   → Location</pre><pre>Person   → Location</pre><p>YOLO tells you both:</p><ul><li><strong>What</strong> the objects are.</li><li><strong>Where</strong> they are located.</li></ul><p>Whenever you compare CNNs and YOLO, remember this simple difference.</p><h3>The Three Main Components of YOLOv8</h3><p>Although we’ll explore the complete architecture in the next chapter, it’s helpful to know that YOLOv8 is built around three major components:</p><h3>1. Backbone</h3><p>The <strong>Backbone</strong> is responsible for extracting important features from the input image.</p><p>It works similarly to a CNN by learning patterns such as:</p><ul><li>Edges</li><li>Corners</li><li>Shapes</li><li>Textures</li><li>Complex object features</li></ul><p>Instead of making predictions, the backbone’s job is simply to understand the image and generate rich feature maps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uHJzO6w0edkNJFo4TS5GTQ.jpeg" /></figure><h3>2. Neck</h3><p>The <strong>Neck</strong> receives the feature maps produced by the backbone.</p><p>Its purpose is to combine information from different scales so that YOLO can detect:</p><ul><li>Small objects</li><li>Medium-sized objects</li><li>Large objects</li></ul><p>This helps the model detect objects of varying sizes more accurately.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G_wFBsYAAbPIc-ddDcH5Ig.jpeg" /></figure><ul><li><strong>Inputs from the Backbone:</strong> The Neck directly receives feature maps at three different resolution scales from the Backbone: <strong>P3</strong> (small objects / high resolution), <strong>P4</strong> (medium objects), and <strong>P5</strong> (large objects / deep semantic features).</li><li><strong>Top-Down Path (Upsampling):</strong> High-level semantic information from deeper layers (like P5) is upsampled and combined with feature maps from earlier layers (P4, P3). This helps lower-resolution features gain contextual knowledge.</li><li><strong>Bottom-Up Path (Downsampling):</strong> Fine-grained spatial detail from earlier layers is downsampled and routed back up through the network using convolutional blocks, strengthening localization accuracy across all scales.</li><li><strong>Feature Concatenation:</strong> At every level, features from different paths are concatenated (joined together) rather than overwritten, preserving both spatial detail and rich semantic context.</li><li><strong>Output to Detection Head:</strong> Once the multi-scale features are fully fused into unified feature maps (represented as <strong>N3/U3</strong>, <strong>N4/U4</strong>, and <strong>N5/U5</strong>), they are passed directly to the <strong>YOLO Head</strong> to predict bounding box coordinates and object classes.</li></ul><h3>3. Detection Head</h3><p>The <strong>Detection Head</strong> is the final stage of YOLOv8.</p><p>Using the feature maps provided by the neck, it predicts:</p><ul><li>The object class (Dog, Cat, Car, Person, etc.)</li><li>The bounding box coordinates</li><li>The confidence score</li></ul><p>This is where the final object detection results are generated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dkXyX7TKpDWnC_IAytipFg.jpeg" /></figure><p>Here is how the Detection Head works in very simple terms:</p><ul><li><strong>It receives features from the Neck:</strong> The Neck gives the Detection Head three different feature maps — one for small objects, one for medium objects, and one for large objects.</li><li><strong>It uses a Decoupled Head structure:</strong> In YOLOv8, predicting <strong>where</strong> an object is and predicting <strong>what</strong> the object is are handled by two separate branches:</li><li><strong>Box Branch (Where):</strong> This part calculates the bounding box coordinates (x,y,w,h) to mark the object’s location.</li><li><strong>Class Branch (What):</strong> This part calculates the probability score for each class to decide what object is inside the box.</li><li><strong>It is Anchor-Free:</strong> Older YOLO versions used preset anchor boxes to guess object shapes, but YOLOv8 directly predicts the center of an object and its distance to the box edges. This makes the model faster and more accurate.</li><li><strong>It outputs the final result:</strong> For every object detected in the image, the head outputs the final bounding box, the object class label, and a confidence score.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/1*oFEE8vuuL7IsSGWOGvfBXw.png" /></figure><h3>How Does YOLOv8 Detect Objects?</h3><p>Now that we’ve seen the three main parts of YOLOv8, let’s understand what actually happens inside the <strong>Detection Head</strong>.</p><p>After the <strong>Backbone</strong> extracts important image features and the <strong>Neck</strong> combines features from different scales, the resulting <strong>feature maps</strong> are passed to the <strong>Detection Head</strong>.</p><p>Instead of working directly on the original image, the Detection Head works on these feature maps because they contain rich information about the objects present in the image.</p><p>To detect objects, the Detection Head conceptually divides each feature map into a <strong>grid</strong>. Each <strong>grid cell</strong> is responsible for checking whether an object is present in its region.</p><p>For every grid cell, the model predicts:</p><ul><li><strong>Is there an object here?</strong> (Confidence Score)</li><li><strong>What is the object?</strong> (Object Class)</li><li><strong>Where is the object located?</strong> (Bounding Box)</li></ul><blockquote>In other words, every grid cell looks at its assigned region and tries to answer these three questions at the same time.</blockquote><p>The Detection Head then combines all these predictions from every grid cell to detect all the objects present in the image.</p><p>For <strong>every object</strong> it finds, YOLO predicts three things simultaneously:</p><ul><li><strong>Object Class</strong> <em>(What is it?)</em></li><li><strong>Bounding Box</strong> <em>(Where is it?)</em></li><li><strong>Confidence Score</strong> <em>(How confident is the model that an object is actually present?)</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cfFkFwWP5EYU4DCdT4bfJg.jpeg" /></figure><p>For example, suppose the model detects a dog.</p><p>Instead of simply predicting:</p><pre>Dog</pre><p>YOLO predicts something like:</p><pre>Dog</pre><pre>Center X = 120<br>Center Y = 90<br>Width = 80<br>Height = 120</pre><pre>Confidence = 98</pre><p>This means:</p><ul><li>The detected object is a <strong>Dog</strong>.</li><li>The rectangle surrounding the dog starts from the predicted center position.</li><li>The bounding box has the given width and height.</li><li>The model is <strong>98% confident</strong> that this prediction is correct.</li></ul><p>This process is repeated for <strong>every object</strong> in the image.</p><p>If an image contains a car, a person, and a bicycle, the Detection Head predicts separate information for each object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5mxZPxTOAO0I1ucd14kXrQ.png" /></figure><h3>Why Doesn’t YOLO Draw Multiple Boxes Around the Same Object?</h3><p>At this point, you might think:</p><blockquote>“If the Detection Head predicts objects, what stops YOLO from predicting the same object multiple times?”</blockquote><p>That’s an excellent question.</p><p>The answer is <strong>Non-Maximum Suppression (NMS).</strong></p><h3>What Is Non-Maximum Suppression (NMS)?</h3><p>The Detection Head doesn’t usually predict <strong>one bounding box</strong> for an object.</p><p>Instead, it often predicts <strong>many overlapping bounding boxes</strong> for the same object.</p><blockquote>For example, suppose YOLO detects a dog.</blockquote><p>Instead of predicting just one box, it may initially produce something like this:</p><pre>Dog → 98%<br>Dog → 95%<br>Dog → 92%<br>Dog → 89%</pre><p>If all of these boxes were displayed, the output would look messy because multiple boxes would surround the same dog.</p><p>Clearly, we don’t want that.</p><p><strong>How Does NMS Solve This Problem?</strong></p><p>NMS removes duplicate detections.</p><p>It follows a simple process:</p><p><strong>Step 1: Keep the Highest Confidence Box</strong></p><p>Suppose YOLO predicts these boxes:</p><pre>   Bounding Box   Confidence <br>   Box A                 98% <br>   Box B                 95% <br>   Box C                 92% </pre><p>The first step is to keep the box with the highest confidence.</p><p>In this case:</p><blockquote><strong>Box A (98%)</strong></blockquote><p><strong>Step 2: Compare the Remaining Boxes</strong></p><p>Now YOLO checks whether the remaining boxes overlap significantly with Box A.</p><p>If Box B and Box C are simply duplicate predictions of the same dog, they are removed.</p><p><strong>Step 3: Keep Only the Best Detection</strong></p><p>After NMS, only one bounding box remains.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_0EdGgMLDZW4o9ta9He0Ag.jpeg" /></figure><h3>Preparing a Custom Dataset for YOLOv8</h3><p>Running a pretrained YOLOv8 model is simple, but training YOLO to detect your own objects requires a properly organized dataset.</p><blockquote>Unlike a CNN, where each image has a single class label, YOLO needs much more information because it must learn both <strong>what</strong> the object is and <strong>where</strong> it is located.</blockquote><p>To achieve this, every training image must have a corresponding annotation file.</p><h3>YOLOv8 Dataset Structure</h3><p>A typical YOLO dataset looks like this:</p><pre>dataset/<br> images/<br>    train/<br>        dog1.jpg<br>        dog2.jpg<br> labels/<br>    train/<br>        dog1.txt<br>        dog2.txt</pre><p>Notice something important.</p><p>The <strong>labels/</strong> folder does <strong>not</strong> contain images.</p><p>Instead, it contains annotation files with the <strong>.txt</strong> extension.</p><p>These files tell YOLO everything it needs to know about the objects present in each image.</p><p>These label data are also called ANNONATED IMAGES .</p><p><strong>One Image = One Label File</strong></p><p>This is one of the most important rules when preparing a YOLO dataset.</p><p>Every image must have a label file with exactly the same filename.</p><pre>dog1.jpg   ↔   dog1.txt</pre><pre>dog2.jpg   ↔   dog2.txt</pre><p>Only the file extension changes.</p><p>This allows YOLO to automatically match every image with its annotations during training.</p><p>If an image doesn’t have a matching label file, YOLO assumes there are no annotated objects in that image.</p><h3>What Is Stored Inside a Label File(ANNONATED DATA)?</h3><p>Each line inside a label file represents <strong>one object</strong>.</p><p>The format is:</p><pre>Class_ID  Center_X  Center_Y  Width  Height</pre><p>For example:</p><pre>0 0.52 0.48 0.30 0.40</pre><p>Let’s understand each value.</p><p>ValueMeaning</p><p><strong>0 is </strong>Class ID</p><p><strong>(0.52,0.48) x,y C</strong>oordinates</p><p><strong>(0.30,0.40)height &amp; width of </strong>Bounding box</p><p>Notice that the coordinates are <strong>normalized</strong>, meaning they are stored as values between <strong>0 and 1</strong> instead of pixel values.</p><p>This allows YOLO to work with images of different sizes without changing the annotation format.</p><h3>One Object = One Line</h3><p>Every object in an image gets its own line.</p><p>Suppose an image contains:</p><ul><li>Dog</li><li>Person</li><li>Car</li></ul><p>The label file might look like this:</p><pre>0 0.25 0.45 0.18 0.30<br>2 0.50 0.60 0.20 0.45<br>3 0.78 0.42 0.25 0.20</pre><p>Here, YOLO reads each line independently and learns that the image contains three different objects.</p><p>The more objects an image contains, the more lines its label file will have.</p><h3>Where Does YOLO Get the Class Names?</h3><p>A common beginner question is:</p><blockquote><strong>“The label file only contains numbers. So how does YOLO know that 0 means Dog or 2 means Person?”</strong></blockquote><p>The answer is simple.</p><p>YOLO gets the class names from the <strong>data.yaml</strong> file.</p><h3>Understanding data.yaml</h3><p>A typical <strong>data.yaml</strong> file looks like this:</p><pre>train: dataset/images/train<br>val: dataset/images/val<br>nc: 4<br>names:<br>  - dog<br>  - cat<br>  - person<br>  - car</pre><p>Let’s understand what each part means.</p><ul><li><strong>train</strong> → Path to the training images.</li><li><strong>val</strong> → Path to the validation images.</li><li><strong>nc</strong> → Total number of classes.</li><li><strong>names</strong> → The class names in order.</li></ul><p>The order inside the <strong>names</strong> list automatically assigns the class IDs.This mapping is created automatically.</p><p>You never manually write:</p><pre>class = &quot;dog&quot;</pre><p>YOLO already knows that:</p><ul><li>0 → Dog</li><li>1 → Cat</li><li>2 → Person</li><li>3 → Car</li></ul><p>because it reads the <strong>data.yaml</strong> file before training starts.</p><h3>How Does YOLO Connect Everything?</h3><p>When training begins, YOLO follows a simple sequence.</p><ol><li>It reads <strong>data.yaml</strong>.</li><li>It loads the class names.</li><li>It reads every image.</li><li>It finds the matching label file.</li><li>It reads the class IDs and bounding box coordinates.</li><li>It learns to predict both the object class and its location.</li></ol><p>All of this happens automatically.</p><h3>Training a YOLOv8 Model</h3><p>Once your dataset is ready, training requires only a few lines of code.</p><pre>from ultralytics import YOLO<br>model = YOLO(&quot;yolov8n.pt&quot;)<br>model.train(<br>    data=&quot;data.yaml&quot;,<br>    epochs=50,<br>    imgsz=640<br>)</pre><p>When you call model.train(), YOLO automatically:</p><ul><li>Reads the dataset paths.</li><li>Loads all images.</li><li>Reads every annotation file.</li><li>Matches images with their labels.</li><li>Learns object classes.</li><li>Learns bounding box locations.</li><li>Updates the model weights after every epoch.</li></ul><p>You don’t need to manually parse annotation files or map class names to IDs. The Ultralytics library handles everything internally.</p><h3>Key Takeaways</h3><p>Let’s quickly recap what you’ve learned in this chapter.</p><ul><li>A traditional CNN performs <strong>image classification</strong>, while YOLO performs <strong>object detection</strong>.</li><li>YOLO predicts three things for every object:</li><li>Object Class</li><li>Bounding Box</li><li>Confidence Score</li><li>The YOLOv8 pipeline consists of four major stages:</li><li>Backbone</li><li>Neck</li><li>Detection Head</li><li>Non-Maximum Suppression (NMS)</li><li>The Detection Head predicts multiple objects by analyzing feature maps and making predictions for each grid cell.</li><li>NMS removes duplicate bounding boxes and keeps only the best detection</li><li>Every training image must have a matching .txt annotation file.</li><li>Each line inside a label file represents one object.</li><li>A label file stores:</li><li>Class ID</li><li>Center X</li><li>Center Y</li><li>Width</li><li>Height</li><li>The <strong>data.yaml</strong> file defines:</li><li>Dataset paths</li><li>Number of classes</li><li>Class names</li><li>Class IDs such as 0, 1, 2, and 3 are simply identifiers that map to class names. They are <strong>not</strong> binary labels.</li><li>YOLO automatically reads the dataset, annotations, and class mappings during training, making the training process simple and efficient.</li></ul><h3>Running Your First YOLOv8 Model in Python</h3><p><strong>Step 1: Install the Ultralytics Library</strong></p><p>If you haven’t installed YOLOv8 yet, run the following command:</p><pre>pip install ultralytics</pre><p>This installs the official Ultralytics package, which includes pretrained YOLOv8 models and all the utilities needed for object detection.</p><p><strong>Step 2: Import the YOLO Class</strong></p><p>After installation, import the YOLO class.</p><pre>from ultralytics import YOLO</pre><p>This class allows you to load pretrained models, train custom models, validate them, and make predictions.</p><p><strong>Step 3: Load a Pretrained YOLOv8 Model</strong></p><p>Now load the pretrained YOLOv8 Nano model.</p><pre>model = YOLO(&quot;yolov8n.pt&quot;)</pre><p>When this line executes for the first time, YOLO automatically:</p><ul><li>Downloads the pretrained <strong>YOLOv8 Nano</strong> weights (if they are not already available).</li><li>Loads the learned weights into memory.</li><li>Prepares the model for inference.</li></ul><p>You don’t need to manually download or configure the model.</p><p><strong>Step 4: Make Predictions</strong></p><p>Detect objects in an image using:</p><pre>results = model.predict(&quot;image.jpg&quot;)</pre><p>That’s it. With just one line of code, YOLO starts the complete object detection pipeline.</p><h3>Conclusion</h3><p>We started by revisiting the limitations of a traditional CNN and learned why image classification alone isn’t enough for many real-world applications. From there, we explored how YOLOv8 solves this problem by detecting <strong>multiple objects</strong>, predicting their <strong>locations</strong> using bounding boxes, and assigning a <strong>confidence score</strong> to each prediction — all in a single forward pass.</p><p>Along the way, we also developed a high-level understanding of the complete YOLOv8 pipeline. We learned the role of the <strong>Backbone</strong> in extracting image features, the <strong>Neck</strong> in combining multi-scale feature maps, the <strong>Detection Head</strong> in predicting object classes, bounding boxes, and confidence scores, and finally how <strong>Non-Maximum Suppression (NMS)</strong> removes duplicate detections to produce clean and accurate results.</p><p>We then looked at how easy it is to use a pretrained YOLOv8 model in Python with the Ultralytics library, making object detection possible with just a few lines of code. Finally, we explored how YOLO datasets are organized, how annotation files are structured, how the data.yaml file maps class IDs to class names, and why class IDs such as 0, 1, 2, and 3 are simply identifiers—not binary labels.</p><p>At this point, you should have a clear understanding of <strong>what YOLOv8 is, why it was developed, what problem it solves, what it predicts, and how all the major components work together at a high level</strong>. This conceptual understanding is the most important step before diving into the model’s internal architecture.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59c0c655014d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DenseNet201 Explained from Scratch: Learn Dense Convolutional Networks the Easy Way (Complete…]]></title>
            <link>https://medium.com/@muhammadsufian602/densenet201-explained-from-scratch-learn-dense-convolutional-networks-the-easy-way-complete-cdf112d8bae9?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdf112d8bae9</guid>
            <category><![CDATA[desnet]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Mon, 20 Jul 2026 13:08:55 GMT</pubDate>
            <atom:updated>2026-07-20T13:08:55.779Z</atom:updated>
            <content:encoded><![CDATA[<h3>DenseNet201 Explained from Scratch: Learn Dense Convolutional Networks the Easy Way (Complete Beginner Guide)</h3><p>In this article, we’ll learn DenseNet201 from scratch in a simple and beginner-friendly way. We will understand the intuition behind DenseNet first, then learn its architecture, and finally build a complete image classification implementation using TensorFlow and a pre-trained DenseNet201 model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*kuaTn1nbduEsqEsJZzasWA.png" /></figure><h3>What is DenseNet?</h3><p><strong>DenseNet</strong> stands for:</p><blockquote><strong><em>Dense Convolutional Network</em></strong></blockquote><p>Despite the new name, DenseNet is still a <strong>Convolutional Neural Network (CNN)</strong>.</p><p>It performs image classification just like a normal CNN.</p><p>The main difference is <strong>not the operations performed</strong>, but <strong>how information flows between layers</strong>.</p><h3>DenseNet is Still a CNN</h3><p>Since DenseNet is a CNN, all the familiar components are still present.</p><p>It still contains:</p><ul><li>Convolution Layers</li><li>ReLU Activation Function</li><li>Pooling Layers</li><li>Batch Normalization</li><li>Global Average Pooling (or sometimes Flatten)</li><li>Fully Connected (Dense) Layer</li><li>Softmax Activation for Classification</li></ul><p>Nothing has been removed.</p><p>Instead, DenseNet introduces a smarter way for convolution layers to communicate with one another.</p><h3>Why Was DenseNet Created?</h3><p>Before understanding DenseNet, let’s first look at how a normal CNN works.</p><p>Imagine a CNN with four convolution layers.</p><pre>Image <br> ↓ <br>Conv1 <br> ↓ <br>Conv2<br> ↓ <br>Conv3<br> ↓ <br>Conv4</pre><p>This is called a <strong>sequential architecture</strong> because every layer only receives information from the layer immediately before it.</p><p>For example:</p><ul><li>Conv2 only receives the output of Conv1.</li><li>Conv3 only receives the output of Conv2.</li><li>Conv4 only receives the output of Conv3.</li></ul><blockquote>Notice something important.</blockquote><p>As we move deeper into the network, the information learned by earlier layers becomes less directly accessible.</p><p>The fine details extracted in the first few layers gradually become diluted as information passes through many intermediate layers.</p><p>This is one of the major limitations of traditional CNN architectures.</p><h3>The Main Idea Behind DenseNet</h3><p>The creators of DenseNet asked a simple question:</p><blockquote>Why should a convolution layer receive information from only the previous layer<strong><em>?</em></strong></blockquote><p>Why not allow it to receive information from <strong>every earlier layer</strong>?</p><p>Instead of this:</p><pre>Image <br> ↓ <br>Conv1 <br> ↓ <br>Conv2<br> ↓ <br>Conv3<br> ↓ <br>Conv4</pre><p>DenseNet connects layers like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Av6H0oTNuyyf0t6MWj6HvA.jpeg" /></figure><p>Now every layer receives:</p><ul><li>Output of Conv1</li><li>Output of Conv2</li><li>Output of Conv3</li><li>And so on…</li></ul><p>Every new layer has access to <strong>all previously learned features</strong>.</p><p>This is why the architecture is called <strong>Dense Convolutional Network</strong>. The connections between layers are <strong>dense</strong>.</p><h3>Understanding Dense Connections</h3><p>The most important concept in DenseNet is the <strong>Dense Connection</strong>.</p><p>A dense connection means:</p><blockquote><em>Every convolution layer receives the outputs of all previous convolution layers.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*Aw8mX1hCPNqT3Cc6AUWl7A.jpeg" /></figure><p>Instead of forgetting earlier features, DenseNet preserves them throughout the entire network.</p><p>This makes learning much more efficient.</p><h3>Why Is This Better?</h3><p>Think about image classification.</p><p>The first convolution layer may detect:</p><ul><li>Edges</li><li>Lines</li><li>Corners</li></ul><p>The middle layers may detect:</p><ul><li>Eyes</li><li>Wheels</li><li>Windows</li><li>Fur</li></ul><p>The deeper layers may detect:</p><ul><li>Faces</li><li>Cars</li><li>Dogs</li><li>Cats</li></ul><p>In a traditional CNN, the final layers mostly receive high-level features.</p><p>In DenseNet, the final layers receive:</p><ul><li>Low-level features</li><li>Mid-level features</li><li>High-level features</li></ul><p>All together.</p><p>This richer feature representation often leads to better performance.</p><h3>Advantages of DenseNet</h3><p>Because every layer has access to all previous feature maps, DenseNet offers several important benefits.</p><h3>1. Better Feature Reuse</h3><p>Earlier features are never discarded.</p><p>Instead of learning similar features repeatedly, later layers simply reuse what earlier layers have already learned.</p><p>This reduces unnecessary computation.</p><h3>2. Stronger Information Flow</h3><p>Information can travel directly from earlier layers to later layers.</p><p>The network does not have to pass everything through many intermediate layers.</p><p>This makes learning easier.</p><h3>3. Better Gradient Flow</h3><p>During backpropagation, gradients can travel directly through dense connections.</p><p>As a result, DenseNet suffers much less from the <strong>vanishing gradient problem</strong>, especially in very deep networks.</p><h3>4. Fewer Parameters</h3><p>One surprising advantage of DenseNet is that it often uses <strong>fewer parameters</strong> than many other deep CNN architectures.</p><p>Since features are reused instead of repeatedly relearned, the network becomes more parameter-efficient.</p><h3>5. Higher Accuracy</h3><p>DenseNet usually achieves excellent performance on image classification tasks because it combines information from every previous layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*naJw86mMBEJL6nt0kYXgPQ.jpeg" /></figure><h3>What is DenseNet201?</h3><p>DenseNet comes in several versions.</p><p>The number represents the total number of layers.</p><p>Popular versions include:</p><ul><li>DenseNet121</li><li>DenseNet169</li><li><strong>DenseNet201</strong></li><li>DenseNet264</li></ul><p>Among these, <strong>DenseNet201</strong> is one of the most popular choices because it provides an excellent balance between depth and performance.</p><p>The “201” simply means the architecture contains approximately <strong>201 layers</strong>. A deeper network allows the model to learn more complex visual patterns while still benefiting from dense connections.</p><h3>Transfer Learning with DenseNet201</h3><p>Training a DenseNet201 model from scratch requires:</p><ul><li>Millions of images</li><li>Powerful GPUs</li><li>A very long training time</li></ul><p>Fortunately, we don’t need to train everything ourselves.</p><p>Instead, we use <strong>Transfer Learning</strong>.</p><p>Transfer Learning means using a model that has already been trained on a massive dataset, such as <strong>ImageNet</strong>, and adapting it to our own classification problem.</p><p>The pre-trained DenseNet201 has already learned useful visual features like:</p><ul><li>Edges</li><li>Shapes</li><li>Textures</li><li>Objects</li><li>Patterns</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uEzE9lo5ohtmhlPERxP3Tw.jpeg" /></figure><p>Instead of learning these again, we simply reuse them and train only the final classifier for our dataset. This makes training much faster while achieving excellent accuracy.</p><h3>Dense Block</h3><p>Now that we understand the idea of <strong>Dense Connections</strong>, let’s see how DenseNet is actually organized internally.</p><p>DenseNet is not simply a long sequence of convolution layers.</p><p>Instead, the network is divided into several groups called <strong>Dense Blocks</strong>.</p><p>A Dense Block is a collection of convolution layers where <strong>every layer is connected to every previous layer inside that block</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*W7E9fXq3hSjBoPzEl5caeA.png" /></figure><h3>Transition Layer</h3><p>After one Dense Block finishes, the feature maps become very large because each layer keeps adding new features.</p><p>If we continue growing the feature maps forever, the model would become too large to train.</p><p>To solve this problem, DenseNet introduces the <strong>Transition Layer</strong>.</p><p>A Transition Layer sits between two Dense Blocks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*zGVwMJMSUt5kxYowcbwDhA.jpeg" /></figure><p>Its main job is to reduce the size of the feature maps before sending them to the next Dense Block.</p><p>A Transition Layer typically contains:</p><ul><li>Batch Normalization</li><li>ReLU Activation</li><li>1×1 Convolution</li><li>Average Pooling</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JzuudvC57o9ijbgKq1Oxog.png" /></figure><p>Its responsibilities are:</p><ul><li>Reduce the number of feature maps.</li><li>Reduce the spatial dimensions (height and width).</li><li>Lower memory usage.</li><li>Speed up training</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3arK1F5aQeKezfymOT53hA.jpeg" /></figure><h3>Growth Rate</h3><p>One unique concept in DenseNet is the <strong>Growth Rate</strong>.</p><p>Let’s understand it with an example. Suppose the input to a Dense Block has <strong>64 feature maps</strong>.</p><p>If every convolution layer produces <strong>32 new feature maps</strong>, then:</p><ul><li>Input = 64 feature maps</li><li>Layer 1 adds 32 → Total = 96</li><li>Layer 2 adds another 32 → Total = 128</li><li>Layer 3 adds another 32 → Total = 160</li><li>Layer 4 adds another 32 → Total = 192</li></ul><p>Notice something important.</p><p>Each layer <strong>does not replace</strong> the previous feature maps.</p><p>Instead, it creates <strong>new feature maps</strong> that are concatenated with all previous ones. The number of new feature maps produced by each layer is called the <strong>Growth Rate</strong>, usually represented by <strong>k</strong>.</p><p>For DenseNet201:</p><ul><li>Growth Rate (k) = <strong>32</strong></li></ul><p>This means every convolution layer contributes <strong>32 new feature maps</strong>.</p><p>As we move deeper into a Dense Block, the total number of feature maps keeps increasing. This gradual increase is called the <strong>growth</strong> of the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_NYsiIWnlPfd5e6RzPlwsA.jpeg" /></figure><h3>Bottleneck Layer</h3><p>You might notice another problem.</p><p>As feature maps continue increasing, every 3×3 convolution would become computationally expensive.</p><p>DenseNet solves this using a <strong>Bottleneck Layer</strong>.</p><p>Instead of directly applying a 3×3 convolution, DenseNet first performs a <strong>1×1 convolution</strong>.</p><p>The bottleneck performs two important tasks:</p><ul><li>Reduces the number of input feature maps.</li><li>Makes the following 3×3 convolution much faster.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*gfyTZv4BEs2sP0O2S3-gHg.png" /></figure><h3>DenseNet201 Implementation in TensorFlow:</h3><h3>Step 1: Import Required Libraries</h3><p>First, import all the required libraries.</p><pre>from tensorflow.keras.preprocessing.image import image_dataset_from_directory<br>from tensorflow.keras.applications import DenseNet201<br>from tensorflow.keras.layers import GlobalAveragePooling2D<br>from tensorflow.keras.layers import Dense<br>from tensorflow.keras.layers import Dropout<br>from tensorflow.keras.models import Sequential</pre><h3>What does each library do?</h3><p><strong>TensorFlow</strong></p><p>TensorFlow is the deep learning framework that helps us build and train neural networks.</p><pre>import tensorflow as tf</pre><p><strong>image_dataset_from_directory</strong></p><p>Loads images directly from folders.</p><pre>image_dataset_from_directory()</pre><p>Instead of manually reading images one by one, TensorFlow automatically:</p><ul><li>Reads images</li><li>Assigns labels</li><li>Creates batches</li><li>Resizes images</li></ul><p><strong>DenseNet201</strong></p><p>Loads the pre-trained DenseNet201 model.</p><pre>DenseNet201</pre><p>This model has already been trained on the ImageNet dataset.</p><p>Instead of training from scratch, we’ll reuse its learned features.</p><p><strong>GlobalAveragePooling2D</strong></p><p>Converts feature maps into a single feature vector.</p><p>Instead of using a Flatten layer, DenseNet commonly uses Global Average Pooling because it requires fewer parameters and helps reduce overfitting.</p><p><strong>Dense Layer</strong></p><p>Creates our custom classifier.</p><p><strong>Dropout</strong></p><p>Randomly disables neurons during training. This helps prevent overfitting.</p><h3>Step 2: Load the Dataset</h3><p>Suppose your dataset is organized like this:</p><pre>dataset/</pre><pre>train/<br>    cats/<br>    dogs/</pre><pre>validation/<br>    cats/<br>    dogs/</pre><pre>test/<br>    cats/<br>    dogs/</pre><p>Load the training dataset.</p><pre>train_dataset = image_dataset_from_directory(<br>    &quot;dataset/train&quot;,<br>    image_size=(224,224),<br>    batch_size=32<br>)</pre><p>Load the validation dataset.</p><pre>validation_dataset = image_dataset_from_directory(<br>    &quot;dataset/validation&quot;,<br>    image_size=(224,224),<br>    batch_size=32<br>)</pre><p>Load the testing dataset.</p><pre>test_dataset = image_dataset_from_directory(<br>    &quot;dataset/test&quot;,<br>    image_size=(224,224),<br>    batch_size=32<br>)</pre><h3>Understanding Every Parameter</h3><h3>dataset/train</h3><p>This is the folder containing all training images.</p><p>TensorFlow automatically assigns labels based on folder names.</p><pre>cats → Label 0</pre><pre>dogs → Label 1</pre><h3>image_size=(224,224)</h3><p>DenseNet201 expects input images of size</p><pre>224 × 224</pre><p>TensorFlow resizes every image automatically.</p><h3>batch_size=32</h3><p>Instead of training on all images at once,</p><p>TensorFlow trains on</p><pre>32 images</pre><pre>↓</pre><pre>Update weights</pre><pre>↓</pre><pre>Next 32 images</pre><p>This is faster and uses less memory.</p><h3>Step 3: Load the Pre-trained DenseNet201 Model</h3><pre>base_model = DenseNet201(<br>    weights=&quot;imagenet&quot;,<br>    include_top=False,<br>    input_shape=(224,224,3)<br>)</pre><p>Let’s understand each parameter.</p><h3>weights=”imagenet”</h3><p>Loads weights already learned from over one million images in the ImageNet dataset.</p><p>Instead of starting with random weights, the model already knows how to detect edges, textures, shapes, and objects.</p><h3>include_top=False</h3><p>The original DenseNet201 was trained to classify 1,000 ImageNet classes.</p><p>We don’t need that classifier.</p><p>So we remove it.</p><p>We’ll build our own classifier.</p><h3>input_shape=(224,224,3)</h3><p>The model expects:</p><ul><li>Height = 224</li><li>Width = 224</li><li>RGB Channels = 3</li></ul><h3>Step 4: Freeze the Base Model</h3><pre>base_model.trainable = False</pre><p>Why?</p><p>The DenseNet201 model already learned useful image features.</p><p>We don’t want to destroy that knowledge.</p><p>So we freeze all its weights.</p><p>Only our new classifier will learn.</p><p>This is called <strong>Transfer Learning</strong>.</p><h3>Step 5: Build Our Own Classifier</h3><pre>model = Sequential([<br>    base_model,</pre><pre>    GlobalAveragePooling2D(),</pre><pre>    Dropout(0.3),</pre><pre>    Dense(128, activation=&quot;relu&quot;),</pre><pre>    Dropout(0.3),</pre><pre>    Dense(1, activation=&quot;sigmoid&quot;)<br>])</pre><p>Let’s understand each layer.</p><h3>Base Model</h3><pre>base_model</pre><p>Extracts powerful image features.</p><h3>GlobalAveragePooling2D</h3><p>Converts feature maps into one feature vector.</p><p>This replaces Flatten.</p><p>It also reduces the number of trainable parameters.</p><h3>Dropout</h3><pre>Dropout(0.3)</pre><p>Randomly turns off 30% of neurons during training.</p><p>This helps reduce overfitting.</p><h3>Dense Layer</h3><pre>Dense(128, activation=&quot;relu&quot;)</pre><p>Learns patterns specific to Cats vs Dogs.</p><h3>Output Layer</h3><pre>Dense(1, activation=&quot;sigmoid&quot;)</pre><p>Since we have only two classes,</p><p>Sigmoid returns a probability between</p><pre>0 → Cat</pre><pre>1 → Dog</pre><h3>Step 6: Compile the Model</h3><pre>model.compile(</pre><pre>    optimizer=&quot;adam&quot;,</pre><pre>    loss=&quot;binary_crossentropy&quot;,</pre><pre>    metrics=[&quot;accuracy&quot;]</pre><pre>)</pre><h3>Optimizer</h3><p>Adam updates model weights during training.</p><h3>Loss Function</h3><p>Binary Crossentropy is used because we have only two classes.</p><h3>Accuracy</h3><p>Measures how many predictions are correct.</p><h3>Step 7: Train the Model</h3><pre>history = model.fit(</pre><pre>    train_dataset,</pre><pre>    validation_data=validation_dataset,</pre><pre>    epochs=10</pre><pre>)</pre><p>What happens internally?</p><p>For every epoch,</p><p>TensorFlow:</p><ul><li>Reads one batch.</li><li>Makes predictions.</li><li>Calculates loss.</li><li>Performs Backpropagation.</li><li>Updates weights.</li><li>Repeats until all batches are processed.</li></ul><p>This process repeats for 10 epochs.</p><h3>Step 8: Evaluate the Model</h3><pre>test_loss, test_accuracy = model.evaluate(test_dataset)</pre><p>This checks how well the model performs on unseen images.</p><p>If the accuracy is high, it means the model has generalized well.</p><h3>Step 9: Save the Model</h3><pre>model.save(&quot;densenet201_cats_dogs.keras&quot;)</pre><p>This saves the trained model so you can use it later without retraining.</p><h3>Step 10: Predict on a New Image</h3><pre>from tensorflow.keras.preprocessing import image<br>import numpy as np<br>img = image.load_img(<br>    &quot;cat.jpg&quot;,<br>    target_size=(224,224)<br>)<br>img_array = image.img_to_array(img)<br>img_array = np.expand_dims(img_array, axis=0)<br>prediction = model.predict(img_array)<br>if prediction &gt; 0.5:<br>    print(&quot;Dog&quot;)<br>else:<br>    print(&quot;Cat&quot;)</pre><p>The image goes through the same pipeline:</p><ol><li>Image is loaded.</li><li>Image is resized to 224 × 224.</li><li>Converted into a NumPy array.</li><li>Batch dimension is added.</li><li>DenseNet201 extracts features.</li><li>The classifier predicts the probability.</li><li>The final output is displayed as <strong>Cat</strong> or <strong>Dog</strong>.</li></ol><h3>Conclusion</h3><p>DenseNet is one of the most successful improvements to the traditional Convolutional Neural Network.</p><p>It does <strong>not replace CNN</strong> — it enhances it.</p><p>The key innovation is simple yet powerful:</p><blockquote><strong>Every convolution layer receives the outputs of all previous convolution layers instead of only the immediately preceding one.</strong></blockquote><p>This dense connectivity allows the network to reuse features, preserve information from earlier layers, improve gradient flow, reduce unnecessary parameters, and achieve higher accuracy on image classification tasks.</p><p>Now that you understand <strong>what DenseNet is</strong>, <strong>why it was introduced</strong>, <strong>how it differs from a traditional CNN</strong>, and <strong>the overall workflow for building a DenseNet201 image classifier</strong>, you’re ready for the next step.</p><p>In the next article, we’ll start implementing DenseNet201 in TensorFlow by loading the dataset and understanding every line of code in detail — exactly as we did while learning CNN.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdf112d8bae9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Convolutional Neural Networks (CNN): A Complete Beginner’s Guide to Image…]]></title>
            <link>https://medium.com/@muhammadsufian602/understanding-convolutional-neural-networks-cnn-a-complete-beginners-guide-to-image-b9cd1da05ec9?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9cd1da05ec9</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[cnn-model]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sun, 19 Jul 2026 18:45:39 GMT</pubDate>
            <atom:updated>2026-07-19T18:45:39.546Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding Convolutional Neural Networks (CNN): A Complete Beginner’s Guide to Image Classification and Deep Learning</h3><p>Learn the complete CNN workflow from input images to backpropagation in the simplest way possible. In this guide, we’ll explore the complete CNN workflow from image preprocessing and feature extraction to backpropagation, prediction, transfer learning, and beyond in a simple, beginner-friendly way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KdNzTMhdNVjCMDnASzDw-w.png" /></figure><h3>What is a Convolutional Neural Network (CNN)?</h3><p>A <strong>Convolutional Neural Network (CNN)</strong> is a deep learning model specifically designed to process images.</p><p>Unlike traditional machine learning algorithms that rely on manually created features, a CNN automatically learns the important features directly from images.</p><p>For example, if you’re building a model to recognize cats and dogs, you don’t have to tell the CNN what ears, eyes, tails, or whiskers look like.</p><p>Instead, the CNN learns these patterns by itself during training. This ability to automatically learn visual features is what makes CNNs so powerful.</p><h3>The Complete CNN Workflow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*_1CEE36G1T4hJgh9Bp86XA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EDhaN0TZ6PByJls_66m35g.jpeg" /></figure><p>This entire process repeats many times during training until the CNN learns to make accurate predictions.</p><p>Now let’s understand each stage.</p><h3>Step 1: Dataset</h3><p>Every CNN begins with a dataset.</p><p>A dataset is simply a collection of labeled images that the CNN will learn from.</p><p>For example:</p><pre>Image             Label<br>🐱                Cat<br>🐶                Dog<br>🐴                Horse</pre><p>The labels tell the CNN the correct answer so it can compare its predictions and gradually improve. The more diverse and high-quality your dataset is, the better your CNN can learn.</p><h3>Step 2: Image Preprocessing</h3><p>Raw images cannot usually be fed directly into a CNN.</p><p>They first need to be prepared so that every image has the same format.</p><p>Common preprocessing steps include:</p><ul><li>Resize every image to the same dimensions (for example, <strong>128 × 128</strong>)</li><li>Convert the image into pixel values</li><li>Normalize pixel values from <strong>0–255</strong> to <strong>0–1</strong></li></ul><p>The workflow becomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3z3cPo48yod68z3Fs1XbAg.jpeg" /></figure><h3>Step 3: Input Layer</h3><p>The Input Layer is the entry point of the CNN.</p><p>It doesn’t perform any calculations. Its only job is to receive the preprocessed image and pass it to the first convolution layer.</p><p>You can think of it as the front door of the CNN.</p><h3>Step 4: Convolution Layer</h3><p>The Convolution Layer is the heart of every CNN.</p><p>This is where feature extraction begins.</p><p>Instead of looking at the entire image at once, the CNN uses small filters (also called <strong>kernels</strong>) that slide across the image.</p><p>As each filter moves, it searches for specific visual patterns such as:</p><ul><li>Edges</li><li>Lines</li><li>Corners</li><li>Curves</li><li>Simple textures</li></ul><p>In deeper layers, these simple features combine into more meaningful features like:</p><ul><li>Eyes</li><li>Nose</li><li>Ears</li><li>Wheels</li><li>Faces</li></ul><p>Eventually, the CNN learns enough features to recognize complete objects.</p><p>Initially, these filters are random. During training, the CNN gradually learns the best filter values for detecting useful patterns.</p><p>This is one of the most important things to remember:</p><blockquote><strong>Filters are not manually designed. They are learned automatically during training.</strong></blockquote><h3>Step 5: ReLU Activation Function</h3><p>After convolution, the feature map passes through an activation function called <strong>ReLU (Rectified Linear Unit).</strong></p><p>Its job is simple.</p><ul><li>Positive values stay the same.</li><li>Negative values become zero.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*iWcbOjflceQlUENUBKZHRw.png" /></figure><p>Mathematically:</p><pre>ReLU(x)<br><br>If x &gt; 0 → x<br><br>If x ≤ 0 → 0</pre><p>ReLU helps the CNN focus on meaningful information while removing unnecessary negative values.</p><p>Without an activation function, CNNs would struggle to learn complex patterns.</p><h3>Step 6: Pooling Layer</h3><p>After important features are detected, the Pooling Layer reduces the size of the feature map.</p><p>This makes the CNN:</p><ul><li>Faster</li><li>More memory efficient</li><li>Less likely to overfit</li></ul><p>The most common pooling method is <strong>Max Pooling</strong>.</p><p>Instead of keeping every value, Max Pooling keeps only the largest value from a small region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zXkjWMQS0WxA8-gkIYvHoA.jpeg" /></figure><p>For example:</p><pre>2 5<br>1 4</pre><p>After Max Pooling:</p><pre>5</pre><p>The important information remains while unnecessary details are removed.</p><h3>Step 8: Flatten Layer</h3><p>After multiple rounds of feature extraction, the CNN has produced many feature maps.</p><p>However, the Dense Layer cannot work directly with these 2D feature maps. The Flatten Layer converts them into a single one-dimensional vector.</p><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*_Cb1dzBhciwRi9y1-J6qjQ.png" /></figure><pre>Feature Maps<br><br>↓<br><br>Flatten<br><br>↓<br><br>1D Feature Vector</pre><p>No learning happens here.</p><p>Flatten simply changes the data format so the next layer can use it.</p><h3>Step 9: Dense (Fully Connected) Layer</h3><p>The Dense Layer acts like the decision-making part of the CNN.</p><p>It receives all the extracted features and determines which class the image most likely belongs to.</p><p>For example, if the extracted features include:</p><ul><li>Pointed ears</li><li>Whiskers</li><li>Cat-shaped face</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iKhT5-V6yZJYFJQtJNLDtQ.jpeg" /></figure><p>The Dense Layer starts assigning higher confidence to the “Cat” class.</p><p>This layer uses <strong>weights</strong> and <strong>biases</strong>, which are also learned during training.</p><h3>Step 10: Output Layer</h3><p>The Output Layer produces the final prediction.</p><p>Suppose your model recognizes three animals.</p><p>The output might look like this:</p><pre>Cat     = 0.82<br>Dog     = 0.12<br>Horse   = 0.06</pre><p>For multi-class classification, CNNs commonly use the <strong>Softmax activation function</strong>.</p><p>Softmax converts the output scores into probabilities that sum to 100%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*Hcueagt-gthZzD7f1ZPjOw.png" /></figure><p>The class with the highest probability becomes the final prediction.</p><p>For example:</p><pre>Cat = 82%</pre><p>Final prediction:</p><p><strong>This image is a Cat.</strong></p><h3>What Happens After Prediction?</h3><blockquote>Making a prediction is <strong>not the end of training</strong>.</blockquote><p>The CNN now needs to check whether its prediction was correct.</p><p>This is done using a <strong>Loss Function</strong>.</p><h3>Understanding the Loss Function</h3><p>The loss function measures <strong>how wrong the CNN’s prediction is</strong>.</p><p>Think of it as the CNN’s report card.</p><p>If the prediction is close to the correct answer:</p><ul><li>Loss is low</li></ul><p>If the prediction is far from the correct answer:</p><ul><li>Loss is high</li></ul><p>During training, the goal is simple:</p><blockquote>Reduce the loss as much as possible</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*Mhyb-jhIrdtvtThbkZst-A.png" /></figure><p>In TensorFlow/Keras, the loss function is usually defined when compiling the model:</p><pre>model.compile(<br>    optimizer=&quot;adam&quot;,<br>    loss=&quot;categorical_crossentropy&quot;,<br>    metrics=[&quot;accuracy&quot;]<br>)</pre><p>Here:</p><ul><li>loss=&quot;categorical_crossentropy&quot; measures how incorrect the model&#39;s prediction is.</li><li>optimizer=&quot;adam&quot; will later use this loss to improve the model.</li></ul><h3>Backpropagation: How the CNN Learns from Its Mistakes</h3><p>Now comes the learning phase.</p><p>After calculating the loss, the CNN asks:</p><blockquote><strong>“Which filters, weights, and biases caused this error?”</strong></blockquote><p>Backpropagation works backward through the network and calculates how much each learnable parameter contributed to the mistake.</p><p>Instead of guessing, it mathematically computes the necessary corrections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DTvz_nQNsZlhBzAWRe0P1Q.jpeg" /></figure><p>In TensorFlow/Keras, this process happens automatically during training you don’t manually write the backpropagation equations. The framework calculates the gradients and updates the model internally.</p><h3>What Does the Optimizer Do?</h3><p>Backpropagation finds <strong>what needs to change</strong>, but it doesn’t actually make the changes.</p><p>That’s the job of the optimizer.</p><p>An optimizer (such as <strong>Adam</strong>) uses the information from backpropagation to update the model’s learnable parameters.</p><p>It improves:</p><ul><li>Convolution filters</li><li>Dense layer weights</li><li>Biases</li></ul><p>These updates help the CNN make better predictions in the next training iteration.</p><p>The complete learning cycle becomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iJe9mHB45LDiaIK4UsT6Gw.jpeg" /></figure><h3>Types of Optimizers</h3><p>There are several optimization algorithms available in deep learning. Although they all have the same goal — <strong>reducing the loss by updating the model’s parameters</strong> — they use different strategies to decide <strong>how much</strong> and <strong>how quickly</strong> to update the weights.</p><p>Some of the most commonly used optimizers are:</p><h3>1. Stochastic Gradient Descent (SGD)</h3><p>SGD updates the model’s parameters using one training example (or a small batch) at a time.</p><p>It is simple, widely used, and gives you more control over the learning process. However, it may require careful tuning of the learning rate and often converges more slowly than newer optimizers.</p><p><strong>Best for:</strong> Large-scale machine learning and when you want more control over training.</p><h3>2. Adam (Adaptive Moment Estimation)</h3><p>Adam is one of the most popular optimizers in deep learning because it combines the advantages of Momentum and RMSProp.</p><p>It automatically adjusts the learning rate for each parameter, allowing the model to converge faster and more efficiently.</p><p>Because of its speed and strong performance, <strong>Adam is the default optimizer in many CNN projects.</strong></p><p><strong>Best for:</strong> Most deep learning applications, especially CNNs.</p><h3>3. RMSProp (Root Mean Square Propagation)</h3><p>RMSProp adapts the learning rate for each parameter based on recent gradients.</p><p>This helps the optimizer perform well on problems where gradients change frequently.</p><p><strong>Best for:</strong> Recurrent Neural Networks (RNNs) and deep learning tasks with varying gradients.</p><h3>4. Adagrad (Adaptive Gradient Algorithm)</h3><p>Adagrad adjusts the learning rate individually for every parameter.</p><p>Parameters that receive frequent updates get smaller learning rates, while infrequent parameters receive larger updates.</p><p>Although effective for sparse data, its learning rate can become very small over time, causing training to slow down.</p><p><strong>Best for:</strong> Sparse datasets, such as text and recommendation systems.</p><h3>5. Adadelta</h3><p>Adadelta was introduced to overcome Adagrad’s problem of continuously decreasing learning rates.</p><p>Instead of accumulating all previous gradients, it uses a moving window of recent gradients, allowing learning to continue effectively throughout training.</p><h3>What Happens After Training? Understanding CNN Prediction on New Unseen Images</h3><p>In the previous section, we learned how a CNN is trained using <strong>forward propagation, loss calculation, backpropagation, and an optimizer</strong>.</p><p>But what happens after training is complete?</p><p>When a <strong>new unseen image</strong> is given to the CNN, does it learn again or simply make a prediction?</p><p>Let’s find out.</p><h3>Training vs Prediction</h3><p>A CNN works in two phases:</p><ul><li><strong>Training Phase</strong> — The model learns from labeled images.</li><li><strong>Prediction (Inference) Phase</strong> — The model uses its learned knowledge to classify new images.</li></ul><p>The key difference is:</p><ul><li><strong>Training = Learning</strong></li><li><strong>Prediction = Using what was learned</strong></li></ul><p>Unlike training, prediction does <strong>not</strong> involve learning or updating the model.</p><h3>What is a Trained CNN?</h3><p>A common misconception is that a trained CNN memorizes all the training images.</p><p>That’s not true.</p><p>A <strong>trained CNN</strong> simply means the model has learned the best values for its <strong>filters, weights, and biases</strong>, and these learned parameters are stored inside the model.</p><p>These learned parameters include:</p><ul><li><strong>Convolution Filters (Kernels):</strong>Detect features like edges, textures, eyes, and ears.</li><li><strong>Weights</strong> :Determine how important each feature is for making predictions.</li><li><strong>Biases</strong> :Help neurons make better decisions.</li></ul><p>Instead of memorizing images, the CNN remembers the patterns it learned from them.</p><h3>Step 1: Preprocess the New Image</h3><p>Before prediction, the new image must be prepared exactly like the training images.</p><pre>Original Image<br>      ↓<br>Resize<br>      ↓<br>Convert to Pixel Matrix<br>      ↓<br>Normalize<br>      ↓<br>Ready for CNN</pre><h3>Step 2: The Image Passes Through the CNN</h3><p>Even after training, the new image still passes through the same CNN architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WXONvUfFE38lgI1MKQ3-rw.jpeg" /></figure><blockquote>Why? Because the CNN must extract features from the new image and compare them with the patterns it learned during training.</blockquote><p>The layers don’t learn anymore they simply apply their learned filters, weights, and biases to make a prediction</p><h3>What Happens During Prediction?</h3><p>During prediction, the CNN performs <strong>only a forward pass</strong>.</p><p>Unlike training:</p><ul><li>❌ No loss calculation</li><li>❌ No backpropagation</li><li>❌ No optimizer</li><li>❌ No weight updates</li></ul><p>The model simply uses its learned knowledge to classify the new image.</p><h3>inal Prediction</h3><p>After the image passes through all the layers, the Output Layer generates a score for each class.</p><p>For example:</p><pre>Cat    = 0.95<br>Dog    = 0.03<br>Horse  = 0.02</pre><p>The <strong>Softmax activation function</strong> converts these scores into probabilities.</p><p>The class with the highest probability becomes the final prediction.</p><pre>Cat = 95%</pre><p><strong>Final Prediction:</strong> This image is a <strong>Cat</strong>.</p><h3>Training vs Prediction</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u4Qdz91camJ-nfk-ptQNEw.jpeg" /></figure><h3>Understanding Learning Rate in CNN: Why It Matters During Training</h3><p>By now, we have learned the complete CNN training process.</p><p>One important question still remains:</p><blockquote><strong>When the optimizer updates the model’s parameters, how much should they change?</strong></blockquote><p>The answer is determined by the <strong>Learning Rate</strong>.</p><h3>What is Learning Rate?</h3><p>The <strong>Learning Rate</strong> is a hyperparameter that controls <strong>how much the optimizer updates the model’s filters, weights, and biases during each training step</strong>.</p><p>Think of it as the <strong>step size</strong> the optimizer takes while trying to reduce the loss.</p><p>Instead of changing the parameters randomly, the optimizer adjusts them by an amount determined by the learning rate.</p><p>In simple words:</p><blockquote><strong><em>Learning Rate decides how big or small each parameter update should be during training.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*JGjt2x_wL3SkcX_a1UE-sg.png" /></figure><h3>Why Do We Need a Learning Rate?</h3><p>After backpropagation calculates the gradients, the optimizer knows <strong>which direction</strong> to move to reduce the loss.</p><p>However, it still needs to know:</p><blockquote><strong>“How far should I move?”</strong></blockquote><p>That’s exactly what the learning rate decides.</p><p>Without a learning rate, the optimizer wouldn’t know whether to make a tiny adjustment or a large one.</p><h3>Learning Rate as a Step Size</h3><p>Imagine you’re standing on a mountain and trying to reach the lowest point in the valley.</p><p>Each step you take represents a parameter update.</p><p>The <strong>learning rate determines the size of each step</strong>.</p><ul><li>Small learning rate → Small steps</li><li>Large learning rate → Large steps</li></ul><p>The goal is to reach the lowest point, where the model has the minimum possible loss</p><h3>What Happens If the Learning Rate Is Too High?</h3><p>If the learning rate is very high, the optimizer takes huge steps.</p><p>Instead of moving smoothly toward the minimum loss, it may jump over the best solution again and again.</p><p>As a result:</p><ul><li>The model may fail to converge.</li><li>The loss may keep increasing or fluctuate.</li><li>Training becomes unstable.</li></ul><p>Think of it like trying to stop at a destination while driving too fast — you keep overshooting it.</p><h3>What Happens If the Learning Rate Is Too Low?</h3><p>If the learning rate is too small, the optimizer makes only tiny adjustments.</p><p>The model still learns, but the learning process becomes very slow.</p><p>This leads to:</p><ul><li>Slow convergence</li><li>Longer training time</li><li>More epochs needed to achieve good performance</li></ul><p>It’s like walking toward your destination using very small steps.</p><p>Eventually, you’ll get there — but it takes much longer.</p><h3>Finding the Right Learning Rate</h3><p>An ideal learning rate should strike a balance.</p><p>It should be:</p><ul><li>Large enough to help the model learn efficiently.</li><li>Small enough to avoid overshooting the optimal solution.</li></ul><p>Choosing a suitable learning rate helps the CNN train faster while maintaining stable and accurate learning.</p><h3>Learning Rate in TensorFlow/Keras</h3><p>When using TensorFlow or Keras, the learning rate is usually specified while creating the optimizer.</p><p>For example:</p><pre>from tensorflow.keras.optimizers import Adam<br><br>optimizer = Adam(learning_rate=0.001)</pre><p>Or directly while compiling the model:</p><pre>model.compile(<br>    optimizer=Adam(learning_rate=0.001),<br>    loss=&quot;categorical_crossentropy&quot;,<br>    metrics=[&quot;accuracy&quot;]<br>)</pre><p>Here:</p><ul><li><strong>Adam</strong> is the optimizer.</li><li><strong>0.001</strong> is the learning rate.</li></ul><p>This means Adam will update the model’s filters, weights, and biases using a step size of <strong>0.001</strong> during every training iteration.</p><pre> Optimizer Typical Learning Rate<br> --------- -------------------- <br> Adam                      0.001 <br> SGD                        0.01 <br> RMSProp                   0.001 </pre><h3>Understanding Gradient Descent: How CNN Finds the Best Solution</h3><h3>What is Gradient Descent?</h3><p>Explain:</p><ul><li>Definition of Gradient Descent</li><li>Why it is needed after backpropagation</li><li>The goal of minimizing the loss function</li><li>Simple intuition (finding the lowest point in a valley)</li></ul><p>Example flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s5O52thuXQ8oDVXPmEHfrA.jpeg" /></figure><blockquote>Gradient Descent is the mathematical technique that helps the optimizer move toward lower loss by following the gradients calculated during backpropagation.</blockquote><h3>How Does Gradient Descent Work?</h3><p>Explain step by step:</p><ol><li>CNN makes a prediction.</li><li>Loss is calculated.</li><li>Backpropagation computes the gradients.</li><li>Gradient Descent determines the direction that reduces the loss.</li><li>The optimizer updates the filters, weights, and biases.</li><li>The process repeats until the loss becomes as small as possible.</li></ol><h3>Types of Gradient Descent</h3><p>Briefly explain the three main types.</p><h4>Batch Gradient Descent</h4><ul><li>Uses the <strong>entire training dataset</strong> before updating the parameters.</li><li>Stable updates but slower on large datasets.</li></ul><h4>Stochastic Gradient Descent (SGD)</h4><ul><li>Updates parameters <strong>after every single training example</strong>.</li><li>Faster but updates are noisier.</li></ul><h4>Mini-Batch Gradient Descent</h4><ul><li>Uses a <strong>small batch of images</strong> (e.g., 32 or 64) before updating the parameters.</li><li>Combines the advantages of Batch and SGD.</li><li><strong>Most commonly used</strong> in modern deep learning and CNN training.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aPysRZt8fZgXoAGClj6Zlg.jpeg" /></figure><h3>Understanding Overfitting and Underfitting in CNN</h3><p>Training a CNN isn’t just about achieving high accuracy on the training dataset. The real goal is to make accurate predictions on <strong>new unseen images</strong>.</p><p>However, this doesn’t always happen. Sometimes the model learns too much, while other times it doesn’t learn enough. These situations are called <strong>Overfitting</strong> and <strong>Underfitting</strong>.</p><h3>Underfitting</h3><p>Underfitting occurs when the CNN fails to learn the important patterns in the training data.</p><p>As a result, the model performs poorly on both the training data and new unseen data.</p><p><strong>Causes:</strong></p><ul><li>Model is too simple.</li><li>Not enough training.</li><li>Learning rate is too high.</li><li>Insufficient features.</li></ul><h3>Overfitting</h3><p>Overfitting happens when the CNN memorizes the training data instead of learning general patterns.</p><p>It performs very well on the training dataset but poorly on new unseen images.</p><p><strong>Causes:</strong></p><ul><li>Too many training epochs.</li><li>Small training dataset.</li><li>Complex model.</li><li>No regularization.</li></ul><h3>How to Reduce Overfitting</h3><p>Some common techniques include:</p><ul><li><strong>Dropout</strong></li><li><strong>Data Augmentation</strong></li><li><strong>Regularization</strong></li><li><strong>Early Stopping</strong></li><li>Using more training data</li></ul><h3>Understanding Padding</h3><p>During convolution, filters reduce the size of the feature map.</p><p><strong>Padding</strong> solves this problem by adding extra pixels (usually zeros) around the image border before convolution.</p><p><strong>Why is Padding Needed?</strong></p><ul><li>Preserves image information at the edges.</li><li>Prevents the feature map from shrinking too quickly.</li><li>Allows deeper CNNs to retain more useful features.</li></ul><h3>Understanding Stride</h3><p><strong>Stride</strong> determines <strong>how many pixels the filter moves</strong> each time while sliding over the image.</p><p>For example:</p><ul><li><strong>Stride = 1</strong> → Filter moves one pixel at a time, capturing more details and producing a larger feature map.</li><li><strong>Stride = 2</strong> → Filter moves two pixels at a time, producing a smaller feature map with less computation.</li></ul><p>In short:</p><ul><li>Small stride → More detail, larger output.</li><li>Large stride → Less detail, smaller output</li></ul><h3>Understanding Batch Normalization</h3><p>As a CNN becomes deeper, training can become slower and unstable.</p><p><strong>Batch Normalization</strong> helps by normalizing the outputs of a layer before passing them to the next layer.</p><h3>Benefits</h3><ul><li>Faster training</li><li>More stable learning</li><li>Allows higher learning rates</li><li>Often improves model accuracy</li></ul><p>It is commonly placed <strong>after the Convolution layer and before the activation function</strong>.</p><h3>Understanding Dropout</h3><p><strong>Dropout</strong> is a regularization technique used to reduce overfitting.</p><p>During training, it randomly turns off some neurons, forcing the remaining neurons to learn independently instead of relying on each other.</p><p><strong>Benefits</strong></p><ul><li>Reduces overfitting</li><li>Improves generalization</li><li>Creates a more robust model</li></ul><blockquote><strong><em>Note:</em></strong><em> Dropout is used </em><strong><em>only during training</em></strong><em>, not during prediction.</em></blockquote><h3>Understanding Transfer Learning</h3><p>Training a CNN from scratch requires a large dataset and significant computational power.</p><p><strong>Transfer Learning</strong> solves this problem by using a <strong>pre-trained CNN</strong> that has already learned useful image features from millions of images.</p><p>Instead of training an entirely new model, we reuse the learned knowledge and fine-tune it for our own dataset.</p><p>Popular pre-trained CNN models include:</p><ul><li><strong>ResNet</strong></li><li><strong>VGGNet</strong></li><li><strong>MobileNet</strong></li><li><strong>EfficientNet</strong></li><li><strong>Inception</strong></li></ul><p>Transfer Learning is one of the most widely used techniques in real-world deep learning because it saves time, requires less data, and often achieves excellent performance.</p><h3>Image Segmentation</h3><p>Image Classification identifies <strong>what</strong> the object is.</p><p>Object Detection identifies <strong>what</strong> and <strong>where</strong> the object is.</p><p><strong>Image Segmentation</strong> goes even further by classifying <strong>every pixel</strong> in the image.</p><p>Instead of drawing a box around a cat, segmentation identifies the exact pixels that belong to the cat.</p><p>Popular segmentation models include:</p><ul><li><strong>U-Net</strong></li><li><strong>Mask R-CNN</strong></li></ul><p>Segmentation is commonly used in medical imaging, satellite image analysis, autonomous driving, and image editing.</p><h3>Conclusion</h3><p>Convolutional Neural Networks have revolutionized computer vision by enabling machines to automatically learn features from images without manual feature engineering. Throughout this guide, we’ve covered the complete CNN pipeline, including data preprocessing, convolution, activation functions, pooling, dense layers, forward propagation, loss calculation, backpropagation, optimizers, learning rate, gradient descent, and prediction on unseen images.</p><p>We also explored important concepts like overfitting, padding, stride, batch normalization, dropout, transfer learning, object detection, and image segmentation. Together, these concepts form the foundation of modern deep learning for computer vision.</p><p>With this knowledge, you now have a solid understanding of how CNNs learn, improve, and make predictions. Whether you’re preparing for interviews, building your first image classification model, or continuing your deep learning journey, mastering these fundamentals will make learning advanced CNN architectures like <strong>LeNet, AlexNet, VGGNet, ResNet, MobileNet, and EfficientNet</strong> much easier. Keep practicing, experiment with real datasets, and you’ll be well on your way to becoming proficient in deep learning and computer vision.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9cd1da05ec9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression Explained from Scratch: A Complete Beginner’s Guide with Examples]]></title>
            <link>https://medium.com/@muhammadsufian602/linear-regression-explained-from-scratch-a-complete-beginners-guide-with-examples-a8a67bfc12f4?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8a67bfc12f4</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machinelearningalgorithms]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sat, 18 Jul 2026 21:01:11 GMT</pubDate>
            <atom:updated>2026-07-18T21:01:42.643Z</atom:updated>
            <content:encoded><![CDATA[<p>In this guide, we’ll build a strong understanding of <strong>how Linear Regression actually works</strong>, not just how to use it with code. By the end, you’ll know how it learns from data, makes predictions, and why it remains one of the most widely used regression algorithms.</p><blockquote><strong>Note:</strong> This article focuses only on <strong>Linear Regression</strong>. We’ll explore its complete workflow from data to prediction in a simple, beginner-friendly way.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*uZN9_LWAFs2ocgiNV6AljQ.jpeg" /></figure><h3>What is Linear Regression?</h3><p>Linear Regression is a <strong>Supervised Machine Learning algorithm</strong> used to solve <strong>regression problems</strong>. Its job is to predict a <strong>continuous numeric value</strong> based on one or more input features.</p><p>Unlike classification algorithms that predict categories such as <strong>Spam/Not Spam</strong> or <strong>Cat/Dog</strong>, Linear Regression always predicts a <strong>number</strong>.</p><p>Some common real-world examples include:</p><ul><li>Predicting house prices</li><li>Predicting employee salaries</li><li>Predicting student exam marks</li><li>Predicting product sales</li><li>Predicting monthly revenue</li><li>Predicting temperature</li></ul><h3><strong>Regression vs Classification</strong></h3><blockquote>Regression</blockquote><p>Regression predicts a <strong>continuous numerical value</strong>.</p><p>Examples:</p><ul><li>House Price = <strong>$250,000</strong></li><li>Student Marks = <strong>87</strong></li><li>Salary = <strong>$75,000</strong></li></ul><blockquote>Classification</blockquote><p>Classification predicts a <strong>category or label</strong>.</p><p>Examples:</p><ul><li>Spam or Not Spam</li><li>Fraud or Not Fraud</li><li>Cat or Dog</li><li>Pass or Fail</li></ul><p>A simple memory trick is:</p><ul><li><strong>Regression → Predict Numbers</strong></li><li><strong>Classification → Predict Categories</strong></li></ul><p>Understanding this difference is essential because Linear Regression is designed <strong>only for regression tasks</strong>.</p><h3>How Does Linear Regression Think?</h3><p>Linear Regression has a very simple objective:</p><blockquote><strong><em>“Can I find the best straight line that represents the relationship between the input and the output?”</em></strong></blockquote><p>Instead of creating rules or memorizing every data point, it tries to discover a line that captures the overall pattern in the dataset.</p><p>Let’s look at a simple example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LA_6rS0gorNSdFRC0D245w.png" /></figure><p>If we plot these points on a graph, we can see they follow a clear upward trend.</p><p>Instead of connecting every point individually, Linear Regression asks:</p><blockquote><strong><em>“Can one straight line represent all of these points as closely as possible?”</em></strong></blockquote><p>The answer is yes. That straight line is called the <strong>Best-Fit Line</strong>.</p><p>Instead of connecting every point individually, Linear Regression asks:</p><blockquote><strong><em>“Can one straight line represent all of these points as closely as possible?”</em></strong></blockquote><p>The answer is yes. That straight line is called the <strong>Best-Fit Line</strong>.</p><h3>Understanding the Best-Fit Line</h3><p>Instead of connecting every point individually, Linear Regression asks:</p><blockquote><strong><em>“Can one straight line represent all of these points as closely as possible?”</em></strong></blockquote><p>The answer is yes.</p><p>That straight line is called the <strong>Best-Fit Line</strong>.</p><h3>Understanding the Best-Fit Line</h3><p>Imagine drawing different lines through the same dataset.</p><p>Some lines will pass too high. Some will pass too low. Some will miss most of the data points.</p><p>Linear Regression keeps adjusting the line until it finds the one that stays <strong>closest to all points overall</strong>.</p><p>This final line is known as the <strong>Best-Fit Line</strong> (also called the <strong>Regression Line</strong>).</p><p>The model does <strong>not</strong> try to make every point lie exactly on the line. Instead, it tries to find the line that produces the <strong>smallest overall prediction error</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*FL9fUdzpZiWxbhTOhEFq9A.jpeg" /></figure><h3>Understanding Residual (Prediction Error)</h3><p>Once the model draws a best-fit line, it starts making predictions.</p><p>However, predictions are rarely perfect.</p><p>For example:</p><ul><li>Actual Marks = 70</li><li>Predicted Marks = 65</li></ul><p>The difference is:</p><pre>70 − 65 = 5</pre><p>This difference is called the <strong>Residual</strong> (or <strong>Prediction Error</strong>).</p><p>Every training example has its own residual because every prediction may differ slightly from the actual value.</p><p>A smaller residual means the prediction is closer to the real value. A larger residual means the prediction is farther from the real value. The overall goal of Linear Regression is simple:</p><blockquote><strong><em>Reduce the residuals for all training examples as much as possible.</em></strong></blockquote><p>In other words, the model keeps improving until the total prediction error becomes as small as it can.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xu-GmCkpi9iH9kInmE9Dtw.jpeg" /></figure><h3>The Goal of Linear Regression</h3><p>The model wants to learn a line that produces the <strong>smallest possible prediction error</strong>.</p><p>You can think of the learning process like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ykHRVa1Z7Qvo5Q25jI8KMA.jpeg" /></figure><h3>What Happens During fit()?</h3><p>When you train a Linear Regression model in Scikit-learn, you’ll usually write:</p><pre>model.fit(X_train, y_train)</pre><p>Although this is just one line of code, a lot happens internally.</p><p>The model follows a sequence of steps to learn from the training data.</p><h3>Step 1: Read the Training Data</h3><p>The algorithm first looks at every training example and studies the relationship between the input features (X_train) and the target values (y_train).</p><h3>Step 2: Start with a Line</h3><p>The model begins with an initial line. This line is only a starting point and is usually not the best one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*YoErlI-M7su6DI7PyScOvg.png" /></figure><h3>Step 3: Make Predictions</h3><p>Using the current line, the model predicts the output for every training example.</p><h3>Step 4: Calculate the Prediction Error</h3><p>Next, it compares every predicted value with the actual value.</p><p>The difference between them is the residual (prediction error).</p><p>The model then calculates the total error across the entire dataset.</p><h3>Step 5: Improve/Adjust the Line</h3><p>If the total error is still large, the model adjusts the line to reduce that error.</p><h3>Step 6: Repeat the Process</h3><p>The algorithm continues repeating these steps until it finds the line with the smallest possible prediction error</p><h3>How Linear Regression Makes Predictions</h3><p>After training is complete, the model has already learned the best-fit line from the data.</p><p>Now comes the real purpose of the model:</p><blockquote><strong><em>Predicting values for unseen data.</em></strong></blockquote><p>In Scikit-learn, predictions are made using:</p><pre>prediction = model.predict(X_test)</pre><p>Although this looks simple, several things happen internally.</p><blockquote>For example, suppose the model learned a relationship between study hours and exam marks.</blockquote><p>If a new student studies for <strong>6 hours</strong>, the model places that value into its learned equation and predicts the expected marks.</p><pre>Hours Studied = 6<br>       ↓<br>Predicted Marks = 90</pre><p>The important thing to remember is that the model does not memorize training data. Instead, it learns a mathematical relationship and uses that relationship to make predictions.</p><h3>The Heart of Linear Regression: The Equation</h3><p>Everything in Linear Regression revolves around one famous equation.</p><p><strong>y=mx+c</strong></p><p>This equation represents the best-fit line learned by the model.</p><p>Let’s understand each part.</p><h3>y = Predicted Value</h3><p>This is the value we want the model to predict.</p><p>Examples:</p><ul><li>House Price</li><li>Salary</li><li>Marks</li><li>Sales</li></ul><h3>x = Input Feature</h3><p>This is the information we provide to the model.</p><p>Examples:</p><ul><li>Hours Studied</li><li>Years of Experience</li><li>House Size</li></ul><h3>m = Slope</h3><p>The slope tells us how much the prediction changes when the input changes.</p><p>A larger slope means the output changes more rapidly.</p><p>For example:</p><pre>Marks = 10 × Hours + 30</pre><p>Every additional study hour increases the marks by 10.</p><p>So:</p><pre>1 Hour → 40 Marks<br>2 Hours → 50 Marks<br>3 Hours → 60 Marks</pre><h3>c = Intercept</h3><p>The intercept is the starting value.</p><p>It represents the prediction when the input value is zero.</p><p>Using the same equation:</p><pre>Marks = 10 × Hours + 30</pre><p>If:</p><pre>Hours = 0</pre><p>Then:</p><pre>Marks = 30</pre><p>The value 30 is the intercept.</p><p>Everything in Linear Regression revolves around one famous equation.</p><p>y=mx+c</p><p>This equation represents the best-fit line learned by the model.</p><p>Let’s understand each part.</p><h3>y = Predicted Value</h3><p>This is the value we want the model to predict.</p><p>Examples:</p><ul><li>House Price</li><li>Salary</li><li>Marks</li><li>Sales</li></ul><h3>x = Input Feature</h3><p>This is the information we provide to the model.</p><p>Examples:</p><ul><li>Hours Studied</li><li>Years of Experience</li><li>House Size</li></ul><h3>m = Slope</h3><p>The slope tells us how much the prediction changes when the input changes.</p><p>A larger slope means the output changes more rapidly.</p><p>For example:</p><pre>Marks = 10 × Hours + 30</pre><p>Every additional study hour increases the marks by 10.</p><p>So:</p><pre>1 Hour → 40 Marks<br>2 Hours → 50 Marks<br>3 Hours → 60 Marks</pre><h3>c = Intercept</h3><p>The intercept is the starting value.</p><p>It represents the prediction when the input value is zero.</p><p>Using the same equation:</p><pre>Marks = 10 × Hours + 30</pre><p>If:</p><pre>Hours = 0</pre><p>Then:</p><pre>Marks = 30</pre><p>The value 30 is the intercept.</p><h3>Single-Feature Linear Regression</h3><p>Example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ILUpgoiSSiqJ1jFdYNFFCw.jpeg" /></figure><p>The model learns a single equation:</p><pre>Marks = 10 × Hours + 30</pre><p>Notice something important. Since there is only one feature, there is only <strong>one slope</strong>.</p><pre>Marks = m × Hours + c</pre><p>One feature means one slope. This is called <strong>Simple Linear Regression</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*36s3j10O6ZHSdnrikCZG-Q.png" /></figure><h3>Multiple-Feature Linear Regression</h3><p>For example:</p><pre>Hours Sleep Attendance Marks<br>4        7     90         ?</pre><p>Now one feature is not enough. The model must consider all features together.</p><blockquote>The equation becomes:</blockquote><pre><br>Prediction =(Hours × Weight1)+(Sleep × Weight2)+(Attendance × Weight3)+Bias</pre><p>Each feature now gets its own (RANDOM )weight.</p><pre>Hours      → Weight1<br>Sleep      → Weight2<br>Attendance → Weight3</pre><h3>How Does the Model Find These Weights?</h3><p>This is where learning actually happens. The model does not know the correct weights at the beginning. It starts with random values.</p><p>For example:</p><pre>Hours Weight = 2<br>Sleep Weight = 5<br>Attendance Weight = 3<br>Bias = 0.1                                                  </pre><p>This creates a random equation:</p><pre>Prediction =2 × Hours+5 × Sleep+3 × Attendance+0.1</pre><p>This is exactly the type of equation a model may start with. At this stage, predictions are usually poor. The model has not learned anything yet.</p><p>This is where learning actually happens.</p><p>The model does not know the correct weights at the beginning.</p><p>It starts with random values.</p><p>For example:</p><pre>Hours Weight = 2<br>Sleep Weight = 5<br>Attendance Weight = 3<br>Bias = 0.1</pre><p>This creates a random equation:</p><pre>Prediction =<br>2 × Hours<br>+<br>5 × Sleep<br>+<br>3 × Attendance<br>+<br>0.1</pre><p>This is exactly the type of equation a model may start with.</p><p>At this stage, predictions are usually poor.</p><p>The model has not learned anything yet.</p><h3>The Complete Learning Process in Multiple Feauters:</h3><h3>Step 1: Dataset Arrives</h3><p>The model receives training data.</p><pre>Dataset<br>   ↓<br>Hours<br>Sleep<br>Attendance<br>Marks</pre><h3>Step 2: Assign Random Weights</h3><p>The model starts with random weights.</p><pre>Hours Weight = 2<br>Sleep Weight = 5<br>Attendance Weight = 3</pre><h3>Step 3: Create a Random Equation</h3><p>The weights form an initial equation.</p><pre>Prediction =2 × Hours+5 × Sleep+3 × Attendance +0.1</pre><h3>Step 4: Predict Every Training Row</h3><p>The model uses the equation to predict outputs for all training examples.</p><h3>Step 5: Calculate Loss</h3><p>The model compares predictions with actual values.Large differences mean high loss. Small differences mean low loss.</p><h3>Step 6: Adjust Weights</h3><p>The model changes the weights to reduce the loss.</p><p>For example:</p><pre>Hours Weight:<br>2 → 8</pre><pre>Sleep Weight:<br>5 → 3</pre><h3>Step 7: Repeat Again and Again</h3><p>The model keeps:</p><pre>Predicting<br>     ↓<br>Calculating Loss<br>     ↓<br>Updating Weights</pre><p>until the loss becomes as small as possible.</p><h3>Step 8: Store the Final Equation</h3><p>Once training is complete, the model stores the best equation. That equation becomes the model’s knowledge.</p><p>Every future prediction uses this learned equation.</p><h3>Implementing Linear Regression in Scikit-Learn</h3><pre>from sklearn.linear_model import LinearRegression<br>model = LinearRegression()<br>model.fit(X_train, y_train)<br>prediction = model.predict(X_test)</pre><h3>Evaluating a Linear Regression Model</h3><p>After training the model, we predict values using:</p><pre>y_pred = model.predict(X_test)</pre><p>Now we compare:</p><ul><li><strong>Actual Values (</strong><strong>y_test)</strong></li><li><strong>Predicted Values (</strong><strong>y_pred)</strong></li></ul><p>The difference between them is called the <strong>error</strong>. The smaller the error, the better the model.</p><h3>Mean Absolute Error (MAE)</h3><p><strong>Mean Absolute Error (MAE)</strong> calculates the average absolute difference between actual and predicted values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*pMIEq6_nVW9UfjcuqvEiQg.png" /></figure><blockquote>For example:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*rQ_m_us8ZTzKzOULiV-ltw.png" /></figure><pre>MAE = (5 + 4 + 2) / 3 = 3.67</pre><h3>Mean Squared Error (MSE)</h3><p>Instead of taking the absolute error, <strong>MSE squares every error before averaging them</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h0MBlpU2z-CI5H8kzkGnFQ.jpeg" /></figure><p>Squaring makes large errors much more important than small ones.</p><pre>Error = 2 → 4 <br>Error = 10 → 100</pre><p>A prediction with a large error receives a much bigger penalty.</p><h3>Root Mean Squared Error (RMSE)</h3><p>RMSE is simply the square root of MSE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*naf5VlPTDXXf2L17CkWinA.jpeg" /></figure><blockquote>Why take the square root?</blockquote><p>Because MSE is in squared units, while RMSE brings the error back to the <strong>original unit</strong>, making it easier to interpret.</p><ul><li><strong>MAE → Average Absolute Error</strong></li><li><strong>MSE → Squares Errors</strong></li><li><strong>RMSE → Square Root of MSE</strong></li></ul><h3>R² Score (Coefficient of Determination)</h3><p>Unlike MAE, MSE, and RMSE, the <strong>R² Score</strong> tells us <strong>how well the model explains the variation in the data</strong>.</p><p>Its value ranges between <strong>0 and 1</strong>.</p><ul><li><strong>1.0 → Perfect model</strong></li><li><strong>0.9 → Excellent</strong></li><li><strong>0.8 → Good</strong></li><li><strong>0.5 → Average</strong></li><li><strong>0 → Poor model</strong></li></ul><p>The closer the R² Score is to <strong>1</strong>, the better the model fits the data.</p><h3>R² Score (Coefficient of Determination)</h3><p>Unlike MAE, MSE, and RMSE, the <strong>R² Score</strong> tells us <strong>how well the model explains the variation in the data</strong>.</p><p>Its value ranges between <strong>0 and 1</strong>.</p><ul><li><strong>1.0 → Perfect model</strong></li><li><strong>0.9 → Excellent</strong></li><li><strong>0.8 → Good</strong></li><li><strong>0.5 → Average</strong></li><li><strong>0 → Poor model</strong></li></ul><p>The closer the R² Score is to <strong>1</strong>, the better the model fits the data.</p><h3>Loss Function</h3><p>The model doesn’t magically find the best equation.</p><p>It needs a way to measure <strong>how bad its current predictions are</strong>. This measurement is called the <strong>Cost Function</strong> (or <strong>Loss Function</strong>).</p><pre>Small Error → Low Cost<br>Large Error → High Cost</pre><p>The model’s goal during training is simple:</p><blockquote>Reduce the cost as much as possible.</blockquote><p>Every time the cost decreases, the model becomes better at making predictions.</p><h3>Gradient Descent</h3><p>Gradient Descent is an optimization algorithm used to reduce the cost function by updating the model’s weights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jDu8pro1eLXR6hssjxA52w.jpeg" /></figure><p>Each iteration improves the weights, bringing the model closer to the best-fit line. Training stops when the cost becomes very small or no longer improves significantly.</p><h3>Learning Rate</h3><p>The <strong>Learning Rate</strong> decides <strong>how big each weight update should be</strong> during Gradient Descent.</p><p>Think of it as the step size while walking toward the minimum cost.</p><blockquote>If the learning rate is too small:</blockquote><ul><li>Training becomes very slow.</li><li>It may take many iterations to converge.</li></ul><blockquote>If the learning rate is too large:</blockquote><ul><li>The model may overshoot the minimum.</li><li>Training can become unstable or fail to converge.</li></ul><p>A good learning rate helps the model learn efficiently and reach the minimum cost faster.</p><h3>Learning Rate</h3><p>The <strong>Learning Rate</strong> decides <strong>how big each weight update should be</strong> during Gradient Descent.</p><p>Think of it as the step size while walking toward the minimum cost.</p><h3>If the learning rate is too small:</h3><ul><li>Training becomes very slow.</li><li>It may take many iterations to converge.</li></ul><h3>If the learning rate is too large:</h3><ul><li>The model may overshoot the minimum.</li><li>Training can become unstable or fail to converge.</li></ul><p>A good learning rate helps the model learn efficiently and reach the minimum cost faster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*gPvrUsKYAlFQTsqeq92OcQ.png" /></figure><h3>Regularization</h3><p>Sometimes a model gives too much importance to certain features by assigning them very large weights.</p><p>Regularization helps by <strong>penalizing large weights</strong>, making the model simpler and more stable.</p><p>There are three common types.</p><blockquote><strong>Ridge Regression (L2)</strong></blockquote><p>Ridge Regression reduces large weights but <strong>does not make them exactly zero</strong>.</p><p>Use Ridge when <strong>most features are useful</strong>, but you want to reduce their influence slightly.</p><blockquote><strong>Lasso Regression (L1)</strong></blockquote><p>Lasso Regression can reduce some weights all the way to <strong>zero</strong>.</p><p>This means it automatically removes less important features.</p><p>Use Lasso when your dataset contains many unnecessary features.</p><blockquote>Elastic Net</blockquote><p>Elastic Net combines the advantages of both Ridge and Lasso.</p><p>It both:</p><ul><li>Reduces large weights.</li><li>Removes unimportant features.</li></ul><p>It is useful when you want the benefits of both techniques.</p><ul><li><strong>Ridge → Shrinks Weights</strong></li><li><strong>Lasso → Removes Features</strong></li><li><strong>Elastic Net → Does Both</strong></li></ul><h3>Feature Scaling</h3><p>Feature Scaling makes all input features have a similar range.</p><p><strong>Consider this dataset:</strong></p><pre>Feature       Value<br>Age            25<br>Salary        500000</pre><p>Salary has much larger values than Age.</p><p>Without scaling, the model may give Salary more importance simply because of its larger numerical values. Feature Scaling solves this issue by bringing all features to a comparable scale.</p><p>Common scaling techniques include:</p><ul><li>Standardization</li><li>Normalization</li></ul><p>Although basic Linear Regression can work without scaling, it becomes especially important when using optimization methods like Gradient Descent or regularized models.</p><h3>Conclusion</h3><p>Linear Regression is much more than drawing a straight line. It introduces many of the core ideas used throughout Machine Learning, including model training, prediction, optimization, evaluation, and feature importance.</p><p>In this two-part guide, you learned:</p><ul><li>What Linear Regression is and when to use it.</li><li>How it finds the best-fit line.</li><li>How fit() and predict() work.</li><li>The meaning of the equation <strong>y = mx + c</strong>.</li><li>The difference between simple and multiple Linear Regression.</li><li>How weights are learned.</li><li>How to evaluate a model using <strong>MAE, MSE, RMSE, and R² Score</strong>.</li><li>The role of the <strong>Cost Function</strong>, <strong>Gradient Descent</strong>, and <strong>Learning Rate</strong>.</li><li>How <strong>Regularization</strong> helps prevent overly complex models.</li><li>Why <strong>Feature Scaling</strong> and the <strong>assumptions of Linear Regression</strong> matter.</li></ul><p>By mastering these concepts, you’ve built a solid foundation for understanding not only Linear Regression but also many advanced Machine Learning algorithms that follow the same core principles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8a67bfc12f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Support Vector Machine (SVM) in Machine Learning: Complete Beginner’s Guide to Theory, Kernels…]]></title>
            <link>https://medium.com/@muhammadsufian602/support-vector-machine-svm-in-machine-learning-complete-beginners-guide-to-theory-kernels-aeba91cff8ad?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/aeba91cff8ad</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[svm]]></category>
            <category><![CDATA[support-vector-machine]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sat, 18 Jul 2026 13:01:58 GMT</pubDate>
            <atom:updated>2026-07-18T13:03:43.349Z</atom:updated>
            <content:encoded><![CDATA[<h3>Support Vector Machine (SVM) in Machine Learning: Complete Beginner’s Guide to Theory, Kernels, Support Vectors, Python, and Scikit-learn</h3><p>Imagine you’re asked to separate two groups with a single line. Many lines would work, but only one creates the largest gap and minimizes future mistakes. That’s exactly the intuition behind Support Vector Machine (SVM), one of the most powerful classification algorithms in Machine Learning.</p><p><strong>Support Vector Machine (SVM)</strong> takes a completely different approach.</p><p>Instead of building trees or asking which feature to split next, SVM tries to answer one simple question:</p><blockquote><strong>“What is the best boundary line that separates different classes?”</strong></blockquote><p>This idea makes SVM one of the most important supervised machine learning algorithms for classification problems.</p><h3><strong>What is Support Vector Machine (SVM)?</strong></h3><p>Support Vector Machine (SVM) is a supervised machine learning algorithm primarily used for <strong>classification</strong>, although it can also be used for <strong>regression</strong>.</p><p>In Scikit-learn:</p><p><strong>Classification</strong></p><pre>from sklearn.svm import SVC</pre><p><strong>Regression</strong></p><pre>from sklearn.svm import SVR</pre><h3>Decision Boundary vs Hyperplane</h3><p>The line That Seprate the classes is called Decison Boundary</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*pK47DPIGNvqG4-6UmwQVJg.png" /></figure><blockquote>THE RED DASHED LINE SHOW DECISON BOUNDARY THAT SEPERTAE CIRCLE AND TRIANGLE SHAPES</blockquote><p>Many beginners get confused between these two terms.The good news is they’re almost the same thing.</p><p>The difference depends only on the number of features.</p><pre> Number of Features    Separator                                <br> <br> 2 Features            Decision Boundary (Line)                 <br> 3 Features            Hyperplane (Plane)                       <br> More than 3 Features  Hyperplane (Higher-dimensional boundary) </pre><ul><li><strong>Decision Boundary</strong> → General name</li><li><strong>Hyperplane</strong> → Mathematical name used in SVM</li></ul><p>So in <strong>2D</strong>:</p><blockquote><strong>Decision Boundary = Hyperplane</strong></blockquote><p>They both refer to the same separating line.</p><h3>Is Every Separating Line Good?</h3><p>Imagine there are several lines that separate the two classes.</p><pre>Line 1<br>---------<br>Line 2<br>------------<br>Line 3<br>-----------------</pre><p>All three separate the classes, But SVM asks a different question:</p><blockquote><strong><em>Which boundary leaves the maximum distance between the two classes?</em></strong></blockquote><p>The answer becomes the final decision boundary.</p><h3>What is Margin</h3><p>The empty space between the two classes is called the <strong>Margin</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*QadRYxYDWStZG1-NR1E7xA.jpeg" /></figure><blockquote>SVM always tries to maximize this margin.</blockquote><p>Because a larger margin usually means better performance on unseen data.</p><p><strong>Large Margin</strong></p><ul><li>Better generalization</li><li>Less chance of overfitting</li></ul><p><strong>Small Margin</strong></p><ul><li>More sensitive to training data</li><li>Higher risk of overfitting</li></ul><h3>Support Vectors</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*KHgVUkfcRMMeiK-bjsg-8w.png" /></figure><p>Only the points closest to the boundary determine where the boundary should be.</p><p>These nearest training samples are called <strong>Support Vectors</strong>. The remaining points have very little influence on the final boundary.</p><p><strong>Why are they called Support Vectors?</strong></p><p>Because they <strong>support</strong> or determine the position of the decision boundary.</p><p>Think about this:</p><ul><li>Remove a far-away point → Almost nothing changes.</li><li>Remove a Support Vector → The boundary may shift completely.</li></ul><p>That is why they are called <strong>Support Vectors</strong>.</p><h3>How Does SVM Think?</h3><p>Unlike Decision Trees, SVM doesn’t split the dataset into branches. Instead, it tries to find the <strong>best possible boundary</strong> that separates different classes.</p><p><strong>1.Imagine a dataset like this:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*xoAZjvpvnHMo1ik_Utzn1Q.png" /></figure><p><strong>2.At The First the SVM draw the Data points just like this :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*qZgfuhPTKseO8PE7bYhIag.png" /></figure><p><strong>3.Identify the Support Vectors</strong></p><p>Then the SVM identifies the training samples that are closest to the class boundary. These critical points are called <strong>Support Vectors</strong> because they determine the position of the boundary.</p><p><strong>4.Compute the Maximum Margin</strong></p><p>SVM calculates the largest possible distance (margin) between the two classes. A larger margin generally leads to better generalization on unseen data.</p><p><strong>5. Draw the Best Decision Boundary (Hyperplane)</strong></p><p>Once the maximum margin is found, SVM places the decision boundary exactly in the middle of that margin, creating the optimal separator between the classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*sSH24h4sl_Wtqh1031-94w.png" /></figure><p><strong>6. Store the Trained Model</strong></p><p>The learned decision boundary, support vectors, and model parameters are stored so they can be reused for future predictions.</p><p><strong>7. Prediction</strong></p><p>For every new sample, SVM checks <strong>which side of the stored decision boundary</strong> it falls on and assigns the corresponding class.</p><h3>Multi-Class Classification in SVM</h3><p>Support Vector Machine (SVM) was originally designed to solve <strong>binary classification</strong> problems, where there are only <strong>two classes</strong> (for example, <em>Spam</em> vs. <em>Not Spam</em>).</p><p>However, many real-world datasets contain <strong>more than two classes</strong>. For example, you might want to classify images as <strong>Cat</strong>, <strong>Dog</strong>, or <strong>Horse</strong>. To handle such problems, Scikit-learn automatically extends SVM to support multi-class classification.</p><p>When you create an SVM model like this:</p><pre>from sklearn.svm import SVC <br>model = SVC()</pre><p>Scikit-learn automatically uses the <strong>One-vs-One (OvO)</strong> strategy.</p><blockquote><strong>What is One-vs-One (OvO)?</strong></blockquote><p>In the <strong>One-vs-One</strong> approach, SVM creates a separate classifier for <strong>every pair of classes</strong> and combines the results to make the final prediction.</p><p>For example, if your dataset has three classes:</p><ul><li>Cat</li><li>Dog</li><li>Horse</li></ul><p>SVM creates classifiers like:</p><ul><li>Cat vs Dog</li><li>Cat vs Horse</li><li>Dog vs Horse</li></ul><p>Each classifier votes for a class, and the class with the most votes becomes the final prediction.</p><blockquote><strong>What is One-vs-rest(OvR) approach?</strong></blockquote><p>If you prefer the <strong>One-vs-Rest (OvR)</strong> strategy, you can wrap the SVM model like this:</p><pre>from sklearn.multiclass import OneVsRestClassifier<br>from sklearn.svm import SVC</pre><pre>model = OneVsRestClassifier(SVC())</pre><p>In <strong>One-vs-Rest (OvR)</strong>, SVM trains one classifier for each class by treating that class as <strong>one group</strong> and all remaining classes as <strong>another group</strong>.</p><blockquote><strong>Note:</strong> In most real-world projects, you don’t need to configure this manually. Simply using <strong>SVC()</strong> is sufficient because Scikit-learn automatically handles multi-class classification using the <strong>One-vs-One (OvO)</strong> strategy.</blockquote><p>In <strong>One-vs-Rest (OvR)</strong>, SVM trains one classifier for each class by treating that class as <strong>one group</strong> and all remaining classes as <strong>another group</strong>.</p><h3>Workflow in Scikit-learn</h3><p>When we write:</p><pre>model.fit(X_train, y_train)</pre><p>Scikit-learn performs the following steps internally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EXl01L2NieS6RnIsSS4BDA.png" /></figure><blockquote>SVM does <strong>not</strong>:</blockquote><blockquote>Build trees</blockquote><blockquote>Create root nodes</blockquote><blockquote>Split features</blockquote><blockquote>Its only goal is to compute the optimal separating boundary.</blockquote><p><strong>What Happens During Prediction?</strong></p><pre>prediction = model.predict(X_test)</pre><p>SVM simply checks which side of the decision boundary the new sample falls on.</p><pre>New Sample<br>      │<br>      ▼<br>Which side of the boundary?<br>      │<br>      ├── Left → Class A<br>      │<br>      └── Right → Class B</pre><h3>Linear vs Nonlinear Data</h3><p>Sometimes one straight line is enough to classify classes</p><pre>● ●<br><br>------------<br><br>▲ ▲</pre><p>This is called <strong>Linear Data</strong>. In such cases, use the <strong>Linear Kernel</strong>.</p><pre>SVC(kernel=&quot;linear&quot;)</pre><p>Unfortunately, not every dataset is this simple.</p><p>Sometimes the classes are arranged like this.</p><pre>▲ ▲ ▲<br><br>● ●<br><br>▲ ▲</pre><p>Now, no straight line can separate the classes. This is called <strong>Nonlinear Data</strong>.</p><p>Then use the rbf or sigmoid or poly kernel ,we learn these below.</p><pre>SVC(kernel=&quot;rbf&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-BQdcSVIPi6sOa11m6yMuw.jpeg" /></figure><h3>The Kernel Trick</h3><p>When the data is not linearly separable, SVM uses something called the <strong>Kernel Trick</strong>.</p><p>Instead of drawing a straight line directly, the kernel function transforms the data into a higher-dimensional space where it becomes easier to separate.</p><p>The important point is:</p><blockquote>You <strong>do not</strong> perform this transformation yourself.</blockquote><p>Scikit-learn does it internally. The most commonly used kernels are:</p><ul><li><strong>linear</strong></li><li><strong>rbf</strong> (Most commonly used)</li><li><strong>poly</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eEeEWTKKPEHZ4ntGL8UNiw.png" /></figure><p>NOTE : There is no single kernel that is best for every dataset. In practice, machine learning engineers experiment with different kernel,such as <strong>Linear</strong>, <strong>RBF</strong>, <strong>Polynomial (Poly)</strong>, and sometimes <strong>Sigmoid</strong> then compare their performance using validation data or cross-validation. The <strong>RBF kernel</strong> is the most commonly used because it performs well on many real-world nonlinear datasets and is often a strong default choice. However, the best kernel is always the one that achieves the highest accuracy and generalizes well to unseen data.</p><h3><strong>Important Parameters in Scikit-learn</strong></h3><h3><strong>Important Parameters in Scikit-learn</strong></h3><p><strong>1. Kernel</strong></p><pre>SVC(kernel=&quot;linear&quot;)<br>OR<br>SVC(kernel=&quot;rbf&quot;)<br>OR<br>SVC(kernel=&quot;poly&quot;)<br></pre><p>Kernel decides <strong>how SVM separates the data.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tf2WHX9dlpNrvXo53IU3dg.png" /></figure><p><strong>2. C Parameter</strong></p><pre>SVC(C=1)</pre><p>The <strong>C parameter</strong> controls the balance between:</p><ul><li>Maximizing the margin</li><li>Minimizing classification errors</li></ul><p><strong>Large C</strong></p><ul><li>Tries to classify every training sample correctly.</li><li>Can lead to overfitting.</li></ul><p><strong>Small C</strong></p><ul><li>Allows a few mistakes.</li><li>Produces a wider margin.</li><li>Usually generalizes better.</li></ul><h3>3. Gamma</h3><p>Gamma is mainly used with the <strong>RBF kernel</strong>.</p><pre>SVC(gamma=&quot;scale&quot;)</pre><p>Gamma controls how much influence each training sample has.</p><p><strong>High Gamma</strong></p><ul><li>Very complex boundary</li><li>Can overfit</li></ul><p><strong>Low Gamma</strong></p><ul><li>Smoother boundary</li></ul><h3>Regression with SVM</h3><p>VM is mainly known for classification, but it also supports regression.</p><p>Instead of:</p><pre>SVC()</pre><p>Use:</p><pre>from sklearn.svm import SVR</pre><pre>model = SVR()</pre><p>The workflow remains almost the same.</p><h3>Complete Scikit-learn Implementation</h3><pre>from sklearn.svm import SVC<br>from sklearn.metrics import accuracy_score<br><br>model = SVC(<br>    kernel=&quot;rbf&quot;,<br>    C=1,<br>    gamma=&quot;scale&quot;<br>)<br><br>model.fit(X_train, y_train)<br><br>prediction = model.predict(X_test)<br><br>print(accuracy_score(y_test, prediction))</pre><h3>Key Takeaways</h3><ul><li>Support Vector Machine (SVM) is primarily used for <strong>classification</strong>, but it also supports <strong>regression</strong> through <strong>SVR</strong>.</li><li>Unlike Decision Trees, SVM does <strong>not</strong> build trees or split features.</li><li>Its objective is to find the <strong>best decision boundary</strong> that separates different classes.</li><li>The <strong>Margin</strong> is the distance between the decision boundary and the nearest training samples.</li><li>The nearest samples are called <strong>Support Vectors</strong>, and they determine the position of the boundary.</li><li>During model.fit(), SVM identifies the support vectors and computes the maximum-margin boundary.</li><li>During model.predict(), classification depends on which side of the boundary a new sample falls.</li><li>The most important parameters are:</li><li><strong>kernel</strong> → How the data is separated.</li><li><strong>C</strong> → Controls the trade-off between maximizing the margin and minimizing training errors.</li><li><strong>gamma</strong> → Controls the complexity of the decision boundary (mainly with the RBF kernel).</li><li>Scikit-learn automatically uses <strong>One-vs-One (OvO)</strong> for multiclass classification.</li><li>In 2D, the <strong>Decision Boundary</strong> and <strong>Hyperplane</strong> refer to the same separating line.</li></ul><h3>Conclusion</h3><p>Support Vector Machine approaches classification from a completely different perspective than tree-based algorithms. Rather than building branches or combining multiple models, it focuses on finding the optimal boundary that maximizes the separation between classes.</p><p>Concepts like <strong>Decision Boundary</strong>, <strong>Margin</strong>, <strong>Support Vectors</strong>, and the <strong>Kernel Trick</strong> are the foundation of understanding SVM. Once these ideas are clear, implementing SVM in Scikit-learn becomes straightforward.</p><p>Whether you’re preparing for machine learning interviews, building classification models, or strengthening your AI fundamentals, SVM remains one of the most essential algorithms every ML engineer should understand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aeba91cff8ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Random Forest Explained Simply: From One Decision Tree to a Forest of Better Predictions]]></title>
            <link>https://medium.com/@muhammadsufian602/random-forest-explained-simply-from-one-decision-tree-to-a-forest-of-better-predictions-4c05b91a0801?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c05b91a0801</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[random-forest]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Fri, 17 Jul 2026 19:09:59 GMT</pubDate>
            <atom:updated>2026-07-17T19:09:59.211Z</atom:updated>
            <content:encoded><![CDATA[<p>Instead of relying on a single Decision Tree, Random Forest combines the predictions of many Decision Trees to produce a more accurate and stable result.</p><h3>What is Random Forest?</h3><p>Random Forest is an ensemble learning algorithm that combines multiple Decision Trees to make a final prediction.</p><ul><li><strong>Random</strong> → Uses randomness while building trees.</li><li><strong>Forest</strong> → A collection of many Decision Trees</li></ul><p>Unlike a single Decision Tree, Random Forest does not depend on just one model. It asks many trees to make predictions and then combines their answers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*BBWDle0D6d3_qIYGM2fhrQ.jpeg" /></figure><h3>Why Was Random Forest Created?</h3><p>A Decision Tree is easy to understand and works well on many problems, but it has one major limitation:</p><blockquote><em>It can easily </em><strong><em>overfit</em></strong><em> the training data.</em></blockquote><p>When a model overfits, it learns the training data too well and struggles to make accurate predictions on unseen data.</p><p>To solve this problem, researchers introduced a simple but powerful idea:=</p><blockquote><em>Instead of trusting one Decision Tree, build many Decision Trees and combine their predictions.</em></blockquote><p>This approach produces a model that is generally more accurate, more stable, and less likely to overfit.</p><h3>How Random Forest Works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i-XCx5OAiHNABst9EKjnKg.png" /></figure><p>Each tree works independently, and the final prediction depends on all of them together.</p><h3>Classification vs Regression</h3><p>Random Forest works for both <strong>classification</strong> and <strong>regression</strong>, but the way it combines predictions is different.</p><h4>Classification</h4><p>Every tree predicts a class label.</p><pre>Tree 1 → Spam<br>Tree 2 → Spam<br>Tree 3 → Not Spam<br>Tree 4 → Spam<br>Tree 5 → Spam</pre><p>Result(Follow Majority Voting):</p><pre>Spam = 4 Votes<br>Not Spam = 1 Vote</pre><h3>Regression</h3><p>Each tree predicts a numerical value.</p><pre>Tree 1→250000<br>Tree 2→255000<br>Tree 3→248000<br>Tree 4→252000<br>Tree 5→245000</pre><p>The Random Forest predicts the <strong>average</strong> of all values.</p><pre>Final Prediction = 250000</pre><h3>Why is it Called “Random”</h3><p>The word <strong>Random</strong> comes from the two sources of randomness used while building the forest.</p><h3>Bootstrap Sampling (Random Rows)</h3><p>Instead of giving every tree the exact same training data, Random Forest creates a different dataset for every tree.</p><p>It randomly selects rows <strong>with replacement</strong>, meaning the same row can appear multiple times while some rows may not appear at all.</p><p>Example: Original Dataset Row1 Row2 Row3 Row4 Row5</p><p>Tree 1 may receive : Row1 Row3 Row3 Row5 Row2</p><p>Tree 2 may receive: Row2 Row4 Row1 Row5 Row5</p><blockquote>Notice:</blockquote><ul><li>Row 3 appears twice.</li><li>Row 4 is missing.</li><li>Row 5 appears twice.</li><li>Row 3 is missing.</li></ul><p>This process is called <strong>Bootstrap Sampling</strong>.</p><p>Its purpose is to make every Decision Tree slightly different.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*pTtvma4ZPNcCGGRhBEDEMg.png" /></figure><blockquote><strong>Why is it important?</strong><br> If every Decision Tree trained on exactly the same rows, they would all learn very similar patterns and produce similar predictions. Bootstrap Sampling introduces diversity among the trees, which helps Random Forest make more accurate and robust predictions while reducing overfitting.\</blockquote><h3>Feature Sampling (Random Features)</h3><p>Random Forest also introduces randomness when selecting features.</p><p>Suppose your dataset contains following Features(Colunm_Name) :</p><pre>Age<br>Salary<br>Experience<br>Education<br>Gender<br>Department</pre><p>One tree might consider:</p><pre>Age<br>Salary<br>Education</pre><p>Another tree might consider:</p><pre>Experience<br>Gender<br>Department</pre><p>Each tree then uses its selected features to build splits using the same Decision Tree criteria (such as Gini or Entropy).This creates diverse trees, which leads to better overall performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7CJpoo9j8A37B8ii7KfDBw.jpeg" /></figure><h3>What Happens During Training?</h3><blockquote>When you call:</blockquote><pre>model.fit(X_train, y_train)</pre><blockquote>Scikit-learn performs the following steps automatically:</blockquote><pre>Training Data<br>      │<br>      ▼<br>Bootstrap Sampling<br>(Random Rows)<br>      │<br>      ▼<br>Feature Sampling<br>(Random Features)<br>      │<br>      ▼<br>Build Multiple Decision Trees<br>      │<br>      ▼<br>Train Every Tree<br>      │<br>      ▼<br>Store the Entire Forest</pre><h3>Making Predictions</h3><blockquote>When you call:</blockquote><pre>prediction = model.predict(X_test)</pre><blockquote>Every tree predicts independently.</blockquote><pre>Tree 1<br>     │<br>Tree 2<br>     │<br>Tree 3<br>     │<br>...<br>     │<br>Combine Predictions<br>     │<br>Classification → Majority Vote<br>Regression → Average</pre><blockquote>The combined prediction becomes the final output.</blockquote><h3>Out-of-Bag (OOB) Samples</h3><p>Because Bootstrap Sampling selects rows with replacement, some training rows are not selected for a particular tree.</p><p>These unused rows are called <strong>Out-of-Bag (OOB) samples</strong>.</p><p>Random Forest can use these samples as a small validation set to estimate model performance without requiring a separate validation dataset.</p><h3>Feature Importance</h3><p>One useful advantage of Random Forest is that it can estimate how important each feature was during training.</p><p>Example:</p><pre>Salary       0.45<br>Experience   0.30<br>Age          0.15<br>Gender       0.10</pre><p>Higher importance means the feature contributed more to the model’s decisions. This is extremely useful for understanding your data and performing feature selection.</p><h3>Important Parameters</h3><pre>RandomForestClassifier(<br>    n_estimators=100,<br>    max_depth=5,<br>    max_features=&quot;sqrt&quot;,<br>    min_samples_split=10,<br>    random_state=42<br>)</pre><ul><li><strong>n_estimators</strong> → decidre how many total Number of Decision Trees.</li><li><strong>max_depth</strong> → decide how Maximum depth of each tree.</li><li><strong>max_features</strong> → Number of random features considered at each split in each tree.</li><li><strong>min_samples_split</strong> → Minimum samples required before splitting a node.</li><li><strong>random_state</strong> → Model prediction must same everytime it runs.</li></ul><h3>Advantages of Random Forest</h3><ul><li>Higher accuracy than a single Decision Tree.</li><li>Reduces overfitting.</li><li>Produces more stable predictions.</li><li>Works for both classification and regression.</li><li>Handles large datasets effectively.</li><li>Provides feature importance scores.</li><li>Performs well even with noisy data.</li></ul><h3>Limitations</h3><ul><li>Training is slower than a single Decision Tree.</li><li>Requires more memory because many trees are stored.</li><li>Harder to interpret compared to one Decision Tree.</li></ul><h3>Implementation in Scikit-learn</h3><pre>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.metrics import accuracy_score<br><br>model = RandomForestClassifier(<br>    n_estimators=100,<br>    max_depth=5,<br>    max_features=&quot;sqrt&quot;,<br>    random_state=42<br>)<br><br>model.fit(X_train, y_train)<br><br>prediction = model.predict(X_test)<br><br>print(accuracy_score(y_test, prediction))</pre><h3>Key Takeaways</h3><ul><li>Random Forest is an <strong>ensemble learning algorithm</strong> built using multiple Decision Trees.</li><li>It improves prediction quality by combining the outputs of many trees instead of relying on one.</li><li><strong>Bootstrap Sampling</strong> creates different training datasets by randomly sampling rows with replacement.</li><li><strong>Feature Sampling</strong> selects random subsets of features while building each tree.</li><li><strong>Classification</strong> uses <strong>Majority Voting</strong>.</li><li><strong>Regression</strong> uses the <strong>Average</strong> of predictions.</li><li><strong>n_estimators</strong> controls how many Decision Trees are built.</li><li>Random Forest provides better generalization and significantly reduces overfitting compared to a single Decision Tree.</li></ul><h3>Conclusion</h3><p>Random Forest extends the idea of a Decision Tree by combining many independent trees into a single powerful model. The combination of <strong>Bootstrap Sampling</strong>, <strong>Feature Sampling</strong>, and <strong>ensemble learning</strong> makes it one of the most reliable algorithms for both classification and regression tasks.</p><p>Whether you’re predicting spam emails, customer churn, house prices, or medical diagnoses, Random Forest is often an excellent baseline model because it offers a strong balance between accuracy, robustness, and ease of use.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c05b91a0801" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K-Nearest Neighbors (KNN) in Machine Learning: A Complete Guide to Concepts, Working, Applications…]]></title>
            <link>https://medium.com/@muhammadsufian602/k-nearest-neighbors-knn-in-machine-learning-a-complete-guide-to-concepts-working-applications-e1b9cb6c51e6?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1b9cb6c51e6</guid>
            <category><![CDATA[knn-algorithm]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Sat, 11 Jul 2026 15:07:44 GMT</pubDate>
            <atom:updated>2026-07-11T15:07:44.756Z</atom:updated>
            <content:encoded><![CDATA[<h3>K-Nearest Neighbors (KNN) in Machine Learning: A Complete Guide to Concepts, Working, Applications, and Implementation</h3><p>Learn the fundamentals of the K-Nearest Neighbors (KNN) algorithm, understand how it makes predictions using distance-based learning, discover its advantages, limitations, and real-world applications, and follow a practical implementation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*QsZ19pN4BrejM4dQjImVpg.png" /></figure><h3>Introduction</h3><p>Machine Learning has become one of the most influential fields in Artificial Intelligence, enabling computers to identify patterns, learn from data, and make predictions with minimal human intervention. Among the many supervised learning algorithms available today, <strong>K-Nearest Neighbors (KNN)</strong> stands out for its simplicity, effectiveness, and intuitive approach to solving both classification and regression problems.</p><p>Unlike algorithms that build complex mathematical models during training, KNN follows a different strategy. Instead of learning a predefined model, it stores the training data and makes predictions by comparing new data points with previously seen examples. This makes KNN a <strong>lazy learning algorithm</strong>, as the actual computation occurs only when a prediction is required.</p><h3>What is K-Nearest Neighbors (KNN)?</h3><p>The <strong>K-Nearest Neighbors (KNN)</strong> algorithm is a <strong>Supervised Machine Learning</strong> algorithm used for both <strong>classification</strong> and <strong>regression</strong> tasks. It is one of the simplest and most intuitive algorithms because it makes predictions based on the similarity between data points rather than learning a complex mathematical model.</p><p>The name <strong>K-Nearest Neighbors</strong> comes from the algorithm’s working principle. The value <strong>K</strong> represents the number of nearest data points (neighbors) considered when making a prediction. Instead of creating decision rules during training, KNN stores all the training data and waits until a new data point needs to be classified or predicted. It then identifies the <strong>K closest neighbors</strong> and uses their information to determine the final output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*AmEiTSeypfKP846h4SR2fA.png" /></figure><blockquote>For Example</blockquote><p>Imagine you have a basket of fruits containing apples, oranges, and bananas. Now, you receive a new fruit and want to identify it. Instead of analyzing its characteristics individually, you compare it with the fruits already in the basket. If most of its nearest fruits are apples, you classify the new fruit as an apple.</p><p>The KNN algorithm follows the same idea. It compares a new data point with the closest examples in the training dataset and predicts the outcome based on the majority class or the average value of those neighbors.</p><p><strong>Key Insight:</strong> KNN does not learn from the data during training. Instead, it learns when a new prediction is requested by finding the nearest neighbors in the dataset.</p><blockquote>KNN can handle two completely different types of tasks:</blockquote><ol><li><strong><em>KNN for Classification</em></strong>:</li></ol><p>KNN Classification is used when the target variable consists of <strong>categories or classes</strong>. The algorithm identifies the <strong>K nearest neighbors</strong> of a new data point and predicts the class that appears most frequently among them using the majority vote.</p><p>For example, if a new email is surrounded mostly by emails labeled as <strong>Spam</strong>, the algorithm classifies the new email as <strong>Spam</strong>.</p><p>2. <strong><em>KNN for Regression:</em></strong></p><p>KNN Regression is used when the target variable is a <strong>continuous numerical value</strong> instead of a category. Instead of taking a majority vote, the algorithm calculates the <strong>average value</strong> of the K nearest neighbors to make the prediction.</p><p>For example, if you want to predict the price of a house, KNN Regression estimates the price by averaging the prices of similar nearby houses.</p><h3>Why is K-Nearest Neighbors (KNN) Important?</h3><p>The <strong>K-Nearest Neighbors (KNN)</strong> algorithm is considered one of the foundational algorithms in Machine Learning due to its simplicity and effectiveness. It provides a straightforward approach to solving prediction problems by relying on the similarity between data points rather than building a complex mathematical model.</p><h3>Why KNN is Widely Used</h3><p>Some of the reasons why KNN remains a popular machine learning algorithm include:</p><ul><li>Simple and easy to understand.</li><li>Requires very little training since it stores the dataset instead of building a model.</li><li>Can solve both classification and regression problems.</li><li>Performs well on small and medium-sized datasets.</li><li>Adapts naturally as new data becomes available.</li><li>Serves as a strong baseline model for comparing other machine learning algorithms.</li></ul><h3><strong>The Step-by-Step Flow</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*vRz-axQWbygK0_gbogcKQg.png" /></figure><h3><strong>How Does K-Nearest Neighbors (KNN) Work?</strong></h3><p>This process allows KNN to classify or predict new observations based on the similarity between data points, making it an intuitive and effective algorithm for many machine learning problems.</p><h3>Working Process of KNN</h3><p>The KNN algorithm follows these steps:</p><ol><li><strong>Store the Training Data</strong><br> The algorithm stores all training samples without building a mathematical model.</li><li><strong>Choose the Value of K</strong><br> Select the number of nearest neighbors to consider for prediction.</li><li><strong>Calculate the Distance</strong><br> Measure the distance between the new data point and every point in the training dataset.</li><li><strong>Find the Nearest Neighbors</strong><br> Identify the K data points with the smallest distances.</li><li><strong>Make the Prediction</strong></li></ol><ul><li>For <strong>classification</strong>, choose the class with the majority vote.</li><li>For <strong>regression</strong>, calculate the average value of the nearest neighbors.</li></ul><h3>Value of K</h3><p>The value of <strong>K</strong> determines how many neighboring data points are considered when making a prediction.</p><ul><li>A <strong>small K</strong> may make the model sensitive to noise and lead to overfitting.</li><li>A <strong>large K</strong> produces smoother predictions but may overlook important patterns, resulting in underfitting.</li></ul><p>Choosing an appropriate value of K is one of the most important steps when building a KNN model.</p><h3>Distance Metrics</h3><p>KNN determines similarity by measuring the distance between data points. Different distance metrics can be used depending on the nature of the dataset.</p><p>The most commonly used distance metrics include:</p><ul><li><strong>Euclidean Distance</strong> : Measures the straight-line distance between two points</li><li><strong>Manhattan Distance</strong> : Measures distance by moving horizontally and vertically, similar to navigating city streets.</li><li><strong>Minkowski Distance: </strong>A generalized distance metric that includes Euclidean and Manhattan distances as special cases.</li></ul><p>The choice of distance metric can significantly affect the model’s performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*10Ax3s9vQzPPYeTTVhlQKg.png" /></figure><h3>Feature Scaling</h3><p><strong>Feature Scaling</strong> is a data preprocessing technique used to bring all numerical features to a similar scale before training a machine learning model. Since KNN relies on distance calculations, all features should be on a similar scale. If one feature has much larger values than others, it can dominate the distance calculation and reduce the model’s accuracy.</p><p>Common feature scaling techniques include:</p><blockquote>Standardization:</blockquote><p>Standardization transforms the data so that it has a <strong>mean of 0</strong> and a <strong>standard deviation of 1</strong>.</p><blockquote>Normalization (Min-Max Scaling):</blockquote><p>Normalization rescales feature values to a fixed range, usually between <strong>0 and 1</strong>.</p><p>Feature scaling is considered an essential preprocessing step when working with KNN.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*0Wnybp4w2myO1glIS-LIsw.png" /></figure><h3>Decision Boundary</h3><p>The <strong>decision boundary</strong> is the line or region that separates different classes in the feature space.</p><p>The value of <strong>K</strong> directly influences the shape of the decision boundary:</p><ul><li>Smaller values of K create more complex boundaries.</li><li>Larger values of K produce smoother and more generalized boundaries.</li></ul><p>Choosing the right value helps balance model complexity and prediction accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*J532e7_ctaSbw95J39Cq8w.jpeg" /></figure><h3>Finding the Perfect K: Error Rate and Accuracy Graphs</h3><p>Data scientists don’t just guess the value of $K$. They run the algorithm multiple times using a range of values (for example, trying every single odd number from 1 to 40) and plot those results onto a line graph to find the sweet spot.</p><blockquote><strong>The Error Rate Graph</strong></blockquote><p>This tracks the percentage of wrong predictions the algorithm makes. When K=1, your error rate is usually high because the model is overreacting to random noise (overfitting). As you slowly increase K, you will watch the line on the graph slope downward as the model gets smarter and more stable. This bottom dip is your model’s sweet spot!</p><p>If you keep increasing $K$ too much, the line will start climbing back up into a high error zone because the model is becoming too simple (underfitting). You want to pick the $K$ value sitting at that lowest valley.</p><blockquote><strong>The Accuracy Graph</strong></blockquote><p>This is simply the mirror image of your error rate graph. Instead of looking for a low valley, you are looking for the highest mountain peak. The value of K that produces the highest, most stable plateau of accuracy is your winner.</p><h3><strong>Advantages of K-Nearest Neighbors (KNN)</strong></h3><ul><li>Simple and easy to understand.</li><li>Easy to implement using popular machine learning libraries.</li><li>Can solve both classification and regression problems.</li><li>No training phase is required since it stores the dataset.</li><li>Makes predictions based on real training examples.</li><li>Works well with small and medium-sized datasets.</li><li>Can model complex decision boundaries without assuming any specific data distribution.</li><li>Easy to update by adding new training data without rebuilding the model.</li></ul><h3><strong>Limitations of K-Nearest Neighbors (KNN)</strong></h3><ul><li>Prediction becomes slower as the dataset grows larger.</li><li>Requires feature scaling because it depends on distance calculations.</li><li>Sensitive to noisy and irrelevant data.</li><li>Choosing the wrong value of <strong>K</strong> can reduce prediction accuracy.</li><li>High memory usage since the entire training dataset must be stored.</li><li>Performance decreases when dealing with high-dimensional datasets (known as the <strong>Curse of Dimensionality</strong>).</li></ul><h3><strong>Practical Implementation of K-Nearest Neighbors (KNN)</strong></h3><p>After understanding the theoretical concepts of the <strong>K-Nearest Neighbors (KNN)</strong> algorithm, the next step was to implement it on a real-world dataset . The objective of this implementation was to build a machine learning model capable of predicting whether an individual’s annual income is <strong>less than or equal to $50K</strong> or <strong>greater than $50K</strong> based on demographic and employment-related information.</p><p>The implementation followed a standard machine learning workflow consisting of <strong>data preprocessing</strong>, <strong>dataset splitting</strong>, <strong>model training</strong>, and <strong>performance evaluation</strong>.</p><h3>Dataset Overview</h3><p>The implementation was performed using the <strong>Adult Income Dataset</strong>, a well-known classification dataset commonly used in machine learning.</p><p>The dataset contains information about individuals, including their age, education, occupation, working hours, marital status, and several other demographic attributes. Using these features, the model predicts whether a person’s annual income falls into one of the following categories:</p><ul><li><strong>Income ≤ $50K</strong></li><li><strong>Income &gt; $50K</strong></li></ul><blockquote>DATASET SOURCE : <a href="https://www.kaggle.com/datasets/wenruliu/adult-income-dataset/data">https://www.kaggle.com/datasets/wenruliu/adult-income-dataset/data</a></blockquote><h4>Dataset Summary</h4><ul><li><strong>Total Records:</strong> 48,842</li><li><strong>Total Features:</strong> 15</li><li><strong>Target Variable:</strong> Income</li><li><strong>Problem Type:</strong> Binary Classification</li></ul><p>This dataset is widely used for evaluating supervised learning algorithms because it contains both numerical and categorical features.</p><h3>Data Preprocessing</h3><p>Before training the KNN model, the dataset required preprocessing to ensure that the data was clean, consistent, and suitable for machine learning.</p><p>The preprocessing stage included the following steps:</p><ul><li>Loading the dataset .</li></ul><pre>df = pd.read_csv(&#39;/content/adult.csv&#39;)<br>print(&quot;Dataset Information:&quot;)<br>df.info()</pre><ul><li>Exploring the dataset to understand its structure and features.</li><li>Checking for missing or inconsistent values.</li></ul><pre># Impute missing categorical values with the mode<br>for col in categorical_cols:<br>    mode_value = df_train[col].mode()[0] # Use mode from training data to avoid data leakage<br>    df_train[col] = df_train[col].fillna(mode_value)<br>    df_test[col] = df_test[col].fillna(mode_value)<br><br>print(&quot;Number of missing values in training set after imputation:&quot;)<br>print(df_train[categorical_cols].isnull().sum())<br><br>print(&quot;\nNumber of missing values in testing set after imputation:&quot;)<br>print(df_test[categorical_cols].isnull().sum())</pre><ul><li>Encoding categorical features into numerical values.</li></ul><pre># Apply One-Hot Encoding to categorical features<br># Initialize OneHotEncoder<br>ohe = OneHotEncoder(handle_unknown=&#39;ignore&#39;, sparse_output=False)<br><br># Fit on training data and transform both training and testing data<br>X_train_encoded = ohe.fit_transform(X_train[feature_categorical_cols])<br>X_test_encoded = ohe.transform(X_test[feature_categorical_cols])<br><br># Create DataFrames from the encoded arrays with proper column names<br>X_train_encoded_df = pd.DataFrame(X_train_encoded, columns=ohe.get_feature_names_out(feature_categorical_cols), index=X_train.index)<br>X_test_encoded_df = pd.DataFrame(X_test_encoded, columns=ohe.get_feature_names_out(feature_categorical_cols), index=X_test.index)<br><br># Drop original categorical columns from X_train and X_test<br>X_train = X_train.drop(feature_categorical_cols, axis=1)<br>X_test = X_test.drop(feature_categorical_cols, axis=1)<br><br># Concatenate numerical and encoded categorical features<br>X_train_processed = pd.concat([X_train, X_train_encoded_df], axis=1)<br>X_test_processed = pd.concat([X_test, X_test_encoded_df], axis=1)</pre><ul><li>Separating the input features (<strong>X</strong>) from the target variable (<strong>y</strong>).</li><li>Applying <strong>Feature Scaling</strong> to normalize numerical features.</li></ul><p>Feature scaling is particularly important for the KNN algorithm because it calculates the distance between data points. Without scaling, features with larger numerical values could dominate the distance calculation and reduce the model’s prediction accuracy.</p><h3>Splitting the Dataset</h3><p>After preprocessing, the dataset was divided into <strong>training</strong> and <strong>testing</strong> subsets using a <strong>70:30 ratio</strong>.</p><ul><li><strong>70% Training Data</strong> :Used to train the KNN model.</li><li><strong>30% Testing Data: </strong>Used to evaluate the model’s performance on unseen data.</li></ul><pre>df_train, df_test = train_test_split(df, test_size=0.3, random_state=42)<br><br>print(f&quot;\nShape of the original dataset: {df.shape}&quot;)<br>print(f&quot;Shape of the training set (70%): {df_train.shape}&quot;)<br>print(f&quot;Shape of the testing set (30%): {df_test.shape}&quot;)<br><br>print(&quot;\nFirst 5 rows of the training set:&quot;)<br>display(df_train.head())<br><br>print(&quot;\nFirst 5 rows of the testing set:&quot;)<br>display(df_test.head())</pre><p>Using separate training and testing datasets helps measure how well the model generalizes to new data instead of simply memorizing the training samples.</p><h3>Training the KNN Model</h3><p>Once the dataset was prepared, the <strong>K-Nearest Neighbors Classifier</strong> was trained using the training dataset.</p><p>During training, the algorithm stored all training instances instead of building a mathematical model. When a new data point needed to be classified, the model calculated the distance between the new observation and every training sample, selected the <strong>K nearest neighbors</strong>, and predicted the final class based on majority voting.</p><p>The value of <strong>K</strong> was chosen carefully to balance model complexity and prediction accuracy.</p><pre>from sklearn.neighbors import KNeighborsClassifier<br>from sklearn.metrics import accuracy_score, classification_report<br><br># Initialize the KNN classifier<br># You can experiment with different values for n_neighbors (e.g., 3, 5, 7)<br>knn = KNeighborsClassifier(n_neighbors=5)<br><br># Train the model using the processed training data<br>print(&quot;Training KNN model...&quot;)<br>knn.fit(X_train_processed, y_train)<br>print(&quot;Training complete.&quot;)</pre><h3>Model Evaluation</h3><p>After training, the model was evaluated using the testing dataset to measure its predictive performance.</p><p>Several evaluation metrics were used to analyze the model:</p><blockquote><strong>Accuracy Score</strong> :Measures the percentage of correctly classified instances.</blockquote><pre>KNN Model Accuracy (after hyperparameter tuning): 0.8464</pre><blockquote><strong>Error Rate</strong> : Represents the proportion of incorrect predictions.</blockquote><blockquote><strong>Classification Report : </strong>Includes Precision, Recall, and F1-Score for each class.</blockquote><pre>Classification Report (after hyperparameter tuning):<br>               precision    recall  f1-score   support<br><br>       &lt;=50K       0.88      0.93      0.90     11233<br>        &gt;50K       0.72      0.56      0.63      3420<br><br>    accuracy                           0.85     14653<br>   macro avg       0.80      0.75      0.77     14653<br>weighted avg       0.84      0.85      0.84     14653</pre><p>These evaluation metrics provide a comprehensive understanding of the model’s strengths and areas for improvement.</p><h3><strong>Accuracy and Error Rate Analysis:</strong></h3><p>To determine the most suitable value of <strong>K</strong>, multiple K values were tested and their corresponding <strong>accuracy</strong> and <strong>error rates</strong> were analyzed.</p><p>The results were visualized using graphs, making it easier to compare the performance of different K values.</p><blockquote>Accuracy Graph</blockquote><p>The accuracy graph illustrates how the model’s accuracy changes as the value of <strong>K</strong> increases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*4Eudc9AADp-vvSxyqzDbCg.png" /></figure><p>A suitable value of <strong>K</strong> provides the highest accuracy while maintaining good generalization on unseen data.</p><blockquote>Error Rate Graph</blockquote><p>The error rate graph shows the relationship between the value of <strong>K</strong> and the prediction error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*9KzzObsJCW0vAzmLPUqwLw.png" /></figure><p>A lower error rate indicates better model performance and helps identify the optimal value of <strong>K</strong>.</p><h3>Confusion Matrix</h3><p>The <strong>Confusion Matrix</strong> was used to evaluate the classification performance of the trained KNN model.</p><p>It summarizes the number of:</p><ul><li><strong>True Positives (TP)</strong> :Correctly predicted positive instances.</li><li><strong>True Negatives (TN)</strong> :Correctly predicted negative instances.</li><li><strong>False Positives (FP)</strong> :Incorrect positive predictions.</li><li><strong>False Negatives (FN)</strong> :Incorrect negative predictions.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*MppShacU24corDLN-lZBGg.png" /></figure><p>By analyzing the confusion matrix, it becomes easier to understand the types of prediction errors made by the model and evaluate its overall reliability.</p><h3>Final Results</h3><p>The experimental results demonstrated that the K-Nearest Neighbors algorithm successfully learned patterns from the Adult Income Dataset and achieved reliable classification performance.</p><p>The combination of proper data preprocessing, feature scaling, an appropriate train-test split, and selecting a suitable value of <strong>K</strong> contributed significantly to the model’s overall performance.</p><p>Through this implementation, We gained practical experience in applying the complete machine learning workflow, from preparing the data and training the model to evaluating its performance using multiple metrics. This project reinforced understanding of how KNN makes predictions based on similarity and highlighted the importance of preprocessing and parameter selection in building accurate machine learning models.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1b9cb6c51e6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decision Trees in Machine Learning: A Complete Guide to Concepts, Working, Applications, and…]]></title>
            <link>https://medium.com/@muhammadsufian602/decision-trees-in-machine-learning-a-complete-guide-to-concepts-working-applications-and-3b0e6fad1803?source=rss-6cf16ace5a71------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b0e6fad1803</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[decision-tree-classifier]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Muhammadsufian]]></dc:creator>
            <pubDate>Fri, 10 Jul 2026 11:18:06 GMT</pubDate>
            <atom:updated>2026-07-10T11:18:06.511Z</atom:updated>
            <content:encoded><![CDATA[<h3>Decision Trees in Machine Learning: A Complete Guide to Concepts, Working, Applications, and Implementation</h3><p>Learn what Decision Trees are, how they work, where they’re used, and how to build your first Decision Tree model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*wMYhvh0XTz0ImAYzbZo3Ug.jpeg" /></figure><h3>Introduction</h3><p>Machine learning allows computers to recognize patterns in data and make predictions without being explicitly programmed for every situation. Among the many algorithms available, Decision Trees stand out because they are simple, easy to understand, and highly effective.</p><p>A Decision Tree works much like the way people make everyday decisions. Instead of jumping straight to an answer, it asks a series of questions, with each answer leading to the next step until it reaches a final decision. This straightforward approach makes Decision Trees one of the easiest machine learning models to learn and explain.</p><p>Because they are both accurate and interpretable, Decision Trees are widely used across industries such as healthcare, finance, education, e-commerce, agriculture, and cybersecurity.</p><h3><strong>What Is a Decision Tree?</strong></h3><p>A <strong>Decision Tree</strong> is a <strong>supervised machine learning algorithm</strong> used for both <strong>classification</strong> and <strong>regression</strong> tasks. It learns from labeled data and predicts outcomes by following a series of logical decisions.</p><p>The model is organized like a tree. It starts at a <strong>Root Node</strong>, where the first decision is made. From there, the data is split into smaller groups based on different conditions. This process continues until the algorithm reaches a <strong>Leaf Node</strong>, which represents the final prediction.</p><p>Since every prediction follows a clear path, it’s easy to understand how the model arrived at its decision.</p><p><strong>Why Are Decision Trees Important?</strong></p><p>One of the biggest strengths of Decision Trees is that they’re easy to interpret. Unlike many advanced machine learning models that operate like “black boxes,” Decision Trees make every decision visible. This transparency is especially valuable in fields where it’s important to explain why a prediction was made.</p><p>Some of their key advantages include:</p><ul><li>Requires little data preprocessing</li><li>Works well with both numerical and categorical data</li><li>Can solve both classification and regression problems</li><li>Produces clear, explainable predictions</li></ul><h3><strong>Decision Tree Algorithms</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Aomw98Mj0FmugPHmfn1mWg.jpeg" /></figure><blockquote><strong>ID3 Algorithm (Iterative Dichotomiser 3)</strong></blockquote><p>The ID3 algorithm is a classic method used to build decision trees for sorting data into categories. It works from the top down by looking at all available features and picking the one that clears up the most confusion, a method calculated using <strong>Entropy</strong> and <strong>Information Gain</strong>. ID3 is designed specifically for <strong>classification tasks</strong> (like predicting “Yes” or “No”) and works best with <strong>categorical data</strong> (like colors or days of the week). When it makes a decision, it can split a node into <strong>multiple branches</strong> at once if a feature has several categories.</p><blockquote><strong>CART Algorithm (Classification and Regression Trees)</strong></blockquote><p>The CART algorithm is a modern, highly versatile method that can be used for both <strong>classification</strong> (sorting into categories) and <strong>regression</strong> (predicting continuous numbers, like a house price). Instead of measuring information gain, it uses a metric called the <strong>Gini Index</strong> to find the cleanest, most pure split in the data. Unlike ID3, CART handles both <strong>numbers and categories</strong> completely on its own. Furthermore, it strictly creates <strong>binary splits</strong>, meaning every single decision point will always branch out into exactly two paths (like True/False or Yes/No).</p><h3><strong>How Does a Decision Tree Work?</strong></h3><p>A Decision Tree learns by repeatedly dividing a dataset into smaller groups based on the feature that best separates the data.</p><p>The process generally follows these steps:</p><p><strong>Step 1: Collect the Dataset</strong></p><p>The algorithm begins with a dataset containing input features and their corresponding target values.</p><p><strong>Step 2: Choose the Best Feature</strong></p><p>It evaluates every feature and selects the one that creates the most effective split. Common measures used to determine the best split include:</p><blockquote>Information Gain</blockquote><p>Information Gain measures how much uncertainty or entropy is reduced after splitting a dataset based on a specific feature.It helps a decision tree decide which question to ask next by choosing the feature that clears up the most confusion.</p><blockquote>Gini Index</blockquote><p>The Gini Index measures the probability of a randomly chosen element being wrongly classified if it was randomly labeled according to the distribution in the dataset. It measures the “impurity” of the data, where a score of 0 means perfect purity (everyone belongs to the same group) and higher scores mean a messy mix.</p><blockquote>Entropy</blockquote><p>Entropy is a measure of the total randomness, disorder, or uncertainty in a dataset. A high entropy means the data is completely mixed up, while a zero entropy means the data is perfectly clean and uniform.</p><p><strong>Step 3: Split the Data</strong></p><p>Using the selected feature, the dataset is divided into smaller subsets.</p><p><strong>Step 4: Repeat the Process</strong></p><p>The algorithm continues evaluating each subset and creating new splits until a stopping condition is met, such as reaching a maximum depth or having too few samples to split further.</p><p><strong>Step 5: Make the Final Prediction</strong></p><p>Once no additional splitting is needed, the remaining nodes become <strong>Leaf Nodes</strong>, each representing the model’s final prediction.</p><h3>Structure of a Decision Tree</h3><p>A Decision Tree is made up of several key components.</p><p><strong>Root Node</strong></p><p>The Root Node is the starting point of the tree. It represents the feature that provides the best initial split in the dataset.</p><p><strong>Decision Node</strong></p><p>A Decision Node contains a condition that divides the data into different branches.</p><p>For example:</p><p><strong>Is Age greater than 30?</strong></p><p>Depending on the answer, the data follows a different path.</p><p><strong>Branch</strong></p><p>A Branch represents the outcome of a decision. Each branch leads either to another decision or directly to a final prediction.</p><p><strong>Leaf Node</strong></p><p>A Leaf Node is the endpoint of the tree. It contains the final predicted class or value and has no further branches. The node That have no any child node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2-8mTt8RKFWS1Db4vpk4oA.jpeg" /></figure><h3>Advantages of Decision Trees</h3><p>Decision Trees remain popular because they combine simplicity with strong predictive performance.</p><p>Some of their main advantages include:</p><ul><li>Easy to understand and explain</li><li>Simple to visualize</li><li>Requires minimal data preparation</li><li>Works with both numerical and categorical features</li><li>Can model non-linear relationships</li><li>Supports both classification and regression tasks</li><li>Helps identify the most important features in a dataset</li></ul><h3>Limitations of Decision Trees</h3><p>Although Decision Trees have many benefits, they also have some drawbacks.</p><ul><li>They can overfit the training data if allowed to grow too deep.</li><li>Small changes in the data may produce a very different tree.</li><li>They may not perform as well as ensemble methods such as Random Forests or Gradient Boosting on complex datasets.</li></ul><p>Fortunately, techniques such as pruning and limiting tree depth can help reduce these issues.</p><h3>Real-World Applications</h3><p>Decision Trees are widely used because they’re easy to interpret and adapt to different types of problems.</p><p>Some common applications include:</p><ul><li><strong>Healthcare:</strong> Diagnosing diseases and predicting patient outcomes.</li><li><strong>Finance:</strong> Assessing loan eligibility and detecting fraud.</li><li><strong>E-commerce:</strong> Recommending products based on customer behavior.</li><li><strong>Education:</strong> Predicting student performance and identifying learning patterns.</li><li><strong>Agriculture:</strong> Forecasting crop yields and detecting plant diseases.</li><li><strong>Cybersecurity:</strong> Identifying suspicious network activity and potential cyberattacks.</li></ul><h3>Conclusion</h3><p>Decision Trees are one of the most beginner-friendly machine learning algorithms. Their simple, logical structure makes them easy to understand while still providing strong predictive performance for many real-world problems.</p><p>Although they have some limitations, such as the risk of overfitting, they remain a valuable tool for both learning machine learning concepts and solving practical classification and regression tasks. As you continue exploring machine learning, mastering Decision Trees will provide a solid foundation for understanding more advanced models such as Random Forests and Gradient Boosting.</p><h3><strong>Practical Implementation of a Decision Tree Model Using Google Colab</strong></h3><p>After understanding the theoretical concepts of Decision Trees, the next step was to implement the algorithm on a real-world dataset using <strong>Google Colab</strong>. The objective was to train a machine learning model capable of predicting whether an individual’s annual salary is <strong>less than or equal to $50K</strong> or <strong>greater than $50K</strong> based on various demographic and employment-related features.</p><p>The implementation followed a standard machine learning workflow, including data preprocessing, dataset splitting, model training, model evaluation, and performance analysis.</p><h3><strong>Dataset Overview</strong></h3><p>The model was trained using the <strong>Adult Salary Dataset</strong>, which contains information about individuals such as their age, education, occupation, work class, marital status, working hours, and other demographic attributes.</p><p>DATASET LINK : <a href="https://www.kaggle.com/datasets/ishank2005/salary-csv">https://www.kaggle.com/datasets/ishank2005/salary-csv</a></p><h3>Dataset Summary</h3><ul><li><strong>Total Records:</strong> 32,561</li><li><strong>Total Features:</strong> 15</li><li><strong>Target Variable:</strong> Salary</li><li><strong>Prediction Classes:</strong></li><li>Salary ≤ 50K</li><li>Salary &gt; 50K</li></ul><p>The dataset is widely used for classification problems and is suitable for demonstrating how Decision Tree algorithms make predictions from structured data.</p><h3>Data Preprocessing</h3><p>Before training the model, the dataset was prepared to ensure that the learning algorithm could process the data effectively.</p><p>The preprocessing stage included several important tasks:</p><ul><li>Loading the dataset into Google Colab.</li><li>Exploring the dataset structure and feature names.</li><li>Checking for missing or inconsistent values.</li><li>Converting categorical variables into numerical representations using appropriate encoding techniques.</li><li>Separating the input features from the target variable.</li><li>Preparing the dataset for machine learning.</li></ul><p>Proper preprocessing is an essential step because the quality of the input data directly influences the performance of the trained model.</p><pre># Separate features (X) and target (y)<br>X = df_salary.drop(&#39;salary&#39;, axis=1)<br>y = df_salary[&#39;salary&#39;]<br># Identify categorical and numerical features<br>categorical_features = X.select_dtypes(include=[&#39;object&#39;]).columns<br>numerical_features = X.select_dtypes(include=[&#39;int64&#39;, &#39;float64&#39;]).columns<br><br># Create a column transformer for one-hot encoding categorical features<br>preprocessor = ColumnTransformer(<br>    transformers=[<br>        (&#39;cat&#39;, OneHotEncoder(handle_unknown=&#39;ignore&#39;), categorical_features)<br>    ],<br>    remainder=&#39;passthrough&#39; # Keep numerical features as they are<br>)<br># Apply preprocessing to features<br>X_processed = preprocessor.fit_transform(X)<br># Encode the target variable<br>label_encoder = LabelEncoder()<br>y_encoded = label_encoder.fit_transform(y)</pre><h3>Splitting the Dataset</h3><p>To evaluate the model fairly, the dataset was divided into two parts:</p><ul><li><strong>Training Set</strong> <strong>(80%):</strong> used to teach the Decision Tree algorithm.</li><li><strong>Testing Set (20%): </strong>used to evaluate how well the trained model performs on unseen data.</li></ul><p>This approach helps measure the model’s ability to generalize rather than simply memorizing the training data.</p><pre># Split the data into training and testing sets (80-20 split)<br>X_train, X_test, y_train, y_test = train_test_split(X_processed, y_encoded, test_size=0.2, random_state=42)</pre><h3>Training the Decision Tree Model</h3><p>Once the dataset was prepared, a Decision Tree Classifier was trained using the training data.</p><p>During training, the algorithm analyzed the relationships between the input features and the target variable. It repeatedly selected the best feature for splitting the data and constructed a tree-like structure consisting of root nodes, decision nodes, branches, and leaf nodes.</p><p>The objective of the model was to learn patterns that distinguish individuals earning more than $50K from those earning $50K or less.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TwcJ0TJjgSVhUt9qldYW4g.png" /></figure><pre>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.metrics import accuracy_score, classification_report<br>import matplotlib.pyplot as plt<br>from sklearn.tree import plot_tree<br><br># Initialize and train the Decision Tree Classifier<br>dt_classifier = DecisionTreeClassifier(random_state=42)<br>dt_classifier.fit(X_train, y_train)<br><br># Make predictions on the test set<br>y_pred = dt_classifier.predict(X_test)<br><br># Evaluate the model<br>accuracy = accuracy_score(y_test, y_pred)<br>print(f&quot;Decision Tree Classifier Accuracy: {accuracy:.4f}&quot;)<br><br>print(&quot;\nClassification Report:&quot;)<br>print(classification_report(y_test, y_pred))<br><br># Visualize the Decision Tree (optional, for smaller trees, consider max_depth)<br># For larger trees, visualization can be complex.<br># We will visualize a simplified tree to understand its structure.<br>plt.figure(figsize=(20,10))<br>plot_tree(dt_classifier,<br>          feature_names=preprocessor.get_feature_names_out(X.columns),<br>          class_names=label_encoder.classes_,<br>          filled=True,<br>          rounded=True,<br>          max_depth=3) # Limiting depth for better readability<br>plt.title(&quot;Decision Tree Classifier (max_depth=3)&quot;)<br>plt.show()</pre><h3>Model Evaluation</h3><p>After training, the model was tested using the testing dataset to evaluate its predictive performance.</p><p>Several evaluation metrics were used to assess the quality of the model, including:</p><ul><li><strong>Accuracy</strong> — Measures the overall percentage of correct predictions.</li><li><strong>Precision</strong> — Indicates how many positive predictions were actually correct.</li><li><strong>Recall</strong> — Measures the model’s ability to correctly identify positive instances.</li><li><strong>F1-Score</strong> — Provides a balanced measure by combining Precision and Recall.</li><li><strong>Confusion Matrix</strong> — Summarizes the number of correct and incorrect predictions for each class.</li></ul><p>These metrics provide a comprehensive understanding of how well the Decision Tree performs beyond simple accuracy alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*Rb_GpX3RJFC7GAvoCgbYGg.png" /></figure><h3>Model Performance</h3><p>The evaluation results demonstrated that the Decision Tree model successfully learned meaningful patterns from the dataset and was able to classify salary categories with good predictive performance.</p><p>The model correctly identified most salary classes while maintaining a balanced performance across the evaluation metrics. The confusion matrix and classification report further helped analyze both correct classifications and prediction errors.</p><p>Overall, the implementation showed that Decision Trees are effective for structured classification problems and provide predictions that are easy to interpret due to their transparent tree-based decision-making process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*EiT0EgusssI0DedOsYgNtA.png" /></figure><h3>Key Learning Outcomes</h3><p>Implementing the Decision Tree model provided valuable practical experience and reinforced several important machine learning concepts.</p><p>Some of the key takeaways include:</p><ul><li>Understanding the importance of data preprocessing before model training.</li><li>Learning how training and testing datasets improve model evaluation.</li><li>Observing how Decision Trees create decision rules from data.</li><li>Understanding the role of evaluation metrics such as Accuracy, Precision, Recall, F1-Score, and the Confusion Matrix.</li><li>Recognizing the importance of evaluating a model on unseen data to measure its real-world performance.</li></ul><p>This practical implementation strengthened understanding of how Decision Trees move from theoretical concepts to real-world machine learning applications and highlighted the complete workflow involved in building an interpretable classification model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b0e6fad1803" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>