<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shubham Singh Tomar on Medium]]></title>
        <description><![CDATA[Stories by Shubham Singh Tomar on Medium]]></description>
        <link>https://medium.com/@shubhamsinghtomar?source=rss-6fa546dc903b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Pe-apkh5LBPKfIbq8Gk7fw.jpeg</url>
            <title>Stories by Shubham Singh Tomar on Medium</title>
            <link>https://medium.com/@shubhamsinghtomar?source=rss-6fa546dc903b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:08:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shubhamsinghtomar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why do we need GPUs for Deep learning?]]></title>
            <link>https://medium.com/knowchow/why-do-we-need-gpus-for-deep-learning-39697059c6e8?source=rss-6fa546dc903b------2</link>
            <guid isPermaLink="false">https://medium.com/p/39697059c6e8</guid>
            <category><![CDATA[parallel-processing]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Shubham Singh Tomar]]></dc:creator>
            <pubDate>Tue, 19 Dec 2017 15:33:17 GMT</pubDate>
            <atom:updated>2020-04-08T05:52:14.249Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the nice properties of neural networks is that they find patterns in the data (features) by themselves. But, how do they do that and how do GPUs help with it?</p><h4>Images and texts can be represented as numbers</h4><p>Here’s an image from the very popular MNIST (“Modified National Institute of Standards and Technology”) dataset, the “Hello World” of computer vision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rgAb1vWt4PEPWK9sNSJUEg.png" /><figcaption>Matrix representation of 0 from MNIST</figcaption></figure><p>On the right you have the matrix representation of the grayscale image on the left. Each element of the matrix represents a pixel of the image and has a value between 0 and 255.</p><p>Similarly, texts can be represented as a term-document matrix.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GKKJJjeRAQO_XjdEAkBzPA.png" /><figcaption>Term-document matrix</figcaption></figure><p>Here, each number in a term-document matrix represents the number of times a word appears in the document.</p><h4>Convolutional Neural Networks</h4><p><a href="https://medium.com/@shubhamsinghtomar/fast-ai-cnn-for-image-classification-72c6ece5518d">CNN for image classification</a> comprises a lot of matrix multiplications. In fact, that’s what a <em>convolution </em>in CNN means — element-wise matrix multiplication.<br><a href="https://twitter.com/jeremyphoward">Jeremy Howard</a> explained this really well in <a href="https://fast.com/">Fast.ai</a>’s Lesson 3. He used an excel spreadsheet to explain a forward pass through a general CNN architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yqM5nitDNQnMVFkPEb668g.png" /></figure><p>You start with an image and multiply it by an image kernel. For example, the kernel in the above image works as an edge detector and finds all the horizontal edges. You iterate over a 3x3 matrix in the image and multiply it by the filter. Then, you use an activation function like ReLU or Softmax.<br>This is followed by more kernels and steps like MaxPooling.</p><p>The objective of MaxPooling is to down-sample an input representation (image, hidden-layer output matrix, etc.), reducing its dimensionality and allowing for assumptions to be made about features contained in the sub-regions binned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qmhsATVkxJ1ZeERq55H6cw.png" /><figcaption>MaxPooling</figcaption></figure><p>Followed by multiplying the MaxPool matrix by arbitrarily initialised weights.<br>The following two layers are called fully-connected layers because we are performing the multiplication of the complete matrix and not parts of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*moV3Iq52VQ5ZM1KYoNjcmA.png" /></figure><p>The next steps, of course, include optimising weights to minimise the loss function.</p><h4>But, how do GPUs do it faster?</h4><p>The main computational module, the CPUs are designed to do computation rapidly on a small amount of data. For example, multiplying a few numbers on a CPU is blazingly fast. But it struggles when operating on a large amount of data. E.g., multiplying matrices of tens or hundreds thousand numbers.</p><p>The following curve shows how computation time increases why multiplying two randomly created matrices as the size of the matrices increase.<br>This graph was created when comparing GTX 1080 and i7.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*wHUC2onsgTz6NziUFwvcig.png" /><figcaption>CPUs vs GPUs</figcaption></figure><p>GPUs take less time for such operation because they are composed of hundreds of cores that can handle thousands of threads simultaneously.<br>This also means that the programming model used to program GPUs are different than the one used for serial programs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bWpozW0PqEZGn-WUjvsIRg.png" /><figcaption>Squaring numbers on a CPU: serial programming</figcaption></figure><p>GPUs divide the task among multiple cores each performing one task at a time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aZhOHodno7QCl6g2Z1ajAA.png" /><figcaption>Squaring numbers on a GPU: Parallel programming</figcaption></figure><p>So, for the above task of squaring numbers, if each operation takes 2 ns, it will take the CPU to 2*64 ns to square all the numbers.<br>On a GPU, however, if each operation takes 10 ns, and there are 64 threads, the whole program would take just 10 ns.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39697059c6e8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/knowchow/why-do-we-need-gpus-for-deep-learning-39697059c6e8">Why do we need GPUs for Deep learning?</a> was originally published in <a href="https://medium.com/knowchow">Knowchow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fast.ai | CNN for image classification]]></title>
            <link>https://medium.com/knowchow/fast-ai-cnn-for-image-classification-72c6ece5518d?source=rss-6fa546dc903b------2</link>
            <guid isPermaLink="false">https://medium.com/p/72c6ece5518d</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Shubham Singh Tomar]]></dc:creator>
            <pubDate>Sun, 03 Dec 2017 16:19:16 GMT</pubDate>
            <atom:updated>2020-04-08T05:51:47.451Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="http://www.fast.ai/about/">About · fast.ai</a></p><p>There are two approaches to learning a subject: Top-Down and Bottom-Up. In the bottom-up learning, you try to study every minute detail of every component of the subject and later learn to put them together. The deep learning analogy of this would be to learn all the linear algebra, probability and calculus you can and then try and understand how different CNN architectures work. Unlike this, Fast.ai employs the top-down approach. In the first lesson of the course, you’ll learn to build a <em>decent-enough</em> <a href="https://www.kaggle.com/c/dogs-vs-cats">Dogs vs. Cats</a> classifier that will put you in the top 1/5th of the leaderboard. As we move along, we’ll learn all the components of a good classifier.</p><h4>Kaggle: Plant Seedlings Classification</h4><p>In this post, we’ll try to build a classifier for <a href="https://www.kaggle.com/c/plant-seedlings-classification">Kaggle’s Plant Seedlings Classification</a> competition. While doing so, we’ll go over the main ideas covered in the first 2 lectures of the course.<br>The course uses a custom <a href="https://github.com/fastai/fastai">Fast.ai library</a> written on top of PyTorch.</p><p>The <a href="https://www.kaggle.com/c/plant-seedlings-classification/data">training dataset</a> contains labeled images of 12 species of plant seedlings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*gQEYbln1UGKeJWtaH26-CQ.png" /><figcaption>Species of plant seedlings</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b9vAu5hGJ9We0M5d1v_tHg.png" /><figcaption>Seedling images</figcaption></figure><h4>ResNet: The classifier</h4><p>We’ll be using a pre-trained model called ResNet50. Instead of building a model from scratch to solve a similar problem, you use the model trained on other problem as a starting point. ResNet50 is a version of the model trained on ImageNet data (1.2 million images and 1000 classes).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HmDynzb7bLNNbe1pIY1U2Q.png" /><figcaption>ResNet50</figcaption></figure><p>In the next sections, we’ll try to optimise some of the important parameters used in the classifier.</p><h4>Data augmentation</h4><p>One of the most important steps to build a good classifier is to give it more data to learn. This also helps in avoiding overfitting. One way to do so is to flip the images along the axes. In this case, because the images were taken from the top, it makes sense to flip them along the horizontal axis. The same goes when you’re trying to build a classifier to identify ships or ice-bergs in satellite imagery. If the image, however, were taken from the side (Dogs vs. Cats), we shouldn’t flip it vertically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*odRNDX2zcIjR_vNMQdChsg.png" /><figcaption>Data augmentation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yNH0SV_Jh2PmAAZrbTrP8Q.png" /><figcaption>Data augmentation: Do</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0BKIRalw_ZWUPRCoEiNkjQ.png" /><figcaption>Flip the cat sideways: Don’t</figcaption></figure><h4>Learning rate</h4><p>Learning rate determines how quickly (or slowly) we want to update the <em>weights</em> after each iteration. One way to pick the best learning rate is to plot the <em>loss vs. learning rate curve</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Le_4MD5q6ZHZVaIcWcccw.png" /><figcaption>Loss vs. Learning rate</figcaption></figure><p>In the graph above, as we increase the <em>learning rate </em>beyond the point between 1e-2 and 1e-3 the loss starts to increase. Because we want to minimise the loss, we should pick a point in the vicinity of the minima. We choose a point slightly less than the minima. So, for the above case, we should choose 1e-2.<br>Too high a learning rate and we risk missing the weights for which the network fits best. And, too low a learning rate, it will take a lot of time to find the best weights.</p><p>There are some other techniques to further tweak the learning rate. One such technique is <em>learning rate annealing.</em> Here, we start with higher learning rates and as we move closer to the optimal weights we start decreasing the rate.</p><p>Another variant of learning rate annealing is known as Stochastic Gradient Descent with Restarts (SGDR) as described in the paper: <a href="https://arxiv.org/abs/1506.01186">Cyclical Learning Rates for Training Neural Networks</a>. In SGDR, we “restart” i.e we make the model jump to a higher learning rate after each mini-batch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8tqZPY9_9KqYQAoxThFD4w.png" /><figcaption>An example of cosine learning rate annealing with restarts</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TgAz1qaKu_SzuRmsO-6WGQ.png" /><figcaption>SGDR</figcaption></figure><p>Another important learning rate optimisation techniques is using differential learning rates. The idea here is that for a pre-trained model like ResNet, we don’t need to fine-tune the earlier layers because they have general-purpose features. So, we’ll use different learning rates for different layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mgB4_YFzZUtw6uc6A2GaBQ.png" /><figcaption>Differential learning rates</figcaption></figure><p>With the above techniques, I was able to get to the 22nd position in the competition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*giaFhaenUKhHOykIsZyn8A.png" /><figcaption>Leaderboard</figcaption></figure><p>The Jupyter notebook containing the above code is <a href="https://nbviewer.jupyter.org/github/Autodidact24/Kaggle/blob/master/Plant%20Seedlings%20Classification/Kaggle%20-%20Plant%20Seedlings%20Classification.ipynb">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72c6ece5518d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/knowchow/fast-ai-cnn-for-image-classification-72c6ece5518d">Fast.ai | CNN for image classification</a> was originally published in <a href="https://medium.com/knowchow">Knowchow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>