<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sashank on Medium]]></title>
        <description><![CDATA[Stories by Sashank on Medium]]></description>
        <link>https://medium.com/@sashankaryal?source=rss-d329802981d1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*pqybkqLKkKlsIjZQVb8ufg.png</url>
            <title>Stories by Sashank on Medium</title>
            <link>https://medium.com/@sashankaryal?source=rss-d329802981d1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:25:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sashankaryal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Traffic Signs Classification: Udacity Self-Driving Cars]]></title>
            <link>https://medium.com/@sashankaryal/traffic-signs-classification-udacity-self-driving-cars-ad1e0ad5fcb2?source=rss-d329802981d1------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad1e0ad5fcb2</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sashank]]></dc:creator>
            <pubDate>Sat, 27 Jan 2018 23:20:42 GMT</pubDate>
            <atom:updated>2018-01-28T03:32:46.823Z</atom:updated>
            <content:encoded><![CDATA[<p>This project involves training a deep neural network to recognize traffic signs, which means creating a function approximator that can map the pixels of traffic sign pictures to their names. The deep learning approach turned out to be really powerful, yielding a 96% test accuracy. This article is a documentation of how I built the pipeline as well as details of the final architecture. Please <a href="https://github.com/sashankaryal/Traffic-signs-classifier">click here to go to the github repo</a>.</p><h3>Motivation</h3><blockquote>Traffic sign recognition is a multi-class classication problem with unbalanced class frequencies. Traffic signs can provide a wide range of variations between classes in terms of color, shape, and the presence of pictograms or text. However, there exist subsets of classes (e. g., speed limit signs) that are very similar to each other.</blockquote><blockquote>The classier has to cope with large variations in visual appearances due to illumination changes, partial occlusions, rotations, weather conditions, etc.</blockquote><blockquote>Humans are capable of recognizing the large variety of existing road signs with close to 100% correctness. This does not only apply to real-world driving, which provides both context and multiple views of a single traffic sign, but also to the recognition from single images.</blockquote><blockquote>- <a href="http://benchmark.ini.rub.de/">http://benchmark.ini.rub.de/</a></blockquote><h3>Dataset summary</h3><p>The GTSRB dataset was an interesting one, particularly because of its imbalance in the number of examples per class. Summary of the data I used for this project:</p><pre>Number of training examples = 34799<br>Number of testing examples = 12630<br>Number of validation examples = 4410<br>Image data shape = (32, 32, 3)<br>Number of classes = 43</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-lejb1e5eyhrjW5dT5A5cw.png" /><figcaption>Distribution of the training, testing and validation data</figcaption></figure><p><strong>Observations:<br></strong>1. The distribution of training, test and validation data is almost identical. That is good.<br>2. The distribution is not uniform as we would have wanted it ideally. Some classes have way more examples than others.</p><h3>Exploratory Analysis</h3><p>First let’s print random images of all classes to get a gist of what data we are working with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*mGhD_KuxaaD03nigScTKag.png" /><figcaption>16 examples, 43 classes</figcaption></figure><p><strong>Observations:<br></strong>Illumination variance. The contrast is very low for most of the images. If we picture the distribution of the intensity of most of these pictures, we would understand two things. First, it is very narrow, which means low contrast. Second, it is either on the far left (very dark images) or on the far right (very bright images) in the spectrum. We can enhance the contrast using a contrast enhancement technique. One of those techniques I’m going to use is <strong>histogram equalization.</strong></p><h4>Vanilla Histogram Equalization</h4><p>This technique is exactly what we need. We need to distribute the frequency of pixels to roughly all intensity values in our image. But there is a problem! Our images are in RGB color space, which doesn’t have any notion of ‘intensity’. The solution is to convert the image from RGB color space to a color space which has a notion of intensity. It could be HSL, HSV, YUV, etc. I’m going with YCbCr for no special reason. After converting the images into the YCbCr color space, we perform histogram equalization on the ‘Y’ plane. In YCrCb color space, ‘Y’ means luminance which is the intensity metric, and Cr and Cb mean red-difference and blue-difference chroma components respectively. From Wikipedia -</p><blockquote><em>Histogram Equalization usually increases the global contrast of many images, especially when the usable data of the image is represented by close contrast values. Through this adjustment, the intensities can be better distributed on the histogram. This allows for areas of lower local contrast to gain a higher contrast. Histogram equalization accomplishes this by effectively spreading out the most frequent intensity values.</em></blockquote><p>Let’s apply histogram equalization to 16 random images of 5 random classes and print them. The steps we take are:</p><ol><li>Convert RGB to YCrCb color space. I tried applying the equalization directly by flattening the RGB channels, but got bad results. It is because RGB is not an ‘intensity model’. Also, channel splitting and equalizing each channels doesn’t work since histogram equalization is a non-linear process that is designed to operate on intensity values.</li><li>Equalize the ‘Y’ channel and leave ‘Cr’ and ‘Cb’ channels intact.</li><li>Merge the channels.</li><li>Convert the image from YcrCb to RGB and return it.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*Ee85E8m7VXbPji8FRvfLhQ.png" /><figcaption>Vanilla Histogram Equalization Applied</figcaption></figure><p>There is a notable improvement in the contrast of the images, but as we can see, there are a lot of rather unwanted artifacts in the image. We will now use another technique called <em>Contrast Limiting Adaptive Histogram Equalization </em>and see how it works.</p><p>From Wikipedia and OpenCV docs,</p><blockquote><em>Adaptive histogram equalization (AHE) is a computer image processing technique used to improve contrast in images. It differs from ordinary histogram equalization in the respect that the adaptive method computes several histograms, each corresponding to a distinct section of the image, and uses them to redistribute the lightness values of the image. It is therefore suitable for improving the local contrast and enhancing the definitions of edges in each region of an image.</em></blockquote><blockquote><em>However, if noise is there, it will be amplified. To avoid this, contrast limiting is applied. If any histogram bin is above the specified contrast limit (by default 40 in OpenCV), those pixels are clipped and distributed uniformly to other bins before applying histogram equalization. After equalization, to remove artifacts in tile borders, bilinear interpolation is applied.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*Rd_b8Hho4QnFQhf36oV_DQ.png" /><figcaption>Contrast Limiting Adaptive Histogram Equalization with Bilinear Interpolation</figcaption></figure><p>This is much better, and the artifacts are much less severe, also to be attributed to the bilinear interpolation operation applied post-processing. Note that you can tweak the clipLimit and tileSize parameters for different results. I got the best results with clipLimit = 2 and tileSize = 4x4 .</p><p><strong>Color vs no color</strong><br>I tried training the model both with color and without color. Surprisingly, I got better validation accuracy without color. When I say “with color”, the images fed into the neural net were in RGB color space. When I say “without color”, the images fed into the neural net were in greyscale mode. Note that the Y channel of YUV or YCrCb is equivalent to the greyscale mode, so that’s what I used.</p><p><strong>One pre-processing pipeline:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f114bb8764c3f5bf21f431e2179addd/href">https://medium.com/media/0f114bb8764c3f5bf21f431e2179addd/href</a></iframe><p>I then apply this pipeline for all training, validation and test datasets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*liOO0UJZGFjPTRY-x5zWWQ.png" /><figcaption>Examples of preprocessed training images</figcaption></figure><h3>Data Augmentation</h3><p>We found out that the distribution of the data is very uneven, so there are two ways we can address this situation:<br>1. Balance dataset to the mean.<br>2. Perform augmentation on the whole dataset.</p><p>Since more data is never going to hurt, I decided to perform augmentations on the whole dataset. I perform 5 augmentations, described below:</p><ol><li>Gaussian blur with sigma randomly selected between 0.25 and 1</li><li>X and Y scaling. Scale factor randomly chosen between 1.3 and 1.5</li><li>Rotation, random between -20 degrees and -5 degrees</li><li>Rotation, random between -5 degrees and -20 degrees</li><li>Shearing by a factor of 15</li></ol><p>I also experimented with a number of other augmentation methods but did not get good results with them. Note that you might be very tempted to use mirroring or any kind of flipping as an augmentation procedure in this project. I was. <strong>Don’t! </strong>It would make sense if it were cat pictures, but since we are dealing with digits and arrows, our data doesn’t have reflection invariance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*8lDxMNUINOeSryGk_dcEVA.png" /><figcaption>Example of data augmentation on 5 random images</figcaption></figure><p>With augmentation, we now have 208795 training examples, 5 times the original dataset!</p><h3>Architecture design</h3><p>Before we go any further into designing our architecture, we should establish a baseline. This is something that is very easy to overlook for early machine learning enthusiasts. Without establishing the baseline, you do not have an idea of what you are working towards. After establishing the baseline, we know our solution has to perform at least as good as it.</p><p><strong>Single Objective Metric:<br></strong>I used Udacity’s given suggestion for single objective metric, accuracy greater than 93% on the validation set. I will later talk about why this was a bad idea.</p><p>My final model consists of 2 convolutional layers with max pooling, and three fully connected hidden layers. I started with vanilla LeNet, simply because of its simplicity, but took an iterative approach and finished off with the following configuration.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5a4cf6fc6aebdaf0d0e42f41737bb52/href">https://medium.com/media/f5a4cf6fc6aebdaf0d0e42f41737bb52/href</a></iframe><p>Parameters that I mostly played with were:</p><ol><li>Cliplimit and tilesize for contrast limiting adaptive histogram equalization, I tried a wide range of values and found the best results when I set both cliplimit and tile size to be 2 and 4x4 respectively</li><li>Image augmentation types. I found success with gaussian blur, scaling, and 3 different rotations. Instead of hardcoding the values for them, I defined a range to add variety. For example, I chose a sigma between 0.25 and 1 for Gaussian Blur.</li><li>Number of epochs, although I observed that a good model would show promising validation accuracy results in the first 10 epochs.</li><li>Dropout probability. I found success in 0.6 and even 0.7 over 0.5. The reason could be that max pooling after convolution results in loss of information, leading to underfitting.</li></ol><p><strong>Hyperparameters</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97402c71f93ec948a140be9d8027a06a/href">https://medium.com/media/97402c71f93ec948a140be9d8027a06a/href</a></iframe><p>I could have done more aggressive hyperparameter tuning. With enough computing power, I would take the Caviar approach and parallel train a lot of models instead of babysitting one model for a long time.</p><p><strong>Adam optimizer</strong></p><p>I used Adam optimization method over simple gradient descent, the ‘momentum’ in Adam optimizer provides much faster convergence. It also means that I do not need to actively tune the learning rate.</p><h3>Performance</h3><p>I trained my model on AWS g2.2xlarge ec2 instance. It took about five minutes.</p><pre>EPOCH 1 ...<br>Validation Accuracy = 0.893<br><br>EPOCH 2 ...<br>Validation Accuracy = 0.938<br><br>EPOCH 3 ...<br>Validation Accuracy = 0.947<br><br>EPOCH 4 ...<br>Validation Accuracy = 0.953<br><br>EPOCH 5 ...<br>Validation Accuracy = 0.955</pre><pre>...<br>...<br>EPOCH 36 ...<br>Validation Accuracy = 0.982<br><br>EPOCH 37 ...<br>Validation Accuracy = 0.977<br><br>EPOCH 38 ...<br>Validation Accuracy = 0.979<br><br>EPOCH 39 ...<br>Validation Accuracy = 0.979<br><br>EPOCH 40 ...<br>Validation Accuracy = 0.978<br><br>Model saved</pre><p>I took an iterative approach and tried a lot of configurations. I altered types and numbers of augmentations, played with color spaces, and tinkered with the network architecture for a long time. After having achieved a good validation accuracy of about 98% (meets the baseline set earlier), I called it done and was ready to test my model!</p><h3>Testing the model</h3><p>This code has been run only one time after the model was developed to find the test accuracy.</p><pre>with tf.Session() as sess:<br>    saver.restore(sess, tf.train.latest_checkpoint(&#39;.&#39;))<br>    test_accuracy = evaluate(X_test_normalized, y_test)<br>    print(&quot;Test Accuracy = {:.3f}&quot;.format(test_accuracy))</pre><pre>Test Accuracy = 0.960</pre><p>That’s it! Test accuracy of 96.0%, much better than I expected. It means the model was not too overfitted and not too underfitted.</p><h3>Testing the model on random pictures from the web</h3><p>I Googled ‘german traffic signs’ and downloaded five signs. I used OpenCV to resize every image to 32x32x3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*l_tK3v7A-4f5O7aO_9qq0Q.png" /></figure><p>Using the pipeline that we developed earlier, I pre-processed the images:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*cOJM-MPTJrynHfuLX7Luiw.png" /></figure><p>I then output the softmax probabilities for every image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*N14hS6idsPBIsDHP98ER2A.png" /></figure><pre>Best 5 guesses:<br>No entry : 99.999%<br>Turn left ahead : 0.001%<br>Stop : 0.001%<br>Keep right : 0.000%<br>Keep left : 0.000%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*g8AB7aiRLP-OVMJ0U7trHw.png" /></figure><pre>Best 5 guesses:<br>Road narrows on the right : 52.745%<br>Children crossing : 42.357%<br>Beware of ice/snow : 2.129%<br>Bicycles crossing : 0.748%<br>Pedestrians : 0.718%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*HjPpnjRC-2AEudrtxWCbVA.png" /></figure><pre>Best 5 guesses:<br>Priority road : 99.999%<br>Yield : 0.001%<br>No passing : 0.000%<br>No vehicles : 0.000%<br>Speed limit (60km/h) : 0.000%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*z-VAup2X1pHPBVdUSTMrJQ.png" /></figure><pre>Best 5 guesses:<br>Yield : 100.000%<br>Priority road : 0.000%<br>Keep left : 0.000%<br>Ahead only : 0.000%<br>Keep right : 0.000%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/271/1*KNdSXtwMYvfYgIJingfWqg.png" /></figure><pre>Best 5 guesses:<br>Slippery road : 99.998%<br>Wild animals crossing : 0.002%<br>Double curve : 0.000%<br>Beware of ice/snow : 0.000%<br>Right-of-way at the next intersection : 0.000%</pre><h4>Reflection:</h4><ol><li>If we calculate the test accuracy for the new 5 images from the web, it is merely 80%. Compare that to the 96% accuracy we got on the test dataset. What is going on here?</li><li>Accuracy is not a good metric for imbalanced datasets, since it is very likely that the prediction performance of smaller classes will be shadowed by performance of any bigger class. One could use, for instance, Kullback-Leibler divergence to measure how the distribution of the data compares to uniform probability distribution and decide if accuracy is the correct metric.</li><li>The ‘Children Crossing’ image that we tested with our classifier was underrepresented in the training data (as we can see in the bar plot). Establishing the baseline with a different evaluation metric like F-1 score would have been a better choice. Then we would be optimizing for something which we would yield better precision and recall.</li></ol><h3>Conclusion</h3><p>This was a really great hands-on opportunity to learn about deep convolutional nets using tensorflow as well as the empirical nature of making deep learning models. Possible improvements to this model could result from:</p><ol><li>Hyperparameter tuning. I did not really experiment with a wide range of values for most hyperparameters. With enough computing power, I would love to take the Caviar approach and parallel train a lot of models instead of babysitting one model for a long time.</li><li>Different network architectures. I confined myself to tiny random variations of LeNet.</li><li>Remove pooling layers altogether and instead use dropout with keep_prob = ~0.9in the convolutional layers for a different regularizing effect.</li><li>Add batch normalization.</li><li>Even out the distribution of training data by even more careful augmentation. This would have justified our use of accuracy as the single objective metric.</li></ol><p>Link to the github repo:<br><a href="https://github.com/sashankaryal/Traffic-signs-classifier">https://github.com/sashankaryal/Traffic-signs-classifier</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad1e0ad5fcb2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>