<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Monish SP on Medium]]></title>
        <description><![CDATA[Stories by Monish SP on Medium]]></description>
        <link>https://medium.com/@monishsp?source=rss-d865c1cc7aed------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*bCZWGtLKYbipkhGF.</url>
            <title>Stories by Monish SP on Medium</title>
            <link>https://medium.com/@monishsp?source=rss-d865c1cc7aed------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 22:51:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@monishsp/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Applying YOLO for traffic light color detection]]></title>
            <link>https://medium.com/@monishsp/applying-yolo-for-traffic-light-color-detection-1a86111e45a?source=rss-d865c1cc7aed------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a86111e45a</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[computer-vi]]></category>
            <dc:creator><![CDATA[Monish SP]]></dc:creator>
            <pubDate>Sun, 15 Oct 2017 20:04:50 GMT</pubDate>
            <atom:updated>2017-10-19T17:34:44.784Z</atom:updated>
            <content:encoded><![CDATA[<p>As my first ever article on Medium I will be sharing my learning experience on applying <em>YOLO</em>(<strong>Y</strong>ou <strong>O</strong>nly <strong>L</strong>ook <strong>O</strong>nce) for traffic light color detection as part of the final capstone for <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Udacity’s Self Driving Car Nanodegree</a> program. <em>YOLO</em> is a state of the art, real-time object detection model. The architecture is really fast and is able to process images at 45 frames a second. Let’s start with an overview!</p><blockquote><strong><em>NOTE</em></strong><em>: A lot of the information I am going to share in this article is from the published paper by the authors which can be found </em><a href="https://arxiv.org/abs/1506.02640"><em>here</em></a></blockquote><h3>Overview</h3><p>Other modern detection models like <em>DPM</em>(Deformable Parts Model) or <em>R-CNN</em> re-purpose classifiers to perform detection. These systems use a classifier for an object and evaluate it at various locations and scales on a test image. This makes the models to have complex pipelines which are slow and hard to optimize.</p><p>The authors of <em>YOLO</em> re-framed the object detection problem as a single regression, straight from single regression to bounding boxes and object class probabilities. The architecture used is a single convolutional network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3QhPv1ZNnRtnfPTE.png" /><figcaption><strong><em>YOLO Architecture</em></strong><em>: Simultaneously predicts bounding boxes and class probabilities for these boxes</em></figcaption></figure><p>The network has 24 convolutional layers followed by 2 fully connected layers. Instead of inception modules(used by GoogLeNet), 1 × 1 reduction layers is used followed by 3 × 3 convolutional layers. <em>Fast YOLO</em> uses a neural network with fewer convolutional layers (9 instead of 24) and fewer filters in those layers. Other than the size of the network, all training and testing parameters are the same between <em>YOLO</em> and <em>Fast YOLO</em>.</p><p><em>YOLO</em> trains on full images and directly optimizes inference performance, making it a unified model. This has the following three benefits:</p><ol><li>It’s extremely fast</li><li>Global reasoning about the image when making predictions</li><li>Model learns generalizable representation of objects</li></ol><h3>How does it work?</h3><p><em>YOLO</em> takes an entire image, splits it into a ‘<em>S</em>x<em>S’</em> grid. Each grid cell predicts ‘<em>B’</em> bounding boxes and confidence scores, Pr(Object) ∗ IOU(truth/pred), for those boxes. These confidence scores reflects the confidence that the box contains an object and how accurate it thinks the box is that it predicts.</p><p>Each grid cell also predicts ‘<em>C’ </em>conditional class probabilities, Pr(Class_i|Object). After combining box and class predictions, Pr(Class_i|Object) ∗ Pr(Object) ∗ IOU(truth/pred) = Pr(Class_i) ∗ IOU(truth/pred), NMS(Non-Maximum Suppression) and thresholding is applied to make predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*mmUHzD_vxDJn5Rfc.gif" /><figcaption><strong><em>YOLO prediction process</em></strong></figcaption></figure><blockquote><em>The official slideshow used for the above .gif can be found </em><a href="https://goo.gl/uc4BeA"><em>here</em></a></blockquote><h3>Udacity’s Self Driving Car Nanodegree Capstone</h3><p>I recently completed Udacity’s Self Driving Car nanodegree program which was nearly 12 months long. The topics and projects were focused on Deep Learning, Computer Vision, Sensor Fusion, Localization, Controllers and Path Planning(will miss the learning 😢). The goal of this project was to use 3 aspects — Perception, Path Planning and Controls to make a simulated car drive autonomously around a simulated track. Then, also on a real track on a real car obeying speed limits and traffic light rules.</p><p>Applying <em>YOLO</em> for traffic light color perception made a lot sense for me and so I did 😄. I was pretty happy and impressed with the results!</p><h3>YOLO using Tensorflow</h3><p>One of the authors of the paper has created an excellent website which has details about the open source framework <a href="https://pjreddie.com/darknet/">darknet</a>, written in C and CUDA, and how to use it. That website lead me to <a href="https://github.com/thtrieu/darkflow">d</a>arkflow which converts darknet to tensorflow framework. The darkflow <a href="https://github.com/thtrieu/darkflow">repo</a> has in-depth details on how to download trained weights and train <em>YOLO</em> on new training data using the tensorflow framework.</p><h3>Applying YOLO model using Darkflow</h3><p>After understanding the <em>YOLO</em> architecture and the darkflow framework, it was about putting all the pieces together and getting things working step by step:</p><h4>Creating annotations for training</h4><p>This step was perhaps the most important and also the most frustrating part of training the <em>YOLO</em> model because I had to:</p><ol><li>Create annotations manually image by image</li><li>Make sure the layout of data folder expected by darkflow was correct(details here in the <a href="https://github.com/thtrieu/darkflow">repo</a>)</li></ol><p>I created annotations for the images using a python based tool, <a href="http://sloth.readthedocs.io/en/latest/installation.html">Sloth</a>. The annotations outputted by Sloth are stored in a .json file. To have the correct data layout for training, I developed a <a href="https://github.com/spgitmonish/SlothAnnotations/blob/master/annotationsinxml.py">custom python script</a>, which laid out the data in the expected format.</p><h4>Training on new data</h4><p>As mentioned by the author in the darkflow <a href="https://github.com/thtrieu/darkflow">repo</a>, to run training on new data, one needs to create a new .cfg and modify the file for the number of classes in the dataset. Then modify the labels.txt file with the dataset&#39;s label information. Once setup, all one needs to do is setup the command to specify the model to use, labels, configuration, dataset &amp; whether to use GPU for training.</p><p>During my training the loss started out pretty high but after even 100 steps, the training and validation loss was less 3%. I trained my model on an AWS g3.8x instance for 30,000 steps(it’s not cheap, so make sure you understand the costs!)</p><blockquote><strong><em>NOTE</em></strong><em>: I ended up using </em>Tiny YOLO<em> even though the accuracy of that model is slightly lower. The dataset I used for training and testing consisted of images and annotations of the traffic lights in Udacity’s </em><a href="https://github.com/udacity/CarND-Capstone/releases/tag/v1.2"><em>simulator</em></a><em> and of the traffic light from Udacity’s </em><a href="https://drive.google.com/file/d/0B2_h37bMVw3iYkdJTlRSUlJIamM/view?usp=sharing"><em>real test track</em></a><em>.</em></blockquote><h4>Results</h4><p>After observing the training and validation loss decay during training, I was really happy with the results and excited to apply my custom trained model for testing. Here are the results of applying <em>YOLO</em> on simulated and real track image data.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FHBJ4aUG0XDI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DHBJ4aUG0XDI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FHBJ4aUG0XDI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/dcfc9e67f776d2710a167f06dc1a6d04/href">https://medium.com/media/dcfc9e67f776d2710a167f06dc1a6d04/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FB0Tqr4EyKH8%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DB0Tqr4EyKH8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FB0Tqr4EyKH8%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d74642e4b92cbd7b76ff9234e4640eab/href">https://medium.com/media/d74642e4b92cbd7b76ff9234e4640eab/href</a></iframe><h3>Limitations of YOLO</h3><p><em>YOLO</em> is a fast and simple model but it does have it’s own limitations:</p><ol><li>The model imposes a spatial constraint which limits the number of nearby objects the model can predict</li><li>Model struggles with small objects which appear in groups, like flocks of birds</li><li>Since annotations are used during training, the model struggles to generalize objects in new or unusual aspect ratios or configurations</li><li>During training, the loss function approximates detection performance, the loss function treats errors the same in small bounding boxes versus large bounding boxes</li></ol><h3>YOLO v2</h3><p>Two of the authors of the original paper implemented a better and faster version of <em>YOLO</em>. The authors reported that adding things like <em>Batch Normalization</em> and using bigger resolutions of images lead to noticeable improvement of <em>mean average precision(mAP)</em>. In-depth details, discussion and results can be found in their <a href="https://pjreddie.com/media/files/papers/YOLO9000.pdf">paper</a>.</p><p>Hopefully this is a useful article and encourages other people to apply <em>YOLO </em>for their own project! The final capstone project repo from my team can be found <a href="https://github.com/d13sl0w/CarND-Capstone">here</a>. Thank you for reading 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a86111e45a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>