<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akhil Suri on Medium]]></title>
        <description><![CDATA[Stories by Akhil Suri on Medium]]></description>
        <link>https://medium.com/@akhilsuri194?source=rss-c3044232e228------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*nfuPam4azK39qVOjnADomQ.jpeg</url>
            <title>Stories by Akhil Suri on Medium</title>
            <link>https://medium.com/@akhilsuri194?source=rss-c3044232e228------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 12:25:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@akhilsuri194/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 4.3:]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-3-2a506e57d830?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a506e57d830</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[autonomous-vehicles]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Sun, 17 Jun 2018 18:47:18 GMT</pubDate>
            <atom:updated>2018-06-17T18:52:04.060Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building an Autonomous Vehicle Part 4.3: Sensor Fusion and Object Tracking using Unscented Kalman Filters</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NNWlF7Qu0PxoQqq7Z-Qw7w.jpeg" /><figcaption>Taken from somewhere in the internet :)</figcaption></figure><p>This post is in continuation to my last two posts on <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-1-sensor-fusion-and-object-tracking-using-kalman-filters-66589e522965">Kalman Filter</a> and <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-2-8049cda5a92">Extended Kalman Filters</a>. Today we will look at another member of Kalman Filter Family: <strong>The Unscented Kalman Filter.</strong></p><h4>Short Summary:</h4><p><strong>Kalman Filter:</strong> It is a tool to predict values using a bunch of mathematical equations under the assumptions that our data is in the form of Gaussian Distribution and we apply linear equations to that Gaussian distribution.</p><p><strong>Extended Kalman Filter</strong>: In real world, we have non linear equations, because we may be predicting in one direction while our sensor is taking reading in some other direction, so it involves angles and sine cosine functions which are non linear. So EKF takes helps of Taylor Series (and Jacobian Matrix further) to linearly approximate a non linear function around the mean of the Gaussian and then predict the values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*CkHS43j3YAywDcO1Ld3O5w.png" /><figcaption>Liner approximation of a non-linear Gaussian distribution using EKF.</figcaption></figure><p>In EKF, we know that the distribution we get after nonlinear transformation is not a Gaussian. Nevertheless, we pretend it as a Gaussian and try to find the best approximation of real distribution. To do that we take the mean of the original Non-Linear Gaussian and pass it through a Non Linear Function to predict the best approximate Gaussian. Since here we approximate around only one point i.e. mean, the Gaussian we get doesn’t give us a good approximation.</p><p>️<strong>Unscented Kalman Filter (UKF)</strong> proposes a solution ✔ to this problem️. In UKF we have a concept of Sigma Points. Here we take some points(known as Sigma Points) on source Gaussian and map them on the target Gaussian after passing those points through some non linear function and then we calculate the new mean and variance of transformed Gaussian.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Q0pBymuFdjfWaRsyN2Jkgg.png" /><figcaption>Sigma points method</figcaption></figure><p><strong>Now you must be wondering why are we not considering all points instead we are taking some random points and transforming them to predict the Gaussian?</strong> 🤔</p><p><em>Good Question</em>… 😋<br>Actually we can also use all points in Original non-linear Gaussian and pass them through non linear function to get the best approximate Linear Gaussian but that would consume a lot of computation power and time 😓 and hence can not be used in real time. Also sigma points method gives us the result pretty close to the expected one 😜.</p><p>Now since we have successfully predicted the position of an object in the surrounding, from here onward we can use Kalman Filter Update equation to update the existing belief of the object’s position and the process of predict and update will continue as discussed in my <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-1-sensor-fusion-and-object-tracking-using-kalman-filters-66589e522965">first blog</a> on Kalman Filters.</p><h4>Conclusion and Discussion: 😎</h4><p>Here the main difference from EKF is that in EKF we take only one point i.e. mean and approximate, but in UKF we take a bunch of points called sigma points and approximate with a fact that more the number of points, more precise our approximation will be!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*AJa5Y_cn-WdrQpvGHquf1g.png" /></figure><p>In addition to sigma points, these points also have weights, so these are <strong>weighted sigma points</strong>. The points near the mean will have more weight the the points away from the mean.</p><p>When a Gaussian is passed through a non linear function, it does not remains a Gaussian anymore but we approximate the Gaussian from the resulting figure, so in UKF a process called <a href="https://en.wikipedia.org/wiki/Unscented_transform"><strong>Unscented Transform</strong></a> helps us to perform this task.</p><p>To summarize here are the below steps the unscented transform performs:<br>1. Compute Set of Sigma Points<br>2. Assign Weights to each sigma point<br>3. Transform the points through non linear function<br>4. Compute Gaussian from weighted and transformed points<br>5. Compute Mean and Variance of the new Gaussian.</p><p>Well, that’s all folks 😋. I have tried to explain the basic overview of the Kalman Filter Family in a very simple manner 😝. Hope you gained something after reading the Posts 😉. If you want to get deeper 🕵️ into mathematical part of the Extended Kalman Filters then I would suggest you to go through <a href="https://www.youtube.com/watch?v=CaCcOwJPytQ&amp;list=PLX2gX-ftPVXU3oUFNATxGXY90AULiqnWT"><strong>these</strong></a><strong> </strong>amazing video lectures series on Kalman Filters by <a href="https://www.youtube.com/channel/UCiGxYawhEp4QyFcX0R60YdQ">Michel van Biezen</a> 👦.</p><p>You can find my implementation on UKF and other <strong>Self-Driving Car</strong> related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Unscented-Kalman-Filter-P7"><em>here</em></a><em> </em>👻<em>.</em> Please feel free to provide me any suggestions, corrections or additions in the comments :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a506e57d830" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 4.2:]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-2-8049cda5a92?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/8049cda5a92</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[sensor-fusion]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Sun, 17 Jun 2018 07:11:59 GMT</pubDate>
            <atom:updated>2018-06-19T08:49:06.734Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building an Autonomous Vehicle Part 4.2: Sensor Fusion and Object Tracking using Extended Kalman Filters</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*GqOI4UpnIrP1L9ZxpUfpIQ.jpeg" /><figcaption>Taken from somewhere in the internet :)</figcaption></figure><p>This post is in continuation to my last post on <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-1-sensor-fusion-and-object-tracking-using-kalman-filters-66589e522965"><strong>Kalman Filter</strong></a><strong>. </strong>In that post we talked about improving the accuracy of the position estimates of other objects in surroundings by combining different sensor measurements. But you must have wondered that where exactly are we combining the data of these sensor measurements?</p><p>Unfortunately, combining data from different sensors is not easy. It’s not as straightforward for the <a href="https://en.wikipedia.org/wiki/Radar_detector">Radar</a> sensor as it gives readings in Polar coordinates and the <a href="https://en.wikipedia.org/wiki/Lidar">Lidar</a> gives data in Cartesian coordinates.</p><p>Before moving forward, let me tell you the difference b/w Cartesian and Polar coordinates system.</p><p><strong>Cartesian Coordinates System:<br></strong> The Cartesian coordinate system is a two-dimensional coordinate system using a <em>rectilinear grid. </em>Using Cartesian Coordinates we mark a point by how far along and how far up it is from origin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*kXlh2gC68XMf0R_Mi5zQQw.png" /><figcaption>Source: (<a href="https://www.mathsisfun.com/polar-cartesian-coordinates.html">Click Here</a>)</figcaption></figure><p><strong>Polar Coordinates System:<br></strong>The polar coordinate system is a two-dimensional coordinate system using a polar grid. Using Polar Coordinates we mark a point by how far away, and what angle it is making to origin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*ChLSyCH3amJGBvc5HAUzZQ.png" /><figcaption>Source: (<a href="https://www.mathsisfun.com/polar-cartesian-coordinates.html">Click Here</a>)</figcaption></figure><p>Our mathematical formulas which we are using in Kalman Filters are all implemented with linear functions of type y = ax + b where the constant ‘a’ is called slope and ‘b’ is called y-intercept. Below in the sample graph of a liner function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*69Z7NRZZuZHvKtPQ1EshUg.gif" /><figcaption>Linear Function: (<a href="http://multidict.net/clilstore/page.php?id=4912">Source</a>)</figcaption></figure><p>A Kalman filter always works with linear functions. But most of the real world problems involve non linear functions. In most cases, the system is looking into some direction and taking measurement in another direction. This involves angles and sine, cosine functions which are non linear functions which then lead to problems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*ECwrOjbA6oOOjO44qQIjZA.gif" /><figcaption>Non Linear Function: (<a href="https://www.varsitytutors.com/hotmath/hotmath_help/topics/graphing-sine-function">Source</a>)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*Sxy82YKE9BTqnlha-3_ORw.png" /><figcaption>Source: Udacity SDC Lecture Videos</figcaption></figure><h4>As I already mentioned that the radar sees the world in polar coordinates. It gives us three measures:</h4><ul><li><strong>ρ (rho)</strong>: The distance to the object that is tracked.</li><li><strong>φ (phi)</strong>: The angle between the x-axis and our object.</li><li><strong>ρ ̇ (rhodot)</strong>: The change of ρ, resulting in a radial velocity.</li></ul><p>These three values make our measurement a nonlinear given the inclusion of the angle <strong>φ</strong>. If we enter non-linear data in a Kalman filter, our result is no longer in uni-modal Gaussian form and we can no longer estimate position and velocity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*2OkLAEJk6yjQ72TwGREl0w.png" /><figcaption>Source: Udacity SDC Lecture Videos</figcaption></figure><p>We can see in above image that the distribution created from the non-linear input is not normal distribution. So our goal here is to convert the data <strong>ρ, φ, ρ</strong> ̇ to Cartesian data (<strong>px</strong>,<strong> py</strong>,<strong> vx</strong>,<strong> vy</strong>).</p><p><strong>Extended Kalman Filter (EKF)</strong> proposes a solution to this problem. The EKF use Taylor expansion to construct a linear approximation of nonlinear function h(x):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*lpe6PDDK13u8St8a1qJJMw.png" /><figcaption>Gaussian distribution after applying a first order Taylor expansion</figcaption></figure><p>We take a point and perform a bunch of derivatives on that point. In case of an EKF, we take mean of the Gaussian on the Non Linear Curve and perform a number of derivatives to approximate it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*sUYr0CzwQOJQMN6mTEkotA.png" /><figcaption>Taylor Series</figcaption></figure><p>For every non linear function, we just draw a tangent around the mean and try to approximate the function linearly. Then we use that approximation as an input in our Kalman Filters equations to improve the accuracy of our position estimates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*KBM6G7eqUl5xlg9ScFBWng.jpeg" /></figure><p>Though this was a brief overview and may be a little boring 😝but if you want to get deeper 🕵️ into mathematical part of the Extended Kalman Filters then I would suggest you to go through <a href="https://www.youtube.com/watch?v=CaCcOwJPytQ&amp;list=PLX2gX-ftPVXU3oUFNATxGXY90AULiqnWT"><strong>these</strong></a><strong> </strong>amazing video lectures series on Kalman Filters by <a href="https://www.youtube.com/channel/UCiGxYawhEp4QyFcX0R60YdQ">Michel van Biezen</a> 👦.</p><p>You can find my implementation of Extended Kalman Filters and other <strong>Self-Driving Car </strong>related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Extended-Kalman-Filter-P6"><em>here</em></a><em> </em>👻. Please feel free to provide me any suggestions, corrections or additions in the comments :)</p><p><em>To be continued…. Part 4.3 — Unscented Kalman Filters coming soon </em>🤞</p><p>Edit-1: Read about Unscented Kalman Filters(Part 4.3) <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-3-2a506e57d830">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8049cda5a92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 4.1: Sensor Fusion and Object Tracking using Kalman Filters]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-1-sensor-fusion-and-object-tracking-using-kalman-filters-66589e522965?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/66589e522965</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Tue, 15 May 2018 17:50:22 GMT</pubDate>
            <atom:updated>2018-06-17T18:48:58.400Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*mQSvkJXZq0MvPdrA6aoXag.jpeg" /><figcaption>Image Source: (<a href="http://www.nvidia.com/object/autonomous-cars.html">Click Here</a>)</figcaption></figure><p>A self-driving car needs a map of the world around it as it drives. It must be able to track car, bikes, trucks, pedestrians and other moving objects on the road continuously and to track these movements we take continuous and dynamic measurements from various sensors like <a href="https://en.wikipedia.org/wiki/Radar">Radars</a>, <a href="https://en.wikipedia.org/wiki/Lidar">Lidars</a> etc. But there is usually a challenge that sensor measurements are uncertain due to reason which includes, but not are limited to, errors from a sensor, discrepant measurement from multiple sensors. So to get the accurate measurements by reducing the error we use an algorithm knows as <a href="https://en.wikipedia.org/wiki/Kalman_filter"><strong><em>Kalman Filters</em></strong></a>.</p><blockquote><em>Before moving forward on Kalman Filters lets read about some sensors that we use in self-driving cars :)</em></blockquote><p><strong>LIDAR</strong> uses a laser for measurement and generates a point cloud of the world around it providing the car with fairly accurate position x and position y values. It is able to detect objects in the vicinity of the car (20–40 m) with very high accuracy. However, LIDAR is not very accurate in poor weather conditions or if the sensor gets dirty. A LIDAR cloud looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*71J6b0FcgmPRj951riYY5g.jpeg" /><figcaption>Source: (<a href="https://www.pcworld.com/article/2043878/driverless-cars-yield-to-reality-its-a-long-road-ahead.html">Click Here</a>)</figcaption></figure><p>The <strong>RADAR, </strong>on the other hand, is less accurate but is able to provide an estimate of both position and velocity of an object. The velocity is estimated using the Doppler effect as seen in the image below. RADAR is able to detect objects up to 200 m from the car. It is also less impervious to weather conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*QURUCqh71qEOBMs5Q8d5Sw.jpeg" /><figcaption>Source: (<a href="https://www.crutchfield.com/S-CSAKSYfxewh/learn/learningcenter/car/radar.html">Click Here</a>)</figcaption></figure><p>Now you have an idea of what these sensors are, we can move on to Kalman Filters :)</p><blockquote>Kalman Filters, also known as linear quadratic estimation (LQE), is an algorithm that helps us to obtain more reliable estimates from sequence of observed measurements(sensor measurements). 😴</blockquote><p>It can be used to track the position and velocity of a moving pedestrian over time and also measure the uncertainty associated with them. It is basically a two-step iterative process.</p><ol><li>Predict 🤔</li><li>Update ✍️</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S5_Kdpz5Uf5bWpkpXv3kaw.png" /><figcaption>Source: (<a href="https://en.wikipedia.org/wiki/Kalman_filter">Click Here</a>)</figcaption></figure><p>In Predict step, we predict the new position of a pedestrian 🚶based on the previous position and assuming they are moving with a constant velocity. Also, we predict the uncertainty/error/variance in our prediction according to the process noise present in the system.</p><p>In Update step, we take into account the actual measurements 📐 coming from the sensors to update our estimates. To do that we first calculate the difference between our predicted value and measured value and then we decide which value to keep by calculating the Kalman Gain. We then calculate the new value(new belief/position) and new uncertainty/error/variance based on our decision made by Kalman Gain. These calculated values will finally be the output of our Kalman Filter and will be used as an input to prediction step for next iteration.</p><p><em>Now you must be wondering what is Kalman Gain? 😳</em></p><p>Actually, <strong>Kalman Gain</strong> is a parameter which decides how much weight should be given to predicted value and the measured value. It checks the uncertainty in both predicted value and measured value and then it decides whether our actual value is close to predicted value or measured value.</p><blockquote><em>K = Error In Prediction / (Error in Prediction + Error in Measurement)</em></blockquote><p><strong>Error In Measurement</strong> is generally given by the sensor manufacturers. When we buy a new sensor, the manufacturer tells us the standard deviation of the measurement that we’ll get from the sensor. It means, let say the standard deviation is 3 and actual measurement is 150, then the sensor can give us the output ranges from 147–153.</p><p><strong>Error In Prediction</strong> is calculated mathematically. We initially start with a wrong belief(large error) and then reduces the error gradually(using Kalman Gain) after taking the first few measurements from the sensor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*u2BH_i6MzU6OEPA_d1dxAA.jpeg" /><figcaption>Source: (<a href="https://www.youtube.com/watch?v=Fuy73n6_bBc">Click Here</a>)</figcaption></figure><p>Kalman Filter comprises of these set of equations. I can explain these equations here also but I feel it would be better if you go through <a href="https://www.youtube.com/watch?v=CaCcOwJPytQ&amp;list=PLX2gX-ftPVXU3oUFNATxGXY90AULiqnWT"><strong>these</strong></a> amazing video lectures series on Kalman Filters by <a href="https://www.youtube.com/channel/UCiGxYawhEp4QyFcX0R60YdQ">Michel van Biezen</a>.</p><h4>Conclusion and Discussion:</h4><p>To summarize all the equations, you can see the below image. Here you can see that we first predict the cart position and then we combine the measured value with predicted value to get the best estimate. Here the Gaussian Distribution shows the probability of cart on a particular position and the width of Gaussian Distribution tells us the possible error/deviation in the predicted/measured value from the actual value. Wider curve means that the belief can be less accurate whereas the narrower curve tells us that the belief is more accurate. So we use Kalman Filters to reduce the width of these Gaussian Distributions to make sure that our final calculated belief(position) is very accurate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Rw6EcgUITlOK_ocnBF1qSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*tURZiXU2dw727a0SnCANVQ.png" /><figcaption>Taken from somewhere in the internet :)</figcaption></figure><p>I would like to thanks <a href="https://medium.com/u/2e1bb83d100c">Infosys</a> for giving me an opportunity to learn this amazing technology and <a href="https://medium.com/u/2929690a28fb">Udacity</a> for helping me to understand these concepts through their lectures.</p><p>You can find my implementation of Kalman Filters and other <strong>Self-Driving Car</strong> related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Extended-Kalman-Filter-P6"><em>here</em></a><em> </em>👻. Please feel free to provide me any suggestions, corrections or additions in the comments :)</p><p><em>To be continued…. Part 4.2 and 4.3 coming soon </em>🤞</p><p>Edit-1: You can read about Extended Kalman Filters(Part 4.2) <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-2-8049cda5a92">here</a>.</p><p>Edit-2: Read about Unscented Kalman Filters(Part 4.3) <a href="https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-4-3-2a506e57d830">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66589e522965" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 3: Using Machine Learning techniques to detect vehicles on road]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-3-using-machine-learning-techniques-to-detect-vehicles-on-road-6cfdfbd926f8?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cfdfbd926f8</guid>
            <category><![CDATA[autonomous-vehicles]]></category>
            <category><![CDATA[svm]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Sun, 15 Apr 2018 11:44:28 GMT</pubDate>
            <atom:updated>2018-04-23T10:15:28.592Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9mR_i5RjAMtzQsY-gZONQ.jpeg" /></figure><p>“<a href="https://forums.tesla.com/forum/forums/why-tesla-betting-cameras-full-self-driving?page=2">You can absolutely be super-human with just cameras</a>” — <a href="https://en.wikipedia.org/wiki/Elon_Musk"><strong><em>Elon Musk</em></strong></a></p><p>Today I am going to discuss about some techniques used to identify vehicles from a front-facing camera on a car. Although in self driving cars we use multiple sensors like <a href="https://en.wikipedia.org/wiki/Radar"><strong><em>Radars</em></strong></a>, <a href="https://en.wikipedia.org/wiki/Lidar"><strong><em>Lidars</em></strong></a><strong><em>, </em></strong><a href="http://education.rec.ri.cmu.edu/content/electronics/boe/ultrasonic_sensor/1.html"><strong><em>Ultrasonics</em></strong></a><strong><em> </em></strong>etc. to detect the surrounding objects in real-time, but camera plays a crucial role in identifying the objects.</p><p>Before starting, let me ask a question….😝</p><blockquote>What is a classifier?<br>The classifier is a mathematical function, implemented by a classification algorithm, that maps input data to a category. <em>😴</em></blockquote><p>In this post, I’ll discuss about some classification techniques like Naive Bayes, Decision Trees, Support Vector Machines and one very famous Computer Vision technique i.e. Histogram of Oriented Gradients(HOG) which can be used to identify vehicles on road.</p><h4><a href="https://en.wikipedia.org/wiki/Naive_Bayes_classifier"><strong><em>Naive Bayes</em></strong></a></h4><p>It is a classification technique based on <a href="https://en.wikipedia.org/wiki/Bayes%27_theorem">Bayes’ Theorem</a> with an assumption of independence among predictors. In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. For example, a fruit may be considered to be an apple if it is red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other features, all of these properties independently contribute to the probability that this fruit is an apple and that is why it is known as ‘Naive’.</p><p>Naive Bayes model is easy to build and particularly useful for very large data sets. Along with simplicity, Naive Bayes is known to outperform even highly sophisticated classification methods.</p><p>Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c), P(x) and P(x|c). Look at the equation below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*DdFz2o7Jt7uFzAGP8Pwdrg.png" /><figcaption>Source: <a href="https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/">https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/</a></figcaption></figure><h4>Decision Tree</h4><p>Decision tree builds classification or regression models in the form of a tree structure. It breaks down a data-set into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with <strong>decision nodes</strong> and <strong>leaf nodes</strong>. A decision node (e.g., Outlook) has two or more branches (e.g., Sunny, Overcast and Rainy). Leaf node (e.g., Play) represents a classification or decision. The topmost decision node in a tree which corresponds to the best predictor called <strong>root node</strong>. Decision trees can handle both categorical and numerical data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*q8_LvmeWB8yQGeFYCwWOnw.png" /><figcaption>Source: <a href="http://www.saedsayad.com/decision_tree.htm">http://www.saedsayad.com/decision_tree.htm</a></figcaption></figure><h4><strong>SVM (Support Vector Machines)</strong></h4><p>“Support Vector Machine” (SVM) is a supervised machine learning algorithm which can be used for both classification or regression challenges. However, it is mostly used in classification problems. In this algorithm, we plot each data item as a point in n-dimensional space (where n is number of features you have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyper-plane that differentiate the two classes very well (look at the below snapshot).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*rU2OUfGtDwx4DBkVdPitwA.png" /><figcaption>Source: <a href="https://www.analyticsvidhya.com/blog/2017/09/understaing-support-vector-machine-example-code/">https://www.analyticsvidhya.com/blog/2017/09/understaing-support-vector-machine-example-code/</a></figcaption></figure><p>The advantages of support vector machines are:</p><ul><li>Effective in high dimensional spaces.</li><li>Still effective in cases where number of dimensions is greater than the number of samples.</li><li>Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.</li><li>Versatile: different <a href="http://scikit-learn.org/stable/modules/svm.html#svm-kernels">Kernel functions</a> can be specified for the decision function. Common kernels are provided, but it is also possible to specify custom kernels.</li></ul><h4>Histogram of Oriented Gradients(HOG)</h4><p>HOG feature descriptors and their extensions remain one of the few options for object detection and localization that can remotely compete with the recent successes of deep neural networks (DNN). Lets, consider building a machine learning system to recognize automobiles from different camera angles. Machine learning using images from traffic cameras requires creating a model to recognize an automobile from a continuous interval of angles (front, side, etc.). Neural networks excel at building such models via learned features. Machine learning on satellite images, however, only requires recognizing the overhead silhouette of a car, assuming a constant nadir stare angle. Furthermore, objects of interest in satellite images are often only a few pixels in size and many features typically used to identify an object may be highly blurred, leaving only object outlines as differentiable. HOG descriptors capture such outline information, and are simpler, less powerful, but faster (~20x) alternatives to neural networks. In addition, HOG features can be extracted via the CPUs of a laptop or computing cluster, and need not rely on high performance graphical processing units (GPU) that may not be available to all users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-iSdZLdozQiONJFJAfEEmw.png" /></figure><p><strong>Ok…So how all these techniques are going to help in identifying position of vehicles on the road?</strong> 🤔</p><p><em>Good Question</em>… 😋<br>So to use these techniques, first we need a good data-set of car and not car images and then after processing the images in the desired format, we can apply these techniques on the processed data-set to extract meaningful information out of it. Below is the step by step guide of my implementation.</p><p><strong>Step 1 — Data Collection:</strong> The data-set I used for my project is was taken from <a href="http://www.gti.ssr.upm.es/data/Vehicle_database.html">GTI vehicle image database</a> and <a href="http://www.cvlibs.net/datasets/kitti/">KITTI vision benchmark suite</a>. Below are some sample images of car and not car images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*yuo0ktUKMmn1juY4iM503g.png" /><figcaption>Sample Car Images</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*PPOUbg4FpyGjfgT7Bi8uAA.png" /><figcaption>Sample Not Car Images</figcaption></figure><p><strong>Step 2 — Data Preprocessing: <br></strong> One of the first steps in data processing is to ensure that the images have the same size and aspect ratio. Resizing the images to a small scale helps the model to run faster, so I resized all the images to 16x16 px using cv2.resize() function of OpenCV library.</p><p>After resizing, I converted all the images from RGB to YCrCb color space.</p><blockquote>Now you must be wondering why?<em> 🤔</em></blockquote><p>Actually, after playing with different color spaces 😝, I found that in <strong>YCrCb </strong>color space most of the information is coming in Y channel, so I discarded the Cr and Cb channel, which helped in reducing the size of my feature vector by 3 times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*UPOrGbWX0UqbD1plV3HqMQ.png" /><figcaption>Histogram of a sample car image in RGB color space</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*n6JenEffU_dSK4dWRIre-A.png" /><figcaption>Histogram of a sample image in YCrCb color space</figcaption></figure><blockquote><strong>Please Note:</strong> You can use any color space, I used YCrCb to improve the processing time of my model. Also it gave me better results than RGB channel.</blockquote><p><strong>Step 3 —Extract HOG Features:<br></strong>After pre-processing, I extraced HOG features from all car and not car images.<br>To extract hog features, I used hog() method of <strong>skimage.feature</strong>, which gave me a one dimensional feature array of the input image. Below is the sample hog output of one car and not car image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*3_MTBTheflP10osBJSakdQ.jpeg" /></figure><p><strong>Step 4 — Training a classifier on the processed output:<br></strong>Now after processing the images and extracting HOG features, I used these features as a input to my classifier. For this project, I used <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html"><strong>SVM</strong></a><strong> </strong>classifier with “<strong>rbf</strong>” Kernel provide by sklearn library as it gave me accuracy of 99.21%.</p><p><strong>Step 5 — Test the classifier on images taken by front facing camera of a car:<br></strong>Till now we have trained our classifier to classify car and not car images, so if we provide the camera images to our classifier, then the chances are that it will identify most of the images as not car images. Also our requirement is to detect vehicle positions on the road, which this classifier can’t detect.</p><p>To do that, I used sliding window algorithm, in which, I created some windows of size 64x64, 96x96 and 128x128 and slided these windows on the original input image with a overlap of 50%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dw5Sa3EMokQyqA5tLqGdug.jpeg" /><figcaption>Sample image with some sliding windows of different size</figcaption></figure><p>For each window, I processed the image patch of that window and used that as a input to my classifier, which in turns classified that patch as car/not car. After that, I discarded all the not car windows and drew the windows which were classified as car images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mjVSGGTHUBIEr90Mx1I6yA.png" /></figure><p><strong>Step 6 — Discard False Positives:</strong><br>Now if you see, apart from the cars, the classifier also classified not car images as car images. These are called as false positives. So to remove these false positives and to make sure that the classified patch contains a car, I created heat maps of this classified output image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R_wNJCKfVQxdhdaGVYvgtA.png" /><figcaption>Sample Image with one false positive on left and corresponding heat map</figcaption></figure><p>To make a heat map we start with a blank grid and “add heat” for all pixels within windows where positive detections are reported by the classifier. The “hotter” the parts, the more likely it is a true positive, and we can impose a threshold to reject areas affected by the false positives. If we integrate a heat map over several consecutive frames of video, areas with multiple detections get “hot” while transient false positives stay “cool”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K_af4BDm6fyvW8TFlDefng.png" /><figcaption>After imposing threshold</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ytoG0ERfnEUw9MvvRybOKQ.jpeg" /><figcaption>Sample final output after rejecting false positives</figcaption></figure><h4>Results:</h4><p>Below the final output of my model on the video taken by front facing camera of a car 🙌.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLb5Y5EYr2lqNE_1F_1n78Ruf7NDVJnJb1%26start%3D0&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DLE9eMDeEq6s&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FLE9eMDeEq6s%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b76cfd1295da5729b90b0470f1d5259b/href">https://medium.com/media/b76cfd1295da5729b90b0470f1d5259b/href</a></iframe><h4>Conclusion and Discussion:</h4><p>Detection of cars from camera is a difficult problem 😴. The sliding window method is expensive, in the sense that it takes too long to process (60 min to process 1 min 😓). For a real-time application, it has to be optimized, say using parallel processing. And this method is likely to find a lot of false positives, even after averaging frames 😕.<br>The described method is likely to fail in cases where it wasn’t trained for: a bicycle 🚴, a pedestrian 🚶, a dog crossing the street 🐕 or a cone in the middle of the road.<br>In real world car driving 🚗, we use the rear mirror to look behind. In the same way, we should have a rear camera to identify cars behind us.<br>I would like to thanks <a href="https://medium.com/u/2929690a28fb">Udacity</a> for providing this high level challenge and valuable guidance on this project.</p><p>You can find this project and other <strong>Self-Driving Car</strong> related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Vehicle-Detection-P5"><em>here</em></a><em> </em>👻<em>.</em> Though this project is completed, I would like to improve it further by using <a href="https://www.youtube.com/watch?v=VOC3huqHrss"><strong>YOLO</strong></a><strong> </strong>architecture which is a <a href="https://en.wikipedia.org/wiki/Artificial_neural_network"><strong>Neural Network</strong></a> based technique to detect objects in real-time. Please feel free to provide me any suggestions, corrections or additions in the comments :).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cfdfbd926f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 2: Using Computer Vision to detect lane lines on road]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-2-using-computer-vision-to-detect-lane-lines-on-road-31ea3cda0cbd?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/31ea3cda0cbd</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Sun, 08 Apr 2018 18:11:51 GMT</pubDate>
            <atom:updated>2018-04-09T04:57:42.454Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ky9atjOIXUVeU9MR0mDzUw.jpeg" /></figure><p>I just finished my 4th project of <a href="https://in.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Udacity Self Driving Car Nano Degree</a>, Advance Lane Lines. In this project I used Python and the very famous computer vision library <a href="https://opencv.org/">OpenCV</a> to identify lane lines on road from the stream of videos. These videos were taken from a camera placed on a vehicle.</p><p>Below is the sample input and output image of my final code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IwN6pF2sd76EnmprvZ4w9w.png" /></figure><p>In a nutshell, I’ve taken the following steps to accomplish this:</p><ul><li>Correct the Camera Distortion</li><li>Filter Lane Pixels</li><li>Get the bird eye view of lane lines</li><li>Calculate and fit the “Best-Fit” Curve</li><li>Project the Lane lines back to the “Vehicle-View” Image</li></ul><p><strong>Step 1: Calibrate camera and remove distortion <br></strong> <a href="https://en.wikipedia.org/wiki/Distortion_(optics)">Image distortion</a> occurs when a camera looks at 3D objects in the real world and transforms them into a 2D image; this transformation isn’t perfect. Distortion actually changes what the shape and size of these 3D objects appear to be. So, the first step in analyzing camera images, is to undo this distortion so that you can get correct and useful information out of them.</p><p><strong>Q) Why do we need to remove distortion?</strong></p><p>We need to remove distortion because</p><ul><li>Distortion can change the apparent size of an object in an image.</li><li>Distortion can change the apparent shape of an object in an image.</li><li>Distortion can cause an object’s appearance to change depending on where it is in the field of view.</li><li>Distortion can make objects appear closer or farther away than they actually are.</li></ul><p>Real cameras use curved lenses to form an image, and light rays often bend a little too much or too little at the edges of these lenses. This creates an effect that distorts the edges of images, so that lines or objects appear more or less curved than they actually are. This is called <strong>radial distortion</strong>, and it’s the most common type of distortion.</p><p>Another type of distortion, is <strong>tangential distortion</strong>. This occurs when a camera’s lens is not aligned perfectly parallel to the imaging plane, where the camera film or sensor is. This makes an image look tilted so that some objects appear farther away or closer than they actually are.</p><p>Images can be undistorted by some known formulation, mapping distorted points to undistorted points. To do this we take some pictures of known shapes in several angles and distances, so we are able to detect image errors. Commonly used are chessboards on a flat surface because chessboards have regular high contrast patterns which makes it easy for automated detection.</p><p>Then I used OpenCV functions <a href="https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#bool%20findChessboardCorners(InputArray%20image,%20Size%20patternSize,%20OutputArray%20corners,%20int%20flags)"><em>findChessboardCorners()</em></a> and <a href="https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#drawchessboardcorners"><em>drawChessboardCorners()</em></a> to automatically find and map the chessboard pattern. Next, I used OpenCV functions <a href="https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#double%20calibrateCamera(InputArrayOfArrays%20objectPoints,%20InputArrayOfArrays%20imagePoints,%20Size%20imageSize,%20InputOutputArray%20cameraMatrix,%20InputOutputArray%20distCoeffs,%20OutputArrayOfArrays%20rvecs,%20OutputArrayOfArrays%20tvecs,%20int%20flags,%20TermCriteria%20criteria)"><em>calibrateCamera()</em></a> and <a href="https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#void%20undistort(InputArray%20src,%20OutputArray%20dst,%20InputArray%20cameraMatrix,%20InputArray%20distCoeffs,%20InputArray%20newCameraMatrix)"><em>undistort()</em></a><em> </em>to compute the calibration matrix and undistort the images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*4pdte1v-_X76OBpZxlkwaQ.png" /><figcaption>Example of an image before and after Distortion Correction</figcaption></figure><p><strong>Step 2: Filter lane line pixels<br></strong>Now the next step is to extract lane lines from the input image. Since the input image can be taken in different light conditions, so to extract data accurately, I first converted the image to <a href="https://en.wikipedia.org/wiki/HSL_and_HSV"><strong><em>HSL</em></strong></a><strong><em> </em></strong>color format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*cMx8xS1G5UQZOAWPP-8YYA.png" /><figcaption>HSL Representation</figcaption></figure><p>As we know that the <em>Hue</em> value is its perceived color number representation based on combinations of red, green and blue. The <em>Saturation</em> value is the measure of how colorful or how dull the pixel is and <em>Lightness </em>is how closer to white the color is. So after changing the image to HSL, I applied thresholding on ‘s’ and ‘l’ channel to remove noise from the image.</p><p>Also, as an added precaution I applied <a href="https://en.wikipedia.org/wiki/Sobel_operator"><strong><em>Sobel operator</em></strong></a> in x direction.<br>A Sobel operator essentially measures the rate of change in value (and its direction) between two locations of the image. This is also technically called as the <em>derivative </em>or <a href="https://en.wikipedia.org/wiki/Image_gradient"><em>gradient</em></a>. This helped me a lot in improving the accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wAK7jebl1Y7GEOrIQfZgXg.png" /></figure><p>Then I used OpenCV libraries to get pixels of the road from the region of interest and remove extra noise from the image. In this case my region of interest is a triangle in lower half as I need pixels of this part only.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4xfqf2JClZPuRsC7mtKP4w.png" /></figure><p><strong>Step 3: Perspective Transformation<br></strong> <a href="https://bl.ocks.org/mbostock/10571478">Perspective transformation</a> is used to obtain a birds-eye view of the line regions in order to detect them and isolate them for further processing.<br>OpenCV <a href="https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#getperspectivetransform"><em>getPerspectiveTransform()</em></a><em> </em>function is used to calculate the perspective transformation matrix, and then I used <a href="https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#warpperspective"><em>warpPerspective()</em></a> function to correctly rectify each image to a “birds-eye view”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qInatg3PAQ_B-LEY0uqGKw.png" /></figure><p><strong>Step 4: Calculate and fit the best fit curve on lane lines<br></strong> To calculate the equation of lane lines I used <a href="https://www.pyimagesearch.com/2015/03/23/sliding-windows-for-object-detection-with-python-and-opencv/">sliding window search</a> algorithm. For that first we need the x coordinates of the lane lines when y = image_height(starting position). So to get the x coordinates, we’ll first draw the histogram of pixel positions on x axis for the bottom half of the warped image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*zi0R4rmZTDeSR2jA_2Kevg.png" /></figure><p>With this histogram I added up the pixel values along each column in the image. In my thresholded binary image, pixels are either 0 or 1, so the two most prominent peaks in this histogram will be good indicators of the x-position of the base of the lane lines(left and right lane lines).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*HLr8aVusYqH1JWAN2DFR3Q.png" /></figure><p>Here we can see that most of the pixels are on x = ~300 and x = ~950 and it is clearly visible that these points are of left and right lane lines. So I used these points as a starting point for where to search for the lines in the sliding window search algorithm to find and calculate the lane line pixel curve equation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1WhhowlwqY39i9W4I7gz0A.png" /></figure><p><strong>Step 5: Project the Lane lines back to the “Vehicle-View” Image<br></strong>After getting the curve equation of lane lines. I applied reverse perspective transformation on the image to transform it back to original state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qp1ag6rxE29GOV9wt3qGag.png" /></figure><p>And now since we have the final position of lane lines, I superimposed this image back on the original image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/403/1*_qPl1Au60CO9xdDn78Jtzw.png" /><figcaption>Final Output</figcaption></figure><p>Here is the final output of the video I processed to extract lane lines.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLb5Y5EYr2lqNE_1F_1n78Ruf7NDVJnJb1&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dvwd6Xz17p4M&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fvwd6Xz17p4M%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4753dd3e6df301c02d59ebf985c35f17/href">https://medium.com/media/4753dd3e6df301c02d59ebf985c35f17/href</a></iframe><p>You can find this project and other Self-Driving Car related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Advanced-Lane-Lines-P4"><em>here</em></a><em>.</em> Though this project is completed, I would like to improve this more by trying different color schemes. Please feel free to provide me any suggestions, corrections or additions in the comments :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31ea3cda0cbd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Autonomous Vehicle Part 1: Using Behavioral Cloning to make a Self-Driving Car drive…]]></title>
            <link>https://medium.com/@akhilsuri194/building-an-autonomous-vehicle-part-1-using-behavioral-cloning-to-make-a-self-driving-car-drive-92622d1c2dfe?source=rss-c3044232e228------2</link>
            <guid isPermaLink="false">https://medium.com/p/92622d1c2dfe</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Akhil Suri]]></dc:creator>
            <pubDate>Sun, 01 Apr 2018 19:26:33 GMT</pubDate>
            <atom:updated>2018-04-07T10:11:37.210Z</atom:updated>
            <cc:license>https://creativecommons.org/publicdomain/mark/1.0/</cc:license>
            <content:encoded><![CDATA[<h3>Building an Autonomous Vehicle Part 1: Using Behavioral Cloning to make a Self-Driving Car drive like yourself</h3><p>This is my first blog post. In this we’ll deep dive on the Deep learning that enables your car to drive autonomously and introduce the concept of behavioral cloning with Convolution Neural Networks (<a href="https://en.wikipedia.org/wiki/Convolutional_neural_network"><strong><em>CNNs</em></strong></a>). CNNs have emerged as the state of the art modelling technique for computer vision tasks, including questions that your car may encounter, such as, “is there a straight road or turn in front of me?”</p><h3>Introduction</h3><p>The objective of this project was to apply deep learning principles to effectively teach a car to drive autonomously in a simulator. The simulator includes both training and autonomous (test 🚔) mode. In training mode you need to use your driving (gaming 😜) skills to drive the car on the track and collect the data which is required to train your network (CNN). The network learns from your driving behavior, i.e when to turn and when to move forward. After completing the learning/training, we need to save the model in a file which will be used by the simulator to test it in Autonomous mode (on a track which it’s never seen).</p><h3>Approach</h3><p>My solution to this project was influenced by <a href="http://yann.lecun.com/exdb/lenet/"><strong>LeNet</strong></a>, <a href="https://arxiv.org/pdf/1604.07316v1.pdf"><strong>NVidia </strong></a>and <a href="https://arxiv.org/pdf/1608.01230.pdf"><strong>comma.ai </strong></a>deep learning architectures for behavior cloning. I also used <a href="https://keras.io/"><strong>Keras</strong></a><strong> </strong>deep learning library which used <a href="https://www.tensorflow.org/"><strong>Tensorflow</strong></a><strong> </strong>as a back-end, to create my model.</p><p>The overall strategy for deriving model architecture was to keep the car running on center of the road.</p><p>My first step was to use a convolution neural network model similar to the <a href="http://yann.lecun.com/exdb/lenet/">LeNet</a> Architecture. I thought this model might be appropriate because it is one of the powerful and first successful image recognition models.</p><p>In order to gauge how well the model was working, I split my image and steering angle data into a training and validation set. I found that my first model had a high mean squared error (<a href="https://en.wikipedia.org/wiki/Mean_squared_error"><strong><em>mse</em></strong></a>) on the training set. To reduce that, I normalized the input image and the mse stared decreasing. After training the network for 5 epochs, mse was low but the results were not up the mark 😔.</p><p>Then I tried <a href="https://arxiv.org/pdf/1604.07316v1.pdf">NVIDIA</a> architecture. In that architecture I cropped and down-scaled the input images so that the model can learn faster. But in this architecture I was getting a low mse on training set but high mse on the validation set. This implied that the model was over fitting 😭.</p><p>So to combat the over fitting, I modified the model by removing some Convolution Layers and Fully Connected Layers 😝.</p><p>After changing the model architecture and training it for 3 epochs, I finally got the training loss as 0.0146% and validation loss of 0.0117% 😋.</p><h4>Creation of the Training Set &amp; Training Process</h4><p>To capture good driving behavior, I first recorded two laps on track one using center lane driving. Here is an example image of center lane driving:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*dU4VyB8aMUwRsNYyKQI3HQ.png" /><figcaption>Images taken from center camera(Track-1)</figcaption></figure><p>Then I tried the model on simulator to see how well the car is driving around track one. There were a few spots where the vehicle fell off the track. To improve the driving behavior in these cases, I augmented the data by flipping the images taken by center camera 🎥, adding more images in the training set taken from left and right cameras with steering angle correction 🛠 of +-0.20 degrees.</p><p>Also, I recorded some images of the vehicle recovering from the left side and right sides of road back to the center so that the vehicle would learn to keep itself in middle of the road on steep turns 🙈 . These images show what a recovery looks like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*QauuEgshflE-r-QwZq9t-w.png" /><figcaption>Recovery Images</figcaption></figure><p>After the collection process, I had 25712 number of data points. I then preprocessed this data by converting the color scheme from BGR to RGB (since cv2 reads the image in BGR color scheme and simulator sends images in RGB format). Then I cropped and re-scaled the image to get the final image shape as 32, 80, 3. This helped in training my model faster 🚴🏻.</p><p>I finally randomly shuffled the data set and put 20% of the data into a validation set and rest in the training set.</p><p>Then I used this training data for training the model. The validation set helped in determining if the model was over or under fitting. The ideal number of epochs was 3 as after 3rd epoch the model started oscillating around 0.014% training loss. Also, I used <a href="https://keras.io/optimizers/#adam">Adam</a> optimizer so that manually training the learning rate wasn’t necessary.</p><h4>Final Model Architecture</h4><p>My model consists of two convolutions neural network layers with 5x5 filter sizes each and depths of 24 and 32 respectively. I also did MaxPooling after both convolution layers.</p><p>The model includes ReLU <a href="https://keras.io/layers/core/#activation">activation</a> function to introduce non-linearity, and the data is normalized in the model using a <a href="https://keras.io/layers/core/#lambda">Keras lambda layer</a>.</p><p>The model flattens the output of last convolution layer and then passes the flattened output through the 3 Dense layers of size 32, 16, and 1 respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*7Rx37s8l8QBn_ToOSF2gbA.png" /><figcaption>Final Model Architecture</figcaption></figure><ul><li><strong>Convolution Layer 1:</strong> 5x5 with 24 filters</li><li><strong>Convolution Layer 2:</strong> 5x5 with 32 filters</li><li><strong>Fully Connected Layer 1</strong></li><li><strong>Fully Connected Layer 2</strong></li><li><strong>Fully Connected Layer 3</strong></li><li><strong>Output Layer</strong></li></ul><h4>Attempts to reduce overfitting in the model</h4><p>The model contains 2 <a href="https://keras.io/layers/core/#dropout">dropout</a> layers in order to reduce over-fitting.</p><p>The model was trained and validated on different data sets to ensure that the model was not over fitting. I used <a href="http://scikit-learn.org/stable/">sklearn</a> library to split the data-set into 2 parts, one for training and another one for validation with the test_size of 20%. The model was tested by running it through the simulator and ensuring that the vehicle should stay on the track.</p><h4>Model parameter tuning</h4><p>The model uses the <a href="https://keras.io/optimizers/#adam">Adam</a> optimizer, so the learning rate was not tuned manually.</p><h4>Results</h4><p>The results came as a pleasant surprise after several nights without any progress. Shown below are the results of the model on the test track.<br>I was surprised how well the car drove on the test track. It recovered successfully from a few critical situations 😮, even though none of those maneuvers had been performed during training 🤔.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLb5Y5EYr2lqNE_1F_1n78Ruf7NDVJnJb1&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DezA0tZGjVmo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FezA0tZGjVmo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/63cf769efb683a79b4266574076c644e/href">https://medium.com/media/63cf769efb683a79b4266574076c644e/href</a></iframe><p>Here for every frame the simulator sends the center camera image to the model in back-end to get the steering angle which is determined in real-time by processing the image and passing it through the network.</p><p><strong>Conclusions</strong><br>Summarizing, this was a really interesting project. It would be interesting to see whether recovery events can also be simulated from the real world data. Although for that we need a big model architecture and very large variety of data to train the network. The project cost me countless of hours of sleep over a week of time including cursing, but the result was well worth it. Deep learning is an exciting field and I feel I am lucky to live in these times of discovery 🙌</p><p>You can find this project and other Self-Driving Car related projects on my <strong><em>GitHub </em></strong><a href="https://github.com/asuri2/CarND-Behavioral-Cloning-P3"><em>here</em></a><em>.</em> Though this project is completed, I would like to improve or maybe train a car to drive on the off road track. Please feel free to provide me any suggestions, corrections or additions in the comments :).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92622d1c2dfe" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>