<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bogdan Djukic on Medium]]></title>
        <description><![CDATA[Stories by Bogdan Djukic on Medium]]></description>
        <link>https://medium.com/@bdjukic?source=rss-97d54424597a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uSdLgTUVRpg-SvA0niqU4g.jpeg</url>
            <title>Stories by Bogdan Djukic on Medium</title>
            <link>https://medium.com/@bdjukic?source=rss-97d54424597a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 11:10:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bdjukic/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to measure glass-to-glass video latency?]]></title>
            <link>https://medium.com/vay-technology/how-to-measure-glass-to-glass-video-latency-13256037e552?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/13256037e552</guid>
            <category><![CDATA[teleoperation]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Fri, 10 Jun 2022 11:08:47 GMT</pubDate>
            <atom:updated>2022-06-10T11:08:47.470Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*efz4tGJ2Rto4uhLG" /></figure><p>At Vay, we deeply believe that there is a better way to optimize utilization rate of car sharing and ride hailing services. We are building a unique mobility service which will challenge private car ownership and introduce a new way of moving in big urban environments. The underlying technology empowering this new approach is teleoperation (remote vehicle control). Vay will be the first company launching a mobility service which is capable of delivering empty vehicles to our customers at the desired pick up location and taking them over once the trip is over. Our vehicle fleet is remotely controlled by remote drivers (teledrivers) who can be at the same city, different city or even in a different country compared to our customers.</p><p>As you might imagine, one of our core modalities is ultra-low latency video streaming. In order to be able to safely control what essentially is a 1.5 ton robot at higher speeds on public streets without a safety driver, we need to be able to achieve glass-to-glass video latency below 200ms.</p><h3>What is glass-to-glass video latency?</h3><p>Glass-to-glass latency refers to the time duration of the entire chain of the video pipeline. From the moment a light source hits the CMOS sensor on the camera mounted on the vehicle to the final rendering of the image on the computer screen of the teledriver. All components in this chain (camera frame capture, ISP post-processing, encoding, network transmission, decoding, screen rendering) add a certain amount of delay to the feedback for the teledriver.</p><p>Depending how vertically integrated you are or how good instrumentation of the video pipeline you have, you might be forced to look at your entire system as a blackbox in order to measure with high accuracy glass-to-glass latency and include all contributors which affect the final latency number.</p><h3>How not to measure glass-to-glass video latency</h3><p>Like with any other measurement methodologies, there are more and less precise ways to do this. The usual method that many use is putting a high precision digital stopwatch in front of the camera and capturing a photo while the stopwatch is being rendered on the screen. This is quite a manual effort with low precision (due to the refresh rate of the stopwatch screen) and does not provide you with a good understanding of the glass-to-glass latency value distribution over time. Some other less scientific methods involve snapping your fingers in front of the camera and looking at the rendered image on the screen in order to have a subjective feeling on the latency of the system.</p><h3>How to do it properly?</h3><p>At Vay, we recognized quite early in the project the importance of having the high precision and reliable tool for measuring glass-to-glass latency. The glass-to-glass latency measuring tool (which is based on this <a href="https://www.researchgate.net/publication/307516190_A_system_for_high_precision_glass-to-glass_delay_measurements_in_video_communication">paper</a>) became important in evaluating and comparing different camera models and hardware compute platforms. Given that we found it quite useful in our context, we decided to <a href="https://github.com/Reemote/glass-to-glass-latency-meter">open source it</a>. We hope you will find it useful as well for understanding your end-to-end system performance and consider it over <a href="https://www.voysys.se/latency">more expensive alternatives</a>.</p><p>Basic principle behind the tool is to centralize the emitting of the light source (LED) on the camera lens and its detection (phototransistor) on the computer screen. The centralization approach eliminates the need for time synchronization which increases the precision of this method. Once that is in place, we are able to calculate the time delta between when the light source is activated (LED triggering is usually measured in microseconds) and when it is detected. Another benefit of this approach is that you are able to execute multiple measurements over time in order to get the latency distribution over time.</p><p>In our repo, you will find an affordable setup which is based on Arduino Uno and related sensors (LED and phototransistor). By running a simple Python script, a measurement test will be kicked off for a predefined number of cycles. The end result is a chart that shows the glass-to-glass video latency distribution.</p><h3>Btw, we are hiring!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nAC6QG6LArLPzLeI" /></figure><p>In case you are excited about working on a deep tech project which includes several engineering disciplines (robotics, perception, video, control, data science embedded, cloud, mobile app development, etc.) and would like to contribute to Vay’s new mobility service concept, do check out <a href="https://vay.io/careers/">Vay’s Career page</a>. We are Berlin based but we are offering remote work options across different roles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13256037e552" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vay-technology/how-to-measure-glass-to-glass-video-latency-13256037e552">How to measure glass-to-glass video latency?</a> was originally published in <a href="https://medium.com/vay-technology">Vay Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an affordable self-driving car development platform]]></title>
            <link>https://medium.com/hackernoon/building-an-affordable-self-driving-car-development-platform-b02c6d7828ab?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b02c6d7828ab</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[diy]]></category>
            <category><![CDATA[automotive]]></category>
            <category><![CDATA[renault]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Fri, 20 Oct 2017 13:40:19 GMT</pubDate>
            <atom:updated>2019-04-26T05:27:33.755Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TMHkCpIC_0_mUklkOri0Dg.jpeg" /><figcaption>Renault Twizy — a good candidate for affordable self-driving car development platform</figcaption></figure><h3>Intro</h3><p>I’ve been planning to write this post for a very long time. My initial idea after doing a research in this area was to work on a real hardware and blog about the progress. But due to personal reasons (I’ll be moving from Stockholm to Berlin soon), I’ve decided to kick things off with this post about current challenges and potential solution for building an affordable <a href="https://hackernoon.com/tagged/self-driving">self-driving</a> car development <a href="https://hackernoon.com/tagged/platform">platform</a>.</p><h3>Goal</h3><p>The goal of this post is to explore a possibility <strong>to build an affordable self driving car platform under 10K USD</strong>. Cost for additional sensors like radars, LIDARs and etc. are not included in the final price. The scope of the effort should only be drive-by-wire system and not the actual brains of the autonomous vehicle.</p><p>The final outcome should be something similar to:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F-bzsfgJJeMs%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D-bzsfgJJeMs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F-bzsfgJJeMs%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c167c61e60b657a413d9c18eef4946f9/href">https://medium.com/media/c167c61e60b657a413d9c18eef4946f9/href</a></iframe><p>Basically, you should be able to control the car with your Xbox joystick.</p><h3>Background</h3><p>One of the main challenges in the self-driving car industry (among other things like technology itself, policy updates, ethical issues, etc.) is the <strong>barrier of entry.</strong> If you are a small start up building a local autonomous delivery service or a single engineer trying out latest deep learning approaches for car/traffic sign detection, it is incredible hard (sometimes even impossible) to get things off the ground and test your solution in the real world setting.</p><p>Some people would argue that simulations would be good enough, but what if you want to make that next step? At the moment, this industry is reserved only for the big league, with heavy financial backing, which is pushing small players out at the very beginning of the game.</p><p>Another side effect of current state is that this technology, which clearly has so many benefits on the society as a whole, will be first developed and deployed in just a few countries around the world. This puts the rest of the world in unequal position where we would need to license the tech from big players and experience even deeper technological and economical divide.</p><p>Current developments in the self-driving car industry are at the moment one big science project for big VC funds. The topic of costs of building products and services in this area is usually not something which is openly discussed. That is why I was surprised once I stumbled upon this interesting video from Raquel Urtasun, current lead of Advanced Technologies Group at Uber, where she covers few ideas on how to build more affordable autonomous systems:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FsW4M7-xcseI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsW4M7-xcseI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FsW4M7-xcseI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/70575a6d14ff5b6e32c4617ddd56d4db/href">https://medium.com/media/70575a6d14ff5b6e32c4617ddd56d4db/href</a></iframe><h3>So what are the costs exactly?</h3><p>There are few vendor companies (<a href="https://autonomoustuff.com/">AutonomousStuff</a> et al) which are specialized in retrofitting cars with drive-by-wire, steer-by-wire and brake-by-wire systems. The costs for their services are between <strong>70K and 100K USD and above</strong> depending whether you are going for electric vehicle or the one with internal combustion engine. Of course, the price goes up in case you want to add advanced sensor set (LIDAR units i.e.). Most of these retrofitting companies are located in USA. Therefore, I needed some alternatives since I’m based in Europe. I’ve reached out to a company in Germany which is doing car retrofits for disabled people (since it’s the same by-wire system), but ended up with similar price tag.</p><p>Recently I’ve seen a British company trying to brake into this market, <a href="http://www.streetdrone.com/">StreetDrone</a>. But the price for their solution is also still too high (<strong>over 60K USD</strong>). Also, their primary customer focus seems to be educational market.</p><p>There are some other advanced concepts like <a href="https://www.osvehicle.com/editselfdrivingcar/">EDIT</a> from OSVehicle, but I could not find any details on the release dates or price ranges.</p><h3>The car</h3><p>After going through many manufactures and models, I’ve ended up with choosing <a href="https://en.wikipedia.org/wiki/Renault_Twizy"><strong>Renault Twizy</strong></a> as an ideal candidate for this project. You can easily get a used one which is several years old for a price tag of <strong>5K USD</strong>. You might want to consider looking at Cargo version of it since you would have additional space in the back of the car for the necessary computing power.</p><h3>Steering</h3><p>Retrofitting steering column in the car is usually the most intrusive part of this procedure. I came up with two different approaches while doing my research for Renault Twizy.</p><h4>More expensive approach</h4><p>I’ve stumbled upon <a href="http://www.dcelectronics.co.uk/Products/EPAS/">DC Electronics EPAS</a> system on <a href="https://autonomoustuff.com/">AutonomousStuff</a> web site. I would assume that they are using this vendor for one of their own electric car retrofits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*BiQvjamBdPB4-LADsSnYFA.png" /><figcaption>Source: <a href="http://www.dcelectronics.co.uk/Products/EPAS/">DC Electronics</a></figcaption></figure><p>The benefits of this system is that it’s replacing complete steering column in Twizy. It would be also possible to control the steering axis with CAN messages which I find quite useful. By reading the documentation of it, I found it straightforward to support the scenario where driver is supposed to take back control of the vehicle.</p><p>Downside of this approach is of course the price. This is the price offer I’ve got from the company:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yfi3NK8Al6mx7VYXA-QBKQ.png" /></figure><p>So, not really the cheapest setup since the price of the new steering column can easily go over the price of the car itself.</p><h4>Cheaper approach</h4><p>After some time searching for the cheaper approach, I’ve discovered the work of <a href="http://www.htw-mechlab.de/index.php/umbau-renault-twizy/">HTW Dresden</a> (Dresden University of Applied Sciences). Basically, the way they approached this problem is replacing the steering column from Renault Twizy with the one from Renault Clio 2. In the video down below they demonstrate their solution where steering column is controlled as an RC unit.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FX5SOeYBpWR4%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DX5SOeYBpWR4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FX5SOeYBpWR4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/749c8c8df7c1ac300db2549cf56ef904/href">https://medium.com/media/749c8c8df7c1ac300db2549cf56ef904/href</a></iframe><p>You can find related Power Point presentation <a href="https://www.htw-mechlab.de/wp-content/uploads/2016/06/dies-academicus_2016.pdf">here</a> and you can also read their <a href="http://www.htw-mechlab.de/wp-content/uploads/2017/01/Diplomarbeit.pdf">paper</a> on this as well where they go into details (it is in German though).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wr_6L9Q1NtSwkPm_zOPeLw.png" /><figcaption>Renault Twizy and Renault Clio 2 steering columns side-by-side</figcaption></figure><p>You can find used Renault Clio 2 power steering column for as much as <a href="http://www.ebay.co.uk/itm/2014-RENAULT-CLIO-MK4-TCe90-POWER-STEERING-COLUMN-488109257R-6700003021A-/322320275248?hash=item4b0bc91730:g:oeAAAOSwB09YIFHj"><strong>65 USD on eBay</strong></a>.</p><h3>Throttle</h3><p>Exposing throttle as by-wire system seems to be the easiest part of the retrofit for this car. According to the web site from Dutch university group <a href="http://autoehv.com/twizy.html">Autonomous Eindhoven</a> (which is taken down recently), sending raw 0–5 voltage signal to Engine Control Module (ECM) will enable you to control the throttle. They were also experimenting with Renault Twizy as their research vehicle. This approach is pretty much the same as <a href="https://news.voyage.auto/under-the-hood-of-a-self-driving-car-78e8bbce62a6">Voyage used on their Ford Fusion</a>.</p><p>The voltage itself can be controlled with an Arduino board or if you want something more fancy you can go for <a href="http://www.axiomatic.com/distributed-control/1-analog-signal-output/">CAN based voltage potentiometer</a>.</p><h3>Brakes</h3><p>Brakes on Reanult Twizy are based on hydraulic system. One way to approach this problem is to put in a brand new braking actuator. I’ve reached out to <a href="https://polysync.io/">Polysync</a> folks asking for help on this since they have solved this for their Kia Soul setup. You can find more info about it <a href="https://github.com/PolySync/oscc/wiki/Hardware-Brake">here</a>. But essentially they have used Electronically Controlled Brake system from Toyota Prius (it’s around <a href="https://www.ebay.com/itm/2004-2009-TOYOTA-PRIUS-BRAKE-ANTI-LOCK-ABS-PUMP-ACTUATOR-MODULE-OEM-44510-47050-/222675297611?epid=657085322&amp;hash=item33d87b654b:g:s1EAAOSwc-tY7Wf0&amp;vxp=mtr">100 USD on eBay</a>). According to Polysync, this approach is quite generic and should work on most of cars with hydraulic braking system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5V-HUJUYcKQH9S85BKTnyg.png" /><figcaption>Source: <a href="https://github.com/PolySync/oscc/wiki/Hardware-Brake">Polysync Wiki</a></figcaption></figure><h3>Conclusion</h3><p>We’ve explored different options for retrofitting Renault Twizy for drive-by-wire scenario. This is far from a complete guide but it gives you a glimpse on the necessary work for building up an affordable platform for developers and startups interested into kicking off their own product/service in self-driving car area.</p><p>The next step would be deciding on the brain for the vehicle. For some basic scenarios <a href="http://www.nvidia.com/object/embedded-systems-dev-kits-modules.html">NVIDIA Jetson TX2</a> would be a good start since it has a low power consumption spec. You could consider putting ROS on it and expose steering, throttle and brakes as ROS nodes. If you are interested in more advanced scenarios which include sensors as well, you should check other options like <a href="http://apollo.auto/">Apollo</a> or <a href="https://github.com/CPFL/Autoware">Autoware</a> which are open source and free for your experimentation.</p><p>If anyone is interested in doing this on a real Twizy, please reach out to me! You can easily find me on Twitter.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b02c6d7828ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/building-an-affordable-self-driving-car-development-platform-b02c6d7828ab">Building an affordable self-driving car development platform</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I built an app on top of Ethereum and IPFS to fight corruption]]></title>
            <link>https://medium.com/coinmonks/i-built-an-app-on-top-of-ethereum-and-ipfs-to-fight-corruption-a2401c133e87?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2401c133e87</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[ipfs]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[corruption]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Tue, 03 Oct 2017 14:22:41 GMT</pubDate>
            <atom:updated>2020-10-13T12:37:20.314Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*HubvLVyVYDGJLhtThy5N6w.gif" /><figcaption>Introducing [Monument] — Corruption reporting app built on top of Ethereum and IPFS</figcaption></figure><h3>Origins of corruption</h3><p><a href="https://antonopoulos.com/">Andreas Antonopoulos</a> in his brilliant talk on <em>Privacy, Identity, Surveillance and Money </em>gave an overview on the flawed institutional/organisational design based on hierarchical principles. Over time, power tends to accumulate in these institutions/organisations, they become more centralized, and with centralized power, corruption comes along.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FVcvl5piGlYg%3Fstart%3D187%26feature%3Doembed%26start%3D187&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DVcvl5piGlYg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FVcvl5piGlYg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e78cd7a2676e6f1850085b86766e6fd/href">https://medium.com/media/6e78cd7a2676e6f1850085b86766e6fd/href</a></iframe><p>As Andreas also notes that products of these power structures (corruption being one of them) are not egalitarian. They are actually in most cases restrictive (I would add destructive as well). They are spreading nativism, nationalism, tribalism, class structure and therefore making the world a smaller place.</p><p>Power structures set up on hierarchical principle lack accountability and transparency, which results in corruption proliferation. On the other hand, these power structures require from us to give up on all our privacy in the name of security, fight against terrorism or to be able to use their services.</p><h3>How can we fix this?</h3><p>What if we flip the roles? What if, instead of few hundred thousand watching 7.5 billion, 7.5 billion stare back (as Andreas points out)? His idea is reversed <a href="https://www.theguardian.com/technology/2015/jul/23/panopticon-digital-surveillance-jeremy-bentham">panopticon</a>. One way to address the opaque nature of power structure is by piercing through its secrecy and by giving a resilient, distributed, transparent, open sourced and secure reporting channel to corruption whistleblowers and leakers.</p><p><strong>[Monument]</strong> is exactly that.</p><h3>What’s out there now?</h3><p>Existing corruption reporting tools lag years behind the current UX trends and reports themselves are usually not accessible and open for everybody who might be affected by the similar corruption cases. There is no strightforward way you could search corruption reports in your area, country or public institutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IiiyvvZj3NUSH_aJ-oDQyA.png" /><figcaption>Source: <a href="https://a032-secure.nyc.gov/ofcomplaint.html">New York City’s Inspector General corruption repot form</a></figcaption></figure><p><strong>[Monument]</strong> is bringing corruption reporting to a mobile app with UX for storytelling similar to Instagram Stories. The idea behind it is to open up corruption cases to a public eye where they can be discussed, shared and easily discoverable. All of this is being done with anonymity of the reporters and by hiding their true identity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*mfhHZ8DE_tHhVpkIkaWO7A.gif" /><figcaption>[Monument] — UX for browsing corruption reports based on Instagram Stories</figcaption></figure><h3>Why Ethereum and IPFS?</h3><p>Corruption report service needs to be accessible at all times. It must be immune to the governmental attacks and bans. System needs to be capable of being noncensorable and its content ought to be permanent.</p><p>These kind of system characteristics can be found in latest, state of the art, distributed systems. More specifically, blockchains (i.e. <a href="https://ethereum.org/">Ethereum</a>) and distributed file systems (i.e. <a href="http://ipfs.io/">IPFS</a>).</p><blockquote><strong>Ethereum</strong> is a <strong>decentralized platform that runs smart contracts</strong>: applications that run exactly as programmed without any possibility of downtime, censorship, fraud or third party interference.</blockquote><p><strong>[Monument]</strong> is utilising Ethereum’s smart contracts and its blockchain technology for storing corruption reports’ meta data (title, description, timestamp, up and down votes, tags, but not the actual video content). Its <a href="https://github.com/bdjukic/monument/blob/master/ReportManager.sol">smart contract exposes simple API</a> for creating new reports, listing existing ones and searching for reports based on a specific query.</p><p>The video content itself is being hosted on IPFS.</p><blockquote><strong><em>IPFS</em></strong><em> is a distributed file system that seeks to connect all computing devices with the same system of files.</em></blockquote><p>As the video content gets uploaded to IPFS, unique hash (identifier) gets assigned to it. This hash is aferwards stored as meta data for corresponding corruption report on Ethereum.</p><p>By using IPFS, <strong>[Monument] </strong>is trying to avoid <a href="https://blog.neocities.org/blog/2015/09/08/its-time-for-the-distributed-web.html">a fundamentally flawed design in HTTP</a> which is centralisation of the content ownership and distribution. What this means is that there is no single point of failure in case governments or any other organizations try to take the corruption report content down. As long as there is a sufficient number of nodes which are able to seed the content, the system itself will stay resiliant to these kinds of attacks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J-XkROFA8qonqM3w_pHePA.png" /><figcaption>We want to move away from centralised to more distributed content distribution to gain on resiliance — <a href="https://blog.neocities.org/blog/2015/09/08/its-time-for-the-distributed-web.html">source</a></figcaption></figure><p>The IPFS nodes could be hosted by NGOs which bring awareness of corruption or by passionate individuals who deeply care about this topic in their own countries.</p><p>In case you need a quick introduction tour of distributed web, Ethererum, and IPFS, I would suggest that you check out Siraj’s video:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FgSQXq2_j-mw%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DgSQXq2_j-mw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FgSQXq2_j-mw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/394f61a2f29ceb0f9e5333c5bfff5217/href">https://medium.com/media/394f61a2f29ceb0f9e5333c5bfff5217/href</a></iframe><h3>Final thoughts</h3><p><strong>[Monument]</strong> is in prototype stage and its initial purpose is to test out a potential of the new distributed paradigm which is posed to disrupt the traditional way of building services with social/political impact.</p><p>Some of the future improvements may include:</p><ol><li>Voice modification (adding distortion, besides video pixalization)</li><li>Being able to up/down vote particular report</li><li>Adding quick share capabilities for social media</li><li>Adding comments capabilities</li><li>Exposing report search through app UI</li><li>Creating Raspberry Pi image which can be used to run IPFS/Ethereum node in order to improve seeding of content</li></ol><blockquote><a href="https://coincodecap.com/?utm_source=coinmonks">Get Best Software Deals Directly In Your Inbox</a></blockquote><figure><a href="https://coincodecap.com/?utm_source=coinmonks"><img alt="" src="https://cdn-images-1.medium.com/max/584/0*OJ-qb5G6i863msBB.png" /></a></figure><p>If you found this to be an interesting read and would like to see further development of <strong>[Monument]</strong>, your tips are more than welcomed:</p><p>ETH: <strong>0x8980154c5C6DDf1c1F759F7D7B7e69BD903231c9</strong><br>BTC: <strong>32sVVk5BPkZF994cwp85sCsNa12s5vVmth</strong></p><p>If you are interested in how <strong>[Monument]</strong> works under the hood or you want to contribute to the project, you should check out:</p><p><a href="https://github.com/bdjukic/monument">bdjukic/monument</a></p><figure><a href="https://cryptofi.co/"><img alt="" src="https://cdn-images-1.medium.com/max/407/1*HrFLDXz92JavTZXjhmvU7g.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2401c133e87" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/i-built-an-app-on-top-of-ethereum-and-ipfs-to-fight-corruption-a2401c133e87">I built an app on top of Ethereum and IPFS to fight corruption</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Road scene understating using Deep Learning]]></title>
            <link>https://medium.com/data-science/road-scene-understating-using-deep-learning-c3610f6b1c4?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3610f6b1c4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Sun, 03 Sep 2017 09:49:58 GMT</pubDate>
            <atom:updated>2017-09-03T20:16:19.272Z</atom:updated>
            <content:encoded><![CDATA[<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FuWT0xHtnNTwlO%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FuWT0xHtnNTwlO%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FuWT0xHtnNTwlO%2F200.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="120" frameborder="0" scrolling="no"><a href="https://medium.com/media/e37e520092ece6fdbae4dda249635bdf/href">https://medium.com/media/e37e520092ece6fdbae4dda249635bdf/href</a></iframe><h3>Scene understanding</h3><p>Semantic segmentation represents a technique in Deep Learning where we assign a meaning to every pixel in the image by assigning it to a predefined class set. From the GIF above, we can see that we have two classes in the semantic segmentation process (<strong>road</strong> and <strong>not road</strong>) which are colored accordingly.</p><p>Using the method of semantic segmentation we are able to decompose the scene into pieces we are particularly interested in. This is especially important for self driving car since it will be able to detect which part of the scene is a drivable area.</p><h3>How does this work?</h3><p>One approach for semantic segmentation is using fully convolution network (FCN). There is a great <a href="https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf">paper</a> from UC Berkley which explains the methodology and architecture in details. Our implementation will be based on this paper as well.</p><p>Typical convolutional networks are built with several convolutional layers followed with fully connected layers. This type of architecture is good for classification problem (for questions like: <em>Is this a road?</em>). But in order to locate the road on an image (<em>Where is the road? </em>question) we would need the network to be able to preserve spacial information. This is where FCN come into play and deliver state of the art semantic segmentation results.</p><h3>How to build FCN?</h3><p>There are three main techniques in building FCN:</p><ol><li>Replace fully connected layers with the 1x1 convolutional layers.</li><li>Introduce up-sampling by using transposed convolutional layers.</li><li>Add skip connections.</li></ol><p>Let’s see how to do this in details.</p><p>Our FCN architecture has two parts: <strong>encoder</strong> and <strong>decoder</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Li8osvpQE-s0AYO8cPumFQ.png" /><figcaption>FCN architecture</figcaption></figure><p>For the <strong>encoder</strong>, we will be using VGG16 model which is pre-trained on ImageNet. Fully connected layers from from VGG16 model will be replaced with the convolutions <strong>(1)</strong>. Encoder extracts features from the image which will be used by the decoder.</p><p>The next step is building up the <strong>decoder</strong> which is done by up-sampling last encoder’s layer using transposed convolutional layers <strong>(2)</strong>. Transposed convolution is essentially a reversed convolution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FhkyU4-fusf4pJaMIm2HKw.png" /><figcaption>FCN architecture based on pre-trained model, replaced fully connected layers and transposed convolutions</figcaption></figure><p>In the very end, we will add skip layers <strong>(3)</strong> between specific layers which enables network to use multiple resolution scales. This in return delivers better segmentation results. Skip connections reduce information loss and enable network to “see the big picture”, since typical convolutional network is focused on parts of the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-K3rRQuCfTkFcOuBLOfDPA.png" /><figcaption>Skip connections in FCN</figcaption></figure><p>We should be careful when adding skip connections since they can drastically impact the size of the model.</p><h3>Final outcome</h3><p>The implementation of FCN for road scene understanding is done as part of the project for <a href="http://udacity.com/drive">Udacity Self Driving nano degree program</a>. I’ve used Python and TensorFlow to build up and train the model. The model has been trained on NVIDIA 1080Ti graphics card in 30 epochs which took less than half an hour.</p><p>This is the output of the inference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*Vsgt82oDuJlSPWGKPcHX1Q.png" /></figure><h3>Conclusion</h3><p>The related source code can be found on my GitHub profile:</p><p><a href="https://github.com/bdjukic/CarND-Semantic-Segmentation">bdjukic/CarND-Semantic-Segmentation</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3610f6b1c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/road-scene-understating-using-deep-learning-c3610f6b1c4">Road scene understating using Deep Learning</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How self driving car navigates on highway?]]></title>
            <link>https://medium.com/@bdjukic/how-self-driving-car-navigates-on-highway-c170cc77346?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c170cc77346</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Thu, 24 Aug 2017 21:15:32 GMT</pubDate>
            <atom:updated>2017-08-24T22:23:04.469Z</atom:updated>
            <content:encoded><![CDATA[<p>In one of my <a href="https://medium.com/towards-data-science/how-self-driving-cars-get-from-a-to-b-f18b1fc12044">previous posts</a> I tried to give an overview of a general process which is used to move self driving car from point A to point B. In this post, we will cover special scenario of that same work-flow applied on highway driving.</p><h3>Quick recap</h3><p>Let’s just quickly go through the work-flow once again. In the picture down below, we can see that <strong>Localization</strong> and <strong>Sensor Fusion</strong> components give us a location of the car and sensory input from another cars on the highway, respectively. These two inputs will end up in <strong>Prediction</strong> and <strong>Behavior</strong> modules which are in charge of deciding what is the next best maneuver (take left lane, take right lane, keep lane, etc.). Once the car figures out what it should do next, <strong>Trajectory</strong> module actually spits out a concrete plan how to do it by creating a path for a car to follow. Trajectory path is then executed by <strong>Motion Control</strong> module (the steering and throttle actuators on the car).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UXHDjQ72_qBT8hfGx8qVOQ.png" /></figure><h3>Problem definition</h3><p>As before, this post is tightly coupled with <a href="http://udacity.com/drive">Udacity Self-driving nano degree</a> project. This time, the goal of the project is to navigate a car on a highway (in simulator) respecting the following:</p><ol><li>The car is able to drive at least <strong>4.32</strong> miles without incident.</li><li>The car drives according to the speed limit (<strong>50 mph</strong>).</li><li>Max Acceleration and Jerk are not exceeded.</li><li>Car does not have collisions.</li><li>The car stays in its lane, except for the time between changing lanes.</li><li>The car is able to change lanes.</li></ol><h3>Architecture overview</h3><p>The class architecture will tightly follow the initial diagram.</p><ol><li><strong>Vehicle </strong>— represents our car on highway and keeps track of current localization and lane.</li><li><strong>SensorFusion</strong> — keeps track of all the vehicles on the highway.</li><li><strong>PredictionPlanner</strong> — is able to figure out how far are other vehicles in the lanes and if our car should slow down or speed up.</li><li><strong>BehaviorPlanner</strong> — takes input from PredictionPlanner and Vehicle modules to figure out what car should do (change or keep lane). This component knows when car should change from one state (center lane for example) to next one (left lane).</li><li><strong>TrajectoryPlanner</strong> — is responsible for finding the right path for changing or keeping the current lane.</li></ol><p>All of these modules work together as new sensory data come in 0.2s cycles from the car and its surroundings.</p><h3>Final State Machine</h3><p>In order to keep track of the current state (lane) of the car and its transition to new one (if necessary) we needed to introduce a lightweight final state machine with three states (left, right and center lane). The state itself is kept in <strong>Vehicle</strong> module, but the transition between different states is accomplished by <strong>BehaviorPlanner</strong> component.</p><p><strong>BehaviorPlanner</strong> is constantly querying <strong>PredictionPlanner</strong> whether the car should slow down or speed up depending on the car in front of us. Once our car slows down, <strong>BehaviorPlanner</strong> will use <strong>cost functions</strong> for current speed and proximity of other cars in other lanes. If speed cost is high given that we are slowing down constantly, <strong>BehaviorPlanner </strong>will evaluate cost for changing lanes. If the cost of changing to right lane for example is low (given that cars in that lane are sufficiently far away), it will update its state to new one. This will cause the car to steer to the new lane. <strong>TrajectoryPlanner </strong>component takes it from here.</p><h3>Path generation</h3><p>The path generation is done by the <strong>TrajectoryPlanner</strong> component. It uses the concept of <strong>anchor points</strong> which represent the future path which the car should follow. The anchor points are constructed from:</p><ol><li>Previous path points which car has not reached yet</li><li>Points which are 30, 60 and 90m from the current vehicle <strong>s</strong> value in <strong>Frenet</strong> coordinate system.</li></ol><p>Once anchor points are defined we would need to shift them to car reference points in order to easy the math. Once that is done, we will use cubic <a href="https://en.wikipedia.org/wiki/Spline_(mathematics)">spline</a> interpolation in order to get concrete points for our car to visit.</p><p>Car simulator acts as a perfect <a href="https://medium.com/towards-data-science/how-self-driving-cars-get-from-a-to-b-f18b1fc12044">MPC</a> and will visit every single point you send to it.</p><h3>Final outcome</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fu8pIO44VqcI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Du8pIO44VqcI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fu8pIO44VqcI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e49628d8ee8c5ea39bab7df1b70ba269/href">https://medium.com/media/e49628d8ee8c5ea39bab7df1b70ba269/href</a></iframe><h3>Wrap up</h3><p>This project is definitely one of the most difficult ones so far in the course. It is quite open-ended and you can easily spend bunch of time trying to make it perfect.</p><p>Source code to my solution can be found here:</p><p><a href="https://github.com/bdjukic/CarND-Path-Planning-Project">bdjukic/CarND-Path-Planning-Project</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c170cc77346" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own self driving (toy) car]]></title>
            <link>https://medium.com/data-science/build-your-own-self-driving-toy-car-ad00a6804b53?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad00a6804b53</guid>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[roses]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Mon, 17 Jul 2017 22:45:16 GMT</pubDate>
            <atom:updated>2017-07-18T13:47:56.735Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are interested in building your own self driving (toy) car and learning some AI concepts along the way, you should stick around and read through.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*fjtxEbMSJSxru9JMYYAiDQ.jpeg" /></figure><h3>Short intro</h3><p>Application of Deep Learning is disrupting many industries today with ever increasing data and computing power. One of its new battlegrounds is robotics. Deep Learning is changing the robotics landscape in the areas of <a href="https://arxiv.org/pdf/1612.07139.pdf">perception and control</a> which is the key for the success of autonomous vehicles and its broader deployments.</p><p>Recent advancements in Deep Learning tools (<a href="https://www.tensorflow.org/">TensorFlow</a>, <a href="https://keras.io/">Keras</a>, etc.) and accessibility of cheap computing platforms (<a href="https://www.raspberrypi.org/">Raspberry Pi</a>) makes experimentation with robotics and AI much more accessible (and fun!) for people outside academia and big tech companies.</p><h3>Goal</h3><p>We’ll take Deep Neural Network described in my <a href="https://medium.com/towards-data-science/using-deep-learning-to-clone-driving-behavior-51f4c9593a57">Behavior Cloning</a> project from <a href="https://www.udacity.com/drive">Udacity Self Driving Car nano degree</a> course and run it on a remote controlled (RC) race car using Robotic Operating System (<a href="http://www.ros.org/">ROS</a>) as a middle-ware. The end goal is to enable robot to autonomously complete the racing track:</p><h3>Bogdan Djukic on Twitter</h3><p>First laps from my second iteration #autonomous #robot car 🚗🤖. Chassis from #donkeycar, but it&#39;s running my software stack on top of #ROS https://t.co/7eJT5UjYg8</p><p>This is just one of many recent projects in this area and there are quite a few of them if you search the Net. There are even meet-ups (most famous one in <a href="http://www.cnbc.com/video/3000609740">Oakland, USA</a>) where people race with these robots and trying to beat the best human time.</p><p>What my project maybe sets a part from others is the usage of ROS as a connecting layer between the software stack and the metal.</p><p>This project is really a great example on how Deep Learning can be used to train the car end-to-end to drive itself and solve both perception and control challenges in robotics using AI.</p><h3>How do I build one?</h3><p>Let’s first look at the hardware to get you started:</p><ol><li><a href="https://www.amazon.com/2-4Ghz-Exceed-RC-Magnet-Electric/dp/9269803775/ref=wl_mb_wl_huc_mrai_2_dp">Remote controller car</a></li><li><a href="https://www.raspberrypi.org/products/raspberry-pi-3-model-b/">Raspberry Pi 3</a></li><li><a href="https://www.raspberrypi.org/products/camera-module-v2/">Raspberry Pi camera</a></li><li><a href="https://www.kiwi-electronics.nl/magnetische-lens-voor-raspberry-pi-groothoek?lang=en">Wide angle lens</a></li><li><a href="https://www.amazon.com/SunFounder-PCA9685-Channel-Arduino-Raspberry/dp/B014KTSMLA/ref=sr_1_1?s=electronics&amp;ie=UTF8&amp;qid=1492275083&amp;sr=1-1&amp;keywords=servo+driver">Servo driver</a></li><li><a href="https://github.com/bdjukic/selfdriving-robot-car/tree/master/3d_print_model">3D printed mount</a></li></ol><p>Since my hardware build is based on <a href="http://www.donkeycar.com/">Donkey Car</a>, you can get more detailed instructions on hardware assembly <a href="https://docs.google.com/document/d/11IPqZcDcLTd2mtYaR5ONpDxFgL9Y1nMNTDvEarST8Wk/edit">here</a>. You’ll notice that there are many options when it comes to choosing your RC car. In my first iteration of this project I went for <a href="https://www.amazon.com/Elegoo-Four-wheel-Ultrasonic-Intelligent-Educational/dp/B01DPH0SWY">Elegoo Robot Car Kit</a> which turned out not be good idea since the kit does not have a dedicated servo for controlling the steering angle. This makes modeling your neural network and later integration with the auto-pilot a bit more difficult. With <a href="https://www.amazon.com/2-4Ghz-Exceed-RC-Magnet-Electric/dp/9269803775/ref=wl_mb_wl_huc_mrai_2_dp">Magnet</a>, you have much better control over the RC car.</p><h3>And the software?</h3><p>If you are in the hurry, you can take software stack from <a href="https://github.com/wroscoe/donkey">Donkey Car</a> as well. I took the other route and tried to build the brains by myself by looking at Donkey Car project and taking learnings from my previous Udacity projects. I like one of the Richard Feynman’s quotes on this topic:</p><blockquote>What I cannot create, I do not understand. Know how to solve every problem that has been solved.</blockquote><p>As I’ve already mentioned, I decided to go for Robotic Operating System (ROS) for the setup as middle-ware between Deep learning based auto-pilot and hardware. It was a steep learning curve, but it totally paid off in the end in terms of size of the complete code base for the project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*sweeRl-Kfdnswkjwex2i6w.png" /></figure><p>ROS offers great publisher/subscriber model, distributed nodes support for each component (camera, steering and throttle control, joystick, etc.). It also comes with great debugging and logging capabilities. If you’re new to ROS, I would highly recommend <a href="https://www.amazon.com/ROS-Robotic-Projects-Lentin-Joseph/dp/1783554711">this book</a> which teaches you the basics through fun projects along the way.</p><p>In my case, I have camera and actuator (steering and throttle) nodes running on the car. Joystick, auto-pilot and training nodes are running on my desktop machine. Inference (neural network predictions) are compute intensive, so had to move it away from Raspberry Pi to beefier machine.</p><p>Remember that you’ll need <a href="https://ubuntu-mate.org/raspberry-pi/">Ubuntu Mate</a> distribution in order to run ROS on your Raspberry Pi. Here’s also a <a href="http://ubiquityrobotics.com/blog/2016/01/24/Raspberry-Pi-Cam.html">link</a> for the ROS node camera support.</p><h3><strong>What about AI?</strong></h3><p>The auto-pilot for the car is based on now famous NVIDIA’s <a href="https://arxiv.org/pdf/1604.07316.pdf">paper</a> which is using <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">Convolutional Neural Network</a> (CNN) to predict the steering angle for the car based only on image input from the camera mounted in front of the car. You can read more on this topic in one of my <a href="https://medium.com/towards-data-science/using-deep-learning-to-clone-driving-behavior-51f4c9593a57">previous posts</a>. I’ve recently stumbled upon this <a href="http://cs231n.stanford.edu/reports/2017/pdfs/626.pdf">paper</a> which is exploring some other Deep Neural Network architectures for the same problem. You should check it out!</p><p>One adjustment I had to do for the CNN compared to my previous Udacity project is introducing <a href="https://msdn.microsoft.com/en-us/library/azure/dn913065.aspx">data binning</a>. You should consider using it if:</p><blockquote>A column of continuous numbers has too many unique values to model effectively, so you automatically or manually assign the values to groups, to create a smaller set of discrete ranges.</blockquote><p>Column of continuous numbers in this case are steering angles for the car which were coming from joystick. I found that this modification improved my model predictions. Throttle on the other hand is fixed at the moment and it is something I would like to use prediction method as well.</p><h3>Conclusion</h3><p>Working on this project was a great way to get to know ROS (both benefits and challenges). It turns out that ROS is used by several autonomous car/minibus companies out there either as their prototyping platform and/or in production.</p><p>Also, working on hardware can reveal some problems when you take your neural network from simulator and put it on a “real” car.</p><p>All the code necessary for you to run ROS nodes on your own robot car can be found in my project repo together with helper scripts:</p><p><a href="https://github.com/bdjukic/selfdriving-robot-car">bdjukic/selfdriving-robot-car</a></p><p>And if you live in Berlin, make sure you drop by upcoming Robot Race on July 29th:</p><p><a href="https://www.meetup.com/Berlin-Self-Driving-Cars-Autonomous-Mobility/events/238595677/">Race Day Berlin - Hack/Race RoboCars</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad00a6804b53" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/build-your-own-self-driving-toy-car-ad00a6804b53">Build your own self driving (toy) car</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How self-driving cars get from A to B?]]></title>
            <link>https://medium.com/data-science/how-self-driving-cars-get-from-a-to-b-f18b1fc12044?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f18b1fc12044</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Wed, 12 Jul 2017 09:31:31 GMT</pubDate>
            <atom:updated>2017-07-12T14:12:05.531Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently I’ve stumbled upon a really interesting <a href="https://arxiv.org/pdf/1604.07446.pdf">paper</a> which makes great overview of motion planning and control techniques for self-driving cars in urban areas. In this post I’ll go over few parts of that paper which relate to my last Term 2 project for <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Udacity Self-driving car Nano degree</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*j2tp2yQ8w7kYuZyO8f6yeA.png" /><figcaption>The final output from last project in Term 2</figcaption></figure><p>One of the common of approaches for building a self-driving car system is to organize sensor perception (radar, LIDAR, camera, etc.) and decision-making process into a hierarchical structure.</p><h3>Hierarchical structure</h3><p>Decision-making module of self-driving car can be represented through four components:</p><ol><li><strong>Route planning </strong>— finding a shortest path between point A and point B. Think of what Google and Here maps are doing when you are searching for a nearby restaurant.</li><li><strong>Behavioral decision making</strong> — once the route has been found, this submodule is responsible for selecting driving behavior (lane following, lane change, turn right, etc.) based on other traffic participants and established traffic regulations. These behaviors are finite set of actions and are usually represented as <em>finite state machines </em>with<em> </em>some heuristics for reflecting current context of the vehicle surroundings (other vehicles, road works, etc.). One of the latest developments in this area is <a href="https://blogs.nvidia.com/blog/2017/07/07/deeptraffic-how-an-mit-simulation-game-uses-deep-learning-to-reduce-gridlock/?adbsc=social_20170710_73234797&amp;adbid=884463089122291712&amp;adbpl=tw&amp;adbpr=740238495952736256">AI approach</a> from MIT for modeling this step in decision making process.</li><li><strong>Motion planning</strong> — once behavior is selected, it needs to be translated into a trajectory which can be followed by a low-level feedback controller. Motion planning module is responsible for finding that trajectory/path.</li><li><strong>Vehicle control </strong>— feedback controller does all the hard work of moving the vehicle along side the selected trajectory by using the proper actuators (steering, throttle, breaks) and correcting the track errors.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*IFcAxf18JA_HKpJZUMDVtA.png" /><figcaption>Decision making process in self-driving cars. <a href="https://arxiv.org/pdf/1604.07446.pdf">Source</a>.</figcaption></figure><h3>A bit more on Motion planning and Vehicle control</h3><p>In the final project for Term 2 of the course, we are asked to steer a car in a simulator using <a href="https://en.wikipedia.org/wiki/Model_predictive_control">Model Predictive Control</a> (MPC in short) which basically covers last two parts of the self-driving car decision process (Motion planing and Vehicle control) from above.</p><p>MPC is able to predict the future states (position, orientation and velocity) of the vehicle for a finite time horizon. It is an iterative model which is constantly updating itself and predicting vehicle state for <em>N</em> time slots (<em>dt</em>). Prediction itself is based on mathematical vehicle model.</p><p>Creating a correct vehicle model is quite a complex task and it comes with trade offs (whether we should go with simple model with lower accuracy or more complex model with high computational requirements). In our project, we will be using <strong>kinematic model</strong> somewhat easier model to understand compared to <strong>dynamic models </strong>(which takes other forces into consideration which impact vehicle dynamics).</p><p>Here’s out vehicle model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*mTeE5nAE6nEgtCk-qS3jNQ.png" /><figcaption>Kinematic vehicle model</figcaption></figure><p>Vehicle coordinates are represented with <em>x</em> and <em>y</em>. <em>ψ </em>and <em>v </em>correspond to orientation and velocity respectively<strong><em>. </em></strong>Vehicle actuators are modeled with steering angle (<em>δ</em>) and acceleration (<em>a</em>).</p><p>Udacity simulator provides us with the way (reference points) of where the vehicle should go. This would come from <em>Route planning</em> step from above hierarchical flow in the real world. A higher order polynomial is being fitted through these points and output is being used to be compared to the MPC predictive model. This would be <em>Motion planning</em> step.</p><p>The way MPC works is that it predicts where the vehicle will be in future and tries to minimize the error rate by correctly adjusting the actuators (steering, throttle and breaks). This would represent <em>Vehicle control</em> step.</p><p>You’ll notice that <em>Behavioral decision making</em> step is missing in the project but it is due to the simplification of the scenario in Udacity simulator (no other traffic on the road, no road block, etc.).</p><h3>Delayed actuations</h3><p>One of the things we should pay attention to is the responsiveness of the actuators in our feedback controller. In real world, the request for changing the steering angle and/or throttle will be executed with a certain delay by the actuators on the car. In the project, an artificial <strong>100ms</strong> delay was introduced in order to simulate this behavior. I have handled this by setting kinematic equations to use the actuations from the previous time step. In this particular case, time step (<em>dt</em>) has the same value as the simulated delay.</p><h3>Final outcome</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FGsd8xn_cbOg%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DGsd8xn_cbOg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FGsd8xn_cbOg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/45816a85969bbe421490a865c15f35e7/href">https://medium.com/media/45816a85969bbe421490a865c15f35e7/href</a></iframe><p>The source code for the project can be found on my Github profile:</p><p><a href="https://github.com/bdjukic/CarND-MPC-Project">bdjukic/CarND-MPC-Project</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f18b1fc12044" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-self-driving-cars-get-from-a-to-b-f18b1fc12044">How self-driving cars get from A to B?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Non-AI approach for steering self-driving car]]></title>
            <link>https://medium.com/data-science/non-ai-approach-for-steering-self-driving-car-e841658ee695?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e841658ee695</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[idp]]></category>
            <category><![CDATA[control-theory]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Thu, 06 Jul 2017 21:09:37 GMT</pubDate>
            <atom:updated>2017-07-07T16:51:14.141Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the last projects in Term 2 of <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Udacity Self-driving car Nano degree program</a> is controlling a car in simulator using concepts from control theory.</p><p>There are currently three approaches for controlling the steering of self-driving car:</p><ol><li><strong>AI approach</strong> (check out solution I <a href="https://medium.com/towards-data-science/using-deep-learning-to-clone-driving-behavior-51f4c9593a57">wrote about before</a> based on behavioral cloning and Deep Neural Networks)</li><li><strong>Non-AI approach</strong> (which is using control theory and math to calculate the steering angle)</li><li>Combination of two above</li></ol><p>In this post we will cover the second one. In control theory, one of the methods which can be applied on autonomous vehicle for steering is called <strong>PID controller</strong> (<strong>P</strong>roportional <strong>I</strong>ntegral <strong>D</strong>erivative controller). <a href="https://en.wikipedia.org/wiki/PID_controller">Wikipedia</a> defines PID controller as:</p><blockquote>A control loop feedback mechanism commonly used in industrial control systems. A PID controller continuously calculates an error value <strong>e(t)</strong> as the difference between a desired setpoint and a measured process variable and applies a correction.</blockquote><p>In the real world out there, PID controllers are used from autonomous ship steering to drone auto pilots. It turns out also <a href="https://medium.com/@comma_ai/how-does-openpilot-work-c7076d4407b3">comma.ai</a> is actually using a variation of this approach for steering in their <a href="https://github.com/commaai/openpilot/blob/v0.3.2/selfdrive/controls/lib/latcontrol.py">openpilot</a> project.</p><h3>But what is PID actually?</h3><p>The general idea of how PID controller works is actually quite simple. In the following block diagram we can look at <strong>r(t) </strong>as desired position in traffic lane which comes from our localization and path planning systems. Position (ground truth) in traffic lane is measured using sensor(s) and is represented by <strong>y(t)</strong> . An error value <strong>e(t) </strong>would be a difference between <strong>r(t)</strong> and <strong>y(t)</strong>.</p><p>The PID controller is trying to minimize the error rate <strong>e(t) </strong>over time by applying an adjustment or control variable <strong>u(t) </strong>(in our case this is steering angle).</p><p>Correction value of <strong>u(t)</strong> is defined with Proportional, Integral and Derivative coefficients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*iA9cP2htfsi6X3oRoAueAQ.png" /><figcaption>Source: <a href="https://en.wikipedia.org/wiki/PID_controller">Wikipedia</a></figcaption></figure><p>The correction value of <strong>u(t)</strong> can be represented formally with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*97OwMMpD49RFhWxaAh5boQ.png" /><figcaption>Source: <a href="https://en.wikipedia.org/wiki/PID_controller">Wikipedia</a></figcaption></figure><h3>PID feedback mechanisms</h3><p>Let’s look at different feedback components of PID controller in more detail.</p><ol><li><strong>Proportional</strong> feedback— the rate of adjustment we need to do is proportional to the error rate <strong>e(t)</strong>. Naive thinking would lead you to ask if we need anything else, right? All we need to do is to set the steering angle to be proportional to the calculated error and we we’re done! Unfortunately it’s not that easy. This component is not taking in account mass and inertia. Using this component only in our controller will lead to so called<em> overshooting</em>. The important thing to remember here is that this component represents the gain (impact) current (present) error rate has on the system. It does not consider the past and the future changes of the error value.</li><li><strong>Integral</strong> feedback — is increasing over time and it’s covering the cases where the error value is to small and hardly detected by the Proportional feedback. It looks in the past and accumulates over time. It is also has a really slow effect on decreasing the error value in case it’s used as a single component in PID controller.</li><li><strong>Derivative</strong> controller — is able to predict the error value and therefore look into the future state of the system.</li></ol><p>There are quite a few videos explaining PID controller but I personally liked this one:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F0vqWyramGy8%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D0vqWyramGy8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F0vqWyramGy8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/06248cb55ad5fb59be45894eba4d5f03/href">https://medium.com/media/06248cb55ad5fb59be45894eba4d5f03/href</a></iframe><p>When it comes to the PID feedback coefficients in my project, I have picked them manually. There is a specific approach called <a href="https://www.youtube.com/watch?v=2uQ2BSzDvXs">twiddle</a> which is automating the process of finding the right parameters for your PID controller.</p><h3>Final outcome</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fqi3P8KShTGw%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dqi3P8KShTGw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fqi3P8KShTGw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/45d0634e93b1ed0d26685a4d9be0c6dd/href">https://medium.com/media/45d0634e93b1ed0d26685a4d9be0c6dd/href</a></iframe><p>The source code for the project can be found in my Github repo:</p><p><a href="https://github.com/bdjukic/CarND-PID-Control-Project">bdjukic/CarND-PID-Control-Project</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e841658ee695" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/non-ai-approach-for-steering-self-driving-car-e841658ee695">Non-AI approach for steering self-driving car</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tracking cars and pedestrians from an autonomous vehicle]]></title>
            <link>https://medium.com/@bdjukic/tracking-cars-and-pedestrians-from-an-autonomous-vehicle-ba1d2194b440?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba1d2194b440</guid>
            <category><![CDATA[sensor-fusion]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[kalman-filter]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Sun, 28 May 2017 21:31:01 GMT</pubDate>
            <atom:updated>2017-05-28T21:31:01.379Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog post we will be going through some of the methodologies autonomous vehicles are using in combination with <a href="https://en.wikipedia.org/wiki/Lidar">LIDAR</a> and <a href="https://en.wikipedia.org/wiki/Radar">radar</a> sensors in order to properly model (track) their surroundings (other cars, pedestrians, etc.).</p><p>Tracking object and its internal (unobservable) states (like position and velocity) using sensor data is a well researched area outside self-driving cars industry. Same problems are found in guidance, navigation and other control systems. In the 1960s, a set of algorithms was developed (namely <a href="https://en.wikipedia.org/wiki/Kalman_filter">Kalman filter</a> and its variations) which were used to model <a href="http://ieeexplore.ieee.org/document/5466132/?reload=true">Apollo</a> spacecraft trajectories to the Moon and back. Same algorithms can be applied in an autonomous vehicle in order to detect the position and velocity of the approaching car for example.</p><h3>Sensor Fusion</h3><p>Combining data from two or more sensors is called <strong>sensor fusion</strong>. Sensor fusion enables self-driving car to model its environment.</p><p>You might be asking yourself “Why do I need more than one sensor in the first place?”. That’s a really good question.</p><p>Generally, none of the sensors will give us a 100% correct observation of the car we are trying to track. All the measurements from the sensors come with the margin of error (noise). Also, not all sensors are working with the same efficiency in all weather conditions. For a more detailed sensor comparison, check out the picture below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wcphGyX62Dh2KB6qQEAUbg.png" /><figcaption>Sensor comparison chart</figcaption></figure><p>Kalman filter is solving this problem for us. It is capable of taking all the sensor data (with the included measurement noise) and estimate unobservable state (position and velocity for example) of the tracking object.</p><h3>Kalman filter in more details</h3><p>Kalman filter represent a series of <strong>prediction</strong> and <strong>update</strong> steps which are executed as the sensor data flows in:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yavba1VahLESVSGMKqSvyA.png" /><figcaption>Kalman filter flow for LIDAR and radar sensors</figcaption></figure><p>Let’s look at the simplified version of the Kalman filter in more details for the <a href="https://en.wikipedia.org/wiki/Linear_motion">linear motion model</a> of the tracking object using LIDAR data only.</p><p>In the <strong>prediction</strong> step, the current state (<strong><em>x′</em></strong>) is being estimated based on the previous state (<strong><em>x</em></strong>), corresponding motion model (<strong><em>F</em></strong>,<em> </em>also called transition state matrix) and process noise (<strong><em>v</em></strong>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/188/1*MJRPJg5kGaEZtcSeA0DTHw.png" /></figure><p>The above equation for the new state which contains information about position (<strong><em>p</em></strong>) and velocity (<strong><em>v</em></strong>) can be written as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/305/1*Y4MhOINZuppV1jpE9rSJXw.png" /></figure><p>This is equivalent to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/195/1*wpAS_2kDwaLFY4wXEcqnEw.png" /></figure><p>The estimated state itself can be written as a vector:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/172/1*ZdNM1s32MXf71Yy1Xb7yUA.png" /></figure><p>We are ignoring the process noise (<strong><em>v</em></strong>) for simplification reasons.</p><p>We would also need to calculate new covariance matrix (<strong><em>P′</em></strong>) (measure of the estimated accuracy of the state estimate) based on the previous one (<strong><em>P</em></strong>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*ZGU-ZTY0TEuXRglBYKvauA.png" /></figure><p>In the <strong>update</strong> step, we are taking the new readings (<strong><em>z</em></strong>) from the sensor and calculating measurement update (<strong><em>y</em></strong>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/167/1*E6AWgAl8rwi9J5pkpJ2uCA.png" /></figure><p><strong><em>H</em></strong> matrix in this case is a transformation matrix (often called measurement matrix) which maps sensor data space into our model space.</p><p>The next step is calculating the Kalman gain (<strong><em>K</em></strong>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*vzmvuUOKN3rkBOpSORiZwQ.png" /></figure><p>Which basically tells us what we should put our trust on, estimated state or sensor measurement:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*xHN_rwMU2hNGzsZ_bRVWJw.png" /></figure><p>If Kalman gain is a bigger number, sensor data will impact more on the calculation of the new estimated state. If it’s smaller number, then we’ll rely more on the mathematical model of the estimated state.</p><h3>Extended Kalman filter</h3><p>Once we introduce radar data into our Kalman filter flow, things get a little bit more complicated since mapping of the radar data into our estimated model space is done through non-linear functions.</p><p>We would need to replace H matrix from:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/167/1*E6AWgAl8rwi9J5pkpJ2uCA.png" /></figure><p>into something which can translate estimated state into measurement space:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*f9VtyYvJ65WPzBjcLrU1rg.png" /></figure><p>This can be accomplished with the following non-linear vector of mapping from polar to Cartesian coordinates:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/1*wsWhdtQUrBB3AmKcNWrFsw.png" /></figure><p>Using <a href="https://en.wikipedia.org/wiki/Taylor_series">Teylor series</a> we can approximate a non-linear vector from above with the linear one.</p><h3>Implementation</h3><p>C++ implementation of the Extended Kalman filter for LIDAR and radar data can be found on my Github profile:</p><p><a href="https://github.com/bdjukic/CarND-Extended-Kalman-Filter-Project">bdjukic/CarND-Extended-Kalman-Filter-Project</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba1d2194b440" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Detecting vehicles using machine learning and computer vision]]></title>
            <link>https://medium.com/data-science/detecting-vehicles-using-machine-learning-and-computer-vision-e319ee149e10?source=rss-97d54424597a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e319ee149e10</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Bogdan Djukic]]></dc:creator>
            <pubDate>Tue, 25 Apr 2017 22:06:24 GMT</pubDate>
            <atom:updated>2017-04-28T19:17:00.896Z</atom:updated>
            <content:encoded><![CDATA[<p>The final project from <a href="http://udacity.com/drive">Udacity self-driving car course</a> is creating a software pipeline which is capable of identifying cars in a video from a front-facing camera on a car.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OoPk0KYO5UV3xai20euEnA.jpeg" /><figcaption>A snapshot from the final output of the project</figcaption></figure><p>The course material is suggesting the usage of somewhat outdated approach for detecting vehicles which I figured out in the middle of the project by reading this <a href="https://t.co/VFxlrhQ70C">great paper</a> on the state-of-the-art computer vision for autonomous vehicles. Small snippet from that paper:</p><blockquote>With the work of Dalal &amp; Triggs (2005), linear Support<br>Vector Machines (SVMs), that maximizes the margin of all<br>samples from a linear decision boundary, in combination with<br>Histogram of Orientation (HOG) features have become popular<br>tools for classification. However, all previous methods rely on<br>hand-crafted features that are difficult to design. <strong>With the renaissance<br>of deep learning, convolutional neural networks have<br>automated this task while significantly boosting performance</strong>.</blockquote><p>As it turns out, Deep Neural Networks are outperforming the approach which I have used (Linear Support Vector Machines in combination with Histogram of Oriented Gradients). I will definitely go back to this project and try out some of the top performers in this list on the same problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1NE59oU2jDTeNMNcgjdngg.jpeg" /><figcaption>Table taken from <a href="https://arxiv.org/pdf/1704.05519.pdf">https://arxiv.org/pdf/1704.05519.pdf</a></figcaption></figure><p>The process of detecting the vehicles on the road could be summed up in the following steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/221/1*IIJPLy-B_9PtgfNVGFX3sg.jpeg" /></figure><h3>Training data analysis</h3><p>Training data is provided by Udacity and it consists of images of <a href="https://s3.amazonaws.com/udacity-sdc/Vehicle_Tracking/vehicles.zip">cars</a> from different angles (8792) and <a href="https://s3.amazonaws.com/udacity-sdc/Vehicle_Tracking/non-vehicles.zip">non-car</a> (8968) objects. Here are two samples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*ThfiUs-X94C0W4LiGlU5AA.jpeg" /><figcaption>Examples from the training data set</figcaption></figure><h3>Feature extraction</h3><p>In order to detect a car on the image, we need to identify <strong>feature(s)</strong> which uniquely represent a car. We could try using simple template matching or relaying on color features but these methods are not robust enough when it comes to changing perspectives and shapes of the object.</p><p>In order to have a robust feature set and increase our accuracy rate we will be using <a href="https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients">Histogram of Oriented Gradients</a> (HOG). This feature descriptor is much more resilient to the dynamics of the traffic. In essence, you should <strong>think of features as thumbprints</strong> of the objects you are interested in.</p><p><a href="http://scikit-image.org/docs/dev/api/skimage.feature.html#skimage.feature.hog">Scikit-image</a> python library provides us with the necessary API for calculating HOG feature. I have used <a href="https://en.wikipedia.org/wiki/YCbCr">YCrCb</a> color space and all its channels as inputs for HOG features extraction. I have tried other color spaces, but YCrCb gave me the best accuracy when training my prediction model. Here’s a sample of vehicle and non-vehicle image with HOG features from the same images as above:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*IhHW2al_-2z188mS8rylBQ.jpeg" /><figcaption>Extracted HOG features from sample training data</figcaption></figure><p>HOG feature extraction was based on 9 orientations, 8 pixels per cell and 2 cells per block. Increasing orientations and pixel per cell parameters did improve prediction time but the accuracy rate of the model went down.</p><h3>Model training</h3><p>In order to detect the car based on our feature set, we would need a prediction model. For this particular case we will be using <a href="https://en.wikipedia.org/wiki/Support_vector_machine">Linear Support Vector Machines</a> (Linear SVMs). It is a supervised learning model which will be able to classify whether something is a car or not after we train it.</p><p>HOG features have been scaled to zero mean and unit variance using <a href="http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html">Standard Scaler</a>.</p><p>I have divided provided data set into training set (80%) and testing set (20%). Images have been shuffled as well before kicking off the training session. In the end Linear SVMs model with the extracted HOG features on YCrCb color space reached <strong>98.06%</strong> accuracy rate.</p><h3>Sliding windows</h3><p>Once we have the prediction model, it’s time to use it on our test images. Prediction model will be applied in a special technique called Sliding Windows. With this technique we will be running prediction model on sub-regions of the images which is divided into a grid.</p><p>In order to increase the robustness of this approach we will be adding multiple grids which will be traversed by the prediction model. We are doing this since cars can appear on the image in various sizes depending on its location on the road.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5VnOJago9j8TTaQ7HUTpPA.jpeg" /><figcaption>Multi-window appraoch for sliding window technique</figcaption></figure><p>This is how this concept looks like when it is applied on our test image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/1*6MxTVwcQW-4SGuYnAs_xTQ.jpeg" /></figure><p>I have used different window sizes (from 128x128 for area closer to the car and 64x64 for area further away from the vehicle). Windows overlap is set to 65%.</p><h3>Eliminating false positives</h3><p>In order to improve the accuracy of the final output we will be trying to find multiple hits for the object of interest in the similar area. This approach is equivalent to creating a heat map.</p><p>The next step is introducing threshold which needs to be met in order for a specific hit count from the heat map to be accepted as a detected car. In our case threshold has a value of 2.</p><p>This is an example of applying the heat map and thresholding it to a specific value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HBdTVrzqGgN9eRPheyu7rw.jpeg" /></figure><h3>Final outcome</h3><p>The final outcome is not perfect but the pipeline itself is showing good potential.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FaP4ZFcPH7wM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DaP4ZFcPH7wM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FaP4ZFcPH7wM%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bfcdc57ad75a7fdd6bd9637d02f159df/href">https://medium.com/media/bfcdc57ad75a7fdd6bd9637d02f159df/href</a></iframe><h3>Conclusion</h3><p>If I would get a chance to re-do this project, I would probably go for a Deep Neural Network approach. I have spent considerable amount of time searching for the right color space, HOG parameters and windows slide’s size.</p><p>The performance of the pipeline is not great and can be improved. Deep Neural Network approach would have better performance numbers.</p><p>Since this car detection approach is based on camera it’s prone to usual challenges with this kind of sensor (bad visibility, reflections, etc.).</p><p>Project can be found on my <a href="https://github.com/bdjukic/CarND-Vehicle-Detection">Github</a> profile:</p><p><a href="https://github.com/bdjukic/CarND-Vehicle-Detection">bdjukic/CarND-Vehicle-Detection</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e319ee149e10" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/detecting-vehicles-using-machine-learning-and-computer-vision-e319ee149e10">Detecting vehicles using machine learning and computer vision</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>