<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Spencer Kelly on Medium]]></title>
        <description><![CDATA[Stories by Spencer Kelly on Medium]]></description>
        <link>https://medium.com/@spencer_kelly?source=rss-68cdee1d3fb3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*-t_Dd-lsApE4XWDLxTyJaA.jpeg</url>
            <title>Stories by Spencer Kelly on Medium</title>
            <link>https://medium.com/@spencer_kelly?source=rss-68cdee1d3fb3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 22:53:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@spencer_kelly/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cards Against Humanity Card Generation (Slightly NSFW)]]></title>
            <link>https://medium.com/data-science/cards-against-humanity-card-generation-slightly-nsfw-a9c132d88345?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9c132d88345</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[funny]]></category>
            <category><![CDATA[cards-against-humanity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Thu, 16 May 2019 22:37:28 GMT</pubDate>
            <atom:updated>2019-05-18T22:52:07.170Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using Char-RNNs for the dumbest reasons</h4><p>(WARNING: Cards Against Humanity is an adult themed game, and includes lots of rude language. Nothing graphic will be shown in this post, only words, but it could still get you some weird looks in the workplace)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*mX49vRdMafEkwhAgRDh3Kg.png" /></figure><p>What is Humor? Can an AI understand what makes something funny? Can writing a Medium post validate the time you spent training a computer to say dirty words?</p><p>I won’t be answering any of those questions today. But hopefully we can all have a bit of fun looking at “AI” generated CAH cards, and getting more insight into what makes this task especially hard.</p><p>Let me start out by saying that I’m not breaking any new ground here. Text generation has been done for a while, and by people much smarter than me. So I’m not going to try to explain in detail deep learning methods I used here. If you’re a deep learning nerd like me, I’ll just say the model structure is <em>Embedding-&gt;LSTM-&gt;Dropout-&gt;Dense (length of char list).</em></p><p>There are some challenging parts of this task though, so i’ll give a short explanation of those below. Feel free to skip further down if you just want to see the cards I generated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*BMkNx8-cpRyjA-MInQbq_w.png" /></figure><h3>Data Problems and Why I Cheated</h3><p>The first step to any deep learning project is finding the data. Thankfully, a group of crazy people had already put in the time to create a <a href="https://docs.google.com/spreadsheets/d/1lsy7lIwBe-DWOi2PALZPf5DgXHx9MEvKfRw1GaWQkzg/edit#gid=2018240023">spreadsheet </a>of every CAH card, including custom expansions created by third parties. This meant I could either train the model with just the ~2k official white cards, or an additional ~19k third party white cards. I’m using the white cards because there are more examples to train on, but I promise to give you a taste of what it looks like when the model tries to create black cards as well.</p><p>The problem with all this beautiful data, is that it’s extremely diverse. At it’s core, deep learning is training a program to recognize patterns, and in this case, replicate them. It’s easy to create a model that generates <a href="https://www.google.com/search?rlz=1C1CHBF_enUS797US797&amp;ei=rxLaXKDWNMuG0wKYyqCADA&amp;q=generating+shakespeare+scripts&amp;oq=generating+shakespeare+scripts&amp;gs_l=psy-ab.3..35i39.2541.7996..8499...2.0..0.92.1075.14......0....1..gws-wiz.......0j0i13j0i13i30j35i304i39.5znL9r5jSWo">Shakespeare scripts</a> because Shakespeare has a consistent voice, talking about relatively similar content. And while CAH does have a distinct voice, it doesn’t have consistent content. And the distinct voice is blurred when bringing in third party cards.</p><p>Think of it this way, you’re learning a new language and I give you two paragraphs to read. One of them is a short movie review, and the other is a list of unrelated short sentences that are meant to make people of that culture laugh when paired with other unseen sentences. If I ask you to translate both, which do you think will be easier? I’d put my money on the former because when you don’t know what a word means, you can rely on the context around it to help you.</p><p>Because of this challenge with diverse data, I had to cheat a bit. I’m not directly just taking the output of the Neural Network and adding it to the list of generated cards. There’s a bit of post processing done on the cards, and human selection to eliminate cards that are just clearly nonsense. I realise this is cheating, but just consider the system more of a “Card Ideas Generator”. I’ll make sure to show you some of the raw output as well though.</p><p>After the network produces potential cards, I run them through a text processor that checks them for similarities with cards in my dataset. Similar cards are inevitable, but I’m trying to eliminate cards that don’t significantly differ in meaning from cards in the database. A bit of necessary overfitting also means that occasionally the network will just produce an exact copy of a card it’s seen.</p><p>Then I run the cards through an automatic spelling corrector. This fixes minor spelling errors that the network made and is still completely automatic, so it’s fair game in my opinion. After this is done, I select the best cards and add them to my list of generated cards.</p><h3>SHOW ME THE CARDS!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*ukK0ACCG45csM7PLvmNH4g.png" /></figure><p>Below is a list 50 white cards I picked out:</p><ul><li>a horny forever</li><li>a hardworking pile of colon stained shit</li><li>an unintentional science at feelings</li><li>big bathrooms</li><li>low branches</li><li>being too late</li><li>coat hair</li><li>white pudding</li><li>secretly tickled meat</li><li>a big fat dead hooker</li><li>a gay</li><li>trying to sex with a large poopy finger</li><li>unicorn sticks</li><li>not flaws</li><li>military decisions</li><li>forgetting the supremacy ideas.</li><li>bully bumpers</li><li>a pile of surprise sex tape.</li><li>that dried medical lube</li><li>borderline inappropriately hanging out.</li><li>your mom’s crush.</li><li>a man in a sippy cup</li><li>good box</li><li>horny white justice.</li><li>a burping ballerina</li><li>making bread red children.</li><li>trump-feeling websites.</li><li>clown cock.</li><li>going to the monkey bar</li><li>not weapons.</li><li>a 6-year-old babysitter.</li><li>my choke hole in my basement</li><li>a range of worn cards</li><li>sad penis.</li><li>dad’s funny bitch.</li><li>my post-travel condoms.</li><li>suffer school</li><li>holding mom</li><li>the sexual vagina that just wanted to communicate</li><li>a couple of boards with booze brains</li><li>the size of a bulge.</li><li>grandma training.</li><li>singing “o tannenbaanana.”</li><li>the 69%</li><li>just a love bag</li><li>m’finger bitch</li><li>sad standards</li><li>buying a virgin on top of the world</li><li>something fried</li><li>sorry.</li></ul><p>The cards that make sense represent about a third of the results the network produced. Admittedly, some of the cards above don’t make sense, but some of the cards that make sense were boring, and part of the fun is creating some nonsensical cards.</p><p>Here’s an idea of what the raw (slightly formatted but unmodified) output of my network looks like:</p><pre>penis.<br>a shit-to-fie<br>the contraction of a burrito.<br>the ring my slowl sorr that tells you with the chinese dogs.	public motherwease.		<br>female social suberious with forgetting to parol into a petrodomined body show.		<br>an erection that lasts for hours.		<br>aming down on a game of minivan		<br>the poor		<br>a female orgasm		<br>subway shaving a single panties.		<br>a keyback horse with a tremendous dingus.		<br>an 80-year-old girlfriend		<br>a burger silence		<br>recaulation	<br>zego for one		<br>magic human remains for your secret school.		inappropriate labor.		<br>having sex with a dead baby fetus.		<br>(rangta butt pullting people.		<br>being kernels for domester of the mongrey.	<br>a flamboyant man that fires a smiling prince.		<br>a valladian memo baby.		<br>idiots		<br>a somen sort of this with the color off the side to.		spending the nuit as every day		<br>smelling sex lives of sexual charases.		<br>getting tape on it.		<br>a doctor with a sopping wet vagina.		<br>a fairy food.		<br>making up for your home.		<br>incurable some sexy tantrum.		<br>buffalo big destroying pop</pre><p>If you play CAH you might recognize a few of these. “penis”, “idiots”, and “the poor” are all direct copies of cards. And “a female orgasm” is only a slight modification on the card “the female orgasm”. This is the network memorizing some cards, and just spitting them back out. That’s why I analyze the cards for their similarity to other cards in the database before accepting them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*g7duZXlUU5ztMUI-FskG8Q.png" /><figcaption>Average number of words in a card was 4.94</figcaption></figure><p>You’ll also note that the list of white cards I posted above hardly has any longer cards in it. And the raw output should show you why. The network couldn’t learn to string together longer sentences that made sense. I blame this mainly on low number of long cards in the dataset.</p><h4>Black Cards (prepare for some garbage)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*qrIwpW05X3WVMj9OS73edQ.png" /></figure><ul><li>every successful man is _________.</li><li>nobody people think i am a powerful wishing with ___.</li><li>the humpty dance is your chance of getting killed by a refugee terrorist.</li><li>the new morning, there was _______ and ______.</li><li>what do with the sound of ______.</li><li>wake up when my husband would say _______</li><li>I knew we would be right!</li><li>the next people invented ______.</li><li>our notorious face of my last position was ________.</li><li>_______ is a sure sign of my life.</li></ul><p>I could keep going but it’s clear that even the cherry picked best examples of black cards make very little sense. They also tend to be more clearly derived from cards the network has tried to memorize.</p><p>Once again, here’s an example of the raw output of the network:</p><pre>nothing is better the good of death?		<br>check out my new podcast about ______ and ______.		<br>ted cruz caused a still want of ______.		<br>the red room for eating ____ for a kirld and gentler ______.		the probilew recommend a trip was completely ruined by ______.		___________. a wedding is a the face.		<br>on cheeristed ______ for his/her dont combrate your signessant-class impressed the only acceptable to have ____ in public.		<br>g go goage ______.		<br>______ is a sure sign of my life.		<br>snatch and ______.		<br>what&#39;s the biggest source of tenish in the charge of ______.		what do you were a parent when i never have to be a proctive in raid of ______.		<br>momes, my imanager compaint for ______.		<br>i wonder if ______ was all over the poil.		<br>immers is now explore the honors using lose in the country black, ive had a horrible vision, father. i saw mountains crumbling, stars falling from _____.		<br>there can a parent that did you have a moment to talk to my kids and do with ______.		<br>i got have ______.</pre><h3>Conclusion time</h3><p>Overall, I’d call this project a somewhat interesting/amusing failure. The white cards were often funny but kind of reminded me of one of those quizzes where you use the first letter of your last and first name to create your “hooker name” or whatever. It just shoved words together and hoped it was funny. And let’s not even talk about the black cards. Clearly more examples were needed there.</p><p>I’m guessing a closer to “state of the art” network could have done better (no shit), but I’m not sure it would have been THAT much better. I think it would have yielded results that made sense more consistently and copied less, but it’s jokes probably wouldn’t have been better. The problem is that in order to make good original cards, you have to be aware of the culture and things around you. And a network like this one only knows cards that are already created.</p><p>The only way I can see of solving this is by training a more advanced network on a broader database of text on many topics, then specializing it for the task of generating cards. But how to do that is currently beyond me.</p><p>If you have any ideas of how to do that, or simply better ways to make this task work, please let me know. I’d love to hear your feedback. I’d also love feedback on the post if you have it. I’m still just learning to write these articles and I could use some constructive criticism.</p><p><a href="https://azala.info/static/index.html?deck=46622">link to deck (whitecards only)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9c132d88345" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/cards-against-humanity-card-generation-slightly-nsfw-a9c132d88345">Cards Against Humanity Card Generation (Slightly NSFW)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lyft Segmentation Challenge]]></title>
            <link>https://medium.com/@spencer_kelly/lyft-segmentation-challenge-9c77bb8b497c?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c77bb8b497c</guid>
            <category><![CDATA[challenge]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[lyft]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Tue, 05 Jun 2018 19:35:02 GMT</pubDate>
            <atom:updated>2018-06-05T19:35:02.510Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*8Nlyvk1dZJNhPixvVTEj2Q.png" /></figure><p>In May, 2018, Udacity and Lyft partnered together to challenge their self driving car students. Each participant had to create a neural network capable of labeling each pixel in an image as belonging to either a car, the road, or something else. This is known as Semantic Segmentation and the models they created were ranked on a leader board. Ranking were based on accuracy, points were taken away if it ran at less than 10FPS, and the top 25 students won an interview with Lyft.</p><p>I’m proud to say, I was in that top 25, just barely… I finished 23rd, and in this post, I’d like to explore some of what I did right to achieve that score, and some of what I could have done better.</p><h3><strong>Time Limitations</strong></h3><p>Since the challenge only lasted a month, time was in short supply. And since I decided to take on the challenge halfway through the month, I only had two weeks to work on it. I also happen to work on weekends, so I only had 10 days. Obviously, this is almost completely my fault, so we can file that under the “things I could have done better” category. Nevertheless, my strategy for this challenge was completely shaped by the small amount of time I had.</p><p>First, I decided to use a Semantic Segmentation library I found on <a href="https://github.com/GeorgeSeif/Semantic-Segmentation-Suite">GitHub</a>. I know first hand how long it can take to create your own network and iteratively improve its architecture, and didn’t want to waste too much time. This had the added advantage of easily being able to try out different models. There were some disadvantages here (namely bugs) and some modification of the library was necessary, but overall, it was an extremely good decision.</p><p>Data for this competition was gathered from Intel’s <a href="http://carla.org/">CARLA </a>Simulator. We were given 1000 images to start our training, and could use the simulator to gather more data. But being short on time, I didn’t want to get bogged down in the trouble shooting that can come with trying to set up <a href="http://carla.org/">CARLA</a>. So I decided to make due with the original training data, plus an additional 3000 images provided by a kind user on the forum. Overall, I think this was a mistake. Assuming it wouldn’t have taken TOO much time to collect, I think creating a larger training dataset would have greatly improved my models performance. It’s no secret that the quality of your model depends on the quality of your dataset, and I would have preferred to have more like 50,000 images to train on. That would take a long time to train on, but as long as I tested my models first on a small fraction of that data, I probably could have made it work.</p><h3>My Approach</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*HRiNDtTo0e4YYwUvhQXCrg.png" /><figcaption>RGB image from my training set</figcaption></figure><p>I took a very iterative approach to this challenge. I trained 16 different models, and created 6 different versions of my dataset that were modified in various ways. To not bore you with the details, let me explain the approach I settled on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*ZG8P551YPEK-IyVzmYQmOw.png" /><figcaption>Labeled image with only road and cars</figcaption></figure><p>Since we only cared about the street and the vehicles, I labeled everything else as “other” (black). I also made sure to label the hood of the car as “other” so that the model ignored it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*WO-Yl4IyT5hemNLb3378Bg.png" /><figcaption>Cropped RGB image</figcaption></figure><p>I then cropped the images to increase the inference speed of the model. I was able to do this because the bottom of the image was always the hood of the car, and the top of the image never contained either road or cars.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*MT7WYxjczDGOiAWObadlrw.png" /><figcaption>Predicted labels</figcaption></figure><p>This is an example of the images my model would output. To use the result, I just have to add black pixels on the top and bottom of the image to bring it back to its original dimensions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*qz56pE4eVxuaUOI7tL0Wog.png" /><figcaption>Prediction combined with original RGB image</figcaption></figure><p>Here’s the prediction overlaid on the original image. As you can see, it does a pretty great job, and I’m fairly happy with the results.</p><h3>The Model</h3><p>I chose to use a <a href="https://arxiv.org/pdf/1706.05587.pdf">DeepLabv3 </a>architecture with a 152 layer pretrained ResNet. ResNet is one of the best networks you can use for any image classification problem, and its skip connections work especially well for Semantic Segmentation. I was hoping to use a much smaller 50 layer ResNet, but I saw significant performance improvements from using the deeper network, and I was still maintain greater than 10 FPS.</p><p><a href="https://arxiv.org/pdf/1606.00915.pdf">DeepLab</a> is a segmentation network developed by Google that relies on Convolutional layers, Atrous Spatial Pyramid Pooling, and <a href="https://arxiv.org/abs/1210.5644">Fully Connected Conditional Random Fields</a> (CRF). By using <a href="https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d">Atrous convolutions</a> with different dilation rates, they were able to capture features in the image at different scales. This makes it much easier for the network perform well on small and large objects alike. And the use of fully connected CRF’s helps the network localize better. This means improved accuracy around the edge of objects. <a href="https://arxiv.org/pdf/1706.05587.pdf">DeepLabv3 </a>improves upon the previous versions by introducing global average pooling, and tweaking the model’s hyper parameters.</p><p>I trained my final model on a 1080ti for 300 epochs with an Adam optimizer and a learning rate of 0.0001. Due to memory constraints I only used a batch size of four. With 4000 images I decided to use 85% for training and 15% for validation. My test set was the data our models were being tested on for the competition.</p><p>Data augmentation also proved to be helpful. I randomly flipped the images horizontally, rotated them by up to 20 degrees, and modified their brightness by a factor up to 2. I tried further augmentation but it didn’t prove useful. I do wish I had more time to get a random crop working, as that is proven to be extremely helpful for Segmentation and Object Detection problems.</p><p>In the end, I achieved a weighted F score of 90.56, at 11.3FPS, and as stated earlier, this put me in 23rd place.</p><h3>Final Thoughts</h3><p>I had a great time participating in this challenge. I had do drop a class and put another on hold, but in my mind it was completely worth it. Classes are fun, but nothing compares to creating your own project, and I don’t know any better way to learn either. Next time an opportunity like this comes around I plan on being prepared, starting early, and using what I learned here to refine my approach.</p><p>Thank you to the amazing teams at Udacity and Lyft for putting on this contest and being very active and helpful on the forum. And thanks to all my fellow students who were giving great advice, and providing extra data to students without the time or capability to collect their own.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c77bb8b497c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Should You Learn After the Self-Driving Car Nanodegree?]]></title>
            <link>https://medium.com/@spencer_kelly/what-to-learn-after-the-self-driving-car-nanodegree-ba25747a1359?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba25747a1359</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[education]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Wed, 28 Feb 2018 04:16:12 GMT</pubDate>
            <atom:updated>2018-03-10T15:37:51.162Z</atom:updated>
            <content:encoded><![CDATA[<p>A guide for students coming from the SDCND and looking to pursue a career in the field. In this article I’ll explore some of the skills you still might want to learn, and give you the resources to do so.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*p7q5jkUCmCuzYUxfo0WVLQ.png" /></figure><p>The Self-Driving Car nanodegree is great but it can’t teach you everything. So where is it lacking? Unlike Udacity, I can’t just call up SDC companies to ask them, so I have to look at what they’re hiring for. I’ve spent a good bit of time sorting through SDC job listing for jobs that strongly relate to the nanodegree. And I’ve tallied up all the times these job listings mentioned a skill. So I now have a list of what self driving car companies look for in candidates, ranked by how many companies listed the skill in the job description.</p><p>So what do companies say they want to see in new hires? Here’s the top 20.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*KgcArFeIiipVsK9a-avTvA.png" /><figcaption>Recognize the style of slide from anywhere? :-)</figcaption></figure><p>Hopefully you’ll recognize almost all of the things listed, and that’s a good sign. But you probably don’t know all of them. I’ll be focusing on the skills we aren’t taught in the nanodegree in this article. A quick not though, I think you should really pay attention here to C++, ROS, AI/Deep Learning, and Math. These skills were required significantly more then skills further down the list, and as such, seem to be more important.</p><p>I should note here as well, that learning a little bit of everything might not be in your best interests. You might want to pick one or two skills and really try to dive deep on those topics. For example, a computer vision job might only require you know computer vision, and math. If you want to focus on getting that job, it would be better to spend all your time improving those skills rather than taking a broader approach to your education.</p><p>Now, lets look at the skills you are already somewhat familiar with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*th13bNuj7p4WkY6effx_Hw.png" /></figure><p>I’ve highlighted here the skills that they teach in the Nanodegree, and I’ve include Python, C++ and Linux because while the nanodegree only touches on them, they are necessary to complete the course.</p><p>From here you can start picking skills you’d like to learn. You can choose by what is asked for the most, or what you’re interested in, or maybe even what’s offered by Udacity?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*kwuHbdSyqCnbunQknA4_ug.png" /><figcaption>Turns out these are pretty easy to make in google slides.</figcaption></figure><h3>Udacity Links:</h3><p><strong>Embedded systems:</strong> <a href="https://www.udacity.com/course/embedded-systems--ud169">Embedded Systems</a></p><p><strong>Algorithms:</strong> <a href="https://www.udacity.com/course/intro-to-algorithms--cs215">Intro to Algorithms</a>, <a href="https://www.udacity.com/course/technical-interview--ud513">Technical interview</a>, <a href="https://www.udacity.com/course/computability-complexity-algorithms--ud061">Computability/Complexity</a></p><p><strong>Reinforcement Learning: </strong><a href="https://www.udacity.com/course/reinforcement-learning--ud600">Reinforcement Learning</a>, <a href="https://www.udacity.com/course/deep-learning-nanodegree--nd101">Deep Learning Nanodegree</a>, <a href="https://www.udacity.com/course/artificial-intelligence-nanodegree--nd889">AI Nanodegree</a></p><p><strong>CUDA/GPU Computing: </strong><a href="https://eu.udacity.com/course/intro-to-parallel-programming--cs344">Intro to Parallel Programming</a></p><h3>Non-Udacity Links:</h3><p><strong>C: </strong><a href="https://www.edx.org/course/cs50s-introduction-computer-science-harvardx-cs50x">CS50</a>, <a href="http://www.learn-c.org/">Learn C</a></p><p><strong>Digest and Implement Research Papers: </strong><a href="https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf">How to read a paper</a>, <a href="http://codecapsule.com/2012/01/18/how-to-implement-a-paper/">How to implement a paper</a></p><p><strong>Robotic Hardware/Sensor Experience: </strong>(links needed)</p><p><strong>SLAM: </strong><a href="https://github.com/kanster/awesome-slam#courses-lectures-and-workshops">Awesome-Slam</a>, <a href="https://www.youtube.com/playlist?list=PLpUPoM7Rgzi_7YWn14Va2FODh7LzADBSm">Claus Brenner Lectures</a>, <a href="https://www.youtube.com/watch?v=U6vr3iNrwRA&amp;list=PLgnQpQtFTOGQrZ4O5QzbIHgl3b1JHimN_">Cyrill Stachniss</a>, <a href="https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-412j-cognitive-robotics-spring-2005/projects/1aslam_blas_repo.pdf">Slam-For-Dummies(pdf)</a></p><h3>Other Links</h3><p><strong>Embedded Systems: </strong><a href="https://ce.uci.edu/areas/engineering/embedded/">ES certification</a>, <a href="https://extension.ucsd.edu/courses-and-programs/embedded-engineering">UCSD courses</a></p><p><strong>ROS: </strong><a href="https://www.udacity.com/course/robotics-software-engineer--nd209">Robotics Nanodegree</a>, <a href="https://www.robotigniteacademy.com/en/accounts/login/">RobotIgnite Academy</a></p><p><strong>Deep Learning: </strong><a href="http://course.fast.ai/">fast-ai</a>, <a href="https://www.coursera.org/specializations/deep-learning">Andrew Ng</a></p><p><strong>Math: </strong><a href="https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw">3blue1brown</a>, <a href="https://www.khanacademy.org/">Khan-Academy</a></p><p>I hope that helps, and I’m always looking for more links to keep this up-to-date and useful. So leave a response if you want to add a link page and I’ll get on it quickly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba25747a1359" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning to Learn (on your own)]]></title>
            <link>https://medium.com/@spencer_kelly/learning-to-learn-on-your-own-8e3866c99886?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e3866c99886</guid>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[mooc]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[online-education]]></category>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Tue, 27 Feb 2018 18:42:59 GMT</pubDate>
            <atom:updated>2018-02-27T18:42:59.283Z</atom:updated>
            <content:encoded><![CDATA[<p>My journey through many a MOOC (massive open online course), and the lessons you can take from it.</p><p>Quick disclaimer, it should be noted that I’ve either accidentally written a divine tome about online education… or this advice will only work for some people. Ultimately, everyone has to decide what works for them, and there’s no wrong way to do it.</p><h3>Background</h3><p>Before the fall of 2016 I didn’t really know what I wanted to do with my life. I had tried acting, taught dancing, ran a Live Action Role Play group, and gotten a marketing certificate from my local Junior College. And after doing all this, I still didn’t like my career options. I wasn’t against college, but it costs a lot of money and I wanted to pursue other options before making that investment.</p><p>Up until this point in my life I’d never really considered Computer Science as a career choice. I mean after all, you had to be super smart to do it right? But I had grown out of that mindset over the years and my Girlfriends parents were both CS people so I decided to try it. Computer Science is also a field where skill matters more than a degree, so I could try to do it without paying for college.</p><h3>Choosing Your First Class</h3><p>I went through a very long and arduous process to decide on what class to take first. You can find my process below :-)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/465/1*1YeFkF14Ua-FE9vGTlV9BQ.gif" /></figure><p>That isn’t to say you shouldn’t put time into deciding what class to take first, but it’s more important that you start learning, then that you take the perfect class first.</p><p>I decided to start with CS50 on Edx. It’s challenging, and the teacher is an amazing speaker.</p><h3>How to Learn and Stay Motivated</h3><p>It’s all well and good to start a class, but just starting it isn’t going to get you anywhere. And sticking with it can be really hard.</p><p>Here’s where the disadvantages of MOOC’s lie. Most of them are not great at providing you any sort of motivation to complete the course. When you go to college, you have money invested in it, you are living in their dorm, and if you don’t do everything on time, you fail. This provides great external motivation and can help people stick with their education. But MOOC’s don’t have any of those things generally, so how do you do it?</p><p>Well unfortunately, the answer is it probably won’t be easy. I’ve been learning online for a year and a half now and there will always be hard times. The most important part is that you <strong>keep going. </strong>I know saying that isn’t really helpful though, so I’ll provide some tips for getting through lack of motivation, or frustration.</p><p>At first, my strategy was just learn when I felt like it and play video games when I didn’t. This was not a good strategy… I ended up spending most of the time on video games, and less and less on coding.</p><h3><strong>Treat Learning Like A Full-Time Job.</strong></h3><p>This obviously can only work for people who don’t already have full-time jobs, but even if you do, I’d recommend treating it like a part time side job. To adopt this way of thinking I started myself on a schedule. In the morning I’d wake up and get breakfast. Then i’d hop on my computer to learn, and besides food breaks, I’d only stop at night when my friends were online to play games.</p><p>But there was a problem with that too. I was getting burned out. I was stressed throughout the day and I started to give up the regiment because of it. Now that I was serious about my education, I needed to learn to…</p><h3><strong>Take breaks</strong></h3><p>Stupid thing to have to learn, I know, but really important. When you get frustrated, go do something else. Preferably something physical like a short walk or even eating something. It really helps relieve the stress and refresh your motivation.</p><p>Stress isn’t necessarily a bad thing though, stress means you’re learning. If you really push yourself to learn, you’ll get stressed. But at this point in my education, I almost like stress because I’ve reframed it my mind as a good thing. If I’m stressed, I know I’m really pushing myself and learning a lot, and so along with the unpleasant feeling of being stressed, I also feel a sense of victory whenever I am able to keep working through it.</p><h3>Changing it up</h3><p>If all else fails, and you can’t keep yourself on task and learning, try changing something important. For instance, changing your environment. When I’m not feeling up to learning some days I’ll head over to starbucks with my laptop and just focus on the task all day. The lively environment (and possibly the caffeine) get me back to my classes in no time. Infact I almost always have my most productive days at starbucks.</p><p>You can also change what you’re learning, switch to a different lesson or topic. I don’t mean change fields completely, but just switch the specifics around a bit. But be careful, switching too often (multiple times a day) can really reduce your productivity, and at least for me, became a form of procrastination.</p><h3>Learn What Motivates You, and Implement It.</h3><p>As you spend a lot of time learning and move through a number of online classes, you’ll probably learn what motivates you, and what doesn’t (if you don’t already know). And this is extremely valuable because you can start using this knowledge to motivate yourself. Maybe it’s having specific goals every day, or having some sort of project to build. I do best when am on a schedule and so I will set my own schedules, and only choose online courses that have fixed time frames.</p><h3>Trust That It Will Get Easier, and Keep Going</h3><p>This is easily the most important thing you have to do to make it in the MOOC world. While I was trying to learn computer science, and later AI, there were many many times when I felt like I couldn’t do it. You can’t let those times get to you. The only time you fail is when you don’t get back up, and push through the hardship.</p><p>When I was going through <a href="https://www.udacity.com/course/deep-learning-nanodegree--nd101">Udacity’s Deep Learning Nanodegree</a>, and later in the <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Self-Driving Car Nanodegree</a> as well. There were many times when I felt way out of my depth in my math knowledge. I was homeschooled through high school and never took math courses at the Junior College, so I didn’t know Calculus or Linear Algebra (or even advanced algebra and geometry). But luckily, I was able to push through it and learn the math I needed too. After all, I was already learning online, why not just focus on math for a bit?</p><p>And it does get easier after a while. You get used to the routine, and being self motivated becomes normal. I still have bad days but I can be sure now that they are just that DAYS, not a pattern of failure or the straw that breaks the camel&#39;s back, but just bad days.</p><p>And that’s what it’s all about, trusting yourself. If you don’t believe that you can learn something, it probably won’t happen. But ultimately, learning is just about hard work, and you have to trust that you’ll get it eventually.</p><h3>GOOD LUCK!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e3866c99886" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Path-Planning with a Finite State Machine]]></title>
            <link>https://medium.com/@spencer_kelly/path-planning-with-a-finite-state-machine-c6794570caff?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6794570caff</guid>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[path-planning]]></category>
            <category><![CDATA[finite-state-machine]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Thu, 01 Feb 2018 22:35:16 GMT</pubDate>
            <atom:updated>2018-02-01T22:35:16.271Z</atom:updated>
            <content:encoded><![CDATA[<p>This project was part of the <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Self-Driving Car Engineer Nanodegree Program</a> and if you want to give it a try yourself, the simulator and starting code are <a href="https://github.com/udacity/CarND-Path-Planning-Project">here</a>.</p><p>I loved this project, it was extremely rewarding, and that was partially because of how challenging it was. I used a whiteboard to plan the project and then tried to implement it all in main file. That was a mistake. After working on it for almost a week I was having trouble keeping track of what my own code was doing. So I decided to scrap almost all my code and implement it in a ‘Car’ class. Then after doing that I spent almost 3 days debugging the finite state machine.</p><p>Once again, it takes way too long for me to realise that a project needs to be broken down into pieces and put in classes… go figure.</p><p>This will be a high level overview of my project, if you want something more in depth, check out the <a href="https://github.com/ablacklama/Highway-Path-Planner">code on github</a>.</p><h4>Project Goals</h4><p>The goal of this project is to safely navigate a simulated car around a highway track. This involves safely navigating traffic and maintaining a reasonable speed by changing lanes if cars ahead are too slow. Doing this requires using sensor fusion and localization data to plan a path for the car.</p><h3>Implementation</h3><p>The parts of this code are explained in the order that makes the most sense to a person. Please look at the code if you want to see how it’s actually assembled.</p><h3>1. CarData Class</h3><p>This class handles most of the interpretation of sensor fusion and localization data. It also keeps track of what state our finite state machine is in. It stores cars detected by the sensor fusion data in Car objects that keep track of what lane that car is in and where it is predicted to be at the end of current path. This helps with functions like ClosestInLane which uses the cars current position to return the closest Car ahead in a given lane. The class also includes a Safe function for checking if it&#39;s safe to change to a lane, and a Print function for debugging.</p><h3>2. Finite State Machine</h3><p>This keeps track of what the car is currently doing. The states are “default”, “following”, “preparing lane change”.</p><p>Default state</p><p>In this state the car will drive straight in its lane at about 49.5 miles an hour (so as to not pass the 50mph speed limit). In this state it is constantly calling the ClosestInLane function to see if a car is within 30 meters ahead. If such a car exists, it transitions to the &quot;follow&quot; state.</p><p>Follow state</p><p>The follow state adjusts the cars speed to match the car ahead at a safe distance. To adjust the speed, we have a maximum speed adjustment that can be made per step (.224). That makes sure our car doesn’t abruptly change speed and cause a lot of jerk. We then adjust the speed by varying amounts depending on whether we are close to the speed we want, and whether we are too close to the car we’re following. Additionally, if the car we’re following is going more than 5mph below the speed limit, we’d like to pass it. So we move into the “prepare lane change” state.</p><p>Prepare Lane Change State</p><p>Now that we’ve determained we’d like to change to a faster lane we need to figure out if a lane change is even safe. This is done by calling the Safe function. Now that we know which lanes are safe we determain which lane is the best by checking which lane has the most free space ahead of us by calling the ClosestInLane function. If no lane is found that&#39;s safe, we simply continue following and looking for a safe lane. We also won&#39;t change lanes if the lane we are currently in is the best lane. If we do find a good lane next to ours, we simply set the desired lane for our path planner to whatever lane that is. If the desired lane is more than one lane away, we just move one lane towards it (assuming that lane is safe). Once we&#39;ve changed lanes, we set the state back to default. We also have a variable in here that makes sure we don&#39;t change lanes again too quickly after a previous change to insure we don&#39;t jerk the car too much.</p><h3>3. Path Planner</h3><p>Now that the finite state machine has determined which lane we should be in, and at what speed, we need to create the waypoints to execute those orders. First map data about the center of the highway is read in and transformed into Frenet coordinates. This makes the distance down the highway our s value and the distance away from the center of the highway our d value (both are in meters). Then knowing what lane to be in, and that each lane is 4 meters across, we can calculate what our desired d value is. We then use the getXY function to transform this into the cartesian coordinate system. Now we have a few points that we want to hit, but as is, the path is made out of long straight lines which if followed completely, would be very jerky. So we feed our future s values to the Spline library and it gives us a smoothed path of very close together waypoints. From there we just have to send these coordinates back to the simulation so that the car will follow them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QBl0WJhRgIGEMa85E5O59w.png" /></figure><p><a href="https://github.com/ablacklama/Highway-Path-Planner">https://github.com/ablacklama/Highway-Path-Planner</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6794570caff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[YOLO Vehicle Detection]]></title>
            <link>https://medium.com/@spencer_kelly/vehicle-detection-using-yolo-612a3b9b54a9?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/612a3b9b54a9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[object-detection]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Fri, 03 Nov 2017 00:11:44 GMT</pubDate>
            <atom:updated>2017-11-03T00:29:26.290Z</atom:updated>
            <content:encoded><![CDATA[<h3>Vehicle Detection Using YOLO</h3><p>This is project 5 of <a href="https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013">Udacity’s Self-Driving Car Engineer Nanodegree.</a></p><p>The goal of the project is to detect and draw squares around cars in dashcam footage. I used a YOLO image detection network to predict the boxes and labels that I put on the image. I choose to use a one-shot approach to this instead of the Udacity recommended approach for a few reasons:</p><ol><li>I love deep learning and I’ll have much more fun playing with Neural networks than I will with more traditional approaches</li><li>I’ve heard models like this are more commonly used in the industry than the HOG/SVM style approach Udacity teaches</li></ol><p>I also choose to implement it in Keras because it’s accessible and makes the nice and small. On that note, I’ve tried to include a more detailed description of YOLO’s output than you can find anywhere else by googling. I spent a long time figuring out what each number meant and hopefully you won’t have to do so. I’ve also put some effort into commenting profusely throughout the <a href="https://github.com/ablacklama/Vehicle-Detection/blob/master/utils/utils.py">utils.py</a> file to make it easy to understand. Feel free to use any parts of this code you want.</p><h3>What is YOLO?</h3><p>“YOLO” or “You Only Look Once” (hah ಠ_ಠ), is a massive Convolutional Neural network for object detection and classification. As a quote from its <a href="https://pjreddie.com/darknet/yolo/">website</a> explains “This network divides the image into regions and predicts bounding boxes and probabilities for each region. These bounding boxes are weighted by the predicted probabilities.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e0VY6U1_WMF2KBoKQNZvkQ.png" /></figure><p>YOLO is a great network and on the cutting edge of object detection, but… it’s really big and it’s really slow. So since I’m not looking to put this network in a car, I decided to use the smaller <a href="https://pjreddie.com/darknet/tiny-darknet/">Tiny-YOLO</a> instead.</p><h3>tiny-YOLO architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fU8mEFhD4cdOaZlKrc24dg.jpeg" /></figure><p>As you can see, tiny-YOLO has 9 convolutional layers and is only tiny when compared to the normal YOLO which has 24.</p><h3>YOLO Output: Overview</h3><p>These next four sections get into the nitty gritty of what the network produces. I’d recommend skipping them if you don’t want to implement this yourself. These outputs are the same for YOLO and Tiny-YOLO.</p><p>This network outputs a vector of 1470 numbers that are divided into three sections.</p><p>YOLO&#39;s sorts it&#39;s output by the 49 grid cells (7x7) that it divides the image into. The cells are represented in this array left to right, then top to bottom. The data is organized into three parts which I will describe below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*cGfWw6lGmV1xUKRsd--JxQ.png" /></figure><h3>YOLO Output: Probability</h3><p>This is the simplest one, there are 20 classes that the network can predict. 20 classes * 49 cells = the 980 numbers that are in this part of the vector. So 0 through 19 of the vector is the relative probabilities that the top left cell is categorized as each class. For this project, I only care about identifying cars, so I only use index 6. This corresponds to the car class in the dataset that tiny yolo was originally trained on.</p><h3>YOLO Output: Confidence</h3><p>Each cell in the grid is responsible for predicting two bounding boxes. Thus there is one number in this section of 98 of each of those bounding boxes in each cell (49*2). Each &quot;confidence score&quot; is the probability that there is an object within that bounding box. This doesn&#39;t tell you anything about the shape of the box, or what&#39;s in it, just whether there is something there.</p><h3>YOLO Output: Box Coordinates</h3><p>This is where the boxes are predicted. Each box is represented with four numbers (x,y,width,height). These numbers are relative to the cell they&#39;re in and have to be scaled to be represented on the entire image. Once again, each cell predicts two bounding boxes so 2 boxes * 4 number per box * 49 cells = 392 numbers.</p><h3>Weights/Training</h3><p>I don&#39;t have a big deep learning computer so even with the tiny-YOLO I need to either use a cpu so it gets stored in RAM or use pre trained weights. I choose the latter. The weights I used are from the darknet site and are from training the model on the VOC2012 Dataset. This is why there are 20 classes to choose from even though we&#39;d be happy with just a car class.</p><p>You can download the weights for yourself <a href="https://pjreddie.com/media/files/yolo.weights">here</a>.</p><h3>Making up for size difference</h3><p>The model I&#39;m using is trained to recognize objects that are generally much closer (and thus bigger) than the cars I&#39;m trying to detect, so I had to work around it. Along with running YOLO on the whole image, I also ran it on subsections. The fixed subsection was zoomed in on the highway ahead to try to get farther cars. But I also kept track of where detected cars were and tried to find them in a zoomed in image of that position again in the next few frames. This also made up for some of the performance loss I got from using tiny-yolo vs the full model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*5faKhD_EMAnNzJ9EN4J3aA.png" /></figure><p>Here&#39;s an example of the standard cropping I do to zoom the image in on the highway ahead. As you can see, padding has been automatically added on the top and bottom to make the image the proper 448x448.</p><h3>Heatmapping</h3><p>I used a heatmap of the past 8 frames to eliminate most false detections. False detections are generally flukes and happen for only a frame or two. The heatmap kept track of the total number of detections in any area of the image for the past 8 frames and if there were less than 3, I discarded that detection.</p><h3>Results</h3><p><a href="https://youtu.be/2EtTBv0DjKc">Result video</a></p><p><a href="https://github.com/ablacklama/Vehicle-Detection">Source Code</a></p><p>And here are a few example images of the output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xgQozspXCoy1h8irNr3voA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NwxKfAwWrWOOOJhOB8Z8Zg.jpeg" /></figure><h3>Reflections</h3><p>Tiny-YOLO works pretty well but I&#39;d love to see how it could perform with the full YOLO or even SDD. In the future I&#39;d love to have the time and GPU&#39;s to train my own YOLO or SDD on open source dashcam datasets.</p><p>Additionally, it would be nice to implement a more robust tracking function that could keep track of the velocity of objects. This would make it easy to estimate their position in upcoming frames.</p><p>My shortcut of running YOLO multiple times per frame is just that, a shortcut. It would be much better to simply train the full YOLO on a highway dataset so that it would know to look for smaller cars. This would also mean i didn&#39;t have to discard 19 out of 20 classes the network produced and would make it more accurate as a result.</p><p>I&#39;d also like to keep track of which car is which and use it&#39;s mean box for the last few frames to eliminate jitter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=612a3b9b54a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Identifying Lane Lines with Gradients]]></title>
            <link>https://medium.com/@spencer_kelly/identifying-lane-lines-with-gradients-750e46224115?source=rss-68cdee1d3fb3------2</link>
            <guid isPermaLink="false">https://medium.com/p/750e46224115</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Spencer Kelly]]></dc:creator>
            <pubDate>Wed, 27 Sep 2017 04:23:37 GMT</pubDate>
            <atom:updated>2017-10-29T06:22:30.070Z</atom:updated>
            <content:encoded><![CDATA[<p>Udacity’s Self-Driving Car Engineer NanoDegree Project 2:</p><p><a href="https://github.com/ablacklama/Lane-Finding">Code here</a></p><p>My goal in this project was to successfully identify and label the lanes the car was driving on. Each frame of the video needed to be analyzed and returned with red lines over the lanes the vehicle was in.</p><p>This project was a bit challenging for me simply because I had to learn about Hough space and Canny edge detection. But I loved the challenge and experimenting with ways to improve it. Most of mine failed, or would fail outside of these test examples, but it was still fun to try them out.</p><p>Here’s how I did it in the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*XmYrsc6SBBr68EgpPK-Z4A.png" /><figcaption>original image</figcaption></figure><p>Each frame went through a pipeline that I created. I started with what you see above, and to make taking the gradient easier, I converted the image to grayscale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*jA4S53yhKWJa9xQHBxrkUA.png" /><figcaption>Converted to greyscale</figcaption></figure><p>From there I apply <a href="https://en.wikipedia.org/wiki/Gaussian_blur">gaussian smoothing</a>. This essentially blurs the image a little bit so that ragged edges are smoothed out and clearer lines can be drawn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*W7Gm2QWXg6UkHcmKEZybRg.png" /><figcaption>Gaussian Blur</figcaption></figure><p>This is where the real fun begins. To find where the lanes are we need to have some way of telling what’s a line and what’s not. Well this requires detecting edges in the image. So the way we do this we take the gradient of every pixel to determine where the greatest changes in color are in the picture. Then we create a new picture with those changes represented in white and everywhere else in black. This is all done with the <a href="http://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html">Canny</a> function of <a href="http://opencv.org/">opencv</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*38OpRdrkmPfpvXCxcTKTsw.png" /><figcaption>Canny Edge Detection</figcaption></figure><p>As you can see, the algorithm has identified the places of highest change in the image, and created a new image with only those portioned represented in white. But at this point there is a lot of information there that we don’t need. The mountains and other lanes aren’t relevant to our goal so lets eliminate any points that aren’t within a cone of interest where we normally find the lane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*Arn-y_JXElQxUdVUcB3kmg.png" /><figcaption>Region of Interest</figcaption></figure><p>Great, now we just need to connect these points into lines. A hough transform is used here to basically connect these points into many possible lines. We then take those lines and separate them into left and right lane line based on whether they have a positive or negative slope. Then we take the average of those two lines and extend it down to the bottom of the image, which results in this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*5-zYRPXOYnoMBfgi3bgHqg.png" /><figcaption>Averaged Hough Line Detection</figcaption></figure><p>And now all we have to do is overlay that onto our original image and we should have our final result!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*N_XqVDkLrf1Lp--m22TGsw.png" /><figcaption>Final Image</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=750e46224115" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>