<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kaspar Sakmann on Medium]]></title>
        <description><![CDATA[Stories by Kaspar Sakmann on Medium]]></description>
        <link>https://medium.com/@ksakmann?source=rss-168a35b2dee1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*7jfuJL6I4oX5KQvW.</url>
            <title>Stories by Kaspar Sakmann on Medium</title>
            <link>https://medium.com/@ksakmann?source=rss-168a35b2dee1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 14:02:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ksakmann/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vehicle tracking using a support vector machine vs. YOLO]]></title>
            <link>https://medium.com/@ksakmann/vehicle-detection-and-tracking-using-hog-features-svm-vs-yolo-73e1ccb35866?source=rss-168a35b2dee1------2</link>
            <guid isPermaLink="false">https://medium.com/p/73e1ccb35866</guid>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <dc:creator><![CDATA[Kaspar Sakmann]]></dc:creator>
            <pubDate>Sat, 25 Feb 2017 15:19:59 GMT</pubDate>
            <atom:updated>2017-02-25T17:27:20.924Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>The <a href="https://github.com/udacity/CarND-Vehicle-Detection">vehicle detection and tracking project</a> of the Udacity Self-Driving Car Nanodegree is a challenge to apply traditional computer vision techniques, such as Histogram of Oriented Gradients (HOG) and other features combined with sliding windows to track vehicles in a video. The ideal solution would run in real-time, i.e. &gt;30FPS. My go at a solution follows the 2005 approach by <a href="https://hal.inria.fr/inria-00548512/document">Dalal and Triggs</a> using a linear SVM and processes video at a measly 3FPS on an i7 CPU. Check this <a href="https://github.com/ksakmann/CarND-Vehicle-Detection">repo</a> for the code and a more technical discussion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sbGY0u0OkjIkyujp5LN5rg.png" /><figcaption>YOLO applied to the Udacity vehicle tracking project video does an excellent job at detecting vehicles.</figcaption></figure><p>For fun I also passed the project video through <a href="https://pjreddie.com/darknet/yolo/">YOLO</a>, a blazingly fast convolutional neural network for object detection. If you are working on a fast GPU (a GTX 1080 in my case) the video gets processed at about 65FPS. Yes! That’s more than 20x faster which is why I made no attempt at vectorizing my SVM+HOG solution. Although there certainly are opportunities, particularly the sliding windows part. I’ll discuss the YOLO results further down.</p><h3>Data sets</h3><p>I used the <a href="http://www.cvlibs.net/datasets/kitti/">KITTI</a> and <a href="http://www.gti.ssr.upm.es/data/Vehicle_database.html">GTI</a> data sets as well as the Extra data that comes along with <a href="https://github.com/udacity/CarND-Vehicle-Detection">the project repository</a> for training. There are only two classes: “cars” and “notcars”. The GTI data is taken from video streams. Therefore blindly randomizing all images and subsequently splitting into train and test sets introduces correlations between training and test sets. I therefore simply split off the final 30% of each data source as validation and test sets. All images (including ones with non-square aspect ratios) are resized to 64x64 pixels for feature extraction.</p><h3>Feature Extraction</h3><p>As a feature vector I used a combination of</p><ul><li>spatial features, which are nothing else but a down sampled copy of the image patch to be checked itself (16x16 pixels)</li><li>color histogram features that capture the statistical color information of each image patch. Cars often come in very saturated colors which is captured by this part of the feature vector.</li><li>Histogram of oriented gradients (HOG) features, that capture the gradient structure of each image channel and work well under different lighting conditions</li></ul><p>You can read more about HOG features in this <a href="http://www.pyimagesearch.com/2014/11/10/histogram-oriented-gradients-object-detection/">blog post</a>, but the idea is to make the feature vector more robust to variations of perspective and illumination by aggregating the gradients on an image in a histogram.<br>Here is a visualization of what HOG features look like for “cars” and “notcars” images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEOEJPRVG8IuixjDAbTJBw.png" /><figcaption><strong>HOG features. </strong>First four columns from the left: typical examples of training data and their channels in HLS color space. Rightmost three columns: visualization of the HOG vectors for each channel.</figcaption></figure><p>The final feature vector contains features extracted in the three different ways quoted above. It is therefore necessary to scale every feature to prevent one of the features being dominant merely due to its value range being at a different scale. I therefore used the Standard.Scaler function of the scikit learn package to standardize features by removing the mean and scaling to unit variance.</p><h3>Training a linear support vector machine</h3><p>Unlike many other classification or detection problems there is a strong real-time requirement for detecting cars. So a trade-off between high accuracy and speed is unavoidable. The two main parameters that influence the performance are the length of the feature vector and the algorithm for detecting the vehicles. A linear SVM offered the best compromise between speed and accuracy, outperforming random forests (fast, but less accurate) and nonlinear SVMs (rbf kernel, very slow). The final test accuracy using a feature vector containing 6156 features was above 98.5% which sounds good, but is much less so when you find out that those remaining 1.5% are frequently appearing image patches, particularly lane lines, crash barriers and guard rails.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R_-8KO38t0JvUL3MXmOtJQ.png" /><figcaption><strong>False Positives</strong>: frequently appearing image patches still get misclassified, despite 98.5% test accuracy.</figcaption></figure><h3>Sliding windows</h3><p>In the standard vehicle detection approach the frames recorded by a video camera the image is scanned using a sliding window. For every window the feature vector is computed and fed into the classifier. As the cars appear at different distances, it is also necessary to search at several scales. Commonly over a hundred feature vectors need to be extracted and fed into the classifier for every single frame. Fortunately this part can be vectorized (not done). Shown below is a typical example of positive detections together with all ~150 windows that are used for detecting cars. As expected there are some false positives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eAEYVev9Yq_CQiDlzPGAHA.png" /></figure><p>For filtering out the false positives I always kept track of the detected windows of the last 30 frames and only considered those parts of the image as positives where more than 15 detections had been recorded. The result is a heatmap with significantly reduced noise, as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6TWq4GNRg976fA337CZqLA.png" /><figcaption><strong>Left: </strong>All detected windows of a frame. <strong>Right: </strong>heatmap of the last 30 frames.</figcaption></figure><p>From the thresholding heatmap the final bounding boxes are determined as the smallest rectangular box that contains all nonzero values of the heatmap</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*erEKaxOy_nfA5tFCIgpVjw.png" /><figcaption><strong>Left: </strong>binarized heatmap thresholded at 15 detections. <strong>Right: </strong>final result of drawn bounding boxes.</figcaption></figure><p>Applying the entire pipeline to a video results in bounding boxes drawn around the cars, as shown here:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6NNDFKoOlaA%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6NNDFKoOlaA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F6NNDFKoOlaA%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/eaad6edd2b15016acadc51699cc9e1d5/href">https://medium.com/media/eaad6edd2b15016acadc51699cc9e1d5/href</a></iframe><h3>Comparison to YOLO and Conclusions</h3><p>The above pipeline using HOG features and a linear SVM is well-known since 2005. Very recently extremely fast neural network based object detectors have emerged which allow object detection faster than realtime. I merely cloned the original <a href="https://github.com/pjreddie/darknet">darknet</a> repository and applied YOLO to the project video. I only needed to do a minor code modification to allow saving videos directly. The result is quite amazing. As no sliding windows are used the detection is extremely fast. A frame is passed to the network and processed precisely once, hence the name YOLO — “you only look once”.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fku2qCp_oIrg%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dku2qCp_oIrg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fku2qCp_oIrg%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/79aaffd6e919190ddc46790da7dc0b37/href">https://medium.com/media/79aaffd6e919190ddc46790da7dc0b37/href</a></iframe><p>A forward pass of an entire image through the network is more expensive than extracting a feature vector of an image patch and passing it through an SVM. Hoever, this operation needs to be done exactly once for an entire image, as opposed to the roughly 150 times in the SVM+HOG approach. For generating the video above I did no performance optimization, like reducing the image or defining a region of interest, or even training specifically for cars. Nevertheless, YOLO is more than 20x faster than the SVM+HOG and at least as accurate. The detection threshold can be set to any confidence level. Here I left it at the default of 50% and at no time were any objects other than cars detected (OK, one time a car was confused for a truck). Note that there are many more classes, like “person”, “cell phone”,”dog” and others. So not only the false positive rate, but also the false negative rate was very good. I find this extremely exciting and will check the possibilities for vehicle detection further in a separate project. This will be fun.</p><p>Thanks for reading,</p><p>Kaspar</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73e1ccb35866" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Behavioral Cloning — make a car drive like yourself]]></title>
            <link>https://medium.com/@ksakmann/behavioral-cloning-make-a-car-drive-like-yourself-dc6021152713?source=rss-168a35b2dee1------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc6021152713</guid>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Kaspar Sakmann]]></dc:creator>
            <pubDate>Sun, 25 Dec 2016 21:02:08 GMT</pubDate>
            <atom:updated>2017-02-26T18:21:33.163Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3aFaK9H6ZsUUaLccc92ANw.png" /></figure><p>This is an article to provide my thoughts on an interesting project I did for the Udacity Self-Driving Car Nanodegree. The code and technical details can be found <a href="https://github.com/ksakmann/CarND-BehavioralCloning">here</a>. The goal is to teach a Convolutional Neural Network (CNN) to drive a car in a simulator provided by Udacity. The car is equipped with three cameras that provide video streams and records the values of the steering angle, speed, throttle and brake. The steering angle is the only thing that needs to be predicted, but more advanced models might also want to predict throttle and brake. This turns out to be a regression task, which is very different from usual applications of CNNs for classification purposes.</p><p>I collected the training data by driving the car on the flat terrain training track. The performance of the CNN can then be checked by letting the car drive autonomously on the same track or ideally on a second track that is considerably more windy with steep hills and that should not be used for training. Below are some pictures from the different cameras on the car on the training track.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*Dn4MWWptj8FyLUcgPaWeHQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*9yUJQLT89tSoJgengERaFQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*ZIfhslbnGa5k3kuL9vOnLA.jpeg" /></figure><p><strong>Unbalanced data<br></strong>While driving the car under normal conditions the steering angle is very close to zero most of the time. This can clearly be seen in the raw training data. Below are the steering angles I recorded while driving the car around the track while staying as close as possible to the middle of the lane. This is all the data I collected for training the final model (~9000 images or driving 3–4 laps).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*tWLG9hCirxCwEw-CWIe8Fg.png" /><figcaption>Histogram of steering angles: the range [-1,1] corresponds to the angle range ±25°</figcaption></figure><p>The left/right skew is due to driving the car around the track in one direction only and can be eliminated by flipping each recorded image and its corresponding steering angle. More troublesome is the bias to driving straight: the rare cases, when a large steering angle recorded are also the most important ones if the car is to stay on the road. One possible solution would be to let the car drift to the edge of the road and recover before a crash occurs. I tried this, but found it an unsatisfactory solution, because in that case the car still goes straight most of the time — direction “off the track” — with a few large steering angles sprinkled on top. As a result a CNN trained on such data typically does not even complete the training track, unless the training data is ‘just right’. I got a CNN to drive the car around the training track this way, but the model failed on the test track.</p><p>Another solution would be to sample the extreme angle events more often than the small angles ones. However, since they are rare, a large amount of training data may need to be collected for the model to avoid overfitting.<br>Inspired by this <a href="https://chatbotslife.com/using-augmentation-to-mimic-human-driving-496b569760a9#.jqcy5fcw6">post</a>, I therefore decided to simulate <em>all </em>recovery events synthetically. For training I drove the car as smoothly as possible right in the middle of the road. The rationale behind this was to record the ideal steering angle at all times. Recovery events were then simulated by distorting and cropping the recorded camera images and adjusting the steering angle. By chaining image distortions, random brightness corrections and crops together a practically infinite number of training images could thereby be generated from the little training data I had gathered. This worked surprisingly well.</p><p><strong>Data augmentation<br></strong>I used images from all three cameras. Images taken from the side cameras are akin to parallel translations of the car. To account for being off-center I adjusted the steering angle for images taken from the side cameras as follows: ignoring perspective distortions one could reason that if the side cameras are about 1.2 meters off-center and the car is supposed to get back to the middle of the road within the next 20 meters the correction to the steering should be about 1.2/20 radians (using tan(𝛼)~𝛼). This turned out to be a quite powerful means to make the car avoid the sides of the road.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*8nadKhf1YQbQOo0vpIuorw.jpeg" /><figcaption>Random camera view. Steering angle: -2.8°, corrected:-3.4°</figcaption></figure><p>In the next stage each image got sheared horizontally. The pixels at the bottom of the image were held fixed while the top row was moved randomly to the left or right. The steering angle was changed proportionally to the shearing angle. This had the effect of making curvy road pieces appear just as often in the training set as straight parts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*QrTDCsDo0xcPzIteKPiTBQ.jpeg" /><figcaption>Random shearing: corrected steering angle: -1.0°</figcaption></figure><p>So far all transformations had been performed on the full 320x160 pixel images coming from the cameras. In the next stage I chose a window of 280x76 pixels that eliminated the bonnet from the bottom of the image and cropped off the part above the horizon in flat terrain at the top. For each image the location of this window was displaced randomly from the center along the x-and y-axes by up to ±20 and ±10 pixels, respectively. The steering angle was changed proportionally to the lateral displacement of the crop. Thereby, also images of the car being in hilly terrain were simulated and a greater variety of images than available from the side cameras could be obtained. The result was then resized to 64x64 pixels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*euudPy83BoNiS08m_dPmlQ.jpeg" /><figcaption>Random crop: corrected steering angle: -2.1°</figcaption></figure><p>Finally, each image was randomly flipped (horizontally) with equal probability in order to make left and right turns appear as frequently. Also brightness was randomly adjusted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*78S3hIrCMWP6MDh0Xt58Gw.jpeg" /><figcaption>Random brightness and flip: corrected steering angle: 2.1°</figcaption></figure><p>Chaining all these transformations can be done efficiently in batches that get generated on the fly during training from the recorded images.</p><p><strong>Model architecture and training<br></strong>The CNN I chose is a pretty standard CNN consisting of 4 convolutional layers with ReLU activations, followed by two fully connected layers with dropout regularization. Finally a single neuron formed the output that predicted the steering angle. One noteworthy thing is the absence of pooling layers. The rationale behind avoiding pooling layers was that pooling layers make the output of a CNN to some degree invariant to shifts of the input, which is desirable for classification tasks, but counterproductive for keeping a car in the middle of the road. I trained the network on an Ubuntu 16.04 system using an NVIDIA GTX 1080 GPU. For any given set of hyperparameters the loss typically stopped decreasing after a few epochs (200000 images each).</p><p><strong>Results<br></strong>The results came as a pleasant surprise after several nights without any progress. The car not only completed the training track, but even the test track. Shown below are the results for one of the earliest incarnations of this model, before hyperparameter tuning.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FDyQkdoPNsYY%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDyQkdoPNsYY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FDyQkdoPNsYY%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bda5a8d6124cc2593931d6658e545f91/href">https://medium.com/media/bda5a8d6124cc2593931d6658e545f91/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F5BTIE_fhReo%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D5BTIE_fhReo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F5BTIE_fhReo%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b86085e1be85da4d5114b2c478d039b4/href">https://medium.com/media/b86085e1be85da4d5114b2c478d039b4/href</a></iframe><p>I was surprised how well the car drove even on the test track. The CNN had never seen this track. The performance on the training track was a little bumpy, but I quite liked it too, because it showed that the car was not merely memorizing the track. It recovered successfully from a few critical situations, even though none of those maneuvers had been performed during training.</p><p><strong>Conclusions<br></strong>Summarizing, this was a really interesting project. It would be interesting to see whether recovery events can also be simulated from real world data. Currently, I can’t see why not. The project cost me countless of hours of sleep over a two week period of time, gray hairs and cursing included, but the result was well worth it. Deep learning is an exciting field and we’re lucky to live in these times of discovery.</p><p>For more details please check out the code and the readme here:<br><a href="https://github.com/ksakmann/CarND-BehavioralCloning">https://github.com/ksakmann/CarND-BehavioralCloning</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc6021152713" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>