<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jannik Zürn on Medium]]></title>
        <description><![CDATA[Stories by Jannik Zürn on Medium]]></description>
        <link>https://medium.com/@jannik-zuern?source=rss-5db0eff2a830------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cZvO2oPwS0ES38YGkV4Q4Q.jpeg</url>
            <title>Stories by Jannik Zürn on Medium</title>
            <link>https://medium.com/@jannik-zuern?source=rss-5db0eff2a830------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 20:43:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jannik-zuern/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why good teaching helps you do better research]]></title>
            <link>https://jannik-zuern.medium.com/why-good-teaching-helps-you-do-better-research-1a93f0ac1db9?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a93f0ac1db9</guid>
            <category><![CDATA[teaching]]></category>
            <category><![CDATA[university]]></category>
            <category><![CDATA[academia]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Sat, 31 Oct 2020 11:51:34 GMT</pubDate>
            <atom:updated>2020-10-31T11:51:34.193Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6lxnj88gFP_zSiUJ3B3_rw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@shubhamsharan?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Shubham Sharan</a> on <a href="https://unsplash.com/s/photos/academic-lecture?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In this blog post, I would like to discuss academic teaching with you; Why teaching is frowned upon by many researchers, and how it can actually help them be better at doing research by teaching great courses.</p><h3>Part 1: The dilemma of teaching</h3><p>Researchers at institutions with teaching assignments often find themselves faced with a dilemma when it comes to teaching: How can they spend precious time teaching that brings them no immediate benefit that would help them with their career (writing academic papers and getting funding)? Don’t get me wrong here; I assume many professors and other teaching staff know perfectly well that teaching is one of if not the most important duties of any university. It serves as a sacred haven of knowledge where the half-gods of science descent into the earthly realms and bless the students with their knowledge and wisdom. Leaving biblical metaphors aside, we can also clear why so many professors frown upon the teaching responsibilities they have, and most probably wish they could assign this responsibility to someone else (which they sometimes do). But how blame them? It is true that not many incentives exist for teaching staff to spend more time on teaching, or — more importantly — do better <em>teaching</em>. Sure, some universities honor great lectures crafted with skill and foresight by highly motivated and talented teaching staff, with teaching prizes, but this is clearly the minority and I argue that most of these great teachers would have done so even without the prices — they are simply motivated by their own account.</p><blockquote>It is a curious fact that professors stand in front of hundreds of young students and try to shovel knowledge into their heads while typically having absolutely no formal qualification for doing so.</blockquote><p>They are good at publishing papers and collecting funds, otherwise, they would probably not have gotten that position, but they don’t know how to explain their research field to newcomers. While school teachers spend many years of their training honing their didactics skills and practicing in actual classes, teaching staff at universities mostly study the academic content they are later expected to digest and pass on to their students.</p><p>That being said, I am wondering how academia can motivate researchers to do better teaching by directly rewarding good teaching? Could “teaching credits” be directly converted to citations, the most important external measurement of success of a researcher? Or even converted to direct funding? I am uncertain if such measures actually help to alleviate the problem of unmotivated and bad teaching at universities. Maybe students should not expect good teaching after all since they should be able to learn concepts without pre-digestion by a teacher anyway? Regardless of what the ideal teaching concepts in academia might look like, I am trying to make the point that making an effort to make good and inspiring lectures can actually help lecturers with their own academic career regardless of extrinsic motivation by the academic system.</p><h3>Part 2: How teaching can help to be a better researcher</h3><p>In part 1, we discussed the deficiencies of today’s academic institutions when it comes to teaching obligations. It is true that there is no obvious path out of this situation and changes will come slowly, as always in academia. However, if you are a teaching researcher, I would like to point you towards possible novel views that you can apply today which might help you to do a better job at teaching.</p><ul><li><strong>Teaching can bring you new ideas that you had not before</strong>: Technical discussions with bright students can be very fruitful both for teachers and students. Every student brings a different background to the table and provides personal insight into the problem. While most questions by students might not cause immediate new ideas, we all know the <em>“Mh… I have never thought of it like that before”</em>-feeling that sometimes leads to a new way of investigating a problem.</li><li><strong>If you can’t explain it simply, you don’t understand it well enough</strong>. This famous Einstein quote, in a nutshell, can serve as a pretty good rule of thumb for differentiating between teachers and <em>great </em>teachers. If you are not able to break a concept down to its core ideas, you know that you deeply understood the concept. Applying this rule to yourself as a teacher can help you detecting deficiencies in your own understanding of the subject that you might otherwise have missed. True experts in their field and great teachers can explain a concept at multiple levels of prior knowledge while still being concise. They have truly mastered their field. For example, take a look at the following video:</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOWJCfOvochA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOWJCfOvochA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOWJCfOvochA%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a06d9cda1a072ed6fe49eb22ab5e8d0b/href">https://medium.com/media/a06d9cda1a072ed6fe49eb22ab5e8d0b/href</a></iframe><ul><li><strong>Teaching is a unique and fun experience</strong>. Teaching can be rewarding in its own way if you are the type for it. Standing in front of a class can be fun if you have motivated and bright students willing to delve into technical discussions with you.</li><li><strong>Teaching helps to repeat the essentials</strong>: There is a reason why you talk about topic X in your lectures, right? Every student in your course should be able to explain this topic to their colleagues after taking your course. And so should you be. Teaching helps to repeat the essentials of your subject in-depth which you might otherwise have forgotten.</li><li><strong>Good teaching draws talent. </strong>In my personal view, this is the most important aspect of doing great teaching. If you are able to give insightful lectures you will note an increase in applications for graduate and post-graduate students. Your lecture serves as one of the most important advertisements for your academic chair — one of the few opportunities for students to know what you are actually doing. It is important to remember that if you cannot recruit great talent, your chair will not be able to hold to its current standards once all the current graduates have left. Which will directly affect the quality of your research and your academic output in the long run.</li></ul><p>Last but not least, I will leave you with the following beautiful quote by infamous Richard Feynman who has an inspiring story to tell about teaching in <strong><em>Surely You’re Joking, Mr. Feynman!</em></strong></p><blockquote>I <em>don’t </em>believe I can really do without teaching. The reason is, I have to have something so that when I don’t have any ideas and I’m not getting anywhere I can say to myself, “At least I’m living; at least I’m <em>doing</em> something; I am making <em>some</em> contribution” — it’s just psychological.</blockquote><blockquote>When I was at Princeton in the 1940s I could see what happened to those great minds at the Institute for Advanced Study, who had been specially selected for their tremendous brains and were now given this opportunity to sit in this lovely house by the woods there, with no classes to teach, with no obligations whatsoever. These poor bastards could now sit and think clearly all by themselves, OK? So they don’t get any ideas for a while: They have every opportunity to do something, and they are not getting any ideas. I believe that in a situation like this a kind of guilt or depression worms inside of you, and you begin to worry about not getting any ideas. And nothing happens. Still no ideas come.</blockquote><blockquote>Nothing happens because there’s not enough real activity and challenge: You’re not in contact with the experimental guys. You don’t have to think how to answer questions from the students. Nothing!</blockquote><blockquote>In any thinking process there are moments when everything is going good and you’ve got wonderful ideas. Teaching is an interruption, and so it’s the greatest pain in the neck in the world. And then there are the <em>longer</em> period of time when not much is coming to you. You’re not getting any ideas, and if you’re doing nothing at all, it drives you nuts! You can’t even say “I’m teaching my class.”</blockquote><p>Thank you for reading! Let me know in the comments how you feel about teaching if you are a researcher yourself, or your perception of teaching staff if you are a student!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a93f0ac1db9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images]]></title>
            <link>https://jannik-zuern.medium.com/bridging-the-day-night-domain-gap-in-semantic-segmentation-with-thermal-images-6d54ffb2ee7f?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d54ffb2ee7f</guid>
            <category><![CDATA[autonomous-driving]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[thermal-imaging]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[semantic-segmentation]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Thu, 12 Mar 2020 13:00:42 GMT</pubDate>
            <atom:updated>2020-03-12T13:00:42.198Z</atom:updated>
            <content:encoded><![CDATA[<p>Today, we will discuss our most recent paper <strong><em>HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images </em></strong><em>(by Johan Vertens, Jannik Zürn, and Wolfram Burgard)</em>. This post serves as a quick and dirty introduction to the topic and to the work itself. For more details, please refer to the original publication, which is available <a href="https://arxiv.org/abs/2003.04645">here</a> and the project website will be available soon at <a href="http://thermal.cs.uni-freiburg.de/">http://thermal.cs.uni-freiburg.de/</a>.</p><h3>Introduction</h3><p>Robust and accurate semantic segmentation of urban scenes is one of the enabling technologies for autonomous driving in complex and cluttered driving scenarios. Recent years have shown great progress in RGB image segmentation for autonomous driving which were predominantly demonstrated in favorable daytime illumination conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*gpugLE9nHDAVHWz7TEF8Fg.png" /><figcaption>During nighttime, thermal images provide important additional data that is not present in the visible spectrum. Note the contrast between vegetation and sky/clouds and the bright spots on the left and right, indicating pedestrians.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*DjUjssG4oWloOgjOvN_UEw.png" /><figcaption>In daytime scenes, thermal images can also provide important information where the dynamic range of standard RGB cameras does not suffice (vegetation in front of the bright sky). Note the temperature gradient between car or truck and road.</figcaption></figure><p>While the reported results demonstrate high accuracies on benchmark datasets, these models tend to generalize poorly to adverse weather conditions and low illumination levels present at nighttime. This constraint becomes especially apparent in rural areas where artificial lighting is weak or scarce. In autonomous driving, to ensure safety and situation awareness, robust perception in these conditions is a vital prerequisite.</p><p>In order to perform similarly well in challenging illumination conditions, it is beneficial for autonomous vehicles to leverage modalities complementary to RGB images. Encouraged by prior work in thermal image processing, we investigate leveraging thermal images for nighttime semantic segmentation of urban scenes. Thermal images contain accurate thermal radiation measurements with a high spatial density. Furthermore, thermal radiation is much less influenced by sunlight illumination changes and is less sensitive to adversary conditions. Existing RGB-thermal datasets for semantic image segmentation are not as large-scale as their RGB-only counterparts. Thus, models trained on such datasets generalize poorly to challenging real-world scenarios.</p><h3>Approach</h3><p>In contrast to previous works, we utilize a semantic segmentation network for RGB daytime images as a teacher model, trained in supervised fashion on the Mapillary Vistas Dataset, to provide labels for the RGB daytime images in our dataset. We project the thermal images into the viewpoint of the RGB camera images using extrinsic and intrinsic camera parameters that we determine using our novel targetless camera calibration approach. Afterward, we can reuse labels from this teacher model to train a multimodal semantic segmentation network on our daytime RGB-thermal image pairs. While the thermal modality is mostly invariant to lighting changes, the RGB modality differs significantly between daytime and nighttime and thus exhibits a significant domain gap. In order to encourage day-night invariant segmentation of scenes, we simultaneously train a feature discriminator that aims at classifying features in the semantic segmentation network to belong either to daytime or nighttime images. This helps aligning the internal feature distributions of the multimodal segmentation network, enabling the network to perform similarly well for nighttime images as for daytime images. Furthermore, we propose a novel training schedule for our multimodal network that helps to align the feature representations between day and night.</p><p>As thermal cameras are not yet available in most autonomous platforms, we further propose to distill the knowledge from the domain adapted multimodal model back into a unimodal segmentation network that exclusively uses RGB images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXnKskQ5MN4hPLUbqElJuA.png" /><figcaption>Our proposed HeatNet architecture uses both RGB and thermal images and is trained to predict segmentation masks in daytime and nighttime domains. We train our model with daytime supervision from a pre-trained RGB teacher model and with optional nighttime supervision from a pre-trained thermal teacher model trained on exclusively thermal images. We simultaneously minimize the cross-entropy prediction loss to the teacher model prediction and minimize a domain confusion loss from a domain discriminator to reduce the domain gap between daytime and nighttime images.</figcaption></figure><h3>Dataset</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*2h1aMsaGfqjhBJnoYdDWYg.png" /><figcaption>Our stereo RGB and thermal camera rig mounted on our data collection vehicle.</figcaption></figure><p>To kindle research in the area of thermal image segmentation and to allow for credible quantitative evaluation, we create the large-scale dataset <strong>Freiburg Thermal. </strong>We provide the dataset and the code publicly available at <a href="http://thermal.cs.uni-freiburg.de/">http://thermal.cs.uni-freiburg.de/</a>. The Freiburg Thermal dataset was collected during 5 daytime and 3 nighttime data collection runs, spanning the seasons summer through winter. Overall, the dataset contains 12051 daytime and 8596 nighttime time-synchronized images using a stereo RGB camera rig (FLIR Blackfly 23S3C) and a stereo thermal camera rig (FLIR ADK) mounted on the roof of our data collection vehicle. In addition to images, we recorded the GPS/IMU data and LiDAR point clouds. The Freiburg Thermal dataset contains highly diverse driving scenarios including highways, densely populated urban areas, residential areas, and rural districts. We also provide a testing set comprising 32 daytime and 32 nighttime annotated images. Each image has pixelwise semantic labels for 13 different object classes. Annotations are provided for the following classes: <em>Road, Sidewalk, Building, Curb, Fence, Pole/Signs, Vegetation, Terrain, Sky, Person/Rider, Car/Truck/Bus/Train, Bicycle/Motorcycle, </em>and <em>Background.</em> We deliberately selected extremely challenging urban and rural scenes with many traffic participants and changing illumination conditions.</p><h4>Camera Calibration</h4><p>For our segmentation approach it is important to perfectly align RGB and thermal images as otherwise, the RGB teacher model predictions would not be valid as labels for the thermal modality. Thus, in order to accurately carry out the camera calibration for the thermal camera, we propose a novel targetless calibration procedure. While in previous works, different kinds of checkerboards or circle boards have been leveraged, our method does not require any pattern. Although for RGB cameras, these patterns can be produced and utilized easily, it still remains a challenge to create patterns that are robustly visible both in RGB and thermal images. In general, the used modalities infrared and RGB entail different information. However, we note that the edges of most common objects in urban scenes are easily observable in both modalities. Thus, in our approach, we minimize the pixel-wise distance between such edges. In the case of aligning two monocular cameras, targetless calibration without any prior information results in ambiguities for the estimation of the intrinsic camera parameters. We, therefore, utilize our pre-calibrated RGB stereo rig in order to provide the missing sense of scale. Due to the target-less nature of our approach, our thermal camera calibration method can be easily deployed in an online calibration scenario.</p><h3>Results</h3><p>We report the performance of HeatNet trained on Freiburg Thermal and tested on Freiburg Thermal, MF, and on the BDD (Berkeley Deep Drive) night test split. We observe that our RGB Teacher model, which is trained on the Vistas dataset, has a high mIoU score of 69.4 in the day domain and an expected low score of 35.7, as the network is neither trained nor adapted to the night domain.</p><p>Our thermal teacher model MN achieves a mIoU score of 57.0, which shows that the domain gap is much smaller for this domain as for RGB. Our final RGB-T HeatNet model achieves with 64.9 the overall best score on our test set. Furthermore, the RGB-only HeatNet reaches a comparable score to our RGB-T variant, proving the efficiency of our distillation approach which leverages the thermal images as a bridge modality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*coWKgtX8reP_BPiQy1F3uQ.png" /></figure><p>We deploy the same distilled RGB network to publish results on the night BDD split. It can be observed that our method boosts mIoU by 50%. In order to compare the performance of our network with the recent RGB-T semantic segmentation approaches MFNet and RTFNet-50, we also fine-tune our model on the 784-image MF training set and report scores on the corresponding test set. We select all classes that are compatible between MF and Freiburg Thermal for evaluation which are the classes Car, Person, and Bike. We train our method only with labels provided by the teacher model MD, while not requiring any nighttime labels or labels from MF in general. Thus, it is expected that MFNet and RTFNet outperform HeatNet as they are trained supervised. However, it can be observed that HeatNet achieves comparable numbers to MFNet.</p><p>We further evaluate the generalization properties of the models trained on MF and tested on our FR-T dataset. We observe that the model performance deteriorates when evaluating MFNet or RTFNet on our FR-T dataset. We conclude that the diversity and complexity of the MF dataset do not suffice to train robust and accurate models for daytime or nighttime semantic segmentation of urban scenes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gj4Mn9qbR8wCqNl0XGRfSQ.png" /><figcaption>Fig. 2 — Qualitative semantic segmentation results of our model variants. We compare segmentation masks of our RGB-only teacher model, HeatNet RGB-only, and HeatNet RGB-T to ground truth. In the first two rows, we show segmentation masks obtained on the Freiburg Thermal dataset. The bottom row illustrates results obtained on the RGB-only BDD dataset. The multimodal approaches cannot be evaluated on BDD and the corresponding images are left blank.</figcaption></figure><h3>Conclusion</h3><p>In this work, we presented a novel and robust approach for daytime and nighttime semantic segmentation of urban scenes by leveraging both RGB and thermal images. We showed that our HeatNet approach avoids expensive and cumbersome annotation of nighttime images by learning from a pre-trained RGB-only teacher model and by adapting to the nighttime domain. We further proposed a novel training initialization scheme by first pre-training our model with a daytime RGB-only teacher model and a nighttime thermal-only teacher model and subsequently fine-tuning the model with a domain confusion loss. We furthermore introduced a first-of-its-kind large-scale RGB-T semantic segmentation dataset, including a novel target-less thermal camera calibration method based on image gradient alignment maximization. We presented comprehensive quantitative and qualitative evaluations on multiple datasets and demonstrated the benefit of the complimentary thermal modality for semantic segmentation and for learning more robust RGB-only nighttime models.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d54ffb2ee7f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Self-Supervised Visual Terrain Classification]]></title>
            <link>https://medium.com/swlh/self-supervised-visual-terrain-classification-6e4f09418328?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e4f09418328</guid>
            <category><![CDATA[semantic-segmentation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Sat, 04 Jan 2020 16:26:57 GMT</pubDate>
            <atom:updated>2020-01-23T07:19:40.547Z</atom:updated>
            <content:encoded><![CDATA[<p>In this post, I present a summary of my findings from my most recent paper <strong><em>Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning</em></strong>. A preprint is available <a href="https://arxiv.org/abs/1912.03227">here </a>and the project website is available at <a href="http://deepterrain.cs.uni-freiburg.de/">http://deepterrain.cs.uni-freiburg.de/</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F4O9kK0hOSqg%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D4O9kK0hOSqg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F4O9kK0hOSqg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c655de9a5dff5b3d846fcca89cc4431b/href">https://medium.com/media/c655de9a5dff5b3d846fcca89cc4431b/href</a></iframe><p>This post is more technical than my usual posts. If you have any questions about the research, please post questions in the comments. Thanks!</p><h3>Introduction</h3><p>Recent advances in robotics and machine learning have enabled the deployment of autonomous robots in challenging outdoor environments for complex tasks such as autonomous driving, last-mile delivery, and patrolling. Robots operating in these environments encounter a wide range of terrains from paved roads and cobblestones to unstructured dirt roads and grass. It is essential for them to be able to reliably classify and characterize these terrains for safe and efficient navigation. This is an <em>extremely challenging problem</em> as the visual appearance of outdoor terrain drastically changes over the course of days and seasons, with variations in lighting due to weather, precipitation, artificial light sources, dirt or snow on the ground. Therefore, robots should be able to actively perceive the terrains and adapt their navigation strategy as solely relying on pre-existing maps is insufficient.</p><p>Most state-of-the-art learning methods require a significant amount of data samples which are often arduous to obtain in supervised learning settings where labels have to be manually assigned to data samples. Moreover, these models tend to degrade in performance once presented with data sampled from a distribution that is not present in the training data. In order to perform well on data from a new distribution, they have to be retrained after repeated manual labeling which in general is unsustainable for the widespread deployment of robots. Self-supervised learning allows the training data to be labeled automatically by exploiting the correlations between different input signals thereby reducing the amount of manual labeling work by a large margin.</p><p>Furthermore, unsupervised audio classification eliminates the need to manually label audio samples. We take a step towards lifelong learning for visual terrain classification by leveraging the fact that the distribution of terrain sounds does not depend on the visual appearance of the terrain. This enables us to employ our trained audio terrain classification model in previously unseen visual perceptual conditions to automatically label patches of terrain in images, in a completely self-supervised manner. The visual classification model can then be fine-tuned on the new training samples by leveraging transfer learning to adapt to the new appearance conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*RLFgI9YaGIU7-rdCYqW5lg.png" /><figcaption>Fig. 1: Our self-supervised approach enables a robot to classify urban terrains without any manual labeling using an on-board camera and a microphone. Our proposed unsupervised audio classifier automatically labels visual terrain patches by projecting the traversed tracks into camera images. The resulting sparsely labeled images are used to train a semantic segmentation network for visually classifying new camera images in a pixel-wise manner.</figcaption></figure><p>In our work, we present a novel self-supervised approach to visual terrain classification by exploiting the supervisory signal from an unsupervised proprioceptive terrain classifier utilizing vehicle-terrain interaction sounds. Fig. 1 illustrates our approach where our robot equipped with a camera and a microphone traverses different terrains and captures both sensor streams along its trajectory. The poses of the robot recorded along the trajectory enables us to associate the visual features of a patch of ground that is in front of the robot initially with its corresponding auditory features when that patch of ground is traversed by the robot. We split the audio stream into small snippets and embed them into an embedding space using metric learning. To this end, we propose a novel triplet sampling method based on the visual features of the respective terrain patches. This now enables the usage of triplet loss formulations for metric learning without requiring ground truth labels. We obtained the aforementioned visual features from an off-the-shelf image classification model pre-trained on the ImageNet dataset. To the best of our knowledge, our work is the first to exploit embeddings from one modality to form triplets for learning an embedding space for samples from an extremely different modality. We interpret the resulting clusters formed by the audio embeddings as labels for training a weakly-supervised visual semantic terrain segmentation model. We then employ this model for pixel-wise classification of terrain that is in front of the robot and use this information to build semantic terrain maps of the robot environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wpeYWwL1DKz0foGVS3qnzA.png" /><figcaption>Fig. 2: The five terrain types along with a birds-eye-view image and the corresponding spectrogram of the vehicle-terrain interaction sound from the five different terrain classes</figcaption></figure><h3>Technical Approach</h3><p>In this section, we detail our proposed self-supervised terrain classification framework. Fig. 3 visualizes the overall information flow in our system. While acquiring the images and audio data, we tag each sample with the robot pose obtained using our SLAM system. We then project the camera images into a birds-eye-view perspective and project the path traversed by the robot in terms of its footprint into this viewpoint. We transform the audio clips into a spectrogram representation and embed them into an embedding space using our proposed Siamese Encoder with Reconstruction loss on audio triplets that uses features in the visual domain for triplet forming. Subsequently, we cluster the embeddings and use the cluster indices to automatically label the corresponding robot path segments in the birds-eye-view images. The resulting labeled images serve as weakly labeled training data for the semantic segmentation network. <strong>Note that the entire approach is executed completely in an unsupervised manner</strong>. The cluster indices can be used to indicate terrain labels such as Asphalt and Grass or in terms of terrain properties.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DZ_nv_57MBfY8msL67Luxg.png" /><figcaption>Fig. 3: Overview of our proposed self-supervised terrain classification framework. The upper part of the figure illustrates our novel Siamese Encoder with Reconstruction loss (SE-R), while the lower part illustrates how the labels obtained from the SE-R are used to automatically annotate data for training the semantic segmentation network. The camera images are first projected into a birds-eye-view perspective of the scene and the trajectory of the robot is projected into this viewpoint. In our SE-R approach, using both the audio clips from the recorded terrain traversal and the corresponding patches of terrain recorded with a camera, we embed each clip of the audio stream into an embedding space that is highly discriminative in terms of the underlying terrain class. This is performed by forming triplets of audio samples using the visual similarity of the corresponding patches of ground obtained with a pre-trained feature extractor. We then cluster the resulting audio embeddings and use the cluster indices as labels for self-supervised labeling. The resulting labeled images serve as a weakly labeled training dataset for a semantic segmentation network for pixel-wise terrain classification.</figcaption></figure><p>We will now briefly discuss the major steps in the processing pipeline.</p><h4>Trajectory Projection Into Image Coordinates</h4><p>We record the stream of monocular camera images from an on-board camera and the corresponding audio stream of the vehicle-terrain interaction sounds from a microphone mounted near the wheel of our robot. We project the robot trajectory into the image coordinates using the robot poses obtained using our SLAM system. We additionally perform perspective warping of the camera images in order to obtain a birds-eye view representation.</p><h4>Unsupervised Acoustic Feature Learning</h4><p>Each terrain patch that the robot traverses is represented by two modalities: <strong>sound </strong>and <strong>vision</strong>. We obtain the visual representation of a terrain patch from a distance using an on-board camera, while we record the vehicle-terrain interaction sounds by traversing the corresponding terrain patch. For our unsupervised acoustic feature learning approach, we exploit the highly discriminative visual embeddings of terrain patch images obtained using a CNN pre-trained on the ImageNet dataset to form triplets of audio samples. To form such discriminative clusters of embeddings, triplet losses have been proposed. We argue that the relative position of a terrain patch image embeddings in embedding space serves as a good approximation for ground truth labels that have previously been relied on for triplet forming. We form triplets of audio clips using this heuristic. Finally, we train our Siamese Encoder with reconstruction loss in order to embed these audio clips into a highly discriminative audio embedding space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UsfZ6KFFxmfLrCGFs6-4-Q.png" /><figcaption>Fig. 4: Two-dimensional t-SNE visualizations of the audio samples embedded with our SE-R approach after 0, 10, 30, and 90 epochs of training. The color of<br>the points indicate the corresponding ground truth class. We observe that clusters of embeddings are clearly separable as the training progresses and they<br>highly correlate with the ground truth terrain class.</figcaption></figure><p>The triplet loss enforces that the embeddings of samples with the same label are pulled together in embedding space and embeddings of samples with different labels are pushed away from each other simultaneously. As the ground truth labels of the audio samples are not available to form triplets, we argue that an unsupervised heuristic can serve as a substitute signal for the ground truth labels for triplet creation: the local neighborhood of the terrain image patch embeddings. We obtain rectangular patches of terrain by selecting segments of pixels along the robot path. The closest neighbor in the embedding space has a high likelihood of belonging to the same ground truth class as the anchor sample. Therefore, for sampling triplets, we select<br>the sample with the smallest euclidean distance in the visual embedding space as a positive sample. We then select negative samples by randomly selecting samples that are in a different cluster in visual embedding space than the anchor sample. Although it cannot be always guaranteed that the negative sample does not have the same ground truth class, it has a high likelihood of belonging to a different class, which we observe in our experiments. Likewise, we argue that visually similar terrain patches have a high likelihood of belonging to the same class. This means that in practice a fraction of the generated triplets are not correctly formed. However, we empirically find that it is sufficient if the majority all triplets have correct class assignments as they outweigh the incorrectly defined triplets.</p><p>Finally, we perform k-means clustering of the embeddings to separate the samples into K clusters, corresponding to the K terrain classes present in the dataset. Our approach only requires us to set the number of terrain classes that are present and assign terrain class names to the cluster indices.</p><h4>Self-Supervised Visual Terrain Classification</h4><p>We use the resulting weakly self-labeled birds-eye-view scenes to train a semantic segmentation network in a self- supervised manner. A self-supervisory signal can be obtained for every image pixel that contains a part of the robot path for which the label is known from the unsupervised audio clustering. Note that the segmentation masks for the traversed terrain types are incomplete as the robot cannot be expected to traverse every physical location of terrain in the view to generate complete segmentation masks. We alleviate this challenge by considering all the pixels in camera images that do not contain the robot path as a background class that does not contribute to the segmentation loss. We deal with the class imbalance in the training set by weighing each class proportional to its log frequency in the training data set.</p><h3>Results</h3><p>We will briefly discuss some of the results reported in the original publication.</p><h4>Qualitative Results</h4><p>Fig. 6 illustrates some qualitative terrain classification results for a small clip in the dataset. We observe that a majority of pixels in each frame are assigned the correct labels. Some errors occur for terrains that are partially covered with objects (bikes in this scene) or have non-favorable lighting conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*D97v_vFaO4xMiCYewfZTKw.gif" /><figcaption>Fig. 6: Qualitative terrain classification results for a small clip in the dataset.</figcaption></figure><p>For more qualitative and quantitative results, please refer to the original publication.</p><h4>Generalization to Terrain Appearance Changes</h4><p>One of the major advantages of our self-supervised approach is that new labels on previously unseen terrains can easily be generated by the robot automatically. While the terrain traversal sounds do not substantially vary with the weather conditions other than rain and winds, the visual appearance of terrain can vary depending on several factors including time of day, season<br>or cloudiness. We record data at dusk with low light conditions and artificial lighting resulting in a variation in terrain hues and substantial motion blur. We qualitatively compare the terrain classification results for a model trained exclusively on the Freiburg Terrains dataset and a model trained jointly on the Freiburg Terrains dataset as well as on the new low light dataset. Qualitative results from this experiment are shown in Fig. 7.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tipiric5f43gM8chA19Log.png" /><figcaption>Fig. 7: Qualitative results on a new low light dataset that was captured at dusk that has a considerable amount of motion blur, color noise, and artificial lighting. We show a comparison between the terrain classification model without and with fine-tuning on training data created using our self-supervised approach</figcaption></figure><h4>Semantic Terrain Mapping</h4><p>We finally demonstrate the utility of our proposed self-supervised semantic segmentation framework for building semantic terrain maps of the environment. To build such a map, we use the poses of the robot that we obtain using our SLAM system and the terrain predictions of the birds-eye-view camera images. We project each image into the correct location in a<br>global map using the 3-D camera pose and we use no additional image blending or image alignment optimization. For each single birds-eye-view image, we generate pixel-wise terrain classification predictions using our self-supervised semantic segmentation model. We then project these segmentation mask predictions into their corresponding locations in the global semantic terrain map, similar to the procedure that we employ for the birds-eye-view images. When there are predictions of a terrain location from multiple views, we choose the class with the highest prediction count for each pixel in the map. We also experimented with fusing the predictions from multiple views using Bayesian fusion which yields similar results.<br>Fig. 8 shows how a semantic terrain map can be built from single camera images and the corresponding semantic terrain predictions of our approach. It can be observed that our self- supervised terrain segmentation model yields predictions that are for the most part globally consistent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*OCPqWK0-YffKVpQ_myanvA.gif" /><figcaption>Fig. 8: Tiled birds-eye-view images and the corresponding semantic terrain maps built from the predictions of our self-supervised semantic terrain segmentation model. We use the SLAM poses of the camera to obtain the 6-D camera poses for each frame.</figcaption></figure><h3>Conclusion</h3><p>In this work, we proposed a self-supervised terrain classification framework that exploits the training signal from an unsupervised proprioceptive terrain classifier to learn an exteroceptive classifier for pixel-wise semantic terrain segmentation. We presented a novel heuristic for triplet sampling in metric<br>learning that leverages a complementary modality as opposed to the typical strategy that requires ground truth labels. We employed this proposed heuristic for unsupervised clustering of vehicle-terrain interaction sound embeddings and subsequently used the resulting clusters formed by the audio embeddings for self-supervised labeling of terrain patches in images. We then<br>trained a semantic terrain segmentation network from these weak labels for dense pixel-wise classification of terrains that are in front of the robot.</p><p>Thanks for reading and happy new year!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e4f09418328" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/self-supervised-visual-terrain-classification-6e4f09418328">Self-Supervised Visual Terrain Classification</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s new in TensorFlow 2.0?]]></title>
            <link>https://medium.com/data-science/whats-new-in-tensorflow-2-0-ce75cdd1a4d1?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce75cdd1a4d1</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Tue, 30 Apr 2019 16:26:02 GMT</pubDate>
            <atom:updated>2019-05-01T21:35:32.742Z</atom:updated>
            <content:encoded><![CDATA[<p>The machine learning library TensorFlow has had a long history of releases starting from the initial open-source release from the Google Brain team back in November 2015. Initially developed internally under the name <strong>DistBelief</strong>, TensorFlow quickly rose to become the most widely used machine learning library today. And not without reason.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*RxfEYcphHNhT6EdUhLYKCA.png" /><figcaption>Number of repository stars over time for the most widely used machine learning libraries</figcaption></figure><h3>TensorFlow 1.XX — where are we today?</h3><p>Before we discuss the most important changes for TensorFlow 2.0, let us quickly recap the some of the essential aspects of TensorFlow 1.XX:</p><h4>Language support</h4><p>Python was the first client language supported by TensorFlow and currently supports the most features within the TensorFlow ecosystem. Nowadays, TensorFlow is available in a multitude of programming languages. The TensorFlow core is written in pure C++ for better performance and is exposed via a C API. Apart from the bindings to Python2.7/3.4–3.7, TensorFlow also offers support for JavaScript (<a href="https://www.tensorflow.org/js">Tensorflow.js</a>), Rust and R. Especially the syntactically simple Python API, compared to the brittle explicitness of C/C++ allowed TensorFlow to quickly overtake the Caffe machine learning library, an early-day competitor.</p><h4><strong>Computation Graph</strong></h4><p>From the start, the core of TensorFlow has been the so-called <strong>Computation Graph</strong>. In this graph model, each operation (Add, Multiply, Subtract, Logarithmize, Matrix-Vector Algebra, Complex functions, broadcasting, …) and also Variables/Constants are defined by a node in a directed graph. The directed edges of the Graph connect nodes to each other and define in which direction information/data flows from one node to the next. There are Input-Nodes where information is fed into the Computation Graph from outside, and Output-Nodes that output the processed data.</p><p>After the Graph has been defined, it can be executed on data that is fed into the Graph. Thus, the data ‘<strong><em>flows</em></strong>’ through the graph, changes its content and shape, and is transformed into the output of the Graph. The data can usually be expressed as a multidimensional array, or <strong><em>Tensor</em></strong>, thus the name <strong><em>TensorFlow</em></strong>.</p><p>Using this model, it is easy to define the architecture of a neural network using these nodes. Each layer of a neural network can be understood as a special node in the computation graph. There are many pre-defined operations in the TensorFlow API, but users can of course define their own custom operations. But keep in mind that arbitrary computations can be defined using a computation graph, not only operations in the context of machine learning.</p><p>Graphs are invoked by as <strong>TensorFlow session</strong>: <em>tf.Session()</em>. A session can take run options as arguments, such as the number of GPUs the graph should be executed on, the specifics of memory allocation on the GPU and what not. Once the necessary data is available, it can be fed into the the computation graph using the tf.Session.run() method in which all the magic happens.</p><h4>Gradients</h4><p>In order to train a neural network, using an optimization algorithm such as Stochastic Gradient Descent, we need the definitions of the gradients of all operations in the network. Otherwise, performing backpropagation on the network is not possible. Luckily, TensorFlow offers automatic differentiation for us, such that we only have to define the forward-pass of information through the network. The backward-pass of the error through all layers is inferred automatically. This feature is not unique with TensorFlow — all current ML libraries offer automatic differentiation.</p><h4>CUDA</h4><p>From the start, the focus of TensorFlow was to let the Computing Graph execute on GPUs. Their highly parallel architecture offers ideal performance for excessive matrix-vector arithmetic which is necessary for training machine learning libraries. The NVIDIA CUDA (<strong>C</strong>ompute <strong>U</strong>nified <strong>D</strong>evice <strong>A</strong>rchitecture) API allows TensorFlow to execute arbitrary operations on a NVIDIA GPU.</p><p>There are also projects with the goal to expose TensorFlow to any OpenCL-compatible device (i.e. also AMD GPUs). However, NVIDIA still remains the clear champion in Deep Learning GPU hardware, not the least due to the success of CUDA+TensorFlow.</p><p>Getting a working installation of CUDA on your machine, including CuDNN and the correct NVIDIA drivers for your GPU can be a <em>painful </em>experience (especially since not all TensorFlow versions are compatible with all CUDA/CuDNN/NVIDIA driver versions and you were too lazy to have a look at the <a href="https://www.tensorflow.org/install/source#linux">version compatibility pages</a>), however, once TensorFlow can use your GPU(s), you will recognize a significant boost in performance.</p><h4>Multi GPU support</h4><p>Large-scale machine learning tasks require access to more than one GPU in order to yield results quickly. Large enough deep neural networks have too many parameters to fit them all into a single GPU. TensorFlow lets users easily declare on which devices (GPU or CPU) the computation graph should be executed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*773QM1sNAeu45utkPtxQiw.png" /><figcaption>Multi-GPU computation model (source: <a href="https://www.tensorflow.org/tutorials/images/deep_cnn">https://www.tensorflow.org/tutorials/images/deep_cnn</a>)</figcaption></figure><h4>Eager execution</h4><p>The TensorFlow Computation Graph is a powerful model for processing information. However, a major point of criticism from the start was the difficulty of debugging such graphs. With statements such as</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8af32536c473ff95594c26c5d5269275/href">https://medium.com/media/8af32536c473ff95594c26c5d5269275/href</a></iframe><p>the content of the variable c is not 4.0, as one might expect, but rather a TensorFlow node with no definite value assigned to it yet. The validity of such a statement (and the possible bugs introduced by the statement) can only be tested after the Graph was invoked and a session was run on the Graph.</p><p>Thus, TensorFlow released the eager execution mode, for which each node is immediately executed after definition. Statements using <em>tf.placeholder</em> are thus no longer valid. The eager execution mode is simply invoked using <em>tf.eager_execution() </em>after importing TensorFlow.</p><p>TensorFlow’s eager execution is an imperative programming environment that evaluates operations immediately, without building graphs: operations return concrete values instead of constructing a computational graph to run later. The advantages of this approach are easier debugging of all computations, natural control flow using Python statements instead of graph control flow, and an intuitive interface. The downside of eager mode is the reduced performance since graph-level optimizations such as common subexpression elimination and constant-folding are no longer available.</p><h4>Debugger</h4><p>The TensorFlow Debugger (<strong><em>tfdbg</em></strong>) lets you view the internal structure and states of running TensorFlow graphs during training and inference, which is difficult to debug with general-purpose debuggers such as Python’s <em>dbg </em>to TensorFlow&#39;s computation-graph paradigm. It was conceived as an answer to criticism regarding the difficulty in debugging TensorFlow programs. There is both a command-line interface and a Debugging plugin for TensorBoard (more info below) that allows you to inspect the computation graph for debugging. For a detailed introduction, please find <a href="https://www.tensorflow.org/guide/debugger">https://www.tensorflow.org/guide/debugger</a>.</p><h4>TensorBoard</h4><p>You can use TensorBoard to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data such as images that pass through it during training or inference. It is definitely the way to go if you wish to visualize any kind of data that is available during within the computation graph. While TensorBoard was originally introduced as part of TensorFlow, it now lives in its own GitHub repository. However, it will be installed automatically when installing TensorFlow itself.</p><p>TensoBoard is not only useful for visualizing training or evaluation data such as losses/accuracies as a function of the number of steps, but also for visualizing image data or sound waveforms. The best way to get an overview of TensorBoard is to have a look at <a href="https://www.tensorflow.org/guide/summaries_and_tensorboard">https://www.tensorflow.org/guide/summaries_and_tensorboard</a>.</p><h4>TPU support</h4><p>TPUs (Tensor Processing Units) are highly-parallel computing units specifically designed to efficiently process multi-dimensional arrays (a.k.a. tensors), which is particularly useful in Machine Learning. Due to their <a href="https://en.wikipedia.org/wiki/Application-specific_integrated_circuit">application-specific integrated circuit</a> (ASIC) design, they are the fastest processors for machine learning applications available today. As of today, Google’s TPUs are proprietary and are not commercially available for any private consumers or businesses. They are part of the Google Compute Engine, where you can rent compute instances that have access to TPUs for your large-scale machine learning needs. Needless to say that Google aims at making every TensorFlow operation executable on a TPU device to further strengthen its position in the ever-growing cloud computing market.</p><p>You can, however, test the performance of a single TPU for yourself in Google Colab, a platform that can host and execute Jupyter Notebooks, with access to CPU/GPU or TPU instances on the Google Compute Engine, for free! For a small introduction, click <a href="https://medium.com/@jannik.zuern/using-a-tpu-in-google-colab-54257328d7da">here</a>.</p><h4>TensorRT</h4><p>While neural network training typically happens on powerful hardware with sometimes multiple GPUs, neural network inference usually happens locally on consumer devices (unless the raw data is streamed to another cloud service, and inference happens there) such as the onboard computers of autonomous cars or even mobile phones. NVIDIA offers a module called <a href="https://developer.nvidia.com/tensorrt">TensorRT </a>that takes a TensorFlow Graph of a trained neural network expressed using the TensorFlow API and converts it to a Computation Graph specifically optimized for <em>inference</em>. This usually results in a significant performance gain compared to inference within TensorFlow itself. For an introduction to TensorRT, <a href="https://medium.com/@jannik.zuern/exploring-the-new-tensorflow-integration-of-nvidias-tensorrt-148ee0d95cd5">click here</a>.</p><h4>tf.contrib</h4><p>TensorFlow has a vibrant community on GitHub that added quite some functionality to the core and the peripherals of TensorFlow(Obviously a strong argument for Google to open-source TensorFlow). Most of these modules are collected in the <a href="https://www.tensorflow.org/api_docs/python/tf/contrib">tf.contrib module</a>. Due to the high market share of TensorFlow, quite a few modules can be found here that you would otherwise have to implement yourself.</p><h4>Tensorflow Hub</h4><blockquote>TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models. A <em>module</em> is a self-contained piece of a TensorFlow graph, along with its weights and assets, that can be reused across different tasks in a process known as transfer learning.<a href="https://www.tensorflow.org/hub">[0]</a></blockquote><h4>More, more, more</h4><p>There is so much more to talk about. Which components of the TensorFlow ecosystem should at least be mentioned?</p><ul><li><strong>TensorFlow Docker container: </strong>Docker containers containing pre-installed TensorFlow, including CUDA compatibility for graph execution on GPUs from within the Docker container</li><li><strong>TensorFlow Lite</strong>: TensorFlow Lite is an open source deep learning framework for on-device inference on devices such as embedded systems and mobile phones.</li><li><strong>TensorFlow Extended (TFX): </strong>TFX is a Google-production-scale machine learning platform based on TensorFlow. It provides a configuration framework and shared libraries to integrate common components needed to define, launch, and monitor your machine learning system.<a href="https://www.tensorflow.org/tfx">[1]</a></li></ul><h4>What sucks?</h4><p>One of the strengths of TensorFlow, the <strong>Computation Graph</strong>, is arguably also one of its weaknesses. While the static computation Graph definitely boosts performance (since graph-level optimizations can happen after the graph is built and before it is executed), it also makes debugging the graph difficult and cumbersome — even with tools such as the TensorFlow Debugger. Also, benchmarks have shown that several other frameworks can compete on equal terms with TensorFlow, while keeping a simpler syntax. Additionally, first building a graph and then instantiating it using <em>tf.Sessions</em> is not very intuitive and definitely scares or bewilders some inexperienced users.</p><p>The TensorFlow API arguably also has weaknesses, i.e. discussed <a href="http://nicodjimenez.github.io/2017/10/08/tensorflow.html">here</a>. Some users complain about the low-level-feeling when using the TensorFlow API, even when solving a high-level task. Much boilder-plate code is needed for simple tasks such as training a linear classifier.</p><h3>TensorFlow 2.0 — What’s new?</h3><p>After delving into the depths of TensorFlow 1.XX, what will change with the big 2? Did the TensorFlow team respond to some of the criticism of the past? And what justifies calling it version 2.0, and not 1.14?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*Zy0gFGzNwZAwjeCgpHhIHw.png" /><figcaption>Image source: <a href="https://www.tensorflow.org/">https://www.tensorflow.org/</a></figcaption></figure><p>In multiple blog posts and announcements, some of the future features of TF2.0 have been revealed. Also, the TF2.0 API reference lists have already been made publicly available. While TF2.0 is still in alpha version, it is expected that the official Beta, Release Candidates, and the final release will be made available later this year.</p><p>Let’s have a closer look at some of the novelties of TF2.0:</p><ul><li><strong>Goodbye <em>tf</em>, hello <em>tf.keras</em></strong></li></ul><p>For a while, TensorFlow has offered the <em>tf.keras</em> API as part of the TensorFlow module, offering the same syntax as the Keras machine learning library. Keras has received much praise for its simple and intuitive API for defining network architectures and training them. Keras integrates tightly with the rest of TensorFlow so you can access TensorFlow’s features whenever you want. The Keras API makes it easy to get started with TensorFlow. Importantly, Keras provides several model-building APIs (Sequential, Functional, and Subclassing), so you can choose the right level of abstraction for your project. TensorFlow’s implementation contains enhancements including eager execution, for immediate iteration and intuitive debugging, and tf.data, for building scalable input pipelines.</p><ul><li><strong>tf.data</strong></li></ul><p>Training data is read using input pipelines which are created using <em>tf.data. </em>This will be the preferred way of declaring input pipelines. Pipelines using <em>tf.placeholders</em> and <em>feed dicts</em> for sessions will still work under the TensorFlow v1 compatibility mode, but will no longer benefit from performance improvements in subsequent tf2.0 versions.</p><ul><li><strong>Eager execution default</strong></li></ul><p>TensorFlow 2.0 runs with eager execution (discussed previously) by default for ease of use and smooth debugging.</p><ul><li><strong>RIP tf.contrib</strong></li></ul><p>Most of the modules in tf.contrib will depreciate in tf2.0 and will be either moved into core TensorFlow or removed altogether.</p><ul><li><strong>tf.function decorator</strong></li></ul><p>The <strong>tf.function</strong> function decorator transparently translates your Python programs into TensorFlow graphs. This process retains all the advantages of 1.x TensorFlow graph-based execution: Performance optimizations, remote execution and the ability to serialize, export and deploy easily while adding the flexibility and ease of use of expressing programs in simple Python. In my opinion, this is the biggest change and paradigm shift from v1.X to v2.0.</p><ul><li><strong>No more tf.Session()</strong></li></ul><p>When code is eagerly executed, sessions instantiating and running computation graphs will no longer be necessary. This simplifies many API calls and removes some boilerplate code from the codebase.</p><ul><li><strong>TensorFlow 1.XX legacy code</strong></li></ul><p>It will still be possible to run tf1.XX code in tf2 without any modifications, but this does not let you take advantage of many of the improvements made in TensorFlow 2.0. Instead, you can try running a conversion script that automatically converts the old tf1.XX calls to tf2 calls, if possible. The detailed <a href="https://www.tensorflow.org/alpha/guide/migration_guide">migration guide</a> from tf1 to tf2 will give you more information if needed.</p><p>I hope you liked this small overview, and see you next time!</p><p>Happy Tensorflowing!</p><h3>Further reading</h3><ul><li><strong>Effective TF2</strong>: <a href="https://www.tensorflow.org/alpha/guide/effective_tf2">https://www.tensorflow.org/alpha/guide/effective_tf2</a></li><li><strong>Whats new in TF2?</strong>: <a href="https://medium.com/tensorflow/whats-coming-in-tensorflow-2-0-d3663832e9b8">https://medium.com/tensorflow/whats-coming-in-tensorflow-2-0-d3663832e9b8</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce75cdd1a4d1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/whats-new-in-tensorflow-2-0-ce75cdd1a4d1">What’s new in TensorFlow 2.0?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[But what is an Autoencoder?]]></title>
            <link>https://jannik-zuern.medium.com/but-what-is-an-autoencoder-26ec3386a2af?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/26ec3386a2af</guid>
            <category><![CDATA[autoencoder]]></category>
            <category><![CDATA[fashion-mnist]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Sun, 24 Feb 2019 14:55:53 GMT</pubDate>
            <atom:updated>2019-02-24T14:55:53.659Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s post I would like to give you a quick-and-dirty introduction into a neural network architecture type called Autoencoders. The post is aimed at Machine Learning beginners who would like get some basic insights into Autoencoders and why they are so useful.</p><h3>Context</h3><p>Autoencoders are structured to take an input, transform this input into a different representation, an <em>embedding</em> of the input. From this embedding, it aims to reconstruct the original input as precisely as possible. It basically tries to copy the input. The layers of the Autoencoder that create this embedding are called the <strong>encoder</strong>, and the layers that try to reconstruct the embedding into the original input are called <strong>decoder</strong>. Usually Autoencoders are restricted in ways that allow them to copy only approximately. Because the model is forced to prioritize which aspects of the input should be copied, it often learns useful properties of the data.</p><p>More formally, an autoencoder describes a nonlinear mapping of an input <strong><em>x</em></strong> into an output <strong>x</strong> using an intermediate representation <strong><em>x_encoded</em></strong><em> </em>= <em>f_encode</em>(<strong>x</strong>), also called an <em>embedding</em>. The embedding is typically denoted as <strong><em>h</em></strong> (h for hidden, I suppose). During training, the encoder learns a nonlinear mapping of <strong><em>x</em></strong> into <strong><em>x_encoded</em></strong><em>.</em> The decoder, on the other hand, learns a nonlinear mapping from <strong><em>x_encoded</em></strong> into the original space. The goal of training is to minimize a loss. This loss describes the objective that the autoencoder tries to reach. When our goal is to merely reconstruct the input as accurately as possible, two major types of loss function are typically used: Mean squared error and Kullback-Leibler (KL) divergence.</p><p>The <strong>mean squared error (MSE)</strong> is (as its name already suggests) defined as the mean of the squared difference between our network output and the ground truth. When the encoder output is a grid of values <em>a.k.a. an image</em>, the MSE between output image <em>I</em>¯ and ground truth image <em>I</em> may be defined as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*evOWI9aYRyK9xuwKPwlAvg.png" /></figure><p>The notion of <strong>KL divergence</strong> comes originally from information theory and describes the relative entropy between two probability distributions <em>p</em> and <em>q</em>. Because the KL divergence is non-negative and measures the difference between two distributions, it is often conceptualized as measuring some sort of distance between these distributions.</p><p>The KL divergence has many useful properties, most notably that it is non-negative. The KL divergence is 0 if and only if <em>p</em>p and <em>q</em> are the same distribution in the case of discrete variables, or equal <em>almost everywhere</em> in the case of continuous variables. It is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*TZXtsOOhz336Rv2xx_aSkw.png" /></figure><p>In the context of Machine Learning, minimizing the KL divergence means to make the autoencoder sample its output from a distribution that is similar to the distribution of the input, which is a desirable property of an autoencoder.</p><h3>Autoencoder flavors</h3><p>Autoencoders come in many different flavors. For the purpose of this post, we will only discuss the most important concepts and ideas for autoencoders. Most Autoencoders you might encounter in the wild are <em>undercomplete</em> autoencoders. This means that the condensed representation of the input can hold less information than the input has. If your input has <em>N</em> dimensions, and some hidden layer of your autoencoder has only <em>X</em>&lt;<em>N</em> dimensions, your autoencoder is undercomplete. Why would you want to hold less information in the hidden layer than your input might contain? The idea is that restricting the amount of information the encoder can put into the the encoded representation forces it to only focus on the relevant and discriminative information within the input since this allows the decoder to reconstruct the input as best as possible. Undercomplete autoencoder <em>boil the information down</em> into the most essential bits. It is a form of <em>Dimensionality reduction</em>.</p><p>Now, let us discuss some flavors of autoencoders that you might encounter <strong>“in the wild”</strong>:</p><h4>Vanilla Autoencoder</h4><p>The most basic example of an autoencoder may be defined with an input layer, a hidden layer, and an output layer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*Bv8dVyCE45Nhk5xrM9jL6A.png" /><figcaption><em>A simple autoencoder (image credit: </em><a href="https://pythonmachinelearning.pro/all-about-autoencoders/"><em>[2]</em></a><em>)</em></figcaption></figure><p>The Input layer typically has the same dimensions as the output layer since we try to reconstruct the content of the input, while the hidden layer has a smaller number of dimensions that input or output layer.</p><h4>Sparse Autoencoder</h4><p>However, depending on the purpose of the encoding scheme, it can be useful to add an additional term to the loss function that needs to be satisfied as well.</p><p>Sparse autoencoders, as their name suggests, enforce sparsity on the embedding variables. This can be achieved by means of a sparsity penalty Ω(<strong>h</strong>) on the embedding layer <strong>h</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*ca5gEnRisLtRCNJwi8M_hA.png" /></figure><p>The operator L denotes an arbitrary distance metric (i.e. MSE or KL-divergence) between input and output. The sparsity penalty may be expressed the <em>L</em>1-norm of the hidden layer weights:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*D_cSGxQXh0Q0PqysBJNy-g.png" /></figure><p>with a scaling parameter <em>λ</em>. Enforcing sparsity is a form of regularization and can improve the generalization abilities of the autoencoder.</p><h4>Denoising Autoencoder</h4><p>As the name suggests, a <em>denoising</em> autoencoder is able to robustly remove noise from images. How can it achieve this property? It finds feature vectors that are somewhat invariant to noise in the input (within a reasonable <a href="https://en.wikipedia.org/wiki/Signal-to-noise_ratio">SNR</a>).</p><p>A denoising autoencoder can very easily be constructed by modifying the loss function of a vanilla autoencoder. Instead of calculating the error between the original input <strong>x </strong>and the reconstructed input <strong>x</strong>̃ , we calculate the error between the original input and the reconstruction of an input <strong>x</strong>̂ that was corrupted by some form of noise:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*4cckN26ke1iGou5gIYPO8Q.png" /></figure><p>Denoising autoencoders learn undo this corruption rather than simply copying their input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*9CiPof4rcAbkSQlxiIKeaA.png" /><figcaption><em>Denoised images (Source: </em><a href="https://blog.keras.io/building-autoencoders-in-keras.html"><em>[1]</em></a><em>)</em></figcaption></figure><h4>Contractive Autoencoder</h4><p>A contrative autoencoder is another subtype of a sparse autoencoder (we impose an additional constraint on the reconstruction loss). For this type of autoencoder, we penalize the weights of the embedding layer by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*Ihx-JUKCmbkmuZ8SBYMztA.png" /></figure><p>The operator ∇ denotes the Nabla-operator, meaning a gradient. Specifically, we penalize large gradients of the hidden layer activations <em>h_i</em> w.r.t the input <strong><em>x</em></strong>. But what purpose might this constraint have?</p><p>Loosely speaking, it lets infinitesimal changes w.r.t. the input <strong><em>x</em></strong> not have any influence on the embedding variables. If make small changes to the pixel intensities of the input images, we do not want any changes to the embedding variables. It is encouraged to map a <strong>local neighborhood of input points</strong> to a <strong>smaller local neighborhood of output points</strong>.</p><p>And what is this useful for, you ask? The goal of the CAE is to learn the manifold structure of the data in the high-dimensional input space. For example, a CAE applied to images should learn tangent vectors that show how the image changes as objects in the image gradually change pose. This property would not be emphasized as much in a standard loss function.</p><h4>Variational Autoencoder</h4><blockquote><em>Variational Autoencoders (VAE) learn a </em><strong><em>latent variable model</em></strong><em> for its input data So instead of letting your neural network learn an arbitrary function, you are learning the parameters of a probability distribution modeling your data. If you sample points from this distribution, you can generate new input data samples: a VAE is a “generative model”. [</em><a href="https://blog.keras.io/building-autoencoders-in-keras.html"><em>1</em></a><em>]</em></blockquote><p>In contrast to a “normal” autoencoder, a VAE turns a sample not into one parameter (the embedding representation), but in two parameters <em>z_μ</em> and <em>z_σ</em>, that describe the mean and the standard deviation of a latent normal distribution that is assumed to generate the data the VAE is trained on.</p><p>The parameters of the model are trained via two loss terms: a reconstruction loss forcing the decoded samples to match the initial inputs (just like in our previous autoencoders), and the KL divergence between the learned latent distribution and the prior distribution, acting as a regularization term.</p><p>For a more in-depth introduction to VAEs, see for instance <a href="https://wiseodd.github.io/techblog/2016/12/10/variational-autoencoder/">[3]</a>.</p><h3>Experiments</h3><h4>2D-Manifold Embedding</h4><p>Let us now see how we can embed data in some latent dimensions. In this first experiment, we will strive for something very simple. We first create a super monotonous dataset consisting of many different images of random blocks with different heights and widths, we will call it the <strong>block image dataset</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*3ZJ8F2fC3O2j-hd1xUIB3g.png" /><figcaption>Exciting random blocks</figcaption></figure><p>Let us train a VAE with only two latent dimensions on 80000 of these block images and see what happens. I chose to use only two latent dimensions because each image can be visualized by the location of its latent embedding vector in a 2-D plane.</p><p>The figure below shows which feature vector in the 2-D plane corresponds to which block image. The block image is drawn at the location where its feature vector lies in the 2-D plane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*SHp49jvuCBSOMUnhXq_r3w.png" /><figcaption><em>Sampling the 2-D latent features on a uniform grid</em></figcaption></figure><p>It is quite obvious that the autoencoder was able to find a mapping that makes a lot of sense for our dataset. Recall that each input datum (a single image) has <em>height</em>⋅<em>width</em>⋅<em>channels</em>=28⋅28⋅1=784 dimensions. The autoencoder was able to reduce the dimensionality of the input to only two dimensions without losing a whole lot of information since the output is visually almost indistinguishable from the input (apart from some minor artefacts). This astounding reconstruction quality is possible since each input image is so easy to describe and does not contain very much information. Each white block can be described by only two parameters: height and width. Not even the center of each block is parametrized since each block is located exactly in the center of each image.</p><p>If you want to play around with this yourself, you can find the code <a href="https://gist.github.com/jzuern/41a6380f885e04a6f1f60e4aa216dd3f">here</a>. Most of the code was taken from the <a href="https://github.com/keras-team/keras/blob/master/examples/variational_autoencoder.py">Keras github repository</a>.</p><h4>Similar image retrieval</h4><p>While embedding those simple blocks might seem like a nice gimmick, let us now see how well an autoencoder actually performs on a real-world dataset: <strong>Fashion MNIST</strong>. Our goal here is to find out how descriptive the embedding vectors of the input images are. Autoencoders allow us to compare visual image similarity by comparing the similarity of their respective embeddings or <strong>features</strong> created by the Autoencoder.</p><p>As the Fashion MNIST images are much more information-dense than the block-images from our last mini experiment, we assume that we need more latent variables in order to express the gist of each of the training images. I chose a different autoencoder architecture with 128 latent dimensions.</p><p>The idea is to to create a feature vector for a query image for which we want similar results from a database. Below we can see an exemplary query image of a (very low-res) pair of jeans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/1*RCQDiyvejp8by_GyBC5RbQ.png" /></figure><p>Using our autoencoder that was trained on the Fashion MNIST dataset, we want to retrieve the images corresponding to the features that are closest to the query image features in the embedding space. But how can we compare the <strong>closeness</strong> of two vectors? For this kind of task, one typically uses the <a href="https://en.wikipedia.org/wiki/Cosine_similarity"><strong>cosine distance</strong></a> between the vectors as a distance metric.</p><p>Here are the four closest neighbors of the query image in feature space:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/963/1*miCVaDq4VXb56R_27MAOUA.png" /></figure><p>Nice! Pants are similar to pants, I guess!</p><p>Lets try again with a new query image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*vFKHkI3f0hgVAJI9Tqd0xw.png" /></figure><p>And here are the four closest neighbors:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*dtcInoZsbaj-nOe4LACYsQ.png" /></figure><p>Cool! The autoencoder was definitely able to encode the relevant information of each input image in the embedding layer.</p><p><strong>Note</strong>: You should bear in mind that the autoencoder was not trained with any labels for the images. It does not “know” that these are images of shirts. In only knows that the abstract features of all of these images are roughly similar and highly descriptive of the actual image content.</p><p>If you want to play around with this yourself, the code may be found <a href="https://gist.github.com/jzuern/c8a1ac0a9516890f7945146c188415db">here</a>.</p><h3>Conclusion</h3><p>We have seen how autoencoders can be constructed and what types of autoencoders have been proposed in the last couple of years.</p><p>The ability of Autoencoders to encode high-level image content information in a dense, small feature vector makes them very useful for unsupervised pretraining. We can automatically extract highly useful feature vectors from input data completely unsupervised. Later we may use these feature vectors to train an off-the-shelf classifier with these features and observe highly competitive results.</p><p>This aspect is especially useful for learning tasks where there is not very much labeled data but very much unlabeled data.</p><p>Thanks for reading and happy autoencoding! 👨‍💻🎉</p><h3>Resources</h3><ul><li>[1]: <a href="https://blog.keras.io/building-autoencoders-in-keras.html">https://blog.keras.io/building-autoencoders-in-keras.html</a></li><li>[2]: <a href="https://pythonmachinelearning.pro/all-about-autoencoders">https://pythonmachinelearning.pro/all-about-autoencoders</a></li><li>[3]: <a href="https://wiseodd.github.io/techblog/2016/12/10/variational-autoencoder/">https://wiseodd.github.io/techblog/2016/12/10/variational-autoencoder/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26ec3386a2af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning to survive in a hostile environment]]></title>
            <link>https://jannik-zuern.medium.com/reinforcement-learning-to-survive-in-a-hostile-environment-3658624a5d83?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/3658624a5d83</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[fun]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[openai]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Sat, 24 Nov 2018 16:05:03 GMT</pubDate>
            <atom:updated>2018-11-24T16:06:20.522Z</atom:updated>
            <content:encoded><![CDATA[<h3>Reinforcement Learning — An introduction</h3><p>Reinforcement Learning (RL) has had tremendous success in many disciplines of Machine Learning. While the results of RL almost look magical, it is surprisingly easy to get a grasp of the basic idea behind RL. In this post, we will discuss the basic principles of RL and we will implement a simple environment in which an agent tries to stay alive as long as possible by means of RL.</p><p>We will use the <a href="https://medium.com/@jannik.zuern/the-c-m-c-approach-towards-a-better-understanding-of-complicated-things-65b4a77d60ff">previously discussed</a> Concept-Math-Code (C-M-C) approach to gain drive our process of understanding RL.</p><h3>Concept</h3><h4>Introduction</h4><p>Broadly speaking, Reinforcement Learning allows an autonomous agent to learn to make intelligent choices about how it should interact with its environment in order to maximize a reward. The environment can be as simple as a single number expressing a measurement the agent takes, or it can be as complex as a screenshot of the game DOTA2, which the agent learns to play (<a href="https://openai.com/">https://openai.com/</a>).</p><p>For every discrete time step, the agent perceives the state <em>s</em> of his environment and chooses an action <em>a</em> according to its policy. The agent then receives a reward <em>r</em> for its action and the environment transitioned into the next state <em>s’.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*X7hncEu656PRZVgDPtoJow.png" /><figcaption>The feedback loop of RL (image source: <a href="https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-into-reinforcement-learning.html">[1]</a>)</figcaption></figure><p>In order to make RL work, the agent does not necessarily need to know the inner workings of the environment (i.e. it does not need a model of its environment predicting future states of the environment based on the current state). However, learning speed increases if the agent incorporates as much knowledge about the environment <em>a priori</em> as possible.</p><h4>Q-Learning</h4><p>Q-Learning was a big breakout in the early days of Reinforcement-Learning. The idea behind Q-Learning is to assign each Action-State pair a value — the Q-value — quantifying an estimate of the amount of reward we might get when we perform a certain action when the environment is in a certain state. So, if we are in a state <em>S</em>, we just pick the action that has the highest assigned Q-value as we assume that we receive the highest reward in return. Once we performed action <em>a</em>, the environment is in a new state <em>S’</em> and we can measure the reward we actually received in return for performing action <em>a</em>. Once we measured the reward for performing action <em>a</em>, we can then update the Q-values of the Action-Space pair since we now know which rewards we actually received by the environment for performing action <em>a</em>. How the Q-values are actually updated after every time step, we will discuss in the Math section this post.</p><p>You might already have noticed a wide-open gap in the Q-Learning algorithm: How the heck are we supposed to know the Q-values of a state-action pair? We might consider updating a table in which we save the Q-values of each state-action pair. Every time we take an action in the environment, we store a new Q-value for a state-action pair. These actions do at first not even have to make sense or lead to high rewards since we are only interested in building up a table of Q-values which we can use to make more intelligent decisions later on. But what if the state S in which an environment has high dimensionality or is sampled from a continuous space? We cannot expect our computer to store infinite amounts of data. What can we do? Neural Networks to the rescue!</p><h4>Deep Q-Learning</h4><p>Instead of updating a possibly infinite table of state-action pairs and their respective Q-values, let’s use a Deep Neural Network to map a state-action pair to a Q-value, hence the name Deep Q-Learning.</p><p>If the network is trained sufficiently well, it is able to tell with high confidence what Q-values certain actions might have given a state <em>S</em> in which the environment currently is. While this sounds super easy and fun, this approach suffers from instability issues and divergence. Two main mechanisms were introduced by <a href="https://storage.googleapis.com/deepmind-data/assets/papers/DeepMindNature14236Paper.pdf"><em>Mnih et al. in 2015</em></a> in order to avoid these issues: <strong>Experience Replay</strong> and <strong>Frozen Optimization Target</strong>. Briefly stated, experience replay allows the network to learn on a single experience e_<em>{t+1}</em> consisting of a state <em>s_t</em>, an action <em>a_t</em>, reward <em>r_t</em>, and new state <em>s_t</em> tuple more than one time. The term “frozen optimization target”, in contrast, refers to the fact that the Q-value estimation network used for predicting future Q-values is not the same as the network used for training. Every <em>N</em> steps, the values of the trained network are copied to the network being used to predict future Q-values. It was found that this procedure leads to much less instability issues during training.</p><h3>Math</h3><p>We have established the concepts behind Q-Learning and why Deep Q-Learning solves the issue with storing possibly infinite numbers of state-action pairs. Let us now briefly dive into some of the math involved in Deep Q Learning.</p><p>During the training of the neural network, the Q-values of each state-value pair is updated using the following equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IUO0Ry3lBiUxC7UrpMhxyQ.png" /><figcaption>Q-value update after every time step</figcaption></figure><p>Let us first discuss the term in square brackets. The variable <em>R_{t+1}</em> denotes the reward given to the agent at time step <em>t+1</em>. The next addend denotes the maximal Q-value for all possible actions <em>a</em> while the environment is in the new state <em>S_{t+1}</em>. Beware that this value can also only be an estimate of the true maximal Q-value as we can only estimate future Q-values of state-action pairs. This maximal Q-value is multiplied by a so-called discount factor (denoted gamma). This factor decides how much weight we assign to future rewards in comparison to the currently achieved reward. If the discount factor equals zero, only the current reward matters to the agent, and no future rewards matter for estimating future Q-values. The discount factor is typically set to a value of about 0.95. The last addend in the square brackets is simply the current Q-value estimate again. Thus, the term in the square brackets as a whole expresses the <strong>difference between the predicted Q-value and our best estimate for the true Q-value</strong>. Bear in mind that obviously also our best estimate for the true Q-value might not be totally perfect since we only know the reward for the next time step <em>R_{t+1}</em> for sure, but this little bit of knowledge helps us to improve the Q-value estimate for the current time step.</p><p>This whole difference-term in square brackets is multiplied by the learning rate alpha that weighs how much we trust this estimated error between the best estimate of the true Q-value and the predicted Q-value. The bigger we choose alpha, the more we trust this estimate. The learning rate is typically between 0.01 and 0.0001.</p><p>With our understanding of the Q-value update, the loss of a Q-value prediction can be described as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VuUwGjdHeDxjWLVX5U4KeA.png" /><figcaption>Loss function in DQN</figcaption></figure><p>Do not let yourself be intimidated by the term before the brackets on the right-hand-side of the equation. This part basically means that we randomly sample a <strong>state-action-reward-new_state</strong> <em>(s,a,r,s’)</em> tuple from the replay memory called <em>D</em>. The term within the square brackets defines the mean squared error between the actually observed reward r added to all expected future rewards beginning from the next time step (including the discount factor gamma) AND the actually by the neural network predicted Q-value. Pretty simple, right? During training, the prediction of the Q-values should become increasingly better (however, strong fluctuations during learning do usually happen).</p><p>These already are the most important bits of math in Deep Q-Learning. We could of course discuss some of the beauty of Linear Algebra and Calculus involved in the Neural Network estimating the Q-values. However, this is beyond the scope of this post and smarter people have done a much better job at explaining this (i.e. the truly awesome <a href="https://www.youtube.com/watch?v=Ilg3gGewQ5U">3Blue1Brown</a>).</p><h3>Problem setup</h3><p>We consider an agent trying to survive in a flappy-bird-like environment. The agent has to find the hole in a moving wall coming its way in order to survive. The goal is to survive as long as possible (sorry, dear agent, but there is no happy end for you). For every time-step the agent receives a reward of 0.1. When the agent learns to maximize its reward, it consequently learns to survive as long as possible and find the holes in the moving walls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/130/1*MeCgiu4Ui0u6FKPyQj-71w.png" /><figcaption>White agent and grey moving walls</figcaption></figure><p>The action space consists of three possible actions: <strong>[Move up, move down, stay]. </strong>The environment consists of an 80-by-80 grey-scale pixel grid. The agent is indicated in grey color, the moving walls are indicated with white color. The agent is supposed to learn optimal behavior by just looking at the raw pixels without any further knowledge about the world. This approach is both the slowest to learn but also the most general approach since no rules have to be hard-coded into the learning system (model-free Reinforcement Learning). During training, the network learns to map the information of the raw pixels of the environment to the Q-values of all three possible actions. The policy we implemented always selects the action with the highest associated Q-value.</p><h3>Code</h3><p>Without further ado, let’s have a look at the code. All relevant bits are (hopefully) commented well enough.</p><h4>Environment</h4><p>We could use one of the many pre-defined environments that come with an installation of the OpenAI gym Python package. However, we could also quickly write our own small environment. As already mentioned, I implemented a crappy Flappy Bird clone featuring a square-shaped robot trying to avoid the oncoming walls.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/27e00cf6b75d6c6ac1b91c206e573a1c/href">https://medium.com/media/27e00cf6b75d6c6ac1b91c206e573a1c/href</a></iframe><h4>Agent</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3d93e236336b70293e6a27ac55efe15c/href">https://medium.com/media/3d93e236336b70293e6a27ac55efe15c/href</a></iframe><h4>Training loop</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1bd52b0543ff61469343c9b454c99969/href">https://medium.com/media/1bd52b0543ff61469343c9b454c99969/href</a></iframe><p>As always, you may find the complete <strong>code</strong> in the <a href="https://github.com/jzuern/robot-rl">project GitHub repository</a>.</p><h3>Results</h3><p>Let us look at one episode of the agent playing without any training:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/120/1*YYiGVb53ej1XkeqvvRb3Xg.gif" /><figcaption>Random actions of untrained agent (sorry about the GIF artefacts)</figcaption></figure><p>The agent selects actions at random as it cannot yet correlate the pixel grid with appropriate Q-values of the agent’s actions.</p><p>Pretty bad performance, I would say. How well does the agent perform after playing 20000 episodes? Let’s have a look:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/120/1*I6y3X8Om-z62xNuRfVDyTg.gif" /></figure><p>We begin to see intelligent behavior. The agent steers towards the holes once it is close enough. However, even after several thousands of episodes, the agent sooner or later crashes into one of the moving walls. It might be necessary to increase the number of training episodes to 100,000.</p><h4>Averaged reward plot</h4><p>The averaged reward plot helps us to understand the training progress of the agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*EDGQN5s_d5YNbchPeHnTcQ.png" /><figcaption>Rolling average of rewards plotted over the episode number</figcaption></figure><p>A clear upward trend of the rolling average of the reward can be made out. Strong fluctuations of rewards are a typical observation in Reinforcement Learning. Other environments may lead to even stronger fluctuations, so do not let yourself be crushed if your rewards do not seem increase during training. Just wait a little longer!</p><p>That’s all I wanted to talk about for today. Please find the following exemplary resources if you want to dive deeper into the topic of Reinforcement Learning:</p><h3>Further reading</h3><p>This post was greatly inspired by the following resources:</p><ul><li>[1] <a href="https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-into-reinforcement-learning.html">Lilian Weng — A (Long) Peek into Reinforcement Learning</a></li><li>[2] <a href="https://karpathy.github.io/2016/05/31/rl/">Andrey Karpathy — Deep Reinforcement Learning: Pong from Pixels</a></li><li>[3] <a href="https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291/ref=sr_1_3?ie=UTF8&amp;qid=1542788485&amp;sr=8-3">Aurélien Géron — Hands-On Machine Learning with Scikit-Learn &amp; TensorFlow</a></li></ul><p>Thanks for reading and happy learning learning!📈💻🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3658624a5d83" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using a TPU in Google Colab]]></title>
            <link>https://jannik-zuern.medium.com/using-a-tpu-in-google-colab-54257328d7da?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/54257328d7da</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ipython]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Wed, 07 Nov 2018 12:46:06 GMT</pubDate>
            <atom:updated>2018-11-07T12:46:06.481Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*EGBvaqdUeOWpUAP5rqIhhQ.jpeg" /><figcaption>TPU unit with 4 cores</figcaption></figure><p><a href="https://medium.com/@jannik.zuern/training-a-cifar-10-classifier-in-the-cloud-using-tensorflow-and-google-colab-f3a5fbdfe24d">Last week</a>, we talked about training an image classifier on the CIFAR-10 dataset using Google Colab on a Tesla K80 GPU in the cloud. This time, we will instead carry out the classifier training on a Tensor Processing Unit (TPU).</p><blockquote>Because training and running deep learning models can be computationally demanding, we built the Tensor Processing Unit (TPU), an ASIC designed from the ground up for machine learning that powers several of our major products, including <a href="https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html">Translate</a>, <a href="https://www.blog.google/products/google-vr/google-lens-real-time-answers-questions-about-world-around-you/">Photos</a>, <a href="https://searchengineland.com/meet-rankbrain-google-search-results-234386">Search</a>, <a href="https://deepmind.com/blog/wavenet-launches-google-assistant/">Assistant</a>, and <a href="https://www.blog.google/products/gmail/subject-write-emails-faster-smart-compose-gmail/">Gmail</a>.</blockquote><p>TPU’s have been recently added to the Google Colab portfolio making it even more attractive for quick-and-dirty machine learning projects when your own local processing units are just not fast enough. While the Tesla K80 available in Google Colab delivers respectable 1.87 TFlops and has 12GB RAM, the <strong>TPUv2 </strong>available from within Google Colab comes with a whopping 180 TFlops, give or take. It also comes with 64 GB High Bandwidth Memory (HBM).</p><h3>Enabling TPU support in the notebook</h3><p>In order to try out the TPU on a concrete project, we will work with a Colab notebook, in which a Keras model is trained on classifying the CIFAR-10 dataset. It can be found <a href="https://colab.research.google.com/drive/1XvhLFD4zECIY6wLQJHDgZORVGGY2GDJa">HERE</a>.</p><p>If you would just like to execute the TPU-compatible notebook, you can find it <a href="https://colab.research.google.com/drive/1ISfhxFDntfOos7cOeT7swduSqzLEqyFn">HERE</a>. Otherwise, just follow the next simple steps to add TPU support to an existing notebook.</p><p>Enabling TPU support for the notebook is really straightforward. First, let&#39;s change the runtime settings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*URqTPp8QPv56MAtf7p5FQQ.png" /></figure><p>And choose <strong>TPU </strong>as the hardware accelerator:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/635/1*YV2dewqM086FnSS5Y2_1LQ.png" /></figure><h3>Code adjustments</h3><p>We also have to make minor adjustments to the Python code in the notebook. We add a new cell anywhere in the notebook in which we check that the TPU devices are properly recognized in the environment:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a472a601e3940fc497f066b88be2a4de/href">https://medium.com/media/a472a601e3940fc497f066b88be2a4de/href</a></iframe><p>This should output a list of 8 TPU devices available in our Colab environment. In order to run the tf.keras model on a TPU, we have to convert it to a TPU-model using the<em> tf.contrib.tpu.keras_to_tpu</em> module. Luckily, the module takes care of everything for us, leaving us with a couple of lines of boilerplate code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f2bdc560343041b43fb6fd5d30918086/href">https://medium.com/media/f2bdc560343041b43fb6fd5d30918086/href</a></iframe><p>In case your model is defined using the recently presented TensorFlow <strong>Estimator API</strong>, you only have to make some minor adjustments to your Estimator’s <em>model_fn</em> like so:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/86693e5dfd140fffe368e66b6345b98f/href">https://medium.com/media/86693e5dfd140fffe368e66b6345b98f/href</a></iframe><p>You can find an example of a TPUEstimator in the <a href="https://github.com/tensorflow/tpu/blob/master/models/experimental/cifar_keras/cifar_keras.py">TensorFlow GitHub repository</a>.</p><p>You should also consider <strong>increasing the batch size</strong> for training and validation of your model. Since we have 8 TPU units available, a batch size of 8*128 might be reasonable — depending on your model’s size. Generally speaking, a batch size of 8*8^n is advised. Due to the increased batch size, you can experiment with <strong>increasing the learning rate</strong> as well, making training even faster.</p><h3>Performance gains</h3><p>Compiling the TPU model and initializing the optimizer shards takes time. Depending on the Colab environment workload, it might take a couple of minutes until the first epoch and all the necessary previous initializations have been completed. However, once the TPU model is up and running, it is <strong>lightning fast</strong>.</p><h4>Using the Resnet model discussed in the <a href="https://medium.com/me/stats/post/f3a5fbdfe24d">previous post</a>, one epoch takes approximately 25 secs compared to the approx. 7 minutes on the Tesla K80 GPU, resulting in a speedup of almost 17.</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54257328d7da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training a CIFAR-10 classifier in the cloud using TensorFlow and Google Colab]]></title>
            <link>https://jannik-zuern.medium.com/training-a-cifar-10-classifier-in-the-cloud-using-tensorflow-and-google-colab-f3a5fbdfe24d?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3a5fbdfe24d</guid>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Tue, 06 Nov 2018 11:14:38 GMT</pubDate>
            <atom:updated>2018-11-06T11:14:38.115Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*SZnidBt7CQ4Xqcag6rd8Ew.png" /></figure><p><a href="https://colab.research.google.com/">Google Colab</a> is a <em>free </em>cloud-based Jupyter notebook environment running completely on Google cloud infrastructure. You can create and execute any Jupyter notebook you want in the browser. Google generously assigns each user a free Tesla K80 with 12GB memory for 12 hours at a time for their small-scale private machine learning needs.</p><p>In this article, we will write a <strong>Jupyter</strong> notebook in order to create a simple object classifier for classifying images from the <strong>CIFAR-10</strong> dataset. The classifier uses the <strong>TensorFlow Keras</strong> API which is an easy-to-use abstraction layer of the TensorFlow API that greatly simplifies machine learning programming while preserving the performance of bare-bones TensorFlow.</p><h3>Prerequisites</h3><p>The only thing you need is a Google account if you want to use Google Colab.</p><h3>Data set</h3><p>We want to train a classifier on the infamous <a href="https://www.cs.toronto.edu/~kriz/cifar.html">CIFAR-10 data set</a>. It consists of 60,000 images of everyday objects and their corresponding classes, namely:</p><p><strong>airplane</strong>, <strong>automobile</strong>, <strong>bird</strong>, <strong>cat</strong>, <strong>deer</strong>, <strong>dog</strong>, <strong>frog</strong>, <strong>horse</strong>, <strong>ship</strong>, and <strong>truck</strong>.</p><p>The images have a resolution of 32 x 32 pixels with 3 color channels (RGB). The data set is split into a training dataset (50k samples), a validation data set (5k samples), and a testing data set (5k samples).</p><h3>Network architecture</h3><p>Since we are being lazy, we use the handy <em>tensorflow.keras</em> API providing us with useful abstractions for specifying a suitable network architecture. We use a variant of the Resnet architecture, which achieved groundbreaking accuracies for image recognition due to a novel approach called <strong>shortcut connections</strong> allowing the network to be deeper with a reduced risk of exploding or vanishing gradients, which typically is a nuisance when training very deep neural networks.</p><p>The network architecture can be found below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcb8389c1d623bf89b2e1cda7ef5d5dd/href">https://medium.com/media/fcb8389c1d623bf89b2e1cda7ef5d5dd/href</a></iframe><h3>The Notebook</h3><p>The Google Colab notebook containing all steps can be found <a href="https://colab.research.google.com/drive/1XvhLFD4zECIY6wLQJHDgZORVGGY2GDJa">HERE</a>.</p><p>In order to run the Notebook on a Google Colab GPU, you should go to <strong>Runtime-&gt;Select Runtime</strong> and choose <strong>Python 3 </strong>as well as ‘<strong>GPU</strong>’ from the Dropdown menu.</p><p>Now you can execute the Notebook by pressing <em>Cmd+F9</em>.</p><h3>Results</h3><p>After training the network for 10 epochs, we arrive at a validation loss of about 1.4956 and a validation accuracy of about 72.86%.</p><p>Below, you can see visualizations of the training loss and training accuracy over the number of training steps (heavy smoothing is applied):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/318/1*MZBuQgJG-USXs_tePEtesg.png" /><figcaption>Training loss</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/325/1*5euvnZpM-cRsBKLL1gfNWQ.png" /><figcaption>Training accuracy</figcaption></figure><p>Using the trained classifier for predicting the classes of the test set images, we arrive at the following <strong>confusion matrix</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*Sy3YX5hlGpyoyCqRmAk7RA.png" /><figcaption>Confusion matrix of CIFAR-10 classifier</figcaption></figure><p>A majority of the labels are being predicted correctly. The more interesting part about the confusion matrix, however, are the mistakes the classifier makes. It occasionally seems to confuse airplanes with birds or automobile with trucks or dogs with cats. These three pairs of classes have many similarities. Airplanes and birds have wings and a elongated body in between. Trucks and automobiles have undoubtedly very similar features making it hard for the classifier to distinguish between the two classes. The same applies to dogs and cats.</p><p>On the <strong>NVIDIA Tesla K80</strong>, the training of the classifier should take about 7 minutes per epoch. With the hyper-parameters selected in the notebook, we already get some pretty good results. However, we are not even close to the state-of-the-art in image recognition which is currently (as of Nov 2018) at a whopping 96.53% Top-1 accuracy.</p><p>Feel free to play around with the hyper-parameters and with the notebook itself in order to find the best set of hyper-parameters. You can fork the notebook on <a href="https://github.com/jzuern/cifar-classifier">https://github.com/jzuern/cifar-classifier</a>.</p><p>Thanks for reading and happy classifying! 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3a5fbdfe24d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The C-M-C approach: Towards a better understanding of complicated things]]></title>
            <link>https://jannik-zuern.medium.com/the-c-m-c-approach-towards-a-better-understanding-of-complicated-things-65b4a77d60ff?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/65b4a77d60ff</guid>
            <category><![CDATA[numerical-analysis]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[problem-solving]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Thu, 25 Oct 2018 22:51:14 GMT</pubDate>
            <atom:updated>2018-10-25T22:51:14.186Z</atom:updated>
            <content:encoded><![CDATA[<p>Lately, I came to the realization that the process of thoroughly understanding a scientific theory or fact may be divided into three tiers:</p><ul><li><strong>Concept (C)</strong></li><li><strong>Math (M)</strong></li><li><strong>Code (C)</strong></li></ul><p>One cannot master a tier until the previous tier has been mastered thoroughly. Let me walk you through these three tiers:</p><h3>Concept</h3><p>A concept or idea of the approach is needed in order wrap your head around the thing you are trying to understand. This concept may be a very vague yet profound statement boiling down some essential scientific findings to a single sentence or a couple of sentences at most. Some scientists even argue that the shorter a scientific theory can be summarized, the more fundamental its nature can be understood. The same goes for good explanations of concepts: The simpler the better. Or, to put it like Albert Einstein:</p><blockquote>If you can’t explain it simply, you don’t understand it well enough.</blockquote><p>Take for instance the problem of heat conduction in materials. A very abstract concept of the conduction of heat may be the following statement: “<strong>Heat flows from hot areas to cold areas</strong>”. Already this simple statement allows us to make meaningful predictions about the future state of a system. If you touch a hot stove, you know that your hand will sooner or later become hot. You cannot make precise predictions about the concrete time span in which your hand will get or the concrete temperature your hand will have but you have a rough idea. Are more advanced concept to grasp may be the wave-particle-duality describing the alleged discrepancy between describing a particle as a wave and as a particle at the same time. Quantum mechanics allowed us to develop a new framework or concept in which a particle as both a wave and a particle at the same time. Having a good grasp of the concept of a theory is the first step in understanding this theory.</p><h3>Math</h3><p>Mathematical equations describe the concept in the most abstract possible way. While the concept draws from your intuition and allows for a qualitative prediction of the future state of a system, mathematical equations allow us to precisely express quantitative properties of the system. In the context of heat conduction in materials, the 3-D heat-diffusion-equation allows us to make precise statements about the temperature u of a material as a function of space and time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*NUd01CMmySeWZ_y_MSxNoA.png" /><figcaption>3-D heat equation</figcaption></figure><p>The constant alpha denotes the <strong>thermal diffusivity</strong> of the material, which is defined as the thermal conductivity k over the product of the material density rho and the specific heat capacity <strong>c_p</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/218/1*dzz1Z-_VeWJapK5hKOMe8Q.png" /><figcaption>Definition of the thermal diffusivity</figcaption></figure><p>Thinking about the math behind a concept allows you to deeply dive into the problem. It allows you to set the different parameters of the system into a relationship with another. In this example, you do not need to immediately see a possible solution to this differential equation. You might, however, understand that the spatial and temporal derivatives of the field quantity temperature are in a special relationship with one another, which is true at every point in space and at every moment in time.</p><p>Of course, other problems might have much more complex equations and might require a much higher mental workload in order for you to hold all the complex ideas in your head. Take, again, our wave-particle duality concept: The mathematical equation describing a quantum particle is a partial differential equation as well: The Schrödinger equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*-vCi-Y1s3uQ9RyZoYCYrgw.png" /><figcaption>Schrödinger equation in all its mathematical beauty</figcaption></figure><p>This equation describes the behavior of the wave function Psi of a quantum particle. While the Schrödinger equation is arguably more complex and difficult to grasp, it has the same fundamental properties as the heat equation: The time derivative of the function we try to obtain is in direct relationship to its second spatial derivative.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*NdajkQbWYbTcHlsrzR4TPg.gif" /><figcaption>Wave function satisfying the Schrödinger equation (Source: <a href="https://en.wikipedia.org/wiki/Schr%C3%B6dinger_equation#/media/File:Wave_packet_(dispersion).gif">Wikipedia</a>)</figcaption></figure><p><strong>Pro-tip</strong>: Do not let yourself be intimidated by mathematical notation. It might sometimes look overwhelming but notation usually is only needed to write down an obvious statement in a water-proof fashion.</p><h3>Code</h3><p>I lied to you in the last paragraph. You do not understand the math until you implemented it in code. Ok, this might be an overstatement but I heavily encourage you to implement your math in code, otherwise you might fall for the illusion of understanding your problem without actually and thoroughly understanding it.</p><p>Writing code is useful. First of all, it helps you to play around with your problem. Mathematical equations are great for expressing complex relationships between objects. However, seeing those relationships in action in your computer adds a whole new level to your understanding of the problem. It allows you to “experience” the mathematics behind the problem. It lets you play with parameters and see how the system state changes due to these differently chosen parameters.</p><p>And secondly, writing your problem in terms of code, lets you view your problem from different angles. It makes you think about the implications of possible solutions. Writing code also allows you to easily visualize your problem and the theory behind it. If you have a concise visualization of your problem, you can easily make a mental model of the system and its behavior. I deeply believe that anyone who understands a scientific theory and says that he does not have some kind of visualization of the problem in his mind does not really tell the truth.</p><p>Let us get back to our heat equation and implement a simple program to get a solution to the equation:</p><p>To make visualization easier, let us focus on one-dimensional heat conduction problems. This boils the differential equation down to the following form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/1*oO1dULgBRwhOcGAkcw_Kvg.png" /><figcaption>1-D heat diffusion equation</figcaption></figure><p>First of all, we need to specify a discrete representation of the continuous differential equation of heat conduction stated previously:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*GkaKLA8VxX2J4h4LbWKmmw.png" /><figcaption>Numerical approximation scheme to the 1-D heat diffusion equation</figcaption></figure><p>The originally continuous temperature u is discretized into discrete points in space and time. The subscripts i and the superscripts n denote the i-th spatial coordinate and n-th time step, respectively. The discrete approximation of the spatial derivative was obtained using the <a href="https://en.wikipedia.org/wiki/Finite_difference#Forward,_backward,_and_central_differences">central differences approach</a> while the discrete approximation of the temporal derivative was obtained using the <a href="https://en.wikipedia.org/wiki/Euler_method">Euler-method</a>.</p><p>In order to find a solution, we solve the above equation for u_i^(n+1) and iterate through all time steps in an outer loop and through all spatial points in an inner loop. We also have to apply initial conditions for u(x, t=0) and boundary conditions for u(x=0,t) and u(x=L,t), while x=0 and x=L denote the left and right ends of the 1-D simulation domain.</p><p>The code get a solution to this equation follows and is fairly straightforward and should be self explaining:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbe26730363b0aa6954c105b0f806993/href">https://medium.com/media/cbe26730363b0aa6954c105b0f806993/href</a></iframe><p>The visualization of the solution to this simple differential equation should add a deeper understanding of the process of heat conduction through materials. The following three GIFS show the change in the material temperature over time. Boundary conditions of u(x=0,t) = 1.0 and u(x=1.0,t) = 0.0 are applied for all three setups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*JFQK__JUgVAW1vQ2kfyUpw.gif" /><figcaption>Slow heat conduction</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*zHs6XQ3LDWjLaudJQIoCvA.gif" /><figcaption>Fast heat conduction</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*MBGd33sluRQeoq-8NqwVCg.gif" /><figcaption>Fast heat conduction with different initial conditions</figcaption></figure><p>Playing around with the heat conduction parameters should make you develop a new intuition regarding the dynamical behavior of the system.</p><p><strong>As a side-note</strong>: You might also encounter some numerical stability issues when selecting certain time step sizes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*hcn8YDeX8eYSemdNVI1kNg.gif" /><figcaption>Numerical divergence of solution due to time step size</figcaption></figure><p>This problem is known as the <a href="https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition">Courant–Friedrichs–Lewy condition</a>, which states that there is an upper limit to any time step size Delta t in finite differences schemes, which depends on the speed of information flow though the simulation domain and the length interval Delta x .</p><p>Using the C-M-C approach, you should have developed an intuitive understanding of the whole problem. Going from an abstract <strong>concept</strong> you were able to understand the <strong>math</strong> behind the problem and were thus able to implement the equations in program <strong>code</strong>. Tinkering with parameters and visualizing your results allowed you to more deeply understand the problem and even find some numerical issues with the finite differences solution scheme.</p><p>Congrats, now you are a real problem-solver!</p><p>Let me know in the comments whether you think that the C-M-C approach can be viewed as a easy-to-follow guide to help you understand difficult problems and if you what approaches to improve your understanding of a topic you use in your daily scientific struggles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=65b4a77d60ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Robot localization with Kalman-Filters and landmarks]]></title>
            <link>https://jannik-zuern.medium.com/robot-localization-with-kalman-filters-and-landmarks-cf97fa44e80b?source=rss-5db0eff2a830------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf97fa44e80b</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[kalman-filter]]></category>
            <dc:creator><![CDATA[Jannik Zürn]]></dc:creator>
            <pubDate>Mon, 17 Sep 2018 08:44:50 GMT</pubDate>
            <atom:updated>2020-11-06T03:56:37.014Z</atom:updated>
            <content:encoded><![CDATA[<p>Meet ‘Robby’. Robby is a robot. Well technically he is a simplistic virtual model of a robot, but that should be fine for our purpose. Robby is lost in his virtual world consisting of a two-dimensional plane and a couple of landmarks. He does have a map of his environment (no mapping is needed), but he does not know where exactly he is located in this environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*NlHDSDDFGtmPwAruWObtOQ.png" /><figcaption>Robby (big red circle) and two landmarks (smaller red circles)</figcaption></figure><p>The purpose of this post is to walk you through the steps of robot localization using landmark detection and Extended Kalman Filtering.</p><h3>Part 1 — Linear Kalman Filters</h3><p>Kalman Filtering can be understood as a way of making sense of a noisy world. When we want to determine where a robot is located, we can rely on two things: We know how the robot moves from time to time since we command it to move in a certain way. This is called state transitioning (i.e. how the robot moves from one state to the other). And we can measure the robot’s environment using its various sensors such as cameras, lidar, or echolot. The problem is that both sets of information are subject of random noise. We do not know exactly how exactly the robot transitions from state to state since actuators are not perfect and we cannot measure the distance to objects with infinite precision. This is where Kalman Filtering comes to play.</p><p>Kalman Filtering allows us to combine the uncertainties regarding the current state of the robot (i.e. where it is located and in which direction it is looking) and the uncertainties regarding its sensor measurements and to ideally decrease the overall uncertainty of the robot. Both uncertainties are usually described by a Gaussian probability distribution or Normal distribution. A Gaussian distribution has two parameters: mean and variance. The mean expresses, what value of the distribution has the highest probability to be true, and the variance expresses how uncertain we are regarding this mean value.</p><p>The algorithm works in a two-step process. In the prediction step, the Kalman filter produces estimates of the current state variables, along with their uncertainties. Once the outcome of the next measurement (necessarily corrupted with some amount of error, including random noise) is observed, these estimates are updated using a weighted average, with more weight being given to estimates with higher certainty. The algorithm is recursive. It can run in real time using only the present input measurements and the previously calculated state and its uncertainty matrix; no additional past information is required.</p><p>Since the Wikipedia image for the information flow in a Kalman Filter is so great, I cannot withhold it here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wR_ID0Ai1fJ_8qzz3BrP4Q.png" /><figcaption>Kalman Filtering. Image grabbed from: <a href="https://upload.wikimedia.org/wikipedia/commons/a/a5/Basic_concept_of_Kalman_filtering.svg">https://upload.wikimedia.org/wikipedia/commons/a/a5/Basic_concept_of_Kalman_filtering.svg</a></figcaption></figure><p>I will not delve into the mathematical details of Kalman Filtering since many smart people already have done so. For a more in-depth explanation, I can recommend a blog post by Tim Babb:</p><p><a href="https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/">How a Kalman filter works, in pictures</a></p><h3>Part 2 — Extended Kalman Filters</h3><p>Extended Kalman Filtering is (as the name suggests) an extension of “Normal” Kalman Filtering. What I did not tell you in the last section is one additional assumption that was made implicitly wenn using Kalman Filters: The state transition model and the measurement model must be linear. From a mathematical standpoint, this means that we can use the simplicity and elegance of Linear Algebra to update the robot’s state and the robot’s measurements. In practice, this means that the state variables and measured values are assumed to change linearly over time. For instance, if we measure the robot’s position in the x-direction. We assume that if the robot was at position x1 at time t1, it must be at position x1+v*(t2–t1) at time t2. The variable v denotes the robot’s velocity in the x-direction. If the robot is actually accelerating or doing any other kind of nonlinear motion (e.g driving around in a circle), the state transition model is slighly wrong. Under most circumstances, it is not wrong by much, but in certain edge cases, the assumption of linearity is simply too wrong.</p><p>Also assuming a linear measurement model comes with problems. Assume you are driving along a straight road and there is a lighthouse right next to the road in front of you. While you are quite some distance away, your measurement of your distance to the lighthouse and the angle in which it lies from your perspective changes pretty much linearly (the distance decreases by roughly the speed your car has and the angle stays more or less the same). But the closer you get and especially while you drive past it, the angle, on one hand, changes dramatically, and the distance, on the other hand, does not change very much. This is why we cannot use Linear Kalman Filtering for Robby when he is navigating his 2-D world with landmarks scattered across his 2-D plane.</p><p>Extended Kalman Filter to the rescue! It removes the restriction of linear state transition and measurement models. Instead, it allows you to use any kind of nonlinear function to model the state transition and the measurements you are making with your robot. In order to still be able to use the efficient and simple Linear Algebra magic in our filter, we do a trick: We linearize the models around the current robot state. This means that we assume the measurement model and the state transition model to be approximately linear around the state at which we are right now (refer to the road / lighthouse example again). But after every time step, we update this linearization around the new state estimate. While this approach forces us to make a linearization of this nonlinear function after every time step, it turns out to be not computationally expensive.</p><p>So there you have it. Extended Kalman Filtering is basically “Normal” Kalman Filtering just with additional linearization of the now nonlinear state transition model and measurement model.</p><p>In our case where Robby is lost and wants to localize in this (arguably) hostile environment, the Extended Kalman Filtering enables Robby to sense the landmarks and update its belief of its state accordingly. If the variance of the state estimate and the measurement estimate is low enough, Robby is very quickly very sure where he is located in respect to the landmarks and since he knows exactly where the landmarks are, he knows where he is!</p><p>His happiness-parameter is skyrocketing!</p><h3>Part 3 — Implementation</h3><p>The implementation in code is fairly straightforward. For visualization purposes, I chose the SDL2 Library for a quick-and-dirty visualization of all necessary objects. It can be downloaded <a href="https://www.libsdl.org/download-2.0.php">here</a>:</p><p>Following an object-oriented programming approach, I implemented the following classes:</p><ul><li>Class <strong>Robot</strong></li></ul><p>The Robot Class’ most important members are the Pose (x position, y position, and direction), and the Velocity (linear and angular velocity) . It can move forward, backward, and rotate left and right. For measuring the landmark positions, it has the method measure Landmarks, which takes the ground-truth landmarks, and overlays their position with fake measurement noise, and returns a new list of measured landmarks.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dba3b96ccc7c7abc4f3ccc4afcdb10de/href">https://medium.com/media/dba3b96ccc7c7abc4f3ccc4afcdb10de/href</a></iframe><ul><li>Class <strong>KalmanFilter</strong></li></ul><p>The KalmanFilter class is arguably the most complex one. Its members are the matrices for state transitioning, measurements, and their respecive covariances. I will gloss over most of the details here, as the code comments give some hints about the purpose of most of the code. The filtering magic is happening in the localization_landmarks() member function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6cd1d3470376b9e52c85070047830ff4/href">https://medium.com/media/6cd1d3470376b9e52c85070047830ff4/href</a></iframe><ul><li>Class <strong>Landmark</strong></li></ul><p>The Landmark class is the most simple of them all. It has a position, an ID (a unique color), and a method for rendering itself to the screen. That’s it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1c89821f181f70beafc9dee186b2705/href">https://medium.com/media/a1c89821f181f70beafc9dee186b2705/href</a></iframe><p>In the main function, all we do is to initialize everything and to run an infinite loop, in which the Robot position is updated according to input from the keyboard, the robot measures its environment and the KalmanFilter does its predict and update step.</p><p>The full code can be found (as always) on my gitHub: <a href="https://github.com/jzuern/robot-localization">https://github.com/jzuern/robot-localization</a></p><p>Happy Filtering! 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf97fa44e80b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>