<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohan Karthik on Medium]]></title>
        <description><![CDATA[Stories by Mohan Karthik on Medium]]></description>
        <link>https://medium.com/@mohankarthik?source=rss-355915c47610------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Mohan Karthik on Medium</title>
            <link>https://medium.com/@mohankarthik?source=rss-355915c47610------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 04:02:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mohankarthik/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Setting up GitHub Actions to Deploy JAVA code on Elastic Beanstalk]]></title>
            <link>https://medium.com/@mohankarthik/setting-up-github-actions-to-deploy-java-code-on-elastic-beanstalk-d8837db7641a?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8837db7641a</guid>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Wed, 23 Mar 2022 04:30:00 GMT</pubDate>
            <atom:updated>2022-03-23T11:02:20.459Z</atom:updated>
            <content:encoded><![CDATA[<p>See the <a href="https://mohankarthik.dev/engineering/2022/03/21/setting-up-github-actions-to-deploy-angular.html">previous post</a> for details on how to understand the basics of GitHub actions and to set it up</p><p>The below is the workflow yml that will build and deploy a maven war file to Elastic Beanstalk. If you’ve questions on how to use this, and why certain things are being used, fire away in the comments.</p><pre>name: &lt;name&gt;<br>on:<br>  push:<br>    branches:<br>      - &lt;branch&gt;<br>  workflow_dispatch:<br>concurrency: &lt;name&gt;<br>jobs:<br>  &lt;name&gt;:<br>    name: &lt;name&gt;<br>    timeout-minutes: &lt;time to wait before timeout&gt;<br>    runs-on: ubuntu-latest<br>    env:<br>      CORRETTO_URL: https://corretto.aws/downloads/latest/amazon-corretto-8-x64-linux-jdk.tar.gz<br>    steps:<br>      - name: Checkout the code<br>        uses: actions/checkout@v2<br>        with:<br>          fetch-depth: 0<br>      - name: Cache jdk binary<br>        uses: actions/cache@v2<br>        with:<br>          path: ${{ runner.temp }}/jdk<br>          key: ${{ runner.os }}-jdk-8<br>          restore-keys: ${{ runner.os }}-jdk<br>      - name: retrieve latest corretto 8 jdk<br>        run: |<br>          test -d ${{ runner.temp }}/jdk || mkdir ${{ runner.temp }}/jdk<br>          test -f ${{ runner.temp }}/jdk/corretto.tar.gz || ( wget -qP ${{ runner.temp }}/jdk $ {{ env.CORRETTO_URL }} )<br>      - name: Set up JDK 8 from file<br>        uses: actions/setup-java@v2<br>        with:<br>          distribution: &#39;jdkfile&#39;<br>          jdkFile: ${{ runner.temp }}/jdk/corretto.tar.gz<br>          java-version: &#39;8&#39;<br>          architecture: x64<br>          cache: &quot;maven&quot;<br>      - name: Enable caching<br>        uses: actions/cache@v2<br>        with:<br>          path: ~/.m2/repository<br>          key: ${{ runner.os }}-m2-${{ hashFiles(&#39;**/pom.xml&#39;) }}<br>          restore-keys: |<br>            ${{ runner.os }}-m2<br>      - name: Create the installer after running the tests<br>        run: mvn clean install<br>      - name: Get the current time<br>        id: time<br>        uses: nanzm/get-time-action@v1.1<br>        with:<br>          timeZone: 8<br>          format: &#39;YYYY-MM-DD-HH-mm-ss&#39;<br>      - name: Configure AWS credentials<br>        uses: aws-actions/configure-aws-credentials@v1<br>        with:<br>          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>          aws-region: us-east-1<br>      - name: Upload the war file to AWS<br>        env:<br>          TIME: &quot;${{ steps.time.outputs.time }}&quot;<br>        run: |<br>          cp ./&lt;project&gt;/target/&lt;filename&gt;.war ./&lt;project&gt;/target/$TIME.war<br>          aws s3 cp ./&lt;project&gt;/target/$TIME.war s3://&lt;s3 bucket name&gt;/<br>          aws elasticbeanstalk create-application-version --application-name &lt;app name&gt; --version-label $TIME --source-bundle S3Bucket=&quot;&lt;s3 bucket name&gt;&quot;,S3Key=&quot;$TIME.war&quot; --auto-create-application<br>          aws elasticbeanstalk update-environment --environment-name &lt;env name&gt; --version-label $TIME</pre><p><em>Originally published at </em><a href="https://mohankarthik.dev/engineering/2022/03/23/setting-up-github-actions-to-deploy-java.html"><em>https://mohankarthik.dev</em></a><em> on March 23, 2022.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8837db7641a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up GitHub Actions to Deploy Front-End Code on AWS S3 and Cloudfront]]></title>
            <link>https://medium.com/@mohankarthik/setting-up-github-actions-to-deploy-front-end-code-on-aws-s3-and-cloudfront-3cadb624921b?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/3cadb624921b</guid>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Mon, 21 Mar 2022 04:30:00 GMT</pubDate>
            <atom:updated>2022-03-23T10:44:22.655Z</atom:updated>
            <content:encoded><![CDATA[<p>AWS S3 and Cloudfront are fantastic infrastructure to deploy your production website (either static, say a simple HTML/CSS, or generated via a static site generator via Gatsby, Hugo, Jekyll, etc…); or a dynamic front-end (say using React, Angular, etc…). This post will cover on how to deploy an angular / react / vue code to S3 and Cloudfront.</p><p>This post assumes that you already have a working JS project (either Angular / React / Vue / etc…), and that you’ve <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html">setup AWS S3 bucket with a website endpoint</a></p><p>The post also assumes that you’ve your code hosted on Github.</p><h3>CI/CD alternatives</h3><p>There are many CI/CD platforms today, including Jenkins, GitHub Actions, Circle CI, Atlassian Bamboo, etc… Given that most of us end up using Github to host the code, and that GitHub provides free minutes to run the pipelines, it’s just a very convenient option without any significant downsides.</p><h3>Github actions can be triggered by many actions</h3><ul><li>Manually</li><li>On a periodic interval via a cronjob</li><li>On pushing a new commit to a branch</li><li>On creating / updating / deleting a pull request</li></ul><p>Depending on the configuration, GitHub would simply create a new container (using very similar concepts to Docker) and do the job that you specify. The job could be anything. You could create a GitHub action to print “Hello World!”. Which is a nice way to test out things and get yourself comfortable with the idea.</p><h3>Structure of the GitHub workflow file</h3><p>A GitHub workflow file is basically a YAML file and follows the <a href="https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html">YAML syntax</a>. This <a href="https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions">page</a> provides an excellent reference for how the GitHub syntax is structured. If you are interested to learn the basics, please go through this before proceeding.</p><h3>GitHub action to deploy Javascript code</h3><p>To create a new GitHub workflow, you create a .github folder in the root of your project and then a workflows folder. Within this folder, you can add a &lt;name&gt;.yml to define your workflow / pipeline. Below is the code that you can directly use, and then we&#39;ll break it down.</p><p>Note the below config uses yarn but you can replace it with npm if that&#39;s your package manager.</p><pre>name: &lt;name&gt;<br>on:<br>  push:<br>    branches: <br>      - master<br>concurrency: &lt;name&gt;<br>jobs:<br>  &lt;name&gt;:<br>    name: &lt;name&gt;<br>    timeout-minutes: &lt;time in minutes after which the job should timeout&gt;<br>    runs-on: ubuntu-latest<br>    env:<br>      NODE_VERSION: &#39;&lt;Node version number&gt;&#39;<br>      S3_BUCKET: &lt;s3 bucket ID&gt;<br>      CDN_DISTRIBUTION_ID: &lt;cloudfront ID&gt;<br>      AWS_REGION: &lt;aws region&gt;<br>    steps:<br>      - name: Checkout the code<br>        uses: actions/checkout@v2<br>        with:<br>          fetch-depth: 0<br>      - name: Install node<br>        uses: actions/setup-node@v2<br>        with:<br>          node-version: ${{ env.NODE_VERSION }}<br>      - name: Get yarn cache directory path<br>        id: yarn-cache-dir-path<br>        run: echo &quot;::set-output name=dir::$(yarn cache dir)&quot;<br>      - name: Cache the dependencies<br>        uses: actions/cache@v2<br>        id: yarn-cache<br>        with:<br>          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}<br>          key: ${{ runner.os }}-yarn-${{ hashFiles(&#39;**/yarn.lock&#39;) }}<br>          restore-keys: |<br>            ${{ runner.os }}-yarn-<br>      - name: yarn install<br>        run: yarn --prefer-offline<br>      - name: build<br>        run: &lt;your yarn command to build your site&gt;<br>      - name: Configure AWS credentials<br>        uses: aws-actions/configure-aws-credentials@v1<br>        with:<br>          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>          aws-region: {{ env.AWS_REGION }}<br>      - name: deploy<br>        run: |<br>          aws s3 rm s3://${{ env.S3_BUCKET }}/ --recursive<br>          aws s3 cp ./target/www s3://${{ env.S3_BUCKET }} --cache-control &#39;public, max-age=31104000&#39; --recursive<br>          aws configure set preview.cloudfront true<br>          aws cloudfront create-invalidation --distribution-id ${{ env.CDN_DISTRIBUTION_ID }} --paths &quot;/*&quot;</pre><p>Let’s break the above down</p><h3>Triggers</h3><p>This <a href="https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows">page</a> contains detailed information. In summary, the most conventional triggers used are as follows:</p><pre>on:<br>  push:<br>    branches: <br>      - master # Matches any push on the master branch<br>      - *asd* # Matches any branch that has asd in the name like basd, basdf, etc...<br>  pull_request:<br>    types:<br>      - opened # When a new PR is opened<br>      - reopened # When a PR is re-opened<br>      - synchronize # When a PR is updated</pre><p>The above are your conventional triggers. So if you want to build and deploy anytime you push / merge to a branch (say the main / master). This is what we do in our original example. If you want to build each time a pull request is updated, so you get to test the bleeding edge work, then you can trigger it on pull_request -&gt; types -&gt; opened, synchronized.</p><h3>Choosing the Runner</h3><p>In the above example, we use ubuntu-latest. This could be the default choice for most people, but look at <a href="https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners">other options</a>.</p><h3>Naming the Job</h3><p>A job has a name</p><pre>jobs:<br>  &lt;name&gt;:<br>    name: &lt;name&gt;</pre><p>Pretty straightforward. Everyone has a name (except if you are <a href="https://www.youtube.com/watch?v=TgXlQlOrSx8">Arya Stark</a>)! It’s just cruel to be nameless!</p><h3>Defining environment variables</h3><p>It’s a good idea to always defining common variables in once place, so you can easily use them in multiple places. Same as <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a></p><pre>env:<br>      NODE_VERSION: &#39;&lt;Node version number&gt;&#39;<br>      S3_BUCKET: &lt;s3 bucket ID&gt;<br>      CDN_DISTRIBUTION_ID: &lt;cloudfront ID&gt;<br>      AWS_REGION: &lt;aws region&gt;</pre><p>In this case, let’s also define what these variables are.</p><ul><li>NODE_VERSION: This is straight forward. Simply put the node version that you’ve locally used to build the project. Also ensure to keep this aligned with your local version, so you don’t get surprises when you deploy</li><li>S3_BUCKET: This is the name of the S3 bucket. Typically you want a name that is unique. In my case it would be dev.mohankarthik.xxx and since I own mohankarthik.dev, the naming convention becomes unique for me.</li><li>CDN_DISTRIBUTION_ID: This is the ID of the cloudfront entry. You can skip cloudfront, but it’s pretty spectacular and I’d suggest enabling it even for hobby projects.</li><li>AWS_REGION: The Region where your AWS account is hosted. us-east-1, etc…</li></ul><h3>Steps</h3><p>This is where the money is. Now let’s dive in.</p><p>Each step has an name and either uses or run. uses essentially uses an existing module directly with no work from us. How cool is that. We get small open-source tasks to run and get us what we want. You can see the list of all existing actions <a href="https://github.com/marketplace?type=actions">here</a>.</p><h3>Checkout the code</h3><pre>- name: Checkout the code<br>        uses: actions/checkout@v2<br>        with:<br>          fetch-depth: 0</pre><p>This piece of code basically checks your code out into the machine that’s building the code. fetch-depth being set to 0 basically means that it will do a shallow pull, only the current commit and not the entire history.</p><h3>Installing node</h3><pre>- name: Install node<br>        uses: actions/setup-node@v2<br>        with:<br>          node-version: ${{ env.NODE_VERSION }}</pre><p>This installs the node version that you’ve specified in the env. For other frameworks / languages, change to the approrpriate setup. There are many <a href="https://github.com/marketplace?type=actions&amp;query=setup+">existing actions</a> to help setup other languages.</p><h3>Caching</h3><p>This is a really cool feature of GitHub (and many other CI/CD tools too). GitHub Actions will save all the dependencies (the infamous node_modules folder) into memory, and then the next time you run your workflow, it&#39;ll restore the local version before trying to get it from the internet. Which will speed up the whole build significantly.</p><pre>- name: Get yarn cache directory path<br>        id: yarn-cache-dir-path<br>        run: echo &quot;::set-output name=dir::$(yarn cache dir)&quot;</pre><p>The above part gets the directory where the cache will get stored and sets it as the yarn cache directory</p><pre>- name: Cache the dependencies<br>        uses: actions/cache@v2<br>        id: yarn-cache<br>        with:<br>          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}<br>          key: ${{ runner.os }}-yarn-${{ hashFiles(&#39;**/yarn.lock&#39;) }}<br>          restore-keys: |<br>            ${{ runner.os }}-yarn-</pre><p>This is the part that actually caches all the dependencies after the first install. You can update this to your language of choice by changing the cache directory to where your framework / language will store the dependencies. .pub-cache for flutter, .m2 for maven, etc...</p><h3>Install dependencies</h3><p>This part is simple</p><pre>- name: yarn install<br>        run: yarn --prefer-offline</pre><p>Call a console command to install all dependencies by calling yarn or npm install. You can do similar for other languages like pip install -r requirements.txt, etc...</p><p>This part will also automatically fetch from the cache specified ahead instead of fetching from internet if the cache already has data.</p><h3>Build</h3><pre>- name: build<br>        run: &lt;your yarn command to build your site&gt;</pre><p>Simple as it is. Call the command line that you want to build the project. And can be customized for the language / framework of choice</p><h3>Deploying to AWS</h3><pre>- name: Configure AWS credentials<br>        uses: aws-actions/configure-aws-credentials@v1<br>        with:<br>          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>          aws-region: {{ env.AWS_REGION }}</pre><p>Before we get to this section, we’ll need to setup an IAM account on AWS dedicated for this GitHub workflow. It’s generally a good idea to not use the same IAM account for multiple purposes, so create one just for this. Copy the Key ID and Access Key to GitHub Settings -&gt; Secrets (the url would be something like https://github.com/&lt;organization&gt;/&lt;repository&gt;/settings/secrets/actions). This will then be accessible to the workflow in run time using the above script.</p><p>This is the code that actually pushes the built distributable onto AWS.</p><ul><li>In this case the first line deletes the existing content on S3 so it’s a clean slate.</li><li>Then the second line copies the entire website content to the S3 bucket. While doing so, it adds headers onto each object so that the browser fetching these information will cache it efficiently.</li><li>The 3rd &amp; 4th line invalidates the cloudfront cache, so the next time a browser requests for the data, it’ll get the latest objects that we just push.</li></ul><p>That’s pretty much it. Now all you need is to add this file into your repository and commit it in and watch the magic happen!</p><p><em>Originally published at </em><a href="https://mohankarthik.dev/engineering/2022/03/21/setting-up-github-actions-to-deploy-angular.html"><em>https://mohankarthik.dev</em></a><em> on March 21, 2022.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3cadb624921b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Battle Unseen]]></title>
            <link>https://medium.com/@mohankarthik/the-battle-unseen-eff2294827a9?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/eff2294827a9</guid>
            <category><![CDATA[mindset]]></category>
            <category><![CDATA[nature]]></category>
            <category><![CDATA[personal-development]]></category>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Tue, 30 Nov 2021 13:51:55 GMT</pubDate>
            <atom:updated>2021-11-30T13:51:55.290Z</atom:updated>
            <content:encoded><![CDATA[<p>For eons, a battle has raged between two mighty forces. The likes of which has never been penned in any fiction nor reminisced in any myth. The likes of which dwarfs the epic battles of our cinemas and novels and make them look like fairy tales. And yet, this battle has all but been oblivious to us “five sensed” earthlings, us who see, yet are blind; who hear and yet are deaf.</p><p>Since our planet first matured, the mountains have grown tall and proud. Even today the mountains continuously evolve with some growing a few centimeters each year due to the continued tectonic pressure. Too high did the mountains climb, so high that their peaks were covered with snow, and crowned white. And while they looked at each other and marveled at their splendid crowns, little did they know that they had just birthed their arch enemies.</p><p>As the sun shone it’s light and warmth on these tall mastiffs, the snow melted down and from there came the water streams. The mountains cared little for the tiny rivulets, as they did nothing more than to add to their proud beauty. As these streams, Brookes and rivulets met, they slowly gave rise to formidable rivers that thundered down these mountains. Even then the mountains cared little for they were confident in their tenacity and their strength. Who among this earth could ever change those majestic mountains, those that were made of earth and stone, those that could host entire forests on their slopes and those that could tower over even the clouds and look down upon the rest of the earth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y8eftbBU-5suWyIptLUWYg.jpeg" /></figure><p>And yet the mountains failed to account that their off-springs would be just as tenacious and persevering as they themselves. The rivers were the great equalizers, and they wanted to chart a course through these tall peaks and level the field. And yet they knew that their objective wasn’t easy. They knew that the gallant and daunting mountains were not easy subjects who would bend to their wills. And so they persevered. At each turn, the rivers would try chipping away at the smallest bit of the mountain. They would fail a million times before they succeeded. And yet they knew that they only had to keep trying until they succeeded.</p><p>And thus began the longest battle that our planet has known. Mountains, who grew ever so slowly and aimed to persevere, and rivers that slowly tried to chip away at the mountains and level them. Who will win this battle is no longer relevant, as we unfortunately wouldn’t be here to witness the end. But there is much to learn from these two titans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lIiWEzlXuDzi3V2Qt-yI8g.jpeg" /></figure><p>The mountains achieve their purpose through being gentle, patient and unmoving. They see the rivers chipping at them, and it would be easy for them to look at their losses and assume that eventually the rivers would beat them down, and yet they don’t lose hope. They stand tall, proud and steady. They persevere.</p><p>The rivers are aggressive, dynamic and yet they too are patient. They see their failures a million times each day as they try and try and fail at chipping away even the tiniest block off a mountain. It would be so easy for them to look at these failures and think that they would never achieve their goals, and yet they don’t give up. They dart this way and that, trying different things each time. They persevere.</p><p>In life too, at every step, in every venture that we take, we face uncertainties, we face hardships and we face losses. It is terribly natural for us to look at these losses and assume that we’ll never succeed. And yet, we see instances all around us, both among nature and people, of success that comes through perseverance. I’m of the belief that along with empathy and rationality, perseverance forms the holy trinity of mental traits that leads to a healthy life. It gives us the strength to fight through hardships and it evens us out during our successes. It keeps us going when there is nothing left to go on. And it’s the stuff that stories are made of.</p><p>This then is our quest. To find things that help us persevere. For me, it’s looking at stars as they chart a billion year life cycle across the cosmos, rivers sculpting their way through mountains, predators prowling all night to hunt their prey, mammals migrating thousands of kilometers to evade harsh weather, humans going to extraordinary lengths to protect each other and nature. Find things that inspire you to persevere and hold onto it. For it shall be the light in the darkest of the days and will make this journey of life a breeze to live on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eff2294827a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unit Testing for Machine Learning]]></title>
            <link>https://medium.com/@mohankarthik/unit-testing-for-machine-learning-d74cc7a5291e?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/d74cc7a5291e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Sat, 21 Oct 2017 03:55:06 GMT</pubDate>
            <atom:updated>2017-10-21T03:55:06.133Z</atom:updated>
            <content:encoded><![CDATA[<p>Would love to share a great article that I read today.</p><p><a href="https://medium.com/@keeper6928/how-to-unit-test-machine-learning-code-57cf6fd81765">How to unit test machine learning code.</a></p><p>Some key points to take away from here.</p><ol><li>You can have a common set of tests (such as trainable parameters, shape validation, etc…) written that can be re-used across all your ML code. Think of it as your common regression suite.</li><li>Look at Udacity’s <a href="https://github.com/udacity/deep-learning">Deep Learning code,</a> most of the projects already have unit test stubs written so that the student can test against it, while developing their code. This is a great idea to continue during your own development as well.</li><li>And since most of ML is written in python anyway, it’s an even greater idea to pair <a href="https://medium.com/u/9e396dff421b">Chase Roberts</a>’s idea with python unit testing frameworks. Here are <a href="https://jeffknupp.com/blog/2013/12/09/improve-your-python-understanding-unit-testing/">some</a> <a href="http://docs.python-guide.org/en/latest/writing/tests/">fantastic</a> <a href="https://cgoldberg.github.io/python-unittest-tutorial/">articles</a> to get you started.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d74cc7a5291e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Path Planning in Highways for an Autonomous Vehicle]]></title>
            <link>https://medium.com/@mohankarthik/path-planning-in-highways-for-an-autonomous-vehicle-242b91e6387d?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/242b91e6387d</guid>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Mon, 31 Jul 2017 03:20:12 GMT</pubDate>
            <atom:updated>2017-07-31T06:44:25.912Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is the write up for my Path Planning project for Term 3, of the Udacity’s Self Driving Car NanoDegreee. The entire article assumes knowledge of how the <a href="https://github.com/udacity/self-driving-car-sim/releases/tag/T3_v1.2">Udacity Term 3 simulator works</a> v1.2.</p><h3><strong>Introduction</strong></h3><p>Path planning is an interesting problem. In essence, it’s the process of finding a safe, comfortable and efficient (in that order) path through a set of dynamic maneuverable objects to reach our goal. Now let’s break it down.</p><p>Traditionally, say if we wanted to find a route between a source and a destination, and we wanted to find the fastest / shortest path, there are tons of algorithms to do this: <a href="https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm">Dijkstra’s</a>, <a href="https://en.wikipedia.org/wiki/Breadth-first_search">Breadth First Search</a>, <a href="https://en.wikipedia.org/wiki/A*_search_algorithm">A*</a>, etc…. But in a road scene, whether it be a highway or urban roads, one of the most important difference is that the scene is not static. So our plan needs to be a dynamic one too.</p><p>What complicates the scenario even further is, while we can track the other objects with Sensor Fusion(dynamic such as other cars, pedestrians, bikes, etc…., and static such as trees, curbs, lamp posts), the dynamic object’s behaviour can dynamically change; i.e: they can be maneuvered. So a simple extrapolation based on the current tracked information, say from a <a href="https://en.wikipedia.org/wiki/Kalman_filter">Kalman Filter</a>, would be insufficient for the task, as the car could suddenly change lane, or the pedestrian could suddenly stop midway during a road cross to message someone (those idiots!! :P).</p><h3><strong>Behaviour prediction of Maneuverable Objects</strong></h3><p>So the first step is to predict the behaviour of all dynamic objects on the scene, and then based on this, predict a trajectory for each of the them. This is a pretty difficult task, since each of the object on the scene have a lot of choices in them. The method to solve them is to use something that is known as <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.9763&amp;rep=rep1&amp;type=pdf">multiple-model algorithms for maneuvering target tracking</a>. These methods allow us to evaluate multiple different models (such as: continue straight, turn left, turn right, slow down, speed up, and combinations of these) simultaneously for each of the object, and then based on the current observation (do we see him speed up, or turn, etc…) the algorithm would assign a probability to each of the model for each of the object. Once the probability reaches a certain threshold, we could assume that behaviour for that object and generate a predicted trajectory for that object based on that behaviour.</p><p>In my current implementation of the path planner (shown at the end of this article), this step is <strong>not </strong>implemented, as an extensive behaviour prediction was not necessary for the case of a simple highway driving. But for urban driving scenes, this almost becomes mandatory. Think of a intersection ;).</p><h3><strong>Behaviour Planning</strong></h3><p>Now that we’ve the predicted trajectories of each of the objects around us, the next step is to plan our behaviour. Do we in turn, speed up, slow down, change lanes, etc… This comes down to two things (and I’m going to be specific to highway scenario).</p><ol><li>Estimating a score for each lane, to determine the best lane for us to be in (efficiency)</li><li>Evaluating the feasibility of moving to that lane in the immediate future (safety &amp; comfort)</li></ol><p>So the three factors that we mentioned in the problem statement are covered here in the two points.</p><p><strong>Ranking Lanes</strong></p><p>The method I took to rank lanes are using the following 3 factors</p><ol><li>The lesser the number of lanes we need to change, the better. Because the lesser we change lanes, the more comfortable the drive is for the passengers (takes care of comfort)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*HgGVM2u-ulmo4xcp3C9SqA.png" /><figcaption>Lane Change Score</figcaption></figure><p>2. The distance of the car ahead of us in that lane. The more the distance, the better the score</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*RGmasaNv7CkM4j3OpaJyzQ.png" /><figcaption>Distance Score</figcaption></figure><p>3. The velocity of the car ahead of us in that lane. The greater the velocity, the faster we can travel in that lane, before being forced to change lane again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*-6bOHLBvZg155NdkC7QCEg.png" /><figcaption>Velocity Score</figcaption></figure><p><em>Note:</em></p><ol><li><em>If a particular lane does not have any car ahead of us, then it would get the full score of the corresponding weights for velocity &amp; distance scores.</em></li><li><em>We could augment this with other factors such as number of cars ahead of us in that lane, slowest car ahead of us in the lane (not just the closest car), include the behaviour prediction we talked about in section 1, etc…</em></li></ol><p><strong>Feasibility Checks</strong></p><p>Now that we’ve ranked the lanes, we now need to check if transition to that lane is feasible or not! This is done with the following flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*Is9ZrUlOV7Hl5ThWV0L-RA.png" /><figcaption>Feasibility Flow</figcaption></figure><ol><li>Loop across the lanes in the order from best to worst lane.</li><li>If the currently looped lane is our current lane, then nothing to do, everything is great, keep moving, break the loop.</li><li>If not, then check if the new lane would be jerk &amp; collision feasible. The sections below explain how this is done.</li><li>If they are not feasible, move onto the next lane.</li><li>If they are feasible, then increment a vote count, just so that we don’t take instantaneous decisions. We want to take decisions after pondering them over several timesteps. So once this vote count reaches a threshold, we can take up the decision.</li><li>Once the votes are there, then we output the desired ‘d: the destination lane’ and the desired ‘s/time: the velocity’.</li><li>The s/time parameter is essentially based on the car ahead of us and if there is no car, then it aims for the maximum velocity without breaking the speed limit.</li></ol><p><strong>Jerk &amp; Collision Feasible checks</strong></p><p>There are many different ways this can be done. The way I chose to do here is the following</p><p>Experimentally, I identified that at the speed limit of 50mph, a single lane change is always jerk feasible. This is automatically taken care by the trajectory generation described below. But a double lane change has chances that it would jerk too much (this depends on the road curvature). If the road is curving in one way, and we try to jump 2 lanes in the other direction, then there is a chance that we would violate the jerk limit. So for jerk checks, I just check that the current speed should be less than a threshold if we are doing more than a single lane change.</p><p>Collision feasibility is more interesting. Essentially, we want to ensure that there is a good 2 or more car length gap between our car and the other cars in the destination lane and all the intermediate lanes (in case of more than a single lane change). We should also consider in the velocity of the other cars while coming up with this window, i.e: a fast car behind us means we need more window behind us, a slow car ahead of us means we need more window ahead of us.</p><h3>Trajectory Generation</h3><p>This is by far the most critical and the most difficult step to implement in this entire project. The solution I implemented is inspired from <a href="https://medium.com/u/8a1f0a75e0b5">John Chen</a>: the idea of converting all coordinates to local car’s co-ordinate system to make things easier.</p><p>In this, I heavily depend on <a href="https://github.com/ttk592/spline">ttk:spline</a> for most of the trajectory generation.</p><p>The Udacity simulator gives us the previous path after removing all used up points. This is a great starting point for the algorithm which essentially is:</p><ol><li>Create a spline of the nearest surrounding waypoints in the car’s reference frame. The waypoints must be converted to the car’s reference and should be aimed for the ‘s/time’ and ‘d’ output from the behaviour planner. Let’s call it the waypointSpline. This essentially gives us the ideal path we want to be in.</li><li>Create another spline with the previous path as the starting points (you could use a subset of the previous path too for more agile response). And add points from the waypointSpline into this. This spline is now the LaneTracker. This now clubs the path that we currently are with the ideal path, making a smooth trajectory from start to finish.</li><li>Create another spline (see what I meant when I said heavily depend), to smoothen the velocity changes from the current velocity (can include all previous path velocities) to the destination velocity (s/time). Let’s call this VelocityTracker. This ensures that we can follow our ideal trajectory without violating jerk / speed constraints.</li><li>Now we can generate the trajectory in the local car’s reference, by estimating the x using VelocityTracker for each time instant. Then use this x as input to the LaneTracker to generate the y.</li><li>Convert these points back to the world coordinates and feed them back to the simulator, and voila you are done.</li></ol><p><strong>Emergency Braking</strong></p><p>During testing the system, once two other cars in front of ours collided and came to a stand still. The fact that I was using the entire previous path as input to the LaneTracker &amp; VelocityTracker spline, meant that the car was too slow to react to this resulting in collision. So I implemented a emergency braking solution, that would ignore the previous path and hard set the trajectory to the maximum braking capable (controlled by jerk) by the car. Fortunately / Unfortunately I never got to test this on the sim, as another accident did not</p><h3>Results</h3><p>The resulting video is the following</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FPqbAUjUfMCo%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DPqbAUjUfMCo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FPqbAUjUfMCo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ed85c3d3f7db9ff3fffe1e4f5709eba/href">https://medium.com/media/7ed85c3d3f7db9ff3fffe1e4f5709eba/href</a></iframe><h3>Future Work</h3><p>There is still a lot of things that can be done to improve this</p><ol><li>Add in behaviour prediction for each dynamic object using multiple-model algorithms that will allow the system to react much better &amp; faster to the dynamic changes.</li><li>Improve the lane scoring using other parameters and see improvements.</li><li>Improve the emergency braking system and test it more (might need simulator upgrades)</li><li>Incorporate Q learning and RL to weave through the traffic (also need simulator upgrades to pass images via telemetry)</li></ol><h3>Acknowledgements</h3><p>Great thanks to the community. Some of these ideas are inspired from discussions on the forums / slack / etc… The stalwarts in the community such as <a href="https://medium.com/u/8a1f0a75e0b5">John Chen</a>, <a href="https://medium.com/u/b783495cc56b">Vivek Yadav</a> have been a continuous inspiration to all of us. Also definitely thanks to the Udacity instructors <a href="https://medium.com/u/8190c86ea791">David Silver</a> and the team from Mercedes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=242b91e6387d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Feature extraction for Vehicle Detection using HOG+]]></title>
            <link>https://medium.com/@mohankarthik/feature-extraction-for-vehicle-detection-using-hog-d99354a84d10?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/d99354a84d10</guid>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Tue, 14 Feb 2017 03:45:25 GMT</pubDate>
            <atom:updated>2017-02-16T10:48:56.454Z</atom:updated>
            <content:encoded><![CDATA[<p>This is how I went about doing the Vehicle Detection project (P5) from Term 1 of the Udacity’s Self Driving Car Nanodegree program. The goals / steps of this project are the following:</p><ul><li>Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier</li><li>Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.</li><li>Note: for those first two steps don’t forget to normalize your features and randomize a selection for training and testing.</li><li>Implement a sliding-window technique and use your trained classifier to search for vehicles in images.</li><li>Run your pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.</li><li>Estimate a bounding box for vehicles detected.</li></ul><p>This article will cover the first two steps.</p><h3>Visualizing the data</h3><p>The first step always is to see what we’ve got. And here is the data from Udacity. Some examples of cars and non-cars that we’ve are</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*2DqoQ0fskGLiEIj2.png" /><figcaption>Car / Non-Car image data</figcaption></figure><p>Some things to note here are that each image is a 64x64. And there are about 8k images of each type. Which considering that the background class is pretty much everything other than car, I guess is a pretty small image set.</p><h3>Image Features</h3><p>The course suggests to use a combination of multiple image features. Once again we are doing this detection using traditional CV techniques as opposed to using modern CNN architectures like YOLO / SSD.</p><p>So we’ve the following image features along with the following parameters for each image features</p><ul><li><strong>Spatial binning</strong> (features directly extracted from the image pixels): Color space of the image (commonly used for other features too); Size of the image to be binned</li><li><strong>Color histogram:</strong> Number of histogram bins</li><li><strong>Histogram of oriented gradients (HOG):</strong> Channel used, Orientations, Pixels per Cells, Cells per block</li></ul><h4>Choosing the parameters</h4><p>In terms of choosing the parameters, we’ve two major rationales. One is speed, we don’t want to wait for ages for the classification to happen, and in the best case, we want it to be real time. The other is accuracy. So the parameter decisions is a trade-off between these two attributes.</p><h4>Visualizing the parameters</h4><p>Once again the shortcut is to visualize the parameters to get an intuition of how they work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nQSaAa3mhwyYSBBc.png" /><figcaption>RGB Cars Color Histogram</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZHvYpDRXmu0jJwgD.png" /><figcaption>RGB Non-Cars Color Histogram</figcaption></figure><p>We can see that for the RGB color space, all the 3 channels give us a very different spectrum across the histogram. So this looks good.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B2jN2ynpjODyQNj9.png" /><figcaption>YCbCr Cars Color Histogram</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BV9q-q9GDnRioCwf.png" /><figcaption>YCbCr Non-Cars Color Histogram</figcaption></figure><p>Here, the Y channel gives a good difference, but the Cb and the Cr channels look almost similar. So does not look like a good choice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RahYNA8PGnwc-WHa.png" /><figcaption>HSV Cars Color Histogram</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rl7EFje-OoILEmRA.png" /><figcaption>HSV Non-Cars Color Histogram</figcaption></figure><p>Here, we see a decent difference in S and V channel, but not much in the H channel. So maybe in terms of color histogram, RGB and the S &amp; V channel of HSV are looking good.</p><p>Great! Now let’s visualize the HOG parameters.</p><p>The combinations used here are</p><ol><li>Orientations = 8, Pixel per cell = (8,8), cells per block = 2</li><li>Orientations = 9, Pixel per cell = (8,8), cells per block = 2</li><li>Orientations = 8, Pixel per cell = (16,16), cells per block = 2</li><li>Orientations = 9, Pixel per cell = (16,16), cells per block = 2</li><li>Orientations = 8, Pixel per cell = (8,8), cells per block = 1</li><li>Orientations = 9, Pixel per cell = (8,8), cells per block = 1</li><li>Orientations = 8, Pixel per cell = (16,16), cells per block = 1</li><li>Orientations = 9, Pixel per cell = (16,16), cells per block = 1</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*7Vl7d2MWQeBWN-p2.jpg" /><figcaption>HOG parameters</figcaption></figure><p>The first option looks great! Nice clear markings for the car image, and a clear difference between the car and non-car image</p><p>Now let’s look at the color spaces within HOG</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/0*NN-M2wcTAUTeZiwS.jpg" /><figcaption>HOG YCbCr</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/0*HMeZhVVqniByOu2T.jpg" /><figcaption>HOG RGB</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/0*e3FS50Gj_Oa6eB0J.jpg" /><figcaption>HOG HSV</figcaption></figure><p>We get almost the same result as we got in the color histogram, which is a great reaffirmation for the data. The YCbCr gives a differential performance only for the Y channel. Whereas all 3 channels in RGB gives us a difference. The Hue in HSV is almost similar, but the S and V show a nice difference.</p><h3>Training the Classifier</h3><p>Before we train, we need to ensure that all the features are normalized. WE can use the StandardScaler in sklearn to get this to work easily.</p><p>Let’s actually train the SVM and check SVM’s accuracy to see if it corresponds to the data that we’ve got till. In the following experiment, I fixed the hog orientations to 8, hog pixel per cell to be 8 and cells per block to be 2. I’ve also fixed the spatial binning size to be 16x16 and the color histogram bins to be 16. The only variations are the color space.</p><ul><li>Y channel of YCbCr (no spatial binning or color hist): 93.53%</li><li>RGB (all 3)(no spatial binning or color hist): 96.07%</li><li>SV channel of HSV(no spatial binning or color hist): 95.97%</li><li>SV channel of HSV (with spatial binning or color hist): 98.51%</li><li>HSV (all 3)(with spatial binning or color hist): 98.86%</li><li>RGB (all 3)(with spatial binning or color hist): 97.73%</li></ul><p>Awesome. So we’ve got our winners. It’s HSV, where the H channel is optional (trading off speed to accuracy). So that covers the feature extraction part of the Vehicle Detection.</p><p>As alternatives, you can also look at a standard CNN as a feature extractor.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d99354a84d10" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloning a car to mimic human driving]]></title>
            <link>https://medium.com/@mohankarthik/cloning-a-car-to-mimic-human-driving-5c2f7e8d8aff?source=rss-355915c47610------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c2f7e8d8aff</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mohan Karthik]]></dc:creator>
            <pubDate>Fri, 23 Dec 2016 03:55:19 GMT</pubDate>
            <atom:updated>2016-12-23T03:55:19.190Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a project for the <a href="https://www.udacity.com/drive">Udacity Self Driving Car Nanodegree</a> program. One of the projects there is to use machine learning to make a car mimic human driving without explicitly programming it to do any of the actions. Similar to how a new driver learns by watching an experienced driver and learning.</p><p>Udacity has provided a stunningly good simulator (similar to a video game). The simulated car is equipped with three cameras which face the front and are mounted on the left, center and the right of the simulated car. They also connect the simulator to a scripting backend so that we can retrieve critical values such as throttle, speed, steering angle, and the images from the simulator during training session. And during ‘autonomous’ mode, the scripting backend can publish a steering angle / throttle to the simulator given an image to allow the car to drive.</p><p>Now let’s jump in…</p><h3>Data</h3><p>As with all machine learning problems, the predominant part of the problem turned out to be getting the right data. This project is extremely true to the phrase; “Garbage In, Garbage Out”.</p><h3>Data source</h3><h4>My own data</h4><p>Collecting data on my own was tortuously difficult considering that my choices were limited between a keyboard or a steam controller to control the simulator. I struggled fruitlessly for over a week to collect the data (although, I’ve to admit that playing with the simulator was a lot of fun to begin with). But this gave me a chance to understand just how important the process of collecting the data is. The problems with collecting data typically are that</p><ul><li>There would be too many entries with the angle set to 0. This is predominantly because of the keyboard entry. Note that in an actual car, this problem won’t be this acute (although it’ll still exist).</li><li>Keyboard driving angles are highly quantized given us huge gradients whenever there is a big turn.</li></ul><p>These are issues because, if we feed in a huge number of zeros to the ML system, it’ll have a tendency towards driving straight and will not be able to make sharp turns.</p><p>Finally came the savior.</p><h4>Udacity data</h4><p>Udacity released their training data on Track 1 couple of weeks back and this really jump started this project for me. As with all ML problems, the first step is to understand the data we are working with.</p><p><strong>Angles over time</strong></p><p>The first was to see how the angles vary over time</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*2Ul6JZEDxyZZqFXS.png" /><figcaption>Udacity data set: Angles over time</figcaption></figure><p>We can see that the data set collected some very sharp turns in the middle, but is otherwise pretty evenly distributed.</p><p><strong>Angle distribution</strong></p><p>Now the important thing to find how the angles are distributed. We know that steering models, due to the large availability of zeros in the datasets typically tend to be biased towards predicting zeros.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*QBKJ2FHUZ4_cJSOz.png" /><figcaption>Udacity data set: Angle distribution</figcaption></figure><p>And we can see that it holds good for this dataset as well. HUGE number of zeros and very very few of all other data. This is definitely going to bias our model towards predicting 0s, if we go ahead as it is.</p><p><strong>Angle difference</strong></p><p>Let’s also see, if we’ve big jumps in the angles over time (like it would if the data were to be recorded via keyboard). Note that recording from an actual steering wheel, we can expect the angle difference between subsequent samples to be quite uniform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*5XCi74GUPO8lJg_j.png" /></figure><p>We can see that sometimes there are pretty large spikes in the difference. This points that the data has a good volatility in it and it’s going to be difficult for the model to get a smooth output from it.</p><p><strong>Sample images</strong></p><p>Finally, let’s plot some sample images from the data for each angle bin. I’m just splitting all the angles into 21 bins of 0.1 size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*3G78GZR7J2ogUOPx.png" /></figure><p>We can see that the some of the images, don’t really make too much sense. For eg:. the -0.8 and the -0.7 images taken at the bridge looks like the car is well centered, and yet we’ve a very large angle. So the dataset has some errors, and we need to think of a way to offset this error.</p><h3>Data Augmentation</h3><p>As with the other ML projects, along with some of the excellent discussions in the slack channel with other students, I came to the conclusion that data augmentation was the way to clean up the data. So the following are the augmentation that I do on the data set. We need to ensure that all of these augmentations are as random as possible to ensure that the model gets a good distribution of input training data.</p><p>These techniques will be extremely helpful and very much applicable when it’s real life driving as well.</p><p><em>Disclaimer: Most of what’s described below has been adapted from </em><a href="https://chatbotslife.com/using-augmentation-to-mimic-human-driving-496b569760a9#.5l28f4q9u"><em>Vivek’s excellent post</em></a><em>.</em></p><h4>Randomly choosing between left, center or right image</h4><p>One predominant problem in steering prediction is that even if the model predicts really really well, the model is bound to make a non zero error on it’s prediction. This means that over time the model will drift off from center. Since we are only training the model to drive correctly and not training the model how to recover (something that we humans know instinctively), the model is handicapped.</p><p>To offset this problem, we choose images randomly between left, right or center images by doing the following. The approach is to add / subtract a static offset from the angle when choosing the left / right image. To get a smoother drive, we can add / subtract an offset that’s weighted by the magnitude of the angle. But this biases the model towards zeros again and is a design tradeoff. I chose the simple static offsets. This simulates the model drifting off the center and trains the model to recover from it’s mistakes. This also gives us a lot more data to work with than by just using the center images alone.</p><pre>img_choice = np.random.randint(3)</pre><pre>if img_choice == 0:<br>    img_path = os.path.join(PATH, df.left.iloc[idx].strip())<br>    angle += OFF_CENTER_IMG<br>elif img_choice == 1:<br>    img_path = os.path.join(PATH, df.center.iloc[idx].strip())<br>else:<br>    img_path = os.path.join(PATH, df.right.iloc[idx].strip())<br>    angle -= OFF_CENTER_IMG</pre><h4>Flipping the image</h4><p>One easy way to get more data is to just flip the image around the horizontal axis and flip the sign on the angle as well. We instantly get twice the data and we don’t inadvertently bias the model towards any one direction.</p><pre>if np.random.randint(2) == 0:<br>    img = np.fliplr(img)<br>    new_angle = -new_angle</pre><h4>Changing brightness</h4><p>Changing brightness allows the model to become robust towards all lighting conditions. So we randomly change the brightness of the image.</p><pre>temp = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)</pre><pre># Compute a random brightness value and apply to the image<br>brightness = BRIGHTNESS_RANGE + np.random.uniform()<br>temp[:, :, 2] = temp[:, :, 2] * brightness</pre><pre># Convert back to RGB and return<br>return cv2.cvtColor(temp, cv2.COLOR_HSV2RGB)</pre><h4>Changing X and Y translation</h4><p>Here we shift the image in the X and Y direction to generate fake images. While it’s great that we get the left and right camera images to train recovery, it is still very less data. Shifting in the X axis and changing the angle correspondingly allows us to train even better recovery.</p><pre># Compute X translation<br>x_translation = (TRANS_X_RANGE * np.random.uniform()) - (TRANS_X_RANGE / 2)<br>new_angle = angle + ((x_translation / TRANS_X_RANGE) * 2) * TRANS_ANGLE</pre><pre># Randomly compute a Y translation<br>y_translation = (TRANS_Y_RANGE * np.random.uniform()) - (TRANS_Y_RANGE / 2)</pre><pre># Form the translation matrix<br>translation_matrix = np.float32([[1, 0, x_translation], [0, 1, y_translation]])</pre><pre># Translate the image<br>return cv2.warpAffine(img, translation_matrix, (img.shape[1], img.shape[0]))</pre><h4>Biasing towards non-0 value</h4><p>From the original dataset, we know that we need to ensure that the model does not get biased towards predicting a close to zero. To do this, we use a bias term as input to the data_generator. We tweak this bias term at the end of each epoch to decreasing the probability that a small angle will be selected for training. It goes like this</p><pre># Choose left / right / center image and compute new angle<br># Do translation and modify the angle again</pre><pre># Define a random threshold for each image taken<br>threshold = np.random.uniform()</pre><pre># If the newly augmented angle + the bias falls below the threshold<br># then discard this angle / img combination and look again<br>if (abs(angle) + bias) &lt; threshold:<br>    return None, None</pre><p>What this will accomplish is that as we decreases bias from 1.0 to 0.0, it’ll with increasing probability drop all the lower angles. Note that we do not set a hard threshold for low angles, but allow it to drop off uniformly.</p><h4>Data Preprocessing</h4><p>We also pre-process the data by removing the top 60 pixels (past the horizon) and the bottom 20 pixels (the hood of the car). And we resize the resulting image to a 64x64 (makes the CNN work much easier when it’s a square image).</p><pre># Remove the unwanted top scene and retain only the track<br>roi = img[60:140, :, :]</pre><pre># Resize the image<br>resize = cv2.resize(roi, (IMG_ROWS, IMG_COLS), interpolation=cv2.INTER_AREA)</pre><pre># Return the image sized as a 4D array<br>return np.resize(resize, (1, IMG_ROWS, IMG_COLS, IMG_CH))</pre><h4>Visualizing data after the augmentation</h4><p>So let’s see what impact we’ve made on the data.</p><p>Bias 1.0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*nJirmaDHkOYrboBC.png" /></figure><p>Bias 0.0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*eP6SXyLLwVVxhOKh.png" /></figure><p>You can see that the when we allow a bias of 1.0, the data is nicely gaussian shaped, but as we reduce bias to 0.0, the 0 angle data is severely punished allowing our model to incrementally learn a more complex hypothesis. Compare this with our original histogram and we can see just how far we’ve already come.</p><p>We can also plot some of the images to see how the images look post augmentation. We can see that we’ve got a variety of images spanning a larger histogram of angles covering shifts, etc…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*mTiRorlwovELL2W-.png" /></figure><h3>Model</h3><p>The model that I finally latched onto was a slightly simplified version of the VGG16 pre-trained model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*GV795Ssg5i1IaDBi76twmw.png" /><figcaption>Network Architecture</figcaption></figure><p>Total params: 17118541</p><p>Things to note are</p><ul><li>I’ve added a lambda layer on the top similar to the comma ai model to normalize the data on the fly</li></ul><pre>model.add(Lambda(lambda x: x/127.5 - .5,<br>                 input_shape=(IMG_ROWS, IMG_COLS, IMG_CH),<br>                 output_shape=(IMG_ROWS, IMG_COLS, IMG_CH)))</pre><ul><li>I’ve added a color space conversion layer (credit to Vivek for the brilliant idea) as the first layer, so the model can automatically figure out the best color space for the hypothesis</li></ul><pre>model.add(Convolution2D(3, 1, 1, border_mode=&#39;same&#39;, name=&#39;color_conv&#39;))</pre><ul><li>The remaining convolutional layers are directly taken from VGG16</li><li>I’ve added 5 fully connected layers in decreasing complexity as the classifier</li><li>I load the pre-trained no-top weights for the VGG16.</li><li>I ensure that the top 3 conv blocks from VGG16 are frozen and the other layers are trainable.</li></ul><h3>Regularization</h3><p>We take care of regularization in three ways</p><ul><li>We augment the data heavily. This more than anything else allows the model to generalize.</li><li>We add dropouts in all the fully connected layers.</li><li>We split the training set to training and cross validation to control the number of epochs in training. We split a small size (BATCH size) of data as validation data and use that to control if we are acutely overfitting the data</li></ul><p>Overall, my impression in this project has been that considering the noise in the data and the complexity of the hypothesis, conventional methods of looking primarily at loss is ineffective at curbing overfitting. But I do end up trying the 3 epochs that scored the lowest validations score during training on the simulator first.</p><p>There is no reason for a test set, since we can test directly by allowing the car to drive.</p><h3>Training</h3><ul><li>Generator: In this training, we use a generator, which randomly samples the set of available images (including left and right) from the CSV file, randomly flips it, randomly changes brightness, randomly shifts it in the X and Y direction (also changing the angle) and pre-processes the images and feeds to the model. This is great because keras ensures that while the GPUs are busy training the model, the CPU can in parallel generate the data using the generator.</li><li>Optimizer: Adam with a learning rate of 1e-5. This was empirically noted as we are transfer learning (fine tuning an existing model). One thing I noted was, having a higher learning rate caused the system to regress to a single output value regardless of the image fed in.</li><li>Bias: We start with a bias of 1.0 (allow all angles) and slowly as the epochs continue, reduce the bias, thereby dropping low angles progressively.</li></ul><pre>bias = 1. / (num_runs + 1.)</pre><p>I trained for about 10 epochs, saving the weights after each epoch. I start testing the models starting with the ones having the least validation loss. My best solution will generally be in the top 3 models with the least validation loss. It takes about a minute to train each epoch on my GTX-1060. But arriving to this point took almost 3 weeks of hypothesis, trials and discussions.</p><h3>Driving</h3><p>During driving, one observation was that the model (because of my intent to smoothen the drive) does not take sharp curves at good speeds. I had two choices, one was to slow down during curves and the other was to multiply the predicted angle by a constant. Both approaches are used here to obtain a clean drive through Track 2. Track 1 was able to run without the above modifications</p><h3>Track 1</h3><p>The model worked directly on track 1 with no changes.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOG080eLUAkQ%3Fstart%3D1%26feature%3Doembed%26start%3D1&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOG080eLUAkQ&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOG080eLUAkQ%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b89cf1419a9f730027423e806317cca/href">https://medium.com/media/5b89cf1419a9f730027423e806317cca/href</a></iframe><p>Note that towards the end, where the two shadow lines (electrical wires) come, the car turns oddly. This suggests that the model is very afraid of shadows.</p><h3>Track 2</h3><p>In lieu of Track 1 results, it was impossible to run the model on Track 2 with shadows enabled. So decreasing the quality to ‘fastest’, and increasing the throttle to 0.3 and multiplying the predicted steering angle by 1.4, we get</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FgevcOvQnhGk%3Fstart%3D15%26feature%3Doembed%26start%3D15&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DgevcOvQnhGk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FgevcOvQnhGk%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f62f62b095be427c5c446e4afedd7185/href">https://medium.com/media/f62f62b095be427c5c446e4afedd7185/href</a></iframe><p>Considering the amount of turns, and steep inclines, the model does spectacularly well.</p><h3>Reflections</h3><p>This project has been a phenomenal journey into the development methodology required for a successful deep learning project. The dependency on data collection, data augmentation, model parameters and the various other entities forces the developer to have a very streamlined process of development, without which the development will easily fall apart. And it instills a wonder and awe towards those working in this field. The complexities of transferring this learning (pun very much intended) to driving a car on the road is inspiring… Can’t wait for that day to come.</p><h3>Future work</h3><ul><li>Incorporate augmented shadows in the training to make it more robust (we need to make the model not afraid of the shadows under the bed)</li><li>Look at the models used in the <a href="https://medium.com/udacity/challenge-2-using-deep-learning-to-predict-steering-angles-f42004a36ff3#.ew71rwadu">Challenge-2</a>, especially <a href="https://github.com/udacity/self-driving-car/tree/master/steering-models/community-models/autumn">optical flow</a> and <a href="https://github.com/udacity/self-driving-car/blob/master/steering-models/community-models/komanda/solution-komanda.ipynb">lstm</a> based (I’ve always loved the idea of RNNs)</li><li>Incorporate advanced lane findings (P4) and object detection (P5) into this project for a complete steering solution</li></ul><h3>Acknowledgements</h3><ul><li><a href="https://chatbotslife.com/using-augmentation-to-mimic-human-driving-496b569760a9">Vivek’s wonderful post on data augmentation</a></li><li><a href="https://arxiv.org/pdf/1604.07316v1.pdf">NVidia’s paper on end-to-end training</a></li><li><a href="https://github.com/commaai/research/blob/master/view_steering_model.py">Comma.ai’s steering model</a></li><li><a href="https://arxiv.org/pdf/1409.1556.pdf">VGG16 paper</a></li><li><a href="https://github.com/fchollet/keras/blob/master/keras/applications/vgg16.py">VGG16 model from keras</a></li><li>All the wonderful classmates in this course who have been extremely helpful and generous with their comments.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c2f7e8d8aff" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>