<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Raju Ahmed Shetu on Medium]]></title>
        <description><![CDATA[Stories by Raju Ahmed Shetu on Medium]]></description>
        <link>https://medium.com/@nashmaniac?source=rss-1af58e55ea31------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*w2EQGY5iCpEfj301nr_peg.png</url>
            <title>Stories by Raju Ahmed Shetu on Medium</title>
            <link>https://medium.com/@nashmaniac?source=rss-1af58e55ea31------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:32:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nashmaniac/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How I operate as Senior Software Engineer everyday]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://nashmaniac.medium.com/how-i-operate-as-senior-software-engineer-everyday-6f93ec474692?source=rss-1af58e55ea31------2"><img src="https://cdn-images-1.medium.com/max/2600/0*Wo2SNTOSkMePh3HD" width="6000"></a></p><p class="medium-feed-snippet">In recent times, I have been asked by several people what I do everyday at work. Most of time, I tried to answer it simple. Working in a&#x2026;</p><p class="medium-feed-link"><a href="https://nashmaniac.medium.com/how-i-operate-as-senior-software-engineer-everyday-6f93ec474692?source=rss-1af58e55ea31------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://nashmaniac.medium.com/how-i-operate-as-senior-software-engineer-everyday-6f93ec474692?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f93ec474692</guid>
            <category><![CDATA[senior-software-engineer]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sat, 20 Jan 2024 13:24:48 GMT</pubDate>
            <atom:updated>2024-01-20T13:24:48.922Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How do you measure happiness?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://nashmaniac.medium.com/how-do-you-measure-happiness-455bb2d5407d?source=rss-1af58e55ea31------2"><img src="https://cdn-images-1.medium.com/max/2600/0*kpj1yPH8ldSf2Orh" width="5472"></a></p><p class="medium-feed-link"><a href="https://nashmaniac.medium.com/how-do-you-measure-happiness-455bb2d5407d?source=rss-1af58e55ea31------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://nashmaniac.medium.com/how-do-you-measure-happiness-455bb2d5407d?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/455bb2d5407d</guid>
            <category><![CDATA[self-care]]></category>
            <category><![CDATA[happiness]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sat, 13 Jan 2024 05:59:16 GMT</pubDate>
            <atom:updated>2024-01-13T13:42:49.948Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The art of remain busy!]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-snippet">Do you feel like you don&#x2019;t have enough time in a day? Ever felt like the thought of workplace remains in your brain after you leave&#x2026;</p><p class="medium-feed-link"><a href="https://nashmaniac.medium.com/the-art-of-remain-busy-99bb1b3c6b33?source=rss-1af58e55ea31------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://nashmaniac.medium.com/the-art-of-remain-busy-99bb1b3c6b33?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/99bb1b3c6b33</guid>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[time-management]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sat, 06 Jan 2024 16:07:03 GMT</pubDate>
            <atom:updated>2024-01-06T16:07:03.163Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Setting goals for 2024]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://nashmaniac.medium.com/setting-goals-for-2024-0322104397c4?source=rss-1af58e55ea31------2"><img src="https://cdn-images-1.medium.com/max/2368/1*bEfG-KPOwDrC8eKW_J5UZg.png" width="2368"></a></p><p class="medium-feed-snippet">How people set goals? Is there a strategy that everyone follows? How high do they target? What is the base minimum for their goals? Every&#x2026;</p><p class="medium-feed-link"><a href="https://nashmaniac.medium.com/setting-goals-for-2024-0322104397c4?source=rss-1af58e55ea31------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://nashmaniac.medium.com/setting-goals-for-2024-0322104397c4?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/0322104397c4</guid>
            <category><![CDATA[self-management]]></category>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[goal-setting]]></category>
            <category><![CDATA[optimism]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Mon, 01 Jan 2024 00:07:11 GMT</pubDate>
            <atom:updated>2024-01-01T00:07:11.317Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[A Bug of six months, finally solved!]]></title>
            <link>https://medium.com/intelligentmachines/a-bug-of-six-months-finally-solved-939e790ef867?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/939e790ef867</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[istio]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Mon, 09 Nov 2020 02:48:12 GMT</pubDate>
            <atom:updated>2020-11-09T03:39:13.136Z</atom:updated>
            <content:encoded><![CDATA[<h3>A bug of six months, finally solved!</h3><h4>Bug!</h4><p>Yes, the one single constant thing that is a nightmare to every engineer or developer out there. The one thing that is capable of making you completely mad and in worst cases even might make you question your intellect.</p><p>I am not a bright guy. I write bad codes. Some of the codes I wrote six months back truly embarrass me now. But one thing I always try to do is not to make the same mistake over and over again. And as you keep writing more and more codes, your codes get better automatically. But what’s the context of it? Here’s the story!</p><p>After I joined the Engineering team from the Analytics team here at <a href="https://intelligentmachin.es/">Intelligent Machines</a>, I started going through the codes far more than ever before. Most of our codes were deployed in the cloud in different manners under different strategies. <strong>My foremost concern at that time was to</strong> <strong>standardize it</strong>, <strong>making all of them automated so that our team can focus more on writing code rather than deploying it</strong>. As the team was already familiar with containerization the next steps towards building a service-oriented architecture were easy.</p><p>We started using <a href="http://istio.io/"><strong>Istio</strong></a> as <strong>a service mesh</strong> a while back in our organization. The <strong>learning curve was</strong> <strong>steep</strong> and was not pleasant at all. <strong>But the after effect was rewarding.</strong> It opened up a bunch of options for us, enabled us to do many things infrastructure wise. Some days later, we started using <a href="https://www.terraform.io/"><strong>Terraform</strong></a>, an infrastructure automation tool. We published three packages to the Terraform registry which we personally used for different infrastructure automation. We were using <strong>GitHub Actions</strong> for our CI/CD pipeline. Everything worked perfectly, we were very happy.</p><p>Looks like a win so far, right? <strong>Yes, certainly was</strong>. I felt never so happy when I wrapped all those things in a single string.</p><p><strong>But for the last 6 months or more, I could not solve one problem which was eating me at my core. </strong>Did I found a work-around of that bug right away at that time?</p><p><strong>Yes!</strong> I needed to keep our application running. But it broke our automation in some way which was my first priority. The statement of the bug was simple.</p><blockquote>“We can not run DB migration in our CI using Kubernetes Job/Deployments”</blockquote><p>Well, this is not partially true, because we could run the migration to our <strong>Cloudsql databases. </strong>The problems started when we migrated our workloads to <strong>Azure. </strong>The migration process was peaceful but the bug was there. So my initial workaround was simple.</p><ul><li>Just log in to the container after deployment and then run the migration manually from one of the pods. As the backend team was small and DB changes were not that frequent, so we lived through the time.</li></ul><p>Our application deployment strategy was simple. Most of our applications contain some deployments to run API, some secrets for config, some services for exposing. We use Istio Gateway and Virtualservices for our routing.</p><h4>#1 Kubernetes Job way</h4><p>As DB migration was always a one-time thing and we did not want to run a pod constantly for that, we tried to run the migration in a batch job so that the pod might terminate when the job is done. But, we moved away from that because the <strong>helm deployment </strong>can not upgrade the job definitions on the fly and it’s not very wise to delete all jobs in a namespace or hard-cord multiple job names to delete.</p><h4>#2 Kubernetes Deployment way</h4><p>So the next step was to convert the jobs into deployments and tweak the command options of a container definition such that it can run the migration first and boot an API server like the other pods for that service which will increase the API pod count to just one. The reason was if the pod is running then run it in a meaningful way and serve a request.</p><p>For <strong>GCP, </strong>the DB migration was running perfectly as most of our <strong>GCP Cloudsql</strong> DB were to be connected via proxy. For <strong>Azure, </strong>our database setup was simple, our DB can only be accessed by the services that were deployed on <strong>Azure.</strong></p><p>At this stage, I was blinded by <strong>Istio</strong>. I could not see the obvious thing that other engineers would see. So I tried running migrations too many times in a namespace where istio-injection was enabled. Every pod was booted with a sidecar proxy named istio-proxy in that namespace and migration was not running. I stumbled upon the <a href="https://istio.io/latest/docs/reference/config/networking/service-entry/">Service Entry documentation</a> on Istio and wrote my own <strong>External Mesh Service Entry</strong> so that my connection to DB that is hosted outside my cluster can be connected via a TCP connection. <strong>It failed!!</strong></p><p>I contacted <strong>Azure Help support </strong>regarding this. I had a fantastic experience with them earlier. As usual, they helped me identify the connection from my pods to DB was okay. Then we exchanged mail sometimes but my problem still exists.</p><p>This time, I had enough. I didn’t want this thing to prolong my suffering further. We tried changing the meshConfig to allow any connections outgoing from our service mesh but it was for nothing. Then I dug down this issue from the Istio official repository named <a href="https://github.com/istio/istio/issues/9157"><strong>Istio sidecar not allowing connection to the database</strong></a><strong>. No luck yet! </strong>After some time I found this piece of gem which eventually help me to look otherwise — <a href="https://github.com/istio/istio/issues/24461#"><strong>Connection to external service with ALLOW_ANY Still drop the connection</strong></a><strong>.</strong></p><p>This actually states that the booting of <strong>Envoy</strong> upon which the istio-proxy container is based <strong>has a random booting uptime</strong> and <strong>our container might request for the migration before the istio-proxy is ready to route the request to an external party</strong>. The author of the issues suggested running such deployments to a namespace where the istio-sidecar-injection is disabled. So I changed the namespace of the deployment to another namespace and ran. <strong>It got failed! </strong>I found my secrets were in the other namespace which the deployment could not access. Classic rookie mistake! I added another secret to the namespace as well and <strong>it ran successfully finally!</strong></p><p>I know, the solution at the end seems very simple. <strong>Just changing the namespace config to another one without any side-car injection enabled</strong>. I am so happy to get to the bottom of this that it compels me to write this article at 2 AM in the morning. Now my services can be truly automated again and I can sleep in peace knowing it’s resolved.</p><p>Looks like, the Istio team still not working on this. It is marked <strong>automatically-closed</strong> by the istio-bot on their GitHub repository. I am currently running on istio-1.7 and the proxy bug still persists. The only way to run this is to run migrations without a side-car.</p><p>Hope this will help to remind each of us to see things more simply and save you from a lot of head-scratching moments.</p><p>Stay bug-free and virus-free at the covid time. Happy coding!!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=939e790ef867" width="1" height="1" alt=""><hr><p><a href="https://medium.com/intelligentmachines/a-bug-of-six-months-finally-solved-939e790ef867">A Bug of six months, finally solved!</a> was originally published in <a href="https://medium.com/intelligentmachines">intelligentmachines</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GitHub Actions: Building and publishing own actions using Python]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/intelligentmachines/github-actions-building-and-publishing-own-actions-using-python-d94e2724b08c?source=rss-1af58e55ea31------2"><img src="https://cdn-images-1.medium.com/max/2158/1*4Hs-SnkVuvxENtVN08SVYw.png" width="2158"></a></p><p class="medium-feed-snippet">When a developer works on CI/CD in different projects, often we see a similar pattern if the technology stack is same or a bit different&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/intelligentmachines/github-actions-building-and-publishing-own-actions-using-python-d94e2724b08c?source=rss-1af58e55ea31------2">Continue reading on intelligentmachines »</a></p></div>]]></description>
            <link>https://medium.com/intelligentmachines/github-actions-building-and-publishing-own-actions-using-python-d94e2724b08c?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/d94e2724b08c</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sun, 07 Jun 2020 17:55:22 GMT</pubDate>
            <atom:updated>2020-06-08T05:15:14.341Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[GitHub Actions: End to end CI/CD Pipeline for Django]]></title>
            <link>https://medium.com/intelligentmachines/github-actions-end-to-end-ci-cd-pipeline-for-django-5d48d6f00abf?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d48d6f00abf</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Fri, 05 Jun 2020 01:32:22 GMT</pubDate>
            <atom:updated>2020-06-05T04:23:38.039Z</atom:updated>
            <content:encoded><![CDATA[<p>As promised I am back! I hope you all enjoyed my last blog on <a href="https://medium.com/intelligentmachines/github-actions-basics-40a4d9b417f8">GitHub Actions: Basics</a>. Today I will show you how you can take earlier knowledge and make an end to end CI/CD pipeline for a django project using GitHub Actions.</p><h3>What will we do in here?</h3><p>What I want to show here is how developers work in their day to day lives. How they work with commits, pushes, pull requests, merges and ci/cd pipelines. For this I will be using a django project. But you all can use any project you like because the basics are all same overall. What might be different is the application specific codes.</p><ul><li>We will open a repo. Create a master and develop branch. Any push to master branch will be deployed to production and develop to the staging.</li><li>All the developers working in the projects should commit their commits to any feature/ branches. For this blog we will be using a branch named feature/feature1</li><li>Generally when the work finishes in the feature/feature1 branch a pull request is submitted to develop branch. On submit request a workflow should run which will check if the formatting of the codes are fine and tests are working alright. You never want to push untested codes to production, right?</li><li>On passing the tests and approval from another reviewer in this case my work account in GitHub <a href="https://github.com/imshetu">@imshetu</a>. I will work on feature branches using my personal account<a href="https://github.com/nashmaniac"> @nashmaniac</a>. When I submit a pull request I will select @imshetu as the reviewer.</li><li>On push to develop branch we will again run the tests and deploy the code to staging server in Google Cloud.</li><li>Later we will submit another pull request from develop to master At this time same code formatting and testing will be checked and approval will be given from a reviewer.</li><li>On push to master we will just run the test and deploy it to production.</li></ul><p>Few things to remember, inour case staging and production are the same server. We didn’t want to complicate things by having two servers to maintain. Let’s see this graphically to understand more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*4tEHLXSs8dAEkUcivcXCfw.png" /></figure><p>So I will be writing a workflow which will be triggered in pull_request is submitted to or code is push to develop or master branch. Our workflow will contain three jobs</p><ul><li>Health Check Job: For all the testing and formatting check.</li><li>Packaging Job: For docker build and update registry</li><li>Deployment Job: Deploying the published image to cloud.</li></ul><p>At this point you might think we have workflows name django and gke in GitHub Actions already, why rewrite it again? I would say if you are comfortable with it, you can definitely do it. In here I will write it step wise so that you can figure out how all these pieces work together.</p><p>I know, it will be a bit long. But I assure you, it’s worth the wait.</p><h3>Application Setup</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*34OAnEwnLSk8hu1MiFESTw.png" /></figure><ol><li>Create a repo in GitHub.</li><li>Clone the repo in your local. I used ssh key for authentication. If you don’t have one use the https link in clone pop up box.</li></ol><pre>git clone git@github.com:nashmaniac/github-django-actions.git</pre><p>3.a. Go to project directory and create a new branch named develop by running git checkout -b develop Now push it to GitHub by running git push -u origin develop</p><p>3.b. Here add some branch protection rules to develop and master branch so that no code get pushed to those branch without review.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rUyzs0e9CJfQMXCWKur0EQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H2PZzSoCpSWd6WnJ22fe6A.png" /><figcaption>Left: Adding Branch Protection Rules Right: Branch Protection Rules added.</figcaption></figure><p>4. Create another branch named feature/feature1 We will do all our work in here.</p><p>5. Initialize a python virtualenv directory, activate it and install django psycopg2-binary pycodestyle gunicorninside the virtualenv and make a requirements.txt by running pip freeze &gt; requirements.txt</p><p>6. Create a .gitignore file and added all the necessary files and folders you don’t want in GitHub.</p><p>7. Initialize django project running django-admin startproject app .</p><p>8. We will create an app in django named users and add some tests there. Create the app using python manage.py startapp users</p><p>9. Add a test in users/test.py file like below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/409fd67daa6eda0ee11afce455bb81b1/href">https://medium.com/media/409fd67daa6eda0ee11afce455bb81b1/href</a></iframe><p>10. We need to change out database config in app/settings.py file to make sure our postgres database works. For this delete DATABASES variable in app/settings.py and paste the code below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/56600ded4397933e9aa6df9bba738465/href">https://medium.com/media/56600ded4397933e9aa6df9bba738465/href</a></iframe><p>12. We will add a Dockerfile to our project directory which will build our application into an image. We won’t be discussing as it is out of scope. You can find it in the repo.</p><p>13. I will be adding a folder named k8s which will contain our helm chart. We will use it during our deployment. For keeping this simple I won’t be discussing it here as it would be out of scope a bit.</p><p>14. Create a workflow file name .github/workflows/ci.yaml</p><h3>Let’s write the workflow!</h3><p>I know it’s a lot of steps earlier. Let’s write our workflow now. After finishing, we will be creating a service account in Google Cloud for deployments and declare few secret in GitHub repo settings and we will test out the whole shebang.</p><p>I will add the whole workflow file here will enough comments to make it understandable. Please go through the comments. It will make it clear.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cecabfdb7dead0c12d1a2929d764c2e3/href">https://medium.com/media/cecabfdb7dead0c12d1a2929d764c2e3/href</a></iframe><p>Great, I hope the upper gist makes the workflow clear to us. Few more steps and we are good for testing.</p><h3>Infrastructure &amp; Secrets Setup</h3><ul><li>We need to create a kubernetes cluster in Google Cloud and a service account with Kubernetes Engine Admin Artifact Registry Writer &amp; Storage Admin permission so that it can push and deploy our application. Make sure you download the service account as json in your local.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pc3KUDR6rexwjlNad-vzZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zc83LJtI7H5sNdD9wIXexw.png" /><figcaption>Left: Kubernetes Create Screen Right: Service Account Creation</figcaption></figure><ul><li>You can setup those with admin permission without any issues because those service accounts credentials won’t be available in logs as those will be set as secrets.</li><li>Go to Settings tab in GitHub Repo and set your secrets.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0MPdfd81GqLiCx4e9_6cRg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nvzEAS0KtJSjRRASKGL-eA.png" /><figcaption>Left: New Secret Add Right: All secrets set</figcaption></figure><ul><li><strong>Few things to remember. GKE_EMAIL is the email of service account key. You will find it as client_email if you open the json file you downloaded and GKE_PASSWORD would be the content of json file.</strong></li></ul><p>Now let’s make a commit in your feature/feature1 branch and push it.</p><h3>Moment of Truth</h3><p>I know you all are waiting for this moment. As our code is pushed to feature/feature1 branch. We are all set. I will be adding my work account to this repo so that I can review the pull request.</p><ul><li>Create a pull request and add a reviewer.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rsjAIDEA2YA5ax5P-U2eHg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zZZrpp96J1RYBOc37PdqYQ.png" /><figcaption>Left: Opening a Pull Request with Reviewer Right: Pull Request Approved &amp; Pipeline Passed.</figcaption></figure><ul><li>When I merged the Pull Request after approval from another account in here, A <a href="https://github.com/nashmaniac/github-django-actions/actions/runs/125406839">workflow</a> was run and failed due to some helm file issue. So I need to submit another pull request from feature/feature1 to develop and it failed again. So I made some changes and pushed. At the <a href="https://github.com/nashmaniac/github-django-actions/pull/3">Pull Request #3</a>, it got passed. After merge this is log of push event on develop branch <a href="https://github.com/nashmaniac/github-django-actions/runs/740611421?check_suite_focus=true">workflow run</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHUoRhNWn9aHrsWK5syvSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eexqciKXzt2n8RwFPwHcpA.png" /><figcaption>Left: Develop Branch Merged Workflow Run Right: Deployed Workload in Google Cloud (Check the commit id and active revision summary id)</figcaption></figure><ul><li>Let’ check what is deployed actually. I have a Cluster IP Service in kubernetes so I will just port forward it to my local port and check. First connect to kubernetes cluster and run kubectl port-forward svc/app-service 8000:80</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EysWqezeFMNKe9LP_lu49A.png" /><figcaption>Response from Home page after port forwarding</figcaption></figure><ul><li>Let’s submit a pull request from develop to master and approve that.</li><li>Merged to master worked fine but workflow failed due to some nasty helm upgrade issue. But I was glad at this point to show you guys the real scenario. Because in real life troubleshooting the bugs in your code is the most common thing we encounter everyday.</li></ul><p>So at last it worked on <a href="https://github.com/nashmaniac/github-django-actions/pull/7">Pull Request #7</a>. The merged code from develop to master ran successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uD5MIaTonI8HJLkBlaZZCA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5QbDw7KGhjjcHZ3LlCW1Nw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UyzVNv4bUHdZjgwP2AJmjA.png" /><figcaption>Left: Pull Request #7 (before Merge) Center: Push Workflow Log Right: Deployment in Google Cloud</figcaption></figure><p>What I really want to highlight here is <a href="https://github.com/nashmaniac/github-django-actions/commit/65d2b9f0837b0272e8a08212d6194b4a2afd6c36">65d2b9f</a> is the short sha of the master branch commit and the docker image deployed in our cloud is also app:sha-65d2b9f which actually means our built image is deployed successfully.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3ohzdIuqJoo8QdKlnW%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Freactionseditor-yes-awesome-3ohzdIuqJoo8QdKlnW&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3ohzdIuqJoo8QdKlnW%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="200" frameborder="0" scrolling="no"><a href="https://medium.com/media/82affdf317e732a79c34adf3102ec317/href">https://medium.com/media/82affdf317e732a79c34adf3102ec317/href</a></iframe><p>So finally all worked out. Yay !!!</p><p>I was super excited while I was planning to write this article. Yes, faced few impediments during writing this due to helm related issue a bit. But I am glad now that I could finish this. You can find all the codes in <a href="https://github.com/nashmaniac/github-django-actions/tree/v1">here</a>.</p><p>Lastly, you can do amazing things with GitHub workflow like opening an issue if workflow fails and assign them to developers, can use CODEOWNERS feature to automatically request review from users and can post slack message too in your slack. This simple automation can make your life lot easier. Hopefully I will write another one on this.</p><p>Stay fine and take care of yourself during this COVID lockdown.</p><p>Happy Coding !!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d48d6f00abf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/intelligentmachines/github-actions-end-to-end-ci-cd-pipeline-for-django-5d48d6f00abf">GitHub Actions: End to end CI/CD Pipeline for Django</a> was originally published in <a href="https://medium.com/intelligentmachines">intelligentmachines</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Github Actions: Basics]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/intelligentmachines/github-actions-basics-40a4d9b417f8?source=rss-1af58e55ea31------2"><img src="https://cdn-images-1.medium.com/max/1174/1*rcuhmTxoiA1KRJQzBKJpAw.png" width="1174"></a></p><p class="medium-feed-snippet">In modern days developing softwares without a CI/CD pipeline is unthinkable. And if it can be tied up with your version control system&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/intelligentmachines/github-actions-basics-40a4d9b417f8?source=rss-1af58e55ea31------2">Continue reading on intelligentmachines »</a></p></div>]]></description>
            <link>https://medium.com/intelligentmachines/github-actions-basics-40a4d9b417f8?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/40a4d9b417f8</guid>
            <category><![CDATA[workflow]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Mon, 01 Jun 2020 15:29:52 GMT</pubDate>
            <atom:updated>2020-06-02T02:46:18.247Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Istio — Part 2: Traffic Management, Consistent Hashing, Canary & Dark Releases]]></title>
            <link>https://medium.com/intelligentmachines/istio-part-2-traffic-management-consistent-hashing-canary-dark-releases-38ee0334321a?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/38ee0334321a</guid>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[service-mesh]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[istio]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sat, 09 May 2020 04:22:27 GMT</pubDate>
            <atom:updated>2020-05-09T16:25:58.729Z</atom:updated>
            <content:encoded><![CDATA[<h3>Istio — Part 2: Traffic Management, Consistent Hashing, Canary &amp; Dark Releases</h3><p>So yeah, I was talking about discovering your api via Istio service mesh in my blog <a href="https://medium.com/intelligentmachines/discovering-services-with-istio-9c48affca17c">Istio — Part 1: Discovering Services</a>. I hope you all read that blog. It will really help you develop your understanding on Istio and some related topic like Gateway, Virtual Services e.t.c. I tend to keep my cloud resources running for sometime so that you all can get my example with code which might be helpful. So I will not tear down the microservice architecture we build on earlier.</p><p>Rather we will write all the codes in a different namespace and will redirect different domains to achieve the same thing as we have in part 1 then move onto our today’s discussion.</p><p>We will be keeping our code in <a href="https://github.com/nashmaniac/istio-demo">here</a>. So what I did is</p><ul><li>Copied the contents fromk8s/1-discovering-services to k8s/2-traffic-mangement folder.</li><li>Changed the name of namespace from default to traffic-management</li><li>Edited each .yaml file to include namespace with value traffic-management Alternatively you can do it by running kubectl config set-context --current --namespace=traffic-management</li><li>We also replaces the domain name part microservice to traffic-mangement So user.microservice.retailvoice.xyz will make another copy with name user.traffic-mangement.retailvoice.xyz</li></ul><p>At this point, we are all set up with micro services deployed in traffic-management namespace exactly identical to our part-1 discussion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SSbBrvQSNduoHkCTBRG2wQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2daLcAE5FF0ey0vUSQcHeQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Y92V3dBBHPS27KG9jBlrg.png" /><figcaption>Deployment snaps from traffic-management namespace</figcaption></figure><h3>Goal</h3><p>We will break down our in different use case and we will change our .yaml config to address each use case.</p><ul><li>Traffic Management</li><li>Canary Release</li><li>Consistent Hashing</li><li>Dark Releases</li></ul><h3><strong>Traffic Management</strong></h3><p>So what is traffic management? The following example is completely fictional, just to give a glimpse of the use case we will be handling. Let’s think we have our user microservice with / route. For different reasons, we want to have a new response format for / route. But we want this version to be available for a certain portion of our user. Let’s say 10%. How could we handle that?</p><p>First of all, we can have a nasty fix. We can have two version of docker image for user micro service named user:v1 and user:v2 Now setting replica count in user:v1 deployment file to 9 and user:v2 deployment file to 1 is enough to achieve the upper assumption. As Kubernetes by default follows ROUND ROBIN method for traffic redirection so 9 pods of user:v1 as opposed to 1 pod of user:v2 will have the correct ratio split. But see, we have 8 redundant pods for user:v1</p><p>But with Istio we can easily have 1 pod of both user:v1 and user:v2 and still get our desired output. Let’s see how to do this.</p><ul><li>First change the k8s/2-traffic-management/2-user-microservice.yaml to add new deployment for user:v2 image and change the image of existing deployment to user:v1</li><li>Add version under labels for both deployment and add another env variable named VERSION</li><li>Set replica count to 1 in each deployment. You can increase if you need. I will use 1 just to demonstrate.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c355b83607b0a91001e600a60cfb54c6/href">https://medium.com/media/c355b83607b0a91001e600a60cfb54c6/href</a></iframe><p>Now apply it using kubectl apply -f k8s/2-traffic-management/2-user-microservice.yaml</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hm4lby0Mevtou7-EJRONBQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xnsvCB5Is3vjQJ-4AiluZQ.png" /><figcaption>Pod List and curl output</figcaption></figure><p>See now we have two pods for user microservice, one with user:v1 &amp; user:v2 And also in the second picture we are seeing 50–50 split in between two pods which is taken care of kubernetes by default. Let’s write a <strong>DestinationRule</strong> to route the traffic.</p><h3>Destination Rule</h3><p>Destination Rule simply define policies where to forward traffic after routing has occurred in service. We will make subsets of our user pods by utilizing the version label we added and then we will use it to redirect traffic in the virtual service named traffic-management/user-virtual-service resource. Let’s edit k8s/2-traffic-management/5-user-virtual-service.yaml</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e04c89b1fe2dbf7f39dcba5b76daf09d/href">https://medium.com/media/e04c89b1fe2dbf7f39dcba5b76daf09d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jwLDClu_DFmRRHQF3r0Y5A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dw_bSBs9BmsbT4qBnI5oHQ.png" /><figcaption>left: curl output of user.traffic-management.retailvoice.xyz &amp; right: curl output of traffic-management.retailvoice.xyz/user</figcaption></figure><p>Now you can see our one domain is working perfectly but other is not that because traffic-management.retailvoice.xyz virtual service is still using the default routing protocol. Let’s make changes in file k8s/2-traffic-management/7-microservice-virtual-service.yaml</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad4501a5f0feae87ea38a64551187210/href">https://medium.com/media/ad4501a5f0feae87ea38a64551187210/href</a></iframe><p>So after applying kubectl apply -f k8s/2-traffic-management/7-microservice-virtual-service.yaml we get this output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWGuKJvPgFFjkGIkrr8qzg.png" /><figcaption>fixed curl output http://traffic-management.retailvoice.xyz/user</figcaption></figure><p>So looks like both our <a href="http://user.traffic-management.retailvoice.xyz/">http://user.traffic-management.retailvoice.xyz</a> and <a href="http://traffic-management.retailvoice.xyz/user">http://traffic-management.retailvoice.xyz/user</a> are having 90–10 split in user:v1 and user:v2 pod as expected.</p><h3>Canary Release</h3><p>As I came across this quote</p><blockquote><strong>Canary release</strong> is a technique that is used to reduce the risk of introducing a new software <strong>version</strong> in production by gradually rolling out the change to a small subgroup of users.</blockquote><p>So above example actually depicts this canary release scenario. Here we only exposed 10% of whole traffic to a service user:v2 and the rest 90% to user:v1 Thus by simply defining a destination rule, adjusting weight ratio and integrating with virtual service will allow to make weighted routing which is also known as <strong>Canary Release.</strong></p><p>As you deploying canary deployment now, soon you will find yourself receiving complains from users on having inconsistent response from user microservice. That because though the routing is weighted that doesn’t actually track which client is actually receiving response from which pods. So a client might make 10 requests and end up served with 8 requests from user:v1 pod and other 2 from user:v2 pod. Remember, <strong>Istio so far took care of server requests weighting but could not stick a request to a client.</strong></p><p>So, what’s the solution? Now here comes <strong>ConsistentHashing</strong></p><h3><strong>ConsistentHashing</strong></h3><p>Consistent Hashing is a mechanism that will allow you to stick a request to a client by generating a hash internally. So if a user requests user microservice and get response from user:v2 pod then in the subsequent requests will also be forwarded to that user:v2 pod.</p><p>But here’s a catch. <strong>ConsistentHashing doesn’t work with weighted routing.</strong> Because, we define hashing mechanism under trafficPolicy in DestinationRule Before generating hash the weighted routing already made sure which service to hit. One more thing, the underlying component of <strong>Istio, Envoy proxy so far doesn’t support weighted routing with consistent hashing.</strong> You can find detail in <a href="https://github.com/envoyproxy/envoy/issues/8167">this github issue</a>.</p><p>So how do we achieve consistent hashing? We have to remove the weighted routing from virtual services and then declare consistentHash mechanism in the <strong>DestinationRule.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/24a0479b2bd0407f2fe6c05cc91d0017/href">https://medium.com/media/24a0479b2bd0407f2fe6c05cc91d0017/href</a></iframe><p>Here we removed the subset v1 and v2 from the <strong>DestinationRule</strong> and added all subset which will have label app: user as we will be targeting all user pods and requests will be redirected based on httpHeaderName with key name my-header</p><p>Now let’s see the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RDN_s6t9rmbFLiIy16pw3Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3edvI18NnER3KsFiZxPsEw.png" /><figcaption>left: my-header=668768768 right: my-header=1233456</figcaption></figure><p>See how the left one with my-header=668768768 redirects to user:v1 and my-header=1233456 redirects to user:v2 Thus we can easily figure out consistent hashing and give your end user a smooth experience.</p><p>Now let’s fix so that consistent hashing also works for <a href="http://traffic-management.retailvoice.xyz/user">http://traffic-management.retailvoice.xyz/user</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f1466c16f97aac1d9917ec8031a00401/href">https://medium.com/media/f1466c16f97aac1d9917ec8031a00401/href</a></iframe><p>Let’s verify if it keeps on sticking to a pod for <a href="http://traffic-management.retailvoice.xyz/user">http://traffic-management.retailvoice.xyz/user</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bnqDzns6o-zDAciiEAj_Mw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eO1HiatzK1tawUAA6qJykA.png" /></figure><p>See as expected the headermy-header=668768768 redirects to user:v1 and my-header=1233456 redirects to user:v2</p><p>One more thing to remember, <strong>You have to forward all the headers from one microservice to another if consistent hashing is based on httpHeaderName otherwise when you make call to another service from your if you don’t pass the header that you receive then you might not get consistent hashing.</strong></p><p>So finally I will wrap this up with Dark Release</p><h3>Dark Release</h3><p>Keeping both staging server and production server with almost same configuration is both costly effort wise and financially. What if I say, we can release our dev code to production without letting the end user know about it so that only developers and engineers can access it. And when the feature is ready we can tweak some istio config and the feature will be live right away. This mechanism is call <strong>Dark Release</strong>. This might sound theoretically very bad but we all know we have done quite a lot of times in our career.</p><p>Remember, in our first part we made a routing based on matching routes. So for matching route we used uri Istio provides many more other that just uri Today I will talk about how you can achieve dark release using http header.</p><p>Let’s say we will pass a header with key mode and value dev So if any request contains a header with mode=dev then it will be forwarded to user:v2 otherwise by default it will be forwarded to user:v1</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4f5e6d9bba20d51b2002e8d28540330/href">https://medium.com/media/a4f5e6d9bba20d51b2002e8d28540330/href</a></iframe><p>See we are matching our routes based on header and at the end defined a default route for version 1. So now the dev can easily make requests using http header where this pod will never be exposed to end user as the end user’s api will not include header mode=dev</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5UzYHRYYrwwaUQ6-yosGyw.png" /><figcaption>curl output of header and non-header request for dark release</figcaption></figure><p>Cool, huh?</p><p>Let’s do this for <a href="http://traffic-management.retailvoice.xyz/user">http://traffic-management.retailvoice.xyz/user</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/556aa8f4ef83af94519e9aae7ec018d2/href">https://medium.com/media/556aa8f4ef83af94519e9aae7ec018d2/href</a></iframe><p>Apply the above yaml and below is the output of it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c5bizlhoN3KUmuHqXkALcA.png" /><figcaption>curl output of header and non header requests</figcaption></figure><p>So, Yay !!!</p><p>Finally we can wrap this up. I know it was a tiring one. I will upload the codes in <a href="https://github.com/nashmaniac/istio-demo/tree/2-traffic-management">here</a> and also will add a <a href="https://github.com/nashmaniac/istio-demo/tree/2-traffic-management/k8s/2-traffic-management">README.md</a>. If you try at my server you might not get the expected result because at the end only dark release update is there. So you won’t find traffic-management and session stickiness example in my server I hosted. But if you follow the instruction, I can assure you will easily get up and running. If you face any problem drop a comment in here or mail me.</p><p>Good bye till the next one.</p><p><strong>Happy Coding !!!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38ee0334321a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/intelligentmachines/istio-part-2-traffic-management-consistent-hashing-canary-dark-releases-38ee0334321a">Istio — Part 2: Traffic Management, Consistent Hashing, Canary &amp; Dark Releases</a> was originally published in <a href="https://medium.com/intelligentmachines">intelligentmachines</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Discovering Services with Istio]]></title>
            <link>https://medium.com/intelligentmachines/discovering-services-with-istio-9c48affca17c?source=rss-1af58e55ea31------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c48affca17c</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[istio]]></category>
            <dc:creator><![CDATA[Raju Ahmed Shetu]]></dc:creator>
            <pubDate>Sat, 02 May 2020 11:14:18 GMT</pubDate>
            <atom:updated>2020-05-08T17:45:13.991Z</atom:updated>
            <content:encoded><![CDATA[<h3>Istio — Part 1: Discovering Services</h3><p>As we start building applications with microservice architecture sooner or later we always find ourselves in a spot where we want to visualize and monitor our entire deployment in a single screen. Also debugging a bug in microservice is really complex and time consuming. To ease it up, everyone felt the need of observability and controllability of their distributed architecture. <a href="https://istio.io/">Istio</a> is one of the tools that actually solves these problems.</p><p>So here I will try to discuss what is Istio and will take a use case based approach to see most of the features of Istio in here and upcoming blogs.</p><h3>What is Istio?</h3><blockquote>Istio is a service-mesh.</blockquote><p>I think this is most simplest and wholesome definition I have come through so far. Let me break this down. Istio gathers a set of tools like <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://grafana.com/">Grafana</a>, <a href="https://prometheus.io/">Prometheus</a> and many others to enable developers to write code in for controlling distributed architecture in a simple manner. Why need Istio, when I have these tools to use independently. <strong>Yes</strong>, you can. But obviously you don’t want it. Istio abstracts all the inner-workings of those tools and made it easy for developers to work with.</p><p>Istio architecture consists of four components.</p><ul><li><strong>Envoy</strong>: It’s the only part of the data plane in Istio. It will be injected as a sidecar proxy to each pod and now any request will be incoming to or outgoing from a pod will pass through Envoy Proxy. Envoy was built by <a href="https://www.lyft.com/">Lyft</a>. You can edit Envoy proxy and can build your own service mesh but I will highly discourage you to do so.</li><li><strong>Pilot</strong>: Responsible mostly service discovery and traffic management. It is a component of control plane.</li><li><strong>Citadel</strong>: Maintains service-to-service and end user authentication by built in authentication and credentials managements. Handles certificates to enable https connection throughout the service-mesh. It is also a part of control plane.</li><li><strong>Galley: </strong>Your Trusted Buddy !!! Does all the validation, parsing and ingestion of yaml configuration we write to make changes in Istio service mesh. It is also a part of control plane.</li></ul><p>Istio is platform independent, can be deployed to Kubernetes, Nomad, Consul. But so far, Kubernetes is the most popular to use for distributed architecture application. Using Istio with Kubernetes is a powerful tool that is always gives edge of simplicity in system management and will ensure a good night sleep.</p><h3>Endgoal</h3><p>Let’s see the use case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t4k0F4jwRMetOZ9-ujhs5A.png" /><figcaption>Expected Architecture</figcaption></figure><p>Let’s say we have this picture for our architecture. In today’s discussion we will use just only the traffic management portion of Istio.</p><p>Here at the very below we have our kubernetes deployment as usual. So what we want is this</p><ul><li>We have two types of multiple pods deployed for docker imageuser and product</li><li>For user we have / and /about route and for product we have / and /latest route</li><li>We also have our service in place one for catering all user pods and other for product pods</li><li>What we want is when we request <a href="https://user.microservice.retailvoice.xyz/">http://user.microservice.retailvoice.xyz</a> then we should be redirected to the user pods</li><li>when we request <a href="https://user.microservice.retailvoice.xyz/">http://product.microservice.retailvoice.xyz</a> then we should be redirected to the products pods</li><li>Lastly we want to have path based routing so when we request <a href="http://microservice.retailvoice.xyz/user">http://microservice.retailvoice.xyz/user</a> and <a href="http://microservice.retailvoice.xyz/product">http://microservice.retailvoice.xyz/product</a>, we want to be redirected to user and product pods respectively</li></ul><h3>Requirements</h3><ul><li>We need to have istioctl installed in your pc. For mac, I used brew install istioctl You can find the installation guide <a href="https://istio.io/docs/setup/install/istioctl/">here</a></li><li>Check the version using istioctl version --remote=false</li><li>Connect your kubernetes cluster via your terminal. Use kubectl config current-context to see if you are in the current cluster</li><li>Let’s install Istio in our cluster. Run istioctl manifest apply --set profile=demo</li><li>If installed successfully, you can check the pods by kubectl get pods -n istio-system</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*buH9yJBBG34w_E2cjEFgEg.png" /><figcaption>Istio Installation Log</figcaption></figure><h3>Steps to success</h3><ul><li>I will be uploading all the codes in <a href="https://github.com/nashmaniac/istio-demo">here</a>. So I cloned it and go into the project directory via terminal.</li><li>Now let’s build the docker image for userusing docker build . -t your_image_name -f Dockerfile.user &amp; publish it to docker registry using docker push your_image_name</li><li>Do the same for product using docker build . -t your_image_name -f Dockerfile.product and publish it to docker registry using docker push your_image_name</li><li>As we said, we need istio proxy per pod to track the requests and other istio functionality. So we need to enable istio-injection per namespace basis. As we will be working in default namespace so let us modify the labels of it via yaml file and use kubectl apply -f k8s/1-discovering-services/1-namespace.yaml (I am assuming we are in work directory)</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7076d7783eae6dbacad4b3254a7b1520/href">https://medium.com/media/7076d7783eae6dbacad4b3254a7b1520/href</a></iframe><ul><li>Now let’s run kubectl apply -f k8s/1-discovering-services/2-user-microservice.yaml which will create user pods and user-service that will handle requests to user pods.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0560c8f3591e08e5115ec3a55699751c/href">https://medium.com/media/0560c8f3591e08e5115ec3a55699751c/href</a></iframe><ul><li>Now let’s run kubectl apply -f k8s/1-discovering-services/2-product-microservice.yaml which will create product pods and product-service that will handle requests to product pods.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/949ce37e1f37a8a0911003d9facc1510/href">https://medium.com/media/949ce37e1f37a8a0911003d9facc1510/href</a></iframe><ul><li>At this point you have a simple kubernetes deployment with proxy per pods basis and cluster ip services to the handle the request. You can simply connect to cluster ip service with port forwarding at point. But we will do better than that. Now we will make some Istio magic to expose our services to the world.</li></ul><h3>Gateway</h3><p>A Gateway is a service that handles the incoming requests to a service mesh. When we installed Istio in our cluster, by default we have a ingressgateway service in istio-systemnamespace.</p><ul><li>Get the list of gateways by running kubectl get gw -n istio-system</li><li>Get the details by kubectl get gw ingressgateway -n istio-system -o yaml</li></ul><p>At this point we will be deleting the default ingressgateway by running kubectl delete gw ingressgateway -n istio-system</p><p>Now let’s write our own gateway. What we will do is we will have a server configuration on port 80 which will accept HTTP request for 3 hosts named microservice.retailvoice.xyz, user.microservice.retailvoice.xyz and product.microservice.retailvoice.xyz.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3c4044ed4d6235ac8164c93f404909c/href">https://medium.com/media/d3c4044ed4d6235ac8164c93f404909c/href</a></iframe><p>Now create our gateway by running kubectl apply -f k8s/1-discovering-services/4-gateway.yaml</p><h3>VirtualService</h3><p>A virtualservice defines some rules that will be applied when the host is addressed. As we have three hosts, we will be writing 3 virtual services.</p><p>We will just forward all the traffic to our cluster ip service when user.microservice.retailvoice.xyz is requested.</p><p>I request to go through the code snippets. Most of the things are commented</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2bb8fce28f013985f84f64cdec207b3f/href">https://medium.com/media/2bb8fce28f013985f84f64cdec207b3f/href</a></iframe><ul><li>Create the virtual service using kubectl apply -f k8s/1-discovering-services/5-user-virtual-service.yaml</li></ul><p>Do the same for product virtual service</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d48f67ec73df3114651f5efddc7c2cf0/href">https://medium.com/media/d48f67ec73df3114651f5efddc7c2cf0/href</a></iframe><ul><li>Create the virtual service using kubectl apply -f k8s/1-discovering-services/6-product-virtual-service.yaml</li><li>You can list all the virtual services by using kubectl get vs</li></ul><p>At this point we need to sort out our dns records. By default with Istio installation we get a load balancer istio-system namespace named istio-ingressgateway We can run kubectl get svc istio-ingressgateway -n istio-system to get the external address of the load balancer. Then change the dns record appropriately to point to external address.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hGTgYDQrq54TWqECL8StqQ.png" /><figcaption>istio-ingressgateway</figcaption></figure><p>I have forwarded my 3 domains to the external ip of the load balancer in my dns manager.</p><p>Now if you want to request the <a href="http://user.microservice.retailvoice.xyz/">http://user.microservice.retailvoice.xyz</a>, <a href="http://user.microservice.retailvoice.xyz/about">http://user.microservice.retailvoice.xyz/about</a>, <a href="http://user.microservice.retailvoice.xyz/">http://product.microservice.retailvoice.xyz</a> and <a href="http://user.microservice.retailvoice.xyz/about">http://product.microservice.retailvoice.xyz/l</a>atest then we should get the responses. I will use curl for this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-YM67fsMxgYhxlugTNf--w.png" /><figcaption>logs for the curl</figcaption></figure><p>Now let’s move on matched route handling. For this let’s write a virtual service for microservice.retailvoice.xyz</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6699c103ea1a2ff6e5458c44ac1608d/href">https://medium.com/media/c6699c103ea1a2ff6e5458c44ac1608d/href</a></iframe><p>Now deploy the virtual service with matched routes to cluster by kubectl apply -f k8s/1-discovering-services/7-microservice-virtual-service.yaml</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ikhpNBL2X2B5eo-XkBTcTw.png" /><figcaption>logs for matching route curl</figcaption></figure><h3><strong>Visualization</strong></h3><p>It’s always better to visual how my distributed system works. <strong>Kiali</strong> is shipped with Istio just to do that. When you install Istio, you get <strong>Kiali</strong> by default.</p><ul><li>Start Kiali dasboard by running istioctl dashboard kiali</li><li>Login using username admin and password admin These are the default one. You can change it if you want.</li></ul><p>Make some requests to the urls and in the graph section under Workload Graph you might follow something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HTwA2yGjxScMSfDctskpwQ.png" /><figcaption>Kiali Dashboard</figcaption></figure><h3>Last Words</h3><p>Looks like it’s all worked. Yay !!!</p><p>You can find all the code in the <a href="https://github.com/nashmaniac/istio-demo">repo</a> with tag 1-discovering-services I am also adding quick link for the tag <a href="https://github.com/nashmaniac/istio-demo/tree/1-discovering-services">here</a>. I am planning to write my next blog on weight routing, canary releases, consistent hashing and dark releases using Istio. Please subscribe to get latest updates.</p><p><strong>Happy Coding !!!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c48affca17c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/intelligentmachines/discovering-services-with-istio-9c48affca17c">Discovering Services with Istio</a> was originally published in <a href="https://medium.com/intelligentmachines">intelligentmachines</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>