<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tait Hoglund on Medium]]></title>
        <description><![CDATA[Stories by Tait Hoglund on Medium]]></description>
        <link>https://medium.com/@tait.hoglund?source=rss-efc3aac42050------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cd5_TLGk4Ht0xKb1THsmAA.jpeg</url>
            <title>Stories by Tait Hoglund on Medium</title>
            <link>https://medium.com/@tait.hoglund?source=rss-efc3aac42050------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:10:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tait.hoglund/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Launching to Production for Site Reliability Engineers]]></title>
            <link>https://medium.com/@tait.hoglund/launching-to-production-for-site-reliability-engineers-14f7f50172ac?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/14f7f50172ac</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[source-code]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Tue, 13 May 2025 21:20:37 GMT</pubDate>
            <atom:updated>2025-05-13T21:20:37.771Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2YFJ0BRv-Lp7AE8YT7mp6w.png" /></figure><p><em>Over my career, I’ve had a number of chances to support the release of applications into public environments. Over each release, I have learned a lot about what can go wrong and how to prevent it.</em></p><p><em>I’m starting a series of articles documenting how I’ve supported those releases to ensure high availability, low error rates, and low latency on public facing applications to ensure an excellent experience for both developers and users.</em></p><p><em>Whether you’re a junior level SRE looking for guidance supporting your first release into production or a seasoned pro looking to sharpen your skills, I’m here to share my experiences across a number of platforms to help you think, plan, and execute better when it comes to your deployments.</em></p><p><em>Consider this part one of a multi-part series. As I expand this series out, I’ll be sure to include links to additional walk-thrus so you can feel free to skip around to a particular area that suits your interests.</em></p><h3>Phase 0 — Scoping the Project</h3><p>As the SRE, my goal is to ensure the stable and reliable release of software and and to develop the infrastructure neeed to support that application.</p><p>To achieve that, there are a number of tools needed to accomplish that goal. Here are my ‘must haves’:</p><ul><li>Centralized Source Control of the Application Code</li><li>CI/CD Pipelines to automate the build, testing, and deployment of that code</li><li>Scalable and secure infrastructure</li><li>Proper ACLs to enable Security</li><li>Caching for better application Performance</li><li>Performance Monitoring and Incident Response</li><li>Logging and Infrastructure Metrics</li></ul><h3>Centralized Source Control of the Application Code</h3><p>Having a centralized repository for your code is great for multiple reasons:</p><p>#1. Keeps a record of every code commit in your project to track changes should a reversion be needed.</p><p>#2. Allows multiple developers to work from the same code base at the same time.</p><p>#3. Ensures proper controls of code review prior to deployment into staging and production environments.</p><p>#4 Allows the automatation of code testing and builds</p><p>#5 Centralizing the configuration and secrets needed by the application</p><p>There are a number of great ways to centralize your application’s code — Github, Gitlab, Bitbucket, AWS CodeCommit, etc.</p><p>You can even host your own machine as a centralized spot for your code. The goal is to find a solution that fits your usecase.</p><p>I’ve set up projects using multiple source control tools but for the purposes of this project, I’m going to walk thru how to set up a remote repository in gitlab and the best practices I use.</p><h3>Prerequisites for this Walk-Thru</h3><ul><li>A gitlab account. Gitlab offers a free account. This walk-thru will be using the enterprise version of gitlab, but the process should be the same if you were to use a free account.</li></ul><h3>Setting up Source Control of your Application Code in Gitlab</h3><ol><li>Login to gitlab with you account.</li><li>In the upper-right corner, click the + and in the drop-down select New Project/repository</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*xm2g3HgCDHMcf5aRG_3bQQ.png" /></figure><p>3. Gitlab as a great feautre that allows you to create project templates so that if you are creating new projects from scratch, you can start from your standard base. For this walk-thru, we’ll create using a blank project</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tXKKbGYUcAbVpEFoW2Yv-g.png" /><figcaption>create using a blank project</figcaption></figure><p>4. Give your project a useful name. Avoid using blank spaces as it can cause problems with automated scripts and commands later on. You can configure a custom project slug if you desire. The visibility setting can be set based on your needs. It’s always good to initialize a project readme that the developer can fill in later. Click create project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V_PUZD6xbl2wOrc0zuSc3w.png" /></figure><p>5. What you should see once the project is created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ov_nX17YhFOgm5ZXGXPuzw.png" /></figure><h3>Create proper branch and merge request rules within the gitlab project settings</h3><p>Branches in source control reflect the current state of the application code in a particular environment. When code changes get commit to a particular branch in the repository, the application state changes and is altered.</p><p>Not all code changes may affect visable changes that the user can detect and see. But each code change should be properly vetted and tested in an isolated environment.</p><p>At my current organization, I maintain the following environments for our application code:</p><ol><li>development- this is used for developers to review their code and see their changes in an isolated development environment as they code prior to merging their changes up into a staging environment for broader QA testing.</li><li>staging — for QA testing of code prior to deployment intoproduction . Changes here are deployed into a seperate production-like staging environment where more extensive QA testing occurs.</li><li>production — this is our live production code and what our public facing users are interfacing with.</li></ol><p>From a source control perspective, these isolated environments that represent various states of your code are contained within the branches of your project.</p><ol><li>developement branch — This is what your developers will branch off of to make their code changes. (ex: development/feature:new_contact_method)</li></ol><p>When they push their changes to the remote gitlab repository, we will see later how we can use a pipeline to automate the deployment of their code into a review app where the developer can see the impact of their code deployed into the application.</p><p>Once they feel confident in their changes via their review app, they can open a merge request to merge their feature branch into the staging branch for further QA.</p><p>2. staging branch — This can also be called integration , this contains the application code to be deployed to a staging environment for further QA and testing.</p><p>3. main or master branch — Code that passess QA within the integration branch gets merged up into main for deployment into production.</p><p>You can set up the branches you need via the code menu in gitlab:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AzpyPoK8iAc7907Uerrwow.png" /></figure><h3>Why Branch and Merge Request Rules Matter</h3><p>Most remote source control tools allow you to control who and under what circumstances can code pushes be made and into which branches. You can imagine a situation where a developer might do the following:</p><pre>git pull main<br>git checkout -b quick_fix<br><br>## Developer makes code changes<br><br>git add .<br>git commit -m &quot;quick bug fix in prod&quot;<br>git push -u origin</pre><p>If you can’t figure out what would happen, this developer just pushed code changes and deployed directly to production.</p><p>To protect this from happening, you institute merge request rules and protected branch status in your gitlab project:</p><p>Let’s start by instituting merge request rules in our project: From the Settings menu, click merge requests:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oJeEQTqeoYTWqdVtJyklfw.png" /></figure><p>Click add approval rule :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ICKoixP-d5icptcGmdM7Ew.png" /></figure><p>I like to select all protected branches, which we will configure after this step.</p><p>If this is your first merge request rule, then you will need to give it a name.</p><p>Elect the number of approvers for the number of engineers you will require approval from before a merge can happen.</p><p>Add the users whom you will designate as reviewers for this rule. Finally click save changes .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1w4FYxm5mg4HBE9XL4HI_g.png" /></figure><p>Here is where I also like to validate the approval settings to prevent an author of merge request from self-approving their own code.</p><p>Gitlab enables this by default, but it’s good to check that these boxes are checked here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UIFDp6InQvFZAN83ktsM8A.png" /><figcaption>Validation that your merge request rule is now in effect.</figcaption></figure><h3>Establishing Protected Branches</h3><p>As discussed before, this will depend on your setting which branches need protecting to prevent accidently or lazy development work. From within the same settings menu, select repository :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HmEjhxTT1cuv4Bi9dWVZow.png" /></figure><p>Scroll to the Protected branches section and establish protections for each branch that corresponds to an environment you need code review for. You will need to repeat this for each branch that needs these protections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W3k0-aZ4WqzCZlytYdjmGg.png" /></figure><p>Once complete all of your protected environments will be protected and ensure proper code review happens before deployment to live environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T53hZoeXxb5kFZ7NWYh05A.png" /></figure><p>That’s all for Part One. In part two, I will be covering the following:</p><ol><li>Setting up a Gitlab Runner to automate the testing and deployment of our application code into a live review development environment</li><li>Setting up the CI/CD Pipeline to automate the testing and deployment of the application code.</li><li>Deploying our code into development, staging, and production environments.</li></ol><p>Until next time, onward!</p><p>If you liked this article, you might like an earlier one I wrote:</p><p><a href="https://medium.com/nerd-for-tech/empowering-your-k8s-cluster-93d23db95fe9">Gitlab Project &amp; GKE Integration</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*GRfI42Vtiap3RfGKYjKJ3Q.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14f7f50172ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to migrate Office 365 Webhook Connectors to Workflow Webhooks in Microsoft Teams]]></title>
            <link>https://medium.com/@tait.hoglund/how-to-migrate-office-365-webhook-connectors-to-workflow-webhooks-in-microsoft-teams-fa117900decc?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa117900decc</guid>
            <category><![CDATA[webhooks]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[workflow-automation]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Mon, 19 Aug 2024 03:03:12 GMT</pubDate>
            <atom:updated>2024-08-27T12:47:14.843Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gSIudH2lf8Ko2HtZYoj_wQ.png" /></figure><p><a href="https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/#:~:text=Starting%20August%2015th%2C%202024%20we,%2C%20flexible%2C%20and%20secure%20way.">Microsoft announced</a> that they are retiring the Office 365 connectors feature.</p><p>My organization uses Webhook connectors to automate the delivery of critical notifcations from platforms we use into a desginated channel to keep our engineers up to date on critical happenings within our cloud platforms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p6qKLDPA72GjOPxgGRF9_A.png" /><figcaption>The Office 365 Connectors feature allows notifcations into Microsoft Teams Channels via a webhook. This feature is deprecated and existing webhook connectors will cease working by Dec 2025.</figcaption></figure><p>What hasn’t been provided by microsoft is a clear path on <em>how to replace existing webhooks that are instituted by the current connectors feature</em> into what Microsoft says is the path forward — power automation workflows.</p><p><a href="https://support.microsoft.com/en-us/office/post-a-workflow-when-a-webhook-request-is-received-in-microsoft-teams-8ae491c7-0394-4861-ba59-055e33f75498#:~:text=You%20can%20post%20to%20a,a%20webhook%20request%20is%20received.&amp;text=next%20to%20the%20channel%20or,a%20webhook%20request%20is%20received.">Microsoft offers a templated solution</a> for “Post to a channel when a webhook request is received” but offers little support for understanding how to formulate your adaptive card correctly to get your dynamic json payloads to display correctly in your channel:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/1*HjYwnl1tdr_b-iH4h3FXTw.png" /></figure><p>After scouring the Power Automate support forums, <a href="https://community.powerplatform.com/forums/thread/details/?threadid=09cf8aa0-224d-ef11-a317-00224832bdc3">posting my own struggles</a> to implement this new solution, I eventually figured it out and wanted to post my findings here to help others who might be struggling.</p><h3><strong>How to properly implement the Microsoft Teams Webhook Workflow to “Post card in a chat or channel”</strong></h3><h3>Prerequisites</h3><ul><li>An established Microsoft Account, Microsoft Team, &amp; Channel.</li><li>Your microsoft teams user account also needs admin permissions in order to create and configure the workflow for the channel you are trying to configure the webhook for.</li><li>External Platform access to paste the webhook into to link your alerts with your microsoft teams channel</li></ul><h3>Step 1: Creating your Workflow in Microsoft Teams</h3><p>Login to your Microsoft Teams account and enter the workflows app menu. There’s a couple ways you can get to it. By clicking ‘Apps’ in the sidebar menu, searching for it, or it might also already be saved as a menu item in your sidebar:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*pPZES2Y9C2VlWEYzYDtRYw.png" /></figure><p>In the workflow menu, click “+ New Flow”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KGD2s-do-uK0LGXGmkC6Ew.png" /></figure><p>At the time of writing, the “Post to a channel when a webhook request is received” is under the popular Teams templates, in case it’s not you can search for it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDqEx9goLT3Q5MMggVDf9Q.png" /></figure><p>Give your Flow a name, make sure you’re signed into Microsoft Teams, and click “Next”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*B8q7lr5NP_FP4dR_UFFAVg.png" /></figure><p>Then select your Microsoft Teams Team and the Channel that you want your webhook to post in, click “Create Flow”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*WUPzvNgyAa6K9Pt9mNTuSA.png" /></figure><p>Your workflow is now created, but we still need to configure it to work with our data correctly. Copy the webhook URL and save it for now:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*P-NyNBxxl1WhJNGcUARl0w.png" /></figure><p>This is as far as Microsoft gets you. But there are a couple items that are preventing your Workflow from working correctly if this is all you configure:</p><p><strong>Problem Area #1:</strong></p><ol><li>The JSON Payload Schema of your data that is received by your webhook.</li></ol><p>If you aren’t aware, platforms that send json data send it within a particular structure. This structure is called a ‘schema’.</p><p>The data itself is usually sent in JSON. But since machines aren’t very smart, engineers you will need to define the structure of your data so your microsoft teams workflow can understand how to format it.</p><p><strong>Problem Area #2:</strong></p><ol><li>The JSON structure of what is sent by your external platforms needs to be understood by your Adapative Card.</li></ol><p>Adaptive Cards provides a graphical presentation of that data in a way that is easily understood by people.</p><p>Without addressing both, even if you are able to get your Workflow to not error and actually post in your designated channel, instead of looking as you would expect it to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*IXpKqNqXKJtSrlQJnmWsOQ.png" /></figure><p>The data will be missing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*HjYwnl1tdr_b-iH4h3FXTw.png" /></figure><p>We need to configure our workflow to address both issues. Once we do, we will get our Post in our Teams channel working as expected</p><h3>Step 2: Addressing Problem Area 1 by Adding a Parce JSON Action to your Workflow</h3><p>By adding a Parce JSON Action to our workflow, we can see the data that is being ingested by our workflow to know how to structure both your schema and how to format your adaptive card.</p><p>I added a ‘Parse JSON’ action to my workflow between the two existing actions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iO0hzR6NaQysVYMUSuxxng.png" /></figure><p>At this point, I added my webhook URL that I was given when creating my workflow into my external platform wehbook integrations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*P-NyNBxxl1WhJNGcUARl0w.png" /></figure><p>This will depend on the platform that you are forwarding notifcations from, but once the webhook is saved, you should start getting notifcations as your webhook is triggerred by your external platform.</p><p>If you’re notifcations are posting as expected with all the data you expect, Great! You’re done! Or if you’re like me and was receiving none, only some, or no data at all, read further:</p><p>If you’re notifications are not posting in your Teams channel as expected, you can visit the ‘Run History’ of any workflow via the workflow console in Microsoft Teams, this will at least let you know that you’ve successfully integrated your webhook workflow within your platform.</p><p>Even after a couple triggers of your workflow URL, you can visit these logs to see where your workflow is failing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SjSYpO5KCdbSLS-OQoZ_LA.png" /></figure><p>The key here is to get a handle on how your data NEEDS to be structured or transformed so that your adaptive card can utilize the data effectively.</p><p>Once you have a couple of failed ‘runs’ of your workflow, examine any/all of your failed runs. With your Parce JSON action added, you will be able to see the data that is being ingested, in order to fine tune your schema. This will be listed under the ‘unputs’ of your Parce JSON action here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6VL1mvAxtALMe-CkNonFOw.png" /><figcaption>If your Parse JSON Action is failing due to an invalid Schema, Copy the Input JSON from a failed Job in the ‘Run History’ log of your Microsoft Teams workflow</figcaption></figure><p>If you’re schema isn’t configured to handle the data being ingested by the inputs, copy any input values in failed runs and look for commonalities.</p><p>In most instances unless you’re webhook is inconsistent data OR from multiple sources that are feeding different data structures, copying the ‘contents’ of your Parse JSON inputs should be enough.</p><p>Once you have the inputs copied, go back to your Workflow Menu (Teams doesn’t have an option to allow you to edit a workflow from the run history at the time of this writing), click edit:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xbGeysAcl6dc9jvROOrIeA.jpeg" /></figure><p>Click into your Parse JSON action within your workflow. Make sure the Conent section is dynamic content of the ‘BODY’ of your previous action — The ‘When a Teams webhook is recieved’ action:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LxvA4i_JMxr96lUTbyzNdw.png" /></figure><p>Next in the ‘schema’ section, click ‘Generate from Sample’ and PASTE in your copied contents from your failed job run:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*abPyLDJxuVjz2C3WHyDv0Q.png" /></figure><p>Click Save.</p><p>And again, allow your webhook to execute more runs. You should see your Parse JSON action now pass in your workflow as your webhook is envoked:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*2Z53_O2wpQ4TZ24-GSWBsw.png" /></figure><p>Addtionally, now that you have a sample JSON payload from failed runs, you don’t have to wait for your webhook to be invoked by your external platform.</p><p>You can use POSTMAN with a POST command to your webook and include your JSON payload within the body to test on demand and get feedback for your workflow in real time.</p><h3>Step 3: Addressing Problem Area 2 by Formatting your Adapative Card correctly.</h3><p>Much like our previous issue of NOT understanding the data that gets ingested, the data that gets passed to our ‘Apply to Each’ needs to be understood by our Adapative Card and our Adaptive Card needs to be formatted in a way that Microsoft Teams understands so that our notifcation gets posted with all the data and fields that we care about.</p><p>Microsoft has a great tool for this in <a href="https://adaptivecards.io/designer/">adaptivecards designer</a>. Which allows you to provide sample data into their input editor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rn9qycGmJEsuTzxpv-rpRw.png" /></figure><ol><li>Copy and Paste any JSON payloads received by your workloads to test in the ‘Sample Data Editor’</li><li>Paste in your Adaptive Card template in the Card Payload Editor</li><li>Preview what your adaptive card in Microsoft Teams will look like in the Preview Mode.</li></ol><p>Note: Make sure that you select ‘Microsoft Teams’ as your ‘host app’ and that you aren’t using a ‘target’ version not supported by Microsoft Teams.</p><p>Once you have your template made (chatgpt is your friend in this regard). Copy your adaptive card from the Adaptive Card designer into your Microsoft Teams “Apply to Each” action within your workflow:</p><p>Here is my adaptive card that i used for my use-case and a bit of an explaination why it’s designed the way it is:</p><pre>{<br>  &quot;type&quot;: &quot;AdaptiveCard&quot;,<br>  &quot;$schema&quot;: &quot;http://adaptivecards.io/schemas/adaptive-card.json&quot;,<br>  &quot;version&quot;: &quot;1.5&quot;,<br>  &quot;body&quot;: [<br>    {<br>      &quot;type&quot;: &quot;TextBlock&quot;,<br>      &quot;text&quot;: &quot;@{coalesce(triggerBody()?[&#39;title&#39;], &#39;No Title Provided&#39;)}&quot;,<br>      &quot;weight&quot;: &quot;Bolder&quot;,<br>      &quot;size&quot;: &quot;Medium&quot;<br>    },<br>    {<br>      &quot;type&quot;: &quot;TextBlock&quot;,<br>      &quot;text&quot;: &quot;@{coalesce(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;activityTitle&#39;], &#39;No Activity Title Provided&#39;)}&quot;,<br>      &quot;wrap&quot;: true<br>    },<br>    {<br>      &quot;type&quot;: &quot;FactSet&quot;,<br>      &quot;facts&quot;: [<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 1), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][0][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 1), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][0][&#39;value&#39;], &#39;&#39;)}&quot;<br>        },<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 2), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][1][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 2), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][1][&#39;value&#39;], &#39;&#39;)}&quot;<br>        },<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 3), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][2][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 3), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][2][&#39;value&#39;], &#39;&#39;)}&quot;<br>        },<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 4), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][3][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 4), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][3][&#39;value&#39;], &#39;&#39;)}&quot;<br>        },<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 5), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][4][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 5), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][4][&#39;value&#39;], &#39;&#39;)}&quot;<br>        },<br>        {<br>          &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 6), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][5][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>          &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 6), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][5][&#39;value&#39;], &#39;&#39;)}&quot;<br>        }<br>      ]<br>    }<br>  ],<br>  &quot;actions&quot;: [<br>    {<br>      &quot;type&quot;: &quot;Action.OpenUrl&quot;,<br>      &quot;title&quot;: &quot;View Incident Details&quot;,<br>      &quot;url&quot;: &quot;@{if(or(equals(triggerBody()?[&#39;potentialAction&#39;], null), equals(triggerBody()?[&#39;potentialAction&#39;][0], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0][&#39;uri&#39;], null)), &#39;https://defaulturl.com&#39;, triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0][&#39;uri&#39;])}&quot;<br>    }<br>  ]<br>}</pre><p>This Adaptive Card template is designed to display dynamic content within a Microsoft Teams message or any other platform that supports Adaptive Cards. The card is structured to adapt its content based on the data received in the JSON payload, using various functions and conditions to ensure that the information is presented even if some fields are missing. Here’s a detailed breakdown of each component:</p><h3>1. Adaptive Card Structure</h3><ul><li><strong>type:</strong> &quot;AdaptiveCard&quot; specifies that this is an Adaptive Card.</li><li><strong>$schema:</strong> This points to the Adaptive Card schema, which defines the card&#39;s structure and is required for validation.</li><li><strong>version:</strong> &quot;1.5&quot; specifies the version of the Adaptive Card schema being used.</li></ul><h3>2. Body</h3><p>The body is where the main content of the card is displayed. It contains various elements, such as TextBlock and FactSet, to present information dynamically.</p><h4>a. TextBlock — Title:</h4><pre>{<br>  &quot;type&quot;: &quot;TextBlock&quot;,<br>  &quot;text&quot;: &quot;@{coalesce(triggerBody()?[&#39;title&#39;], &#39;No Title Provided&#39;)}&quot;,<br>  &quot;weight&quot;: &quot;Bolder&quot;,<br>  &quot;size&quot;: &quot;Medium&quot;<br>}</pre><ul><li><strong>type:</strong> &quot;TextBlock&quot; indicates that this element is a block of text.</li><li><strong>text:</strong> The value is determined by the coalesce function:</li><li>coalesce(triggerBody()?[&#39;title&#39;], &#39;No Title Provided&#39;): This function checks if triggerBody()?[&#39;title&#39;] exists and is not null. If it exists, it displays the title; if not, it defaults to &quot;No Title Provided&quot;.</li><li><strong>weight:</strong> &quot;Bolder&quot; makes the text bold, emphasizing the title.</li><li><strong>size:</strong> &quot;Medium&quot; sets the font size to medium.</li></ul><h4>b. TextBlock — Activity Title:</h4><pre>{<br>  &quot;type&quot;: &quot;TextBlock&quot;,<br>  &quot;text&quot;: &quot;@{coalesce(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;activityTitle&#39;], &#39;No Activity Title Provided&#39;)}&quot;,<br>  &quot;wrap&quot;: true<br>}</pre><ul><li><strong>text:</strong> This uses a similar coalesce function to check for the activityTitle within the first section (sections[0]):</li><li>If activityTitle exists, it is displayed; otherwise, it defaults to &quot;No Activity Title Provided&quot;.</li><li><strong>wrap:</strong> true allows the text to wrap onto multiple lines if it&#39;s too long to fit on one line.</li></ul><h4>c. FactSet:</h4><pre>{<br>  &quot;type&quot;: &quot;FactSet&quot;,<br>  &quot;facts&quot;: [<br>    {<br>      &quot;title&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 1), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][0][&#39;name&#39;], &#39;&#39;)}:&quot;,<br>      &quot;value&quot;: &quot;@{if(greaterOrEquals(length(triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;]), 1), triggerBody()?[&#39;sections&#39;]?[0]?[&#39;facts&#39;][0][&#39;value&#39;], &#39;&#39;)}&quot;<br>    },<br>    // Additional fact items...<br>  ]<br>}</pre><ul><li><strong>type:</strong> &quot;FactSet&quot; is used to display a set of key-value pairs.</li><li><strong>facts:</strong> This is an array of facts, where each fact consists of a title and a value.</li></ul><p>Each fact is dynamically generated based on the data in the facts array within the first section (sections[0]). The logic uses conditional checks to determine if a fact exists and should be displayed:</p><ul><li><strong>title:</strong> The if function checks if there is at least one fact (greaterOrEquals(length(...), 1)):</li><li>If true, it displays the name of the first fact (facts[0][&#39;name&#39;]); otherwise, it displays an empty string.</li><li><strong>value:</strong> Similarly, the value field checks if the fact exists and then displays its value or leaves it empty.</li></ul><p>This pattern is repeated for up to six facts, making the card flexible to handle varying numbers of facts without breaking.</p><h3>3. Actions:</h3><pre>{<br>  &quot;type&quot;: &quot;Action.OpenUrl&quot;,<br>  &quot;title&quot;: &quot;View Incident Details&quot;,<br>  &quot;url&quot;: &quot;@{if(or(equals(triggerBody()?[&#39;potentialAction&#39;], null), equals(triggerBody()?[&#39;potentialAction&#39;][0], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0], null), equals(triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0][&#39;uri&#39;], null)), &#39;https://defaulturl.com&#39;, triggerBody()?[&#39;potentialAction&#39;][0][&#39;targets&#39;][0][&#39;uri&#39;])}&quot;<br>}</pre><ul><li><strong>type:</strong> &quot;Action.OpenUrl&quot; defines an action that opens a URL when clicked.</li><li><strong>title:</strong> &quot;View Incident Details&quot; is the text displayed on the action button.</li><li><strong>url:</strong> The URL is determined by an if function that checks if the potentialAction array and its nested properties exist:</li><li>If any part of the path to potentialAction[0][&#39;targets&#39;][0][&#39;uri&#39;] is missing, it defaults to &quot;https://defaulturl.com&quot;.</li><li>Otherwise, it uses the provided URL from triggerBody().</li></ul><h3>My Logic Behind the Structure</h3><ul><li><strong>Dynamic Content Handling:</strong> The template is designed to be flexible, handling cases where certain data might be missing. The use of coalesce, if, and greaterOrEquals functions ensures that the card can gracefully fall back to default values if needed.</li><li><strong>Conditional Display:</strong> By checking the existence and length of arrays, the template can display different numbers of facts without causing errors or leaving awkward gaps.</li><li><strong>User Experience:</strong> The card ensures that essential information is always shown, either from the provided data or from sensible defaults, making it reliable and user-friendly.</li></ul><p>Overall, the structure and logic ensure that the Adaptive Card can adapt to varying input scenarios while maintaining a consistent and informative presentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AXD3xqqjrOAZ8qNgutY64g.png" /><figcaption>For the Apply to Each section of your workflow, your selected output shold be the `sections` from the previous ‘Parse JSON’ action</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dUe_VuZXnUt9L9TIDZ3fCg.png" /></figure><p>And as you can see, it’s bee very flexible to date and I have had no workflows fail as a result of this Adapative Card template for my use-case:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tY9SESpMtJrWQ4vO_FZGJg.png" /></figure><h3>Step 3: QA</h3><p>Now that you’re workflow is configured, you will want to observe it for failures. For me, it passed and was presenting the correct data about 50% of the time on my first deployment.</p><p>Each failure will give you an Error log to help you refine why it failed. I kept going back to my Adaptive Card template and the Adaptive Card designer to modifying my template to make it more dynamic.</p><p>Eventally, my workflow worked and the out of the box “Post to a channel when a webhook request is received” was working as expected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*i8X3AOWSalIIi_S5sTCuzQ.png" /></figure><p>If you found this article helpful, then you might also like a previous article I wrote about <a href="https://medium.com/aws-in-plain-english/kubernetes-deployments-for-micro-service-applications-fa3236592c9f">Kubernetes Deployments for Micro-service Applications.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa117900decc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering URL Redirects: A Step-by-Step Guide using EasyRedir & NS1]]></title>
            <link>https://medium.com/@tait.hoglund/mastering-url-redirects-a-step-by-step-guide-using-easyredir-ns1-84922d16bfaa?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/84922d16bfaa</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Wed, 13 Dec 2023 19:38:06 GMT</pubDate>
            <atom:updated>2023-12-13T19:38:06.363Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lvw_p0Xh2BRvdnzUlXVRAg.png" /></figure><p>Recently I was charged with removing some legacy websites that were no longer needed. Those sites had been migrated to a new hosting platform while the old websites were still receiving traffic.</p><p>From a purely cost saving perspective, it made sense to just delete the old sites, however there could have been a negative hit to our organization’s SEO score if we also gave up the URLs that were hosting those sites.</p><p>Therefore, It made sense to set-up some URL redirects that would take traffic that was being sent to the old URL and forward to the location of the new site.</p><p>It can take years for websites to build up their SEO reputation and older, more visited URLs often get higher search scores on search engines.</p><p>The process is pretty simple and depending on your DNS settings can be done in just a few hours. But as with anything regarding changing DNS records, changing their settings should be done with caution as there is no easy way to rollback mistakes when a setting gets misconfigured.</p><p>In my case, there was plenty of documentation for EasyRedir, which is whom manages the redirects for my organization, but it was hard to find a good tutorial for how to change the DNS records in NS1.</p><p>In this walk-thru, I will show you how to set-up simply URL Redirects using EasyRedir &amp; NS1.</p><h3>Requisites</h3><ul><li>Fundamental understanding of DNS concepts, records, and TTL</li><li>Basic knowledge of HTTPS protocols. URLs, and browsers</li><li>Accounts in NS1 &amp; EasyRedir</li></ul><h3>Step 1: Setting up your URL Redirect in EasyRedir</h3><p>Login to your EasyRedir Account and click “Create New Redirect</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*g55ucxzspfdPqoKQ7dDgEA.png" /></figure><p>Define the <em>source url</em>, this is the url that you want to forward traffic <em>from</em>. Define the <em>target url</em>, this is the url that EasyRedir will forward traffic <em>to</em>. You can select the redirect type, to be either<em> permanent</em> or <em>temporary</em>. Click “Create Redirect”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*5ePAhyuAvGVyQe0Yd4GGqg.png" /></figure><p>You will receive a confirmation from EasyRedir, confirming that your redirect is created, but you still need to configure your DNS records with your DNS hosting provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Pq3seuNqGJtenol9B3tZQQ.png" /></figure><p>In order to retrieve the EasyRedir settings that you will need for your DNS record, from your EasyRedir dashboard, navigate to the <em>redirects</em> sub-menu, click, <em>view by source URL</em> and then search for the redirect that you just created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*UzxpWsMlRVu9KqX3NoEoIA.png" /></figure><p>From the detailed DNS &amp; SSL status submenu of your newly created EasyRedir, you will need the following:</p><p>1. The registered domain name, which needs to be a a domain that you have access to.<br>2. The host record, which is generated based on your inputs from when you configured your redirect.<br>3. The Recommended DNS Record Type.</p><p>4. The Recommended DNS values, which is a customized EasyRedir link unique to your account and redirect link.</p><p>All of these values will be needed for when you log into your NS1 account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*3rQ8QHZyQQvPwJPFP_psxA.png" /></figure><h3>WARNING BEFORE PROCEEDING TO STEP 2:</h3><p>Proceeding to step 2 is where you can <em>REALLY</em> screw up your website.</p><p>You should not be making DNS record changes unless you are ABSOLUTELY sure of what those changes will produce.</p><p>Here’s why:</p><p>DNS records are the link between a website’s name (www.example.com) and the servers that host it (ex: 169.0.0.0.12). When a user types the name of your website in a browser, it’s the DNS records that exist on servers all over the world that link that name to it’s server so that users can access your website.</p><p>These records can be changed for say when you want to change the name of your website, you migrate servers, you want to add a caching layer to improve UX. But what links the two are DNS records.</p><p>DNS records are configured with a TTL or ‘Time To Live”, it’s an expiry build into the record that the DNS servers monitor, once a record hits it’s expiry, the DNS server will then “fetch” a new copy of the record and replicate it across the internet.</p><p>Typically DNS records don’t change often and DNS hosting providers charge by the query, which means if your DNS record has a TTL of `60`, this means that every 60 seconds, their servers are going to fetch a fresh copy of that DNS record (and charge you for it).</p><p>Conversely, if you have a high TTL, that means that NS1 DNS servers will wait to fetch a new copy of those records until the TTL expires.</p><p>If your existing DNS record is set for 604,800, that means the DNS servers won’t fetch a new record until the current record expires in 7 Days.</p><p>Can you see the potential risk in a misconfiguration? If you misconfigure your DNS record, you potentially break traffic coming to your site and there isn’t a way to rollback like in typical IaaC workflows or rolling updates.</p><p>Which is why unless you are absolutely sure of what changing a DNS record will do, you should double-check, have the changes peer-reviewed, etc.</p><h3>Step 2: Re-configuring your DNS records in NS1</h3><p>Login to your NS1 account, and click “DNS” in the top menu.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xIDNRYP1WjRRQEL3ds-tQw.png" /></figure><p>Within the zone name search bar, type your root domain. For instance, if your redirect source url was <em>resources.example.com</em>, you would search <em>example.com</em>, provided that you were the owner of that domain.</p><p>This will show you the domain and all records associated with it, click the purple <em>records</em>, link.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*GoEvsLM5lwe54NXAxHCl2A.png" /></figure><p>Next, within your zones menu, search within the domain name for the host record value to make sure that a record currently doesn’t exist for your redirect.</p><p>We want to be sure there isn’t an existing record for <em>resources.</em></p><p>If there is AND it’s the same record type (i.e. CNAME), then we can modify that record with the values from our EasyRedir configuration from step 1.</p><p>If another record currently exists for the Host Record value from your EasyRedir, you will need to consider deleting that record OR edit your EasyRedir to use a different <em>Host Record</em> value so as to not conflict with your current records.</p><p>Click either the existing CNAME record to edit your existing record, or if a CNAME record doesn’t currently exist, click the purple plus icon to create a new one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oRpqJFp9LdJLWkcf2GIf-Q.png" /></figure><h3>Updating Existing CNAME Records</h3><p><em>*If you will be creating a new CNAME record for your redirect, you can skip this section</em></p><p>If you are updating a current CNAME record, I would recommend FIRST changing the TTL to 60 seconds.</p><p>This will allow you to make a change once you change your record settings and the servers will update every 60 seconds so if you make a mistake, you can change the record values and know the the DNS servers will begin to update those records every 60 seconds.</p><p>NOTE: As of present writing, NS1 says it can still take 24–48 hours for DNS records to full propagate through out it’s network, which means even with a low TTL, you could still see intermittent failures if you implement a bad DNS record.</p><p>For my instance, I was updated previous records that already existed that had a TTL of 86400. So the day before I planned the changes, I only updated the TTL of the existing records to 60 because I knew it would take a full 24 hours before that TTL would update and waited before changing my existing CNAME records.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1YClcpID1OyAbE8u2d_rnw.png" /></figure><p>This ensured that if I screwed up, I could make a new record change and the the TTL would refresh the record within 60 seconds. Once I could confirm the changes were successful, I would reset the TTL back to the original 24 hour period knowing that I had not killed traffic to the web server.</p><h3>Step 3: Creating a new CNAME record in NS1</h3><p>If you will be creating a new CNAME record for your redirect, from your NS1 DNS dashboard, locate your domain name by searching in the <em>zone menu</em>. This will show your domain listed under “zones”, you’ll want to click the purple “records” link to be brought to your existing DNS records:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_8PUPXzAF85SujM-ekx3fw.png" /></figure><p>From within your domain menu within NS1, click the purple “Add Record” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/148/1*ZEJt1srDDTPaIYMoQptF4Q.png" /></figure><p>You will now need to retrieve the DNS set-up information from your EasyRedir configuration from your redirect and import that data into this DNS record:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*3rQ8QHZyQQvPwJPFP_psxA.png" /></figure><p>You need to be within the registered domain name in NS1, which you should already be. Copy the values from the Hostname setup information over to your “Add Record” in NS1.</p><p>For the “answers” field in NS1, put the information from the “Recommended DNS Values” in EasyRedir. Keep the TTL at 60 for now, you can update to longer once you can confirm it’s working.</p><p>Once all your values are added to the record, click the purple “Save Record”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xziRNWuEkUJQkprOCKV2hg.png" /></figure><p>You should see your new DNS record within your DNS zone dashboard for your domain with your EasyRedir value under the answers tab.</p><p>Back in your EasyRedir dashboard, within a couple minutes, you should see a notification that your redirect is set up correctly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ua9T28rq0kBcLm7R8tGaEA.png" /></figure><p>Finally, you should be able to test the integration of your redirect by typing the redirect URL in your browser and seeing your browser redirect to the target URL that you set up in EasyRedir.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*fxX67LYmlko57RW8hC3Vhg.png" /></figure><p>Great job! You successfully created and implemented a URL redirect using EasyRedir and NS1.</p><p>If you likes this tutorial, then you might also like a previous tutorial I made about <a href="https://medium.com/towards-aws/effortless-scaling-your-web-applications-in-the-cloud-e564368a4a9">A/B Website testing using Kubernetes on AWS</a>.</p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more engineers.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84922d16bfaa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps Pipeline Scaling with Kubernetes]]></title>
            <link>https://medium.com/@tait.hoglund/devops-pipeline-scaling-with-kubernetes-693104319f2a?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/693104319f2a</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Tue, 17 Oct 2023 16:59:00 GMT</pubDate>
            <atom:updated>2023-10-17T16:59:00.645Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How to deploy a GitLab runner using Helm on a Kubernetes cluster using a Kubernetes Agent</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rHNeNx-HTli-Dzvm3d1fpg.png" /></figure><p><em>This is part-two of my series on Auto-Dev-Ops- Scaling your CI/CD builds. Read part one </em><a href="https://medium.com/@tait.hoglund/empowering-your-k8s-cluster-93d23db95fe9"><em>here.</em></a></p><p><strong>Introduction</strong></p><p>Cloud engineers are often responsible for ensuring the efficiency and scalability of the CI/CD pipelines of their organization. When you support a team of developers, your job is to ensure that they can build and deliver their software effcicently and effectively.</p><p>In my previous article, I showed you how you can integrate the GitLab Kubernetes agent with your GKE cluster. In this article, we are going to take it a step further show show you how you can deploy your GitLab runner using a helm chart to your Kubernetes cluster.</p><p>This has a number of benefits:</p><ul><li>The agent can automatically deploy your runner when you launch your Kubernetes cluster.</li><li>The GitLab runner can dynamically scale based on the number of pipeline jobs.</li></ul><p>Which means you no longer have to have your GitLab runner sit idle in a VM waiting for jobs, since the GitLab runner deploys new workloads on a per job basis.</p><p>Say you have multiple developers deploying code, the GitLab runner will scale a pod per pipeline job and scale workloads to as many jobs as required.</p><p>It also kills the pods once the job is completed so you only pay for the compute resources as your pipeline jobs are running.</p><p>So without further ado, Let’s jump in and show you how you can use the Kubernetes Agent to deploy scalable GitLab runners in your GKE cluster:</p><h3>Prerequisites</h3><ul><li>A Gitlab Account with at least developer access to a project</li><li>A Deployed Cluster to GCP with Cluster Admin Access</li><li>Helm installed on your local IDE.</li><li>IaC knowledge (preferably in terraform and .yaml)</li><li>A google cloud account and CLI configured on your local machine</li></ul><p><strong>Step 1: Create a “runner-chart-values.yaml” file with your values. <br></strong>Create this file locally on your local machine.<br>Do NOT push this file to your GitLab project since it contains sensitive values like your gitlabURL endpoint and your runner registration token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wzIXfR-QinKr9PWmHsnnzg.png" /></figure><pre>gitlabUrl: https://&lt;YOUR_URL&gt;<br>runnerRegistrationToken: &quot;&lt;YOUR_RUNNER_REGISTRATION TOKEN&quot;<br>rbac:<br>    create: true<br>runners:<br>    privileged: true</pre><p>NOTE: You have to also determine your TAGs if the Runner is going to be tagged at this step.</p><p>Tags are used in GitLab to ensure that your Runner only runs certain jobs.</p><p>This is helpful if you work on a large team or if you want to limit the types of pipeline jobs that your runner executes on.</p><p>Once a runner is registered with a token and that runner’s .yaml was not configured to use a tag, you will have to create a new RUNNER REGISTRATION TOKEN in you GitLab project and re-register your runner with GitLab.</p><p><strong>Step 2: Use the following Helm command locally to create a single manifest file on your local machine.</strong> This file is your configuration for the Git Lab runner that will be installed on your cluster via the Kubernetes Agent.</p><p>Replace the namespace value with a namespace name already in your Kubernetes cluster. Replace the &lt;YOUR_runner-chart-values.yml&gt; with your actual .yml file name from Step 1.</p><pre>helm template --namespace &lt;YOUR_CLUSTER_NAMESPACE&gt; gitlab-runner -f &lt;YOUR_runner-chart-values.yml&gt; gitlab/gitlab-runner &gt; runner-manifest.yaml</pre><p>This helm command will produce a new yaml file called “runner-manifest.yaml”</p><p>This is the file you need to commit to your GitLab repository so that your Kubernetes agent deploys your Git Lab runner.</p><pre>---<br># Source: gitlab-runner/templates/service-account.yaml<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: gitlab-runner<br>  namespace: &quot;int-poc-k8s-gitlab--runner-agent&quot;<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>---<br># Source: gitlab-runner/templates/secrets.yaml<br>apiVersion: v1<br>kind: Secret<br>metadata:<br>  name: &quot;gitlab-runner&quot;<br>  namespace: &quot;int-poc-k8s-gitlab--runner-agent&quot;<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>type: Opaque<br>data:<br>  runner-registration-token: &quot;&lt;BASE64_ENCODED_TOKEN&gt;&quot;<br>  runner-token: &quot;&quot;<br>---<br># Source: gitlab-runner/templates/configmap.yaml<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: gitlab-runner<br>  namespace: &quot;&lt;YOUR_NAME_SPACE&gt;&quot;<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>data:<br>  entrypoint: |<br>    #!/bin/bash<br>    set -e<br><br>    export CONFIG_PATH_FOR_INIT=&quot;/home/gitlab-runner/.gitlab-runner/&quot;<br>    mkdir -p ${CONFIG_PATH_FOR_INIT}<br>    cp /configmaps/config.toml ${CONFIG_PATH_FOR_INIT}<br><br>    # Set up environment variables for cache<br>    if [[ -f /secrets/accesskey &amp;&amp; -f /secrets/secretkey ]]; then<br>      export CACHE_S3_ACCESS_KEY=$(cat /secrets/accesskey)<br>      export CACHE_S3_SECRET_KEY=$(cat /secrets/secretkey)<br>    fi<br><br>    if [[ -f /secrets/gcs-applicaton-credentials-file ]]; then<br>      export GOOGLE_APPLICATION_CREDENTIALS=&quot;/secrets/gcs-applicaton-credentials-file&quot;<br>    elif [[ -f /secrets/gcs-application-credentials-file ]]; then<br>      export GOOGLE_APPLICATION_CREDENTIALS=&quot;/secrets/gcs-application-credentials-file&quot;<br>    else<br>      if [[ -f /secrets/gcs-access-id &amp;&amp; -f /secrets/gcs-private-key ]]; then<br>        export CACHE_GCS_ACCESS_ID=$(cat /secrets/gcs-access-id)<br>        # echo -e used to make private key multiline (in google json auth key private key is oneline with \n)<br>        export CACHE_GCS_PRIVATE_KEY=$(echo -e $(cat /secrets/gcs-private-key))<br>      fi<br>    fi<br><br>    if [[ -f /secrets/azure-account-name &amp;&amp; -f /secrets/azure-account-key ]]; then<br>      export CACHE_AZURE_ACCOUNT_NAME=$(cat /secrets/azure-account-name)<br>      export CACHE_AZURE_ACCOUNT_KEY=$(cat /secrets/azure-account-key)<br>    fi<br><br>    if [[ -f /secrets/runner-registration-token ]]; then<br>      export REGISTRATION_TOKEN=$(cat /secrets/runner-registration-token)<br>    fi<br><br>    if [[ -f /secrets/runner-token ]]; then<br>      export CI_SERVER_TOKEN=$(cat /secrets/runner-token)<br>    fi<br><br>    # Register the runner<br>    if ! sh /configmaps/register-the-runner; then<br>      exit 1<br>    fi<br><br>    # Run pre-entrypoint-script<br>    if ! bash /configmaps/pre-entrypoint-script; then<br>      exit 1<br>    fi<br><br>    # Start the runner<br>    exec /entrypoint run \<br>      --working-directory=/home/gitlab-runner<br><br>  config.toml: |<br>    concurrent = 10<br>    check_interval = 30<br>    log_level = &quot;info&quot;<br><br><br>  config.template.toml:   |<br>    [[runners]]<br>      [runners.kubernetes]<br>        namespace = &quot;&lt;YOUR_NAMESPACE&gt;&quot;<br>        image = &quot;ubuntu:22.04&quot;<br><br><br>  register-the-runner: |<br>    #!/bin/bash<br>    MAX_REGISTER_ATTEMPTS=30<br><br>    # Reset/unset the not needed flags when an authentication token<br>    RUN_UNTAGGED=&quot;&quot;<br>    ACCESS_LEVEL=&quot;&quot;<br><br>    if [ ! -z &quot;$REGISTRATION_TOKEN&quot; ] &amp;&amp; [ &quot;${REGISTRATION_TOKEN#glrt-}&quot; != &quot;$REGISTRATION_TOKEN&quot; ]; then<br>      RUN_UNTAGGED=&quot;&quot;<br>      ACCESS_LEVEL=&quot;&quot;<br>      unset REGISTER_LOCKED<br>      unset RUNNER_TAG_LIST<br>    fi<br><br>    for i in $(seq 1 &quot;${MAX_REGISTER_ATTEMPTS}&quot;); do<br>      echo &quot;Registration attempt ${i} of ${MAX_REGISTER_ATTEMPTS}&quot;<br>      /entrypoint register \<br>        ${RUN_UNTAGGED} \<br>        ${ACCESS_LEVEL} \<br>        --template-config /configmaps/config.template.toml \<br>        --non-interactive<br><br>      retval=$?<br><br>      if [ ${retval} = 0 ]; then<br>        break<br>      elif [ ${i} = ${MAX_REGISTER_ATTEMPTS} ]; then<br>        exit 1<br>      fi<br><br>      sleep 5<br>    done<br><br>    exit 0<br><br>  check-live: |<br>    #!/bin/bash<br>    if /usr/bin/pgrep -f .*register-the-runner; then<br>      exit 0<br>    elif /usr/bin/pgrep gitlab.*runner; then<br>      exit 0<br>    else<br>      exit 1<br>    fi<br><br>  pre-entrypoint-script: |<br>---<br># Source: gitlab-runner/templates/role.yaml<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: &quot;Role&quot;<br>metadata:<br>  name: gitlab-runner<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>  namespace: &quot;&lt;NAMESPACE&gt;&quot;<br>rules:<br>- apiGroups: [&quot;&quot;]<br>  resources: [&quot;*&quot;]<br>  verbs: [&quot;*&quot;]<br>---<br># Source: gitlab-runner/templates/role-binding.yaml<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: &quot;RoleBinding&quot;<br>metadata:<br>  name: gitlab-runner<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>  namespace: &quot;&lt;NAMESPACE&gt;&quot;<br>roleRef:<br>  apiGroup: rbac.authorization.k8s.io<br>  kind: &quot;Role&quot;<br>  name: gitlab-runner<br>subjects:<br>- kind: ServiceAccount<br>  name: gitlab-runner<br>  namespace: &quot;&lt;NAMESPACE&gt;&quot;<br>---<br># Source: gitlab-runner/templates/deployment.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: gitlab-runner<br>  namespace: &quot;&lt;NAMESPACE&gt;&quot;<br>  labels:<br>    app: gitlab-runner<br>    chart: gitlab-runner-0.57.0<br>    release: &quot;gitlab-runner&quot;<br>    heritage: &quot;Helm&quot;<br>spec:<br>  replicas: 1<br>  revisionHistoryLimit: 10<br>  selector:<br>    matchLabels:<br>      app: gitlab-runner<br>  template:<br>    metadata:<br>      labels:<br>        app: gitlab-runner<br>        chart: gitlab-runner-0.57.0<br>        release: &quot;gitlab-runner&quot;<br>        heritage: &quot;Helm&quot;<br>      annotations:<br>        checksum/configmap: 4f81c3d957d79b3f60c520791f111723a267799262a6047ade9b18b020377290<br>        checksum/secrets: b3e6bfd4031623a9a3a43d401fbf9f7814d50a6d570950cbe023dbe39f0e7ede<br>    spec:<br>      securityContext:<br>        fsGroup: 65533<br>        runAsUser: 100<br>      terminationGracePeriodSeconds: 3600<br>      serviceAccountName: gitlab-runner<br>      containers:<br>      - name: gitlab-runner<br>        image: registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v16.4.0<br>        imagePullPolicy: &quot;IfNotPresent&quot;<br>        securityContext:<br>          allowPrivilegeEscalation: false<br>          capabilities:<br>            drop:<br>            - ALL<br>          privileged: false<br>          readOnlyRootFilesystem: false<br>          runAsNonRoot: true<br>        lifecycle:<br>          preStop:<br>            exec:<br>              command: [&quot;/entrypoint&quot;, &quot;unregister&quot;, &quot;--all-runners&quot;]<br>        command: [&quot;/usr/bin/dumb-init&quot;, &quot;--&quot;, &quot;/bin/bash&quot;, &quot;/configmaps/entrypoint&quot;]<br>        env:<br><br>        - name: CI_SERVER_URL<br>          value: &quot;https://in.thewardro.be/&quot;<br>        - name: RUNNER_EXECUTOR<br>          value: &quot;kubernetes&quot;<br>        - name: REGISTER_LOCKED<br>          value: &quot;true&quot;<br>        - name: RUNNER_TAG_LIST<br>          value: &quot;test-poc&quot;<br>        - name: RUN_UNTAGGED<br>          value: &quot;false&quot;<br>        livenessProbe:<br>          exec:<br>            command: [&quot;/bin/bash&quot;, &quot;/configmaps/check-live&quot;]<br>          initialDelaySeconds: 60<br>          timeoutSeconds: 1<br>          periodSeconds: 10<br>          successThreshold: 1<br>          failureThreshold: 3<br>        readinessProbe:<br>          exec:<br>            command: [&quot;/usr/bin/pgrep&quot;,&quot;gitlab.*runner&quot;]<br>          initialDelaySeconds: 10<br>          timeoutSeconds: 1<br>          periodSeconds: 10<br>          successThreshold: 1<br>          failureThreshold: 3<br>        ports:<br>        - name: &quot;metrics&quot;<br>          containerPort: 9252<br>        volumeMounts:<br>        - name: projected-secrets<br>          mountPath: /secrets<br>        - name: etc-gitlab-runner<br>          mountPath: /home/gitlab-runner/.gitlab-runner<br>        - name: configmaps<br>          mountPath: /configmaps<br>        resources:<br>          {}<br>      volumes:<br>      - name: runner-secrets<br>        emptyDir:<br>          medium: &quot;Memory&quot;<br>      - name: etc-gitlab-runner<br>        emptyDir:<br>          medium: &quot;Memory&quot;<br>      - name: projected-secrets<br>        projected:<br>          sources:<br>            - secret:<br>                name: &quot;gitlab-runner&quot;<br>                items:<br>                  - key: runner-registration-token<br>                    path: runner-registration-token<br>                  - key: runner-token<br>                    path: runner-token<br>      - name: configmaps<br>        configMap:<br>          name: gitlab-runner</pre><p>If you want to “tag” the git lab runner to only run certain jobs, then find and update the following line in the code below from within your “runner-manifest.yaml”:</p><pre>#Repalce this block:<br><br>- name: RUNNER_TAG_LIST<br>  value: &quot;&quot;<br><br># With<br><br>- name: RUNNER_TAG_LIST<br>  value: &quot;test-poc&quot;</pre><p><strong>Step 3: Commit your runner-manifest.yaml into your GitLab project Repository — </strong>This should be committed into the project that your Kubernetes Agent is integrated with from <a href="https://medium.com/@tait.hoglund/empowering-your-k8s-cluster-93d23db95fe9">part one.</a></p><p>Remember from my previous article where we configured our Kubernetes Agent to monitor a particular file in a particular project. We told it to only look for changes and commits to a file called, “runner-manifest.yaml”:</p><pre>gitops:<br>  manifest_projects:<br>  - id: &quot;&lt;_PATH_TO_YOUR_GITLABPROJECT_&quot;<br>    paths:<br>    - glob: &#39;runner-manifest.yml&#39;</pre><p>This is why we had to name our helm chart, “runner-manifest.yml”. This is because when we commit our GitLab runner yaml into our project repository, our Kubernetes agent will see the commit and automatically deploy our GitLab runner into our cluster as a workload.</p><p>After you commit the file to your GitLab Project, It might take up to 3 minutes but eventually you should be able to see your gitlab-runner running in your GKE cluster under workloads:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ESTica1ed0CYGV79-US_sw.png" /></figure><p>You can also confirm the Git Lab Runner link by visiting the CI/CD settings within your Git Lab Project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*evo2p4UlQyspR8w1xTlCmg.png" /></figure><p><strong>Step 4 — Test our GitLab Runner Scaling by Running Tagged and Untagged Pipeline Jobs — </strong>Once a commit is pushed, our GitLab Runner should pick up the new commits and execute them based on ‘jobs’ defined within the .gitlab-ci.yml file in our project repository.</p><p>To test, we will configure a sample .gitlab-ci.yml file with the mostly “tagged” jobs and one “untagged” job to ensure that we have configured our gitlab runner correctly:</p><p>In the following pipeline file, all stages should pass except the “verify” stage since it is the only job not tagged with our “test-poc” tag:</p><pre>stages:<br>  - build<br>  - test<br>  - deploy<br>  - verify <br><br>build_job:<br>  stage: build<br>  script:<br>    - echo &quot;This is the build stage.&quot;<br>    - mkdir build/<br>    - echo &quot;Build completed on $(date)&quot; &gt; build/output.txt<br>  tags:<br>    - test-poc<br>  artifacts:<br>    paths:<br>      - build/output.txt<br><br>test_job:<br>  stage: test<br>  script:<br>    - echo &quot;This is the test stage.&quot;<br>    - cat build/output.txt<br>    - echo &quot;Test completed successfully.&quot;<br>  tags:<br>    - test-poc<br><br>deploy_job:<br>  stage: deploy<br>  script:<br>    - echo &quot;This is the deploy stage.&quot;<br>    - echo &quot;Deployment completed on $(date)&quot;<br>  tags:<br>    - test-poc<br><br># This un-tagged job should fail based on how we&#39;ve configured out runner.<br>regular_job:<br>  stage: verify<br>  script:<br>    - echo &quot;This is a regular job&quot;</pre><p>Once we save our .gitlab-ci.yml file into the root of our project directory, we can run a “test” Pipeline to verify that our Runner only runs tagged jobs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sx-t4PX7eZVl9UeC1BvSUA.png" /></figure><p>And when we drill into our verify stage, we can see the un-tagged job got “stuck” because there is a lack of runners:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IV6hQfUhgOa_bULPq9DaIg.png" /></figure><p><strong>Step 5 — Test the scaling of our GitLab Runner — </strong>Now that we know our GitLab runner is successfully running pipeline jobs and ONLY the pipeline jobs we want it to, we can change our .gitlab-ci.yaml file and include more jobs to see how the workloads scale within our Kubernetes Cluster:</p><p>This sample ci file includes multiple jobs to run in parallel ensuring that our runner will scale depending on the number of jobs needed to run at the same time:</p><pre>stages:<br>  - build<br>  - test<br>  - deploy<br>  - verify<br><br>build_job:<br>  stage: build<br>  script:<br>    - echo &quot;This is the build stage.&quot;<br>    - mkdir build/<br>    - echo &quot;Build completed on $(date)&quot; &gt; build/output.txt<br>  tags:<br>    - test-poc<br>  artifacts:<br>    paths:<br>      - build/output.txt<br><br>build_job_parallel_1:<br>  stage: build<br>  script:<br>    - echo &quot;This is the parallel build job 1.&quot;<br>    - mkdir build_parallel_1/<br>    - echo &quot;Parallel Build 1 completed on $(date)&quot; &gt; build_parallel_1/output.txt<br>  tags:<br>    - test-poc<br>  artifacts:<br>    paths:<br>      - build_parallel_1/output.txt<br><br>build_job_parallel_2:<br>  stage: build<br>  script:<br>    - echo &quot;This is the parallel build job 2.&quot;<br>    - mkdir build_parallel_2/<br>    - echo &quot;Parallel Build 2 completed on $(date)&quot; &gt; build_parallel_2/output.txt<br>  tags:<br>    - test-poc<br>  artifacts:<br>    paths:<br>      - build_parallel_2/output.txt<br><br>test_job:<br>  stage: test<br>  script:<br>    - echo &quot;This is the test stage.&quot;<br>    - cat build/output.txt<br>    - echo &quot;Test completed successfully.&quot;<br>  tags:<br>    - test-poc<br><br>deploy_job:<br>  stage: deploy<br>  script:<br>    - echo &quot;This is the deploy stage.&quot;<br>  tags:<br>    - test-poc<br><br>verify_job:<br>  stage: verify<br>  script:<br>    - echo &quot;This is the verify stage.&quot;</pre><p>Once we update our .gitlab-ci.yml file within the root of our project repository, we can run another pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*icbl7tsSMfFvbLxSYY9wag.png" /><figcaption>A new Gitlab Runner deployment is created for each job requested and the Runner automatically shuts down once it’s job is completed.</figcaption></figure><p>We can also see these workloads in real-time via the GCloud Console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sSqmXb1qZv6DeX_xj1ErCA.png" /></figure><p>That’s it! You have successfully scaled your CI/CD Pipelines to match developer demand and save on compute costs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*r3g-9HW1GscE-2qmUbOfFQ.jpeg" /></figure><p>If you found this article helpful, then you might also like a previous article I wrote about <a href="https://medium.com/faun/modeling-ci-cd-pipelines-with-aws-services-using-a-dynamic-resume-5d4ce4aaddd4">configuring CI/CD pipelines on on AWS.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=693104319f2a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gitlab Project & GKE Integration]]></title>
            <link>https://medium.com/@tait.hoglund/empowering-your-k8s-cluster-93d23db95fe9?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/93d23db95fe9</guid>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Sat, 07 Oct 2023 16:30:15 GMT</pubDate>
            <atom:updated>2023-10-23T11:48:14.314Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to authenticate your GitLab project within a GKE Cluster using the Kubernetes Agent</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tQa73mRg20oQ8TfdWf2qvQ.png" /></figure><p><em>*This is part-one of of a two-part series on Auto-Dev-Ops- Scaling your CI/CD builds.</em></p><p><strong>Introduction</strong></p><p>*Gitlab runs CI/CD pipelines through a a program that you define called a Gitlab runner. These runners can be managed by Gitlab (for a fee) on their servers or you can configure and set up your own self-managed one. Gitlab runners are configured to run your CI/CD Pipeline jobs for your codebase.</p><p>Recently my organization experienced a corruption of our container registry during a services upgrade. This forced us to manually re-build our container images in our registry via our GitLab runners.</p><p>Unfortunately, this task took about 3 hours because we only had four configured runners available to run pipeline jobs so while the task of running 25 pipelines was a relatively simple task, we were stuck waiting because our Gitlab runners were deployed on GCP compute instances, what should have taken <em>an hour</em> at most for our production environment, took over three as we waited for our Gitlab runners to become available to process our queue of pipeline jobs.</p><p>This made me this, is there a better way to deploy our existing runners to make them more scalable?</p><p>Certainly having multiple runners sitting idle in compute instances isn’t the most cost-effective either.</p><p>Which brought me to deploying Gitlab runner instead within a Kubernetes Cluster using a Kubernetes Agent.</p><p><strong>What is a GitLab Kubernetes Agent?</strong> <br>Simply put, if you are looking to make your Gitlab runners more scalable, cost-effective, and move away from Gitlab’s <a href="https://www.notion.so/GitLab-1e70f0b5d30344dfa29457928883777e?pvs=21">deprecated certificate-based authentication</a> method of connecting your GitLab project into your Kubernetes cluster, then it starts with authenticating your Kubernetes cluster with a Kubernetes Agent.</p><p>GitLab Kubernetes Agents not only are the way forward for authenticating your Kubernetes clusters with Gitlab but they also can be configured to automate the deployment of your Gitlab runners within your GKE Kubernetes cluster.</p><p>And since I just completed this project, I’m going to share a walk-thru with you on how you can automate the deploy your own Gitlab Kubernetes agent into your cluster using a helm chart and terraform.</p><p>Let’s get started:</p><h3><strong>Prerequisites</strong></h3><ul><li>A Gitlab Account with at least developer access to a project</li><li>A Deployed Cluster to GCP with Cluster Admin Access</li><li>Helm installed on your local IDE.</li><li>IaC knowledge (preferably in terraform and .yaml)</li><li>A google cloud account and CLI configured on your local machine</li></ul><p>Note: If you anticipate utilizing your Kubernetes Agent for multiple projects, then you should configure it within it’s own project.</p><p><strong>Step 1: Create a new GitLab Project:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWxw-z7WNLit_3KLT4Jhhw.png" /></figure><p><strong>Step 2: Create an agent configuration file — </strong>In your default branch, create a Kubernetes Agent configuration file with the following specs:</p><pre># File needs to be in this file structure. You can name the &lt;agent-name&gt; whatever you want<br><br>.gitlab/agents/&lt;agent-name&gt;/config.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DaE_uQbK7-GoozOOGw4gxQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GtGnOtWYdUK2e5U01pb5IA.png" /></figure><p>The config.yaml should contain the PATH the Gitlab project that you want to integrate with your agent and the name of your .yaml which will eventually be the configuration for your Gitlab runner <em>(I will walk through the process in a later article).</em></p><pre>gitops:<br>  manifest_projects:<br>  - id: &quot;&lt;_PATH_TO_YOUR_GITLABPROJECT_&quot;<br>    paths:<br>    - glob: &#39;runner-manifest.yml&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cz2V7G6cMxAGGhg2wYqc0w.png" /></figure><p><strong>Step 3: Create an Agent Access Token</strong> — Within your project settings, you need to create an Agent Access Token. On the left-side menu, under “Operate”, click “Kubernetes Clusters” link. Click the “agents” tab and when it prompts you to select an agent, select your agent that you created in the previous step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ynUD2AEq1imAtDW4IiAeeA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q2zuhcWypIDGTzgWMPJf8w.png" /></figure><p>Next GitLab will show you your ‘agent access token’ &amp; your Kubernetes Agent Server, which is managed by GitLab. SAVE THEM.</p><p>The token is not reproducible, if you lose it. and it’s what gets configured on your GKE Cluster when you deploy the agent.</p><p>The token is what allows your agent on your cluster to maintain communication with your repository to monitor for any changes in your repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ml28q43YJVWOdQdeUOagsQ.png" /><figcaption><strong>NOTE: YOU MUST REGISTER THE AGENT AND DO THE SET UP IN GITLAB BEFORE YOU DEPLOY THE AGENT TO THE CLUSTER.</strong></figcaption></figure><p>In the coming steps I will show you the terraform configuration needed to deploy your agent into your cluster using helm.</p><p>However, if you deployed your agent BEFORE you configured your token, you will need to redeploy the agent in order for the agent to deploy and integrate correctly with your Gitlab project.</p><p><strong>2. Create a Terraform configuration file with your KAS &amp; token values </strong>This creates our Kubernetes agent via a Helm Chart, which will be used in our terraform to deploy the Kubernetes agent into our cluster.</p><p>I chose to create this .tf in the same directory as my cluser configuration so that it deploys it directly on my cluster when I run a <em>terraform apply</em>.</p><pre>// Sample k8s_agent_helm.tf file:<br><br>resource &quot;helm_release&quot; &quot;gitlab_agent&quot; {<br>  name = &quot;gitlab-agent&quot;<br><br>  repository = &quot;https://charts.gitlab.io&quot;<br>  chart      = &quot;gitlab-agent&quot;<br><br>  create_namespace = true<br>  namespace        = var.agent_namespace<br><br>  set {<br>    name  = &quot;config.kasAddress&quot;<br>    value = var.kas_address<br>  }<br><br>  set {<br>    name  = &quot;config.token&quot;<br>    value = var.agent_token<br>  }<br><br>  set {<br>    name  = &quot;image.tag&quot;<br>    value = &quot;v16.3.0&quot;<br>  }<br>}</pre><p>Make sure to define the values for your variables in your variables.tf file:</p><pre>//Sample variable blocks for values in your variables.tf file<br><br>variable &quot;agent_token&quot; {<br>  type        = string<br>  default     = &quot;&lt;YOUR_RUNNER_TOKEN_VALUE&gt;&quot;<br>  description = &quot;Agent token (provided when registering an Agent in GitLab)&quot;<br>  sensitive   = true<br>}<br><br>variable &quot;kas_address&quot; {<br>  type        = string<br>  default     = &quot;wss://&lt;YOUR_KAS_AGENT_SERVER&gt;&quot;<br>  description = &quot;Agent Server address (provided when registering an Agent in GitLab&quot;<br>}</pre><p><strong>Step 3: Run ‘Terraform Apply” and confirm Kubernetes Agent Deployment</strong></p><p>Save you file and run a terraform apply:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tGuZtcl7K9TsUDUxvDE1qA.png" /></figure><p>Verify the agent deployment via the GCloud Console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GfyCjOBxCBNeT4BpW_oJXw.png" /></figure><p>Finally, we can verify the Kubernetes agent integration with our GitLab project by re-visiting the agent settings under the “Operate &gt; Kubernetes Clusters &gt; Agent” link within our project</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uhfiJ6sFbwyzcncHRm3TGw.png" /></figure><p>Congrats! You have successfully deployed a Kubernetes Agent within your GKE Cluster. This is part one of my three part-series — auto-dev-ops.</p><p>Follow me so that you get notified when I post my next tutorial — How to use the Kubernetes Agent to deploy your GitLab runners for CI/CD Jobs.</p><p>If you found this article helpful, then you might also like a previous article I wrote about <a href="https://medium.com/aws-in-plain-english/kubernetes-deployments-for-micro-service-applications-fa3236592c9f">Kubernetes Deployments for Micro-service Applications.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*DAcrnMzoM_UEhCaCt8KhnA.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93d23db95fe9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[PagerDuty 101: The Ultimate Guide for First-Time Site Reliability Engineers]]></title>
            <link>https://blog.devgenius.io/pagerduty-101-the-ultimate-guide-for-first-time-site-reliability-engineers-c8864dceebf0?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8864dceebf0</guid>
            <category><![CDATA[incident-response]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[sre]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[pagerduty]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Mon, 21 Aug 2023 16:21:29 GMT</pubDate>
            <atom:updated>2023-08-22T18:01:16.086Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Empowering Your On-Call Experience: A Step-by-Step Introduction to Streamlining Incident Management with PagerDuty</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ghb7J_RJTciTPcuF4RUKA.png" /></figure><p>As a first-time Site Reliability Engineer (SRE), navigating the complex landscape of incident management can be daunting. Enter PagerDuty, a powerful platform designed to streamline on-call duties and enhance real-time collaboration.</p><p>Whether you’re just starting your journey in the world of SRE or looking to sharpen your existing skills, this guide will walk you through the essentials of PagerDuty.</p><p>From setting up schedules to integrating with your existing tools, we’ll cover everything you need to know to become proficient in managing incidents and ensuring the reliability of your systems.</p><p>Let’s dive into the world of PagerDuty and explore how it can become a vital part of your SRE toolkit.</p><h3>What is PagerDuty and Why it’s Helpful</h3><p>PagerDuty is a digital operations management platform that plays a crucial role in Site Reliability Engineering (SRE) practices.</p><p>It’s designed to provide real-time alerts and incident tracking, ensuring that the right team members are notified of any issues as soon as they arise for online platforms.</p><p>By integrating with various monitoring tools, PagerDuty centralizes alerts and streamlines the response process, reducing the time it takes to identify and resolve incidents.</p><p>For SRE Teams, this means quicker reaction times, more efficient collaboration, and a more proactive approach to maintaining system reliability.</p><p>The platform’s ability to automate routine tasks and provide insightful analytics also empowers SRE teams to focus on continuous improvement and innovation.</p><p>In essence, PagerDuty acts as a force multiplier for SRE practices, enhancing both the responsiveness and resilience of digital systems.</p><p>The goal of this article to give a brief overview of the PagerDuty platform, it’s terms and features and how you can start incorporating it into your everyday SRE practices.</p><p>Let’s jump in:</p><h3>Pre-Requisites</h3><ul><li>Basic understanding of incident management</li><li>Experience with monitoring tools</li><li>Familiarity with DevOps practices</li><li>Access to a PagerDuty account</li><li>Programming knowledge</li><li>Understanding of cloud platforms</li><li>A collaborative devOps mindset</li></ul><h3>Overview</h3><p>Pager duty is designed to ingest data from monitoring tools and platforms to initiate <em>alerts. </em>Alerts in the PagerDuty platform can automatically initiate incident response to your developers, system administrators, and engineers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6N6-B2isETjuZJKWctgy6Q.png" /><figcaption>PagerDuty follows a predictable pattern of events to help technical teams respond to events in their Cloud native applications. (Screenshot Source: PagerDuty.com)</figcaption></figure><p>Alerts are initiated by digital signals, which are typically events that are set up by your platform engineers within your monitoring tools set up.</p><p>These signals initiate Alerts to inform a team of designated responders of status changes to services in their deployments that is monitored by the PagerDuty Platform.</p><p>Each alert contain a severity value:</p><ul><li><strong>Critical</strong> — A failure in the system’s primary application.</li><li><strong>Error </strong>— Any error which is fatal to the operation, but not the service or application.</li><li><strong>Warning</strong>- May indicate that an error will occur if an action is not taken.</li><li><strong>Info</strong>- Normal operational messages that require not action.</li><li><strong>Unknown</strong>- Automatically chosen when a monitoring tool is not setting, or cannot set the severity.</li></ul><p>Alerts are sent to Users that are assigned in PagerDuty. Once paged, users can then assign a <em>classification</em> to determine the response needed. Classifications are graded on a P1-P5 (Priority 1-Priority 5) scale OR a SEV1-SEV5 (Severity 1- Severity 5) scale.</p><p>The lower the number, the HIGHER the priority or severity of the incident.</p><h3>Three States of an Incident Response</h3><p>An incident progresses through three stages- Triggered, Acknowledged, Resolved:</p><p><strong>Triggered</strong> — When an incident is triggered, a notification is sent to a team member of a triggered state of the incident. The responding teammate then needs to Acknowledge that they have received the Alert and are now looking into the incident.</p><p>Once an incident is ‘acknowledged’, other response team members are notified that their co-worker is aware and working on a solution.</p><p><strong>Acknowledge</strong> — Once a team member receives that notification, they must acknowledge that they have received the notification of the event and are working on diagnosing the problem.</p><p><strong>Resolved </strong>— Means that the team members responding to the incident have resolved the issue that triggered the incident.</p><h3>Responding to an Incident</h3><p>When an incident is triggered in PagerDuty the first step is for the on-call person to click and “Acknowledge” the Trigger via the Incident Dashboard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*SI_vE2G5Nlq55-U0EiQtrQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>You can sort incidents by Open, Triggered, Acknowledged, Resolved, or Any Status to see all incidences sorted in descending order by occurrence.</p><p>You can also click into any incident by clicking the link within the Title column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cbjcda7mCnJvXEl--tTUVg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>Clicking the link will bring you into the Incident Page for that particular incident.</p><p>Within the Incident page, you can see the status of the incident:</p><ul><li>Status: (Triggered, Acknowledged, or Resolved)</li><li>The impacted service with a link to it.</li><li>Responder information</li><li>Payload information about the alert</li><li>Links to your external monitoring tools for more logs</li></ul><p>When responding to an incident, make you you click “Acknowledge” so others know that you are responding to the incident.</p><p>You can also set policies for other parties to be informed if an incident isn’t acknowledged within a set period of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*7LCn_n5Zq54X9MrJ8iLj4Q.png" /><figcaption>When you click the “Show Details” the payload information will be displayed. (Screenshot Source: PagerDuty.com)</figcaption></figure><h3>Escalating an Incident</h3><p>When you don’t have the resources alone to resolve an incident. You should escalate to other team members who can.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UqcZuqUyuFf9LScwRui8VA.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>When you escalate an incident, it places the incident back into a “Triggered State”.</p><p>Immediately after escalating the incident to another team member, you should also place a note in the incident informing the assignee that you are passing response on to them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n79GojPX9qEz1lbON18O-Q.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p><em>Priority levels</em> can be manually assigned by the responder once the incident enters a “Acknowledged” status or it can be automatically assigned via policy.</p><h3>Adding Responders to an Incident</h3><p>When responding if additional team members are needed in order to resolve and incident, the team members can be added into the <em>responders</em> category.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oSgVljsdfxFi-XMdAX8U1g.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>You can also add team members into the incident response via assigned roles such as <em>Internal Liaison</em>, <em>Scribe</em>, or <em>Incident Commander.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1kvfNFnT9RVxppafKV4Elg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>You can also add a message that will be sent and add a conference bridge if needed to bring all the needed team members into a meeting for urgent matters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*H5uD9PRY7PUOzsfrC7Ctwg.png" /><figcaption>When clicking save, a request will be sent to the responding team. (Screenshot Source: PagerDuty.com)</figcaption></figure><p>The status of receiving, acknowledging, or whether they’ve jumped into the incident response meeting via invite will be displayed on the Incident Dashboard:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*av-Orv9QcLiGrzPzHia6wQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*F98Irqkny1Kydkl3J20mTQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>You can manually add responders when manually triggering an incident or automatically by using <em>Response Plays</em>.</p><p><em>Response Plays</em> are a feature in PagerDuty that allow you to automate specific actions when an incident occurs. Think of them as predefined workflows or playbooks that can be triggered during an incident to save time and streamline the response process.</p><p>When a user is added as a responder to an incident from a private team, that user will be able to see the incident and take action on it. However, members of that user’s team will only be able to see the incident but not the response options.</p><h3>Adding Subscribers to an Incident</h3><p>When an incident occurs, it’s crucial to keep relevant people within the organization in the loop. By adding subscribers to an incident, you are essentially informing key stakeholders about what’s going wrong and what steps are being taken to fix it.</p><p>Subscribers give those who need to know a direct view into the problem’s impact and the progress toward solving it.</p><p>Adding subscribers does more than just keeping people informed of the incident; it also helps the team dealing with the incident.</p><p>When the stakeholders know what’s happening, the responders can concentrate on addressing the issue itself rather than spending time answering individual questions about the status.</p><p>This can speed up the process of resolving the issue, making your system more reliable, and ultimately enhancing the user experience.</p><p>Subscriber updates can be posted and found within the <em>Status/Updates</em> tab within the incident dashboard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JVO2JSfmhKNi9uwCp3-39Q.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>Additionally, you can manager the subscribers who will receive these updates within the Status/Updates sub-menu.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FgHcKbFh_iMs-HzG6ujGTg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*thfw0D6xJfd2q-eJfL6UPA.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>In the Status Updates is where you can post messages that will be sent to all subscribers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tEe0Wsh8YRBdxCT8ZYCKTg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>The timeline tab keeps a record of all the incident status updates in a timestamped format:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wt7ESkHH2kIqrWw3BxRyzA.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><h3>Snoozing an Incident</h3><p>By default, PagerDuty incidents will issue another alert of an incident every 30 minutes after an incident enters an “Acknowledged State” that go unresolved.</p><p><em>Snoozing</em> gives the responder time to work on the incident without it returning to a <em>triggered state</em> and re-notifying them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wxE9IDDq1X6QW_Ppm5VKbQ.png" /><figcaption><em>Snoozing</em> within the incident dashboard overrides the acknowledgement timeout of an incident with a user-specified duration. (Screenshot Source: PagerDuty.com)</figcaption></figure><p>If another user acknowledges a <em>snoozed</em> incident, the incident exits the snoozed state and the acknowledgement timeout starts fresh.</p><h3>Merging Incidents</h3><p>Think of handling incidents like dealing with a puzzle. Sometimes, you might find that different pieces (aka incidents) are actually related to the same underlying problem.</p><p>Merging these related incidents into one is like putting those puzzle pieces together. It simplifies the response process by gathering all the alerts and information into a single place.</p><p>Imagine all the clues leading to the root cause and the overall impact of the problem being assembled together.</p><p>This not only helps the responders to see the bigger picture but also assists them in identifying exactly what went wrong and how it’s affecting the system.</p><p>By consolidating everything into one incident, you’re streamlining the approach to a solution, making it easier and more efficient to get everything back on track.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ozRAwIHvjQrN-KaQYFdRTQ.png" /><figcaption>You can merge incidences within the Incident Dashboard, under the “More” submenu (Screenshot Source: PagerDuty.com)</figcaption></figure><p>You must know and enter the number or ID of the incident you want to merge into:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O6m5QzRVUgaDG5VD8YU4jA.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p><em>Merging</em> incidents combine separate threads into one unified conversation.</p><p>When you <em>merge</em> and incident with another, the responders of the target incident remain in their roles.</p><p>They become the go-to team for all the alerts grouped under that merged incident. It’s like gathering different tasks and assigning them to an already coordinated team, ensuring that the same people who were dealing with the original issue continue to manage everything related to it.</p><p>This consistency helps maintain clear communication and a focused approach to resolving the problem.</p><h3>Custom Incident Actions</h3><p><em>Custom incident actions</em> are useful when a quick fix to the issue cannot be discovered in a short about of time (say :30 or less).</p><p>Custom Actions are useful when you need to immediately restore service to a previously usable state and cannot figure out a simple solution to resolve the issue.</p><p><em>Using a Custom incident action</em>, incident responders can revert changes in platforms back to an original state to rollback issues that may have occurred to cause the incident to take place</p><p><em>Custom incident actions</em> can be run via an incident’s dashboard under the <em>more </em>submenu:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p5CZf1Sn9lX2BQmUCAcyWg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>Once a custom incident action is run, a confirmation message will display confirming that the custom action was successful:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6abWTuMCf3W4l4fZ31zuXg.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>Custom Actions can be used to inform team about the reversion.</p><h3>Reassigning an Incident</h3><p>Transferring ownership of an incident, or reassigning it, enables you to hand it over to another individual within your team, or even to a different escalation policy.</p><p>This functionality is particularly valuable when various teams are responsible for different aspects of an incident, or if an incident has mistakenly been allocated to the incorrect team.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*moXUt8uScKqxE17vcDzzIQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>When you re-assign an incident, you can select a <em>user, escalation policy</em>, or <em>team</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6GhidmaZ6MSyH_KxEQXeQQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>The benefit of assigning it to a team rather than an individual means it can be routed to the correct on-call person from that team when the exact team member is not known by the current assignee.</p><p>Once the incident is <em>reassigned. </em>The incident re-enters a <em>triggered</em> state and will send a notification to that assigned user or team.</p><p>When you reassign an incident, the duration timer does not reset because the duration timer counts the time past since the incident at acknowledged not since the incident was assigned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dHFxSu4Edxr0hBHPi1auow.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>It’s best practice to also include a <em>note</em> and <em>status update</em> when re-assigning an incident to keep stakeholders updated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*JGM4V8WmWl0t9wEGR0n0qA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PUjlVhOa-l-WFIV57Y_-rQ.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>If an incident originating from a private team is handed over to someone outside that team, the person receiving the incident (the assignee) will have the ability to view and act upon it.</p><p>In contrast, the assignee’s teammates will only have visibility of the incident and won’t be able to take any responsive actions.</p><p>This limitation also holds true when someone is designated as a responder to an incident associated with a private team; they can see it, but their team members cannot respond to it.</p><h3>Resolving an Incident</h3><p>When an incident is marked as <em>resolved</em>, it signifies that the problem has been addressed and corrected.</p><p>Following its resolution, the incident is officially closed, and no further alerts or notifications will be dispatched.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IfZQh3_5PisDMSqwcIw8aA.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>When you click resolve in an incident’s dashboard, you will be prompted to add a <em>resolution note</em> and post it as a <em>status update</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kkbVafMLNEyix0vKt2masw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9m2ZwomcvI_QUTtQq9h91Q.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><p>You can also use the <em>edit</em> feature next to the <em>duration timer</em> to edit the actual effort took to resolve the incident, if needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9m2ZwomcvI_QUTtQq9h91Q.png" /><figcaption>(Screenshot Source: PagerDuty.com)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*4MOfQlTgxBgJyc9Y5CSEVA.jpeg" /></figure><p>If you found this article helpful helpful, then you might also like a previous article I wrote about <a href="https://medium.com/nerd-for-tech/smooth-sailing-anchoring-your-help-desk-with-docker-containers-on-aws-services-3bcbb36d2ba2">creating a help desk on docker containers.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8864dceebf0" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/pagerduty-101-the-ultimate-guide-for-first-time-site-reliability-engineers-c8864dceebf0">PagerDuty 101: The Ultimate Guide for First-Time Site Reliability Engineers</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Smooth Sailing: Anchoring Your Help Desk with Docker Containers on AWS Services]]></title>
            <link>https://medium.com/@tait.hoglund/smooth-sailing-anchoring-your-help-desk-with-docker-containers-on-aws-services-3bcbb36d2ba2?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/3bcbb36d2ba2</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Sat, 13 May 2023 21:28:28 GMT</pubDate>
            <atom:updated>2023-07-26T11:29:22.095Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How to Build an agile Help Desk using Docker Containers on AWS Services</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dgBmnRWnpZloCshdZnXR7A.png" /></figure><h3>Introduction</h3><p>In today’s fast-paced digital landscape, businesses need robust and efficient solutions to handle customer support and technical assistance. One such solution that combines the power of containerization and cloud computing is the creation of a Help Desk using Docker containers on AWS services.</p><p>In this tech-walkthrough, we will embark on a journey to build a scalable and flexible Help Desk infrastructure that will revolutionize your customer support operations.</p><p>Let’s get building!</p><h3>Pre-Requisites</h3><ul><li>Basic understanding of Docker and containerization concepts.</li><li>Familiarity with AWS services such as Amazon ECS, IAM, VPC, RDS, and CloudWatch.</li><li>Access to an AWS account with sufficient permissions to create and manage resources.</li><li>Prior knowledge of networking principles and configuring security groups in AWS.</li><li>Proficiency in using the command line interface (CLI) for Docker and AWS.</li></ul><h3>Step 1 — Setting up our Instance</h3><p>Login to your AWS account and navigate to the EC2 instance dashboard. Click “Launch Instance”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DXcmh0gAGz2hwKJM6Gdu3w.png" /></figure><p>Give your instance a name. Select an OS and other settings as needed. Finally click “Launch Instance”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*JJVKq7rlnchwAeAQL0QiJQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/1*YwvDqQVGZPwCvFKDvXnL7A.png" /></figure><p>Once launched you will receive a confirmation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*d3zidplsDA7RcrGbypL5VQ.png" /></figure><p>Navigate back to your Instance Dashboard, select your newly created instance and click the “Connect” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tv_1XnwfpvGKPyKFT8URqQ.png" /></figure><p>Under the SSH client tab, locate the ssh command that will allow you to SSH into your instance via your local terminal. Copy the command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*IUPiivJ2dmZPtyj6SLC27A.png" /></figure><p>Paste the command into your local terminal to login to your instance remotely.</p><p>Note: If you used a key pair to create your instance, your .pem key-pair file needs to be in the same directory of running this command. Otherwise, you will receive an “access denied” error.</p><p>Type “yes” when prompted to connect to the instance. And you will be logged into your cloud instance via your remote machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*Zp8RXAljKBMdWfdrSGF-ZA.png" /></figure><h3>Step 2- Updating our Instance &amp; Installing Docker</h3><p>Now that we are logged in remotely, we need to update the packages on our machine by running <em>sudo apt update</em>:</p><pre>sudo apt update</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*QjFwpPtnPGPChisLFxWSWQ.png" /></figure><p>Next, we need to upgrade our local OS to reflect those updated changes by running <em>sudo apt upgrade</em>. You might be prompted to ‘reset’ your system by the CLI.</p><pre>sudo apt upgrade</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*rzEUvKwExq2uhRiPdQM88A.png" /></figure><p>Now that our Ubuntu packages and systems have been downloaded and installed, we need to install docker and docker compose to be able to run containers on our instance. We start by running <em>sudo apt install docker.io</em></p><pre>sudo apt install docker.io</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*neXzkZO-M0hpJ-_FtHQIGg.png" /></figure><p>Next, we install docker-compose by running <em>sudo apt install docker-compose -y</em>.</p><pre>sudo apt install docker-compose -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*0jl3dgzaOygcEVacz81GmA.png" /></figure><p>Now that docker is installed, we can run <em>docker ps</em> to see if any containers are running.</p><p>Since we only just installed docker, there should be no containers listed.</p><pre>sudo docker ps</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*fgEefi6Gki4BGxa8mWU3Bg.png" /></figure><p>Now our environment is ready for us to install our help desk.</p><h3>Step 3: Installing and Configuring our Help Desk</h3><p>Now we need to download our Help Desk and run it in a docker container. Today, I have chosen to build a <a href="https://github.com/Peppermint-Lab/peppermint">Peppermint Help Desk.</a></p><p>Peppermint is a help desk solution similar to ServiceNow, Zendesk, or SysAid, or Zoho.</p><p>Peppermint has a great documentation on how to launch using Docker Containers. Here is a screen shot from the GitHub site. This template is slightly dated, but a good starting point. We are going to use this as our base ‘<em>docker-compose.yml’</em> file for configuring our Peppermint help desk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*tmOPU4tCoALjzbCsXaY8Sw.png" /></figure><p>Starting in our home directory, we need to make a peppermint directory and navigate to it:</p><pre>mkdir peppermint<br>cd peppermint</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*nKfkjODKxMoKaCCEq8vEEg.png" /></figure><p>Now we need to make our <em>docker-compose.yml</em> file which will contain the configuration for installing our Peppermint Help Desk using docker containers.</p><pre>nano docker-compose.yml</pre><p>And paste the following YAML configuration into the file. I’ve updated it a bit from the Peppermint GitHub to reflect updated versions of Docker.</p><p>I also changed the base URL since I am not running this locally. You should also change the “BASE_URL” to to your website DNS.</p><pre>version: &quot;3.8&quot;<br><br>services:<br>  postgres:<br>    container_name: postgres<br>    image: postgres:latest<br>    restart: always<br>    volumes:<br>      - ./docker-data/db:/var/lib/postgresql/data<br>    environment: <br>      - POSTGRES_USER=peppermint<br>      - POSTGRES_PASSWORD=1234<br>      - POSTGRES_DB=peppermint<br><br>  client:<br>    container_name: peppermint<br>    image: pepperlabs/peppermint:latest<br>    ports:<br>      - 5000:5000<br>    restart: on-failure<br>    depends_on:<br>      - postgres<br>    environment:<br>      - PORT=5000<br>      - DB_USERNAME=peppermint<br>      - DB_PASSWORD=1234<br>      - DB_HOST=postgres<br>      - BASE_URL=http://http://ec2-18-224-29-122.us-east-2.compute.amazonaws.com:5000/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*bP_yNjz7gvv0xd_7nRPv4w.png" /></figure><p>Now we only need to run one command to set up our first Peppermint Help Desk container</p><pre>sudo docker-compose up -d</pre><p>If this is your first time pulling down the docker image, it might take a first minutes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*hMNxqRLZ5w8dCqdOGSt6sw.png" /></figure><p>Once completed, you should get a similar confirmation in your CLI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*0kU-3y1N8W1aM2b0ei6pqw.png" /></figure><p>Next, we run docker ps to retrieve the port # to test our new help desk in the browser.</p><pre>sudo docker ps</pre><p>We can see our Help Desk is running on port :5000, which we configured in our docker-compose.yml file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*wxpfb_V7KKNUro9drpF28g.png" /></figure><p>If you were running this locally on your machine, you could type “localhost:5000” in your browser to check to see if your help desk is running.</p><p>Since we launched our via AWS EC2, we need to location our public DNS name and add our port number :5000 to check our results. We can locate this in the EC2 dashboard and paste that Public DNS name in our browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DBnLeZxwERivP9CkH3Jblw.png" /></figure><p>Wait! Why can’t reach our help desk?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8gZ7hcky53u9yHxOixaNBg.png" /></figure><p>Remember our Firewall settings? We forgot to allow ingress traffic from port :5000. Let’s revisit our security group settings and fix that now</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bflq0AsDZJKloE5Oh4YDpA.png" /></figure><p>Now let’s try visiting our public DNS name with the correct port number in our browser to see if our firewall fix changed our results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RPiYih4GPEfu0reJZ8iAkg.png" /><figcaption>With the updated firewall settings, we can now access the login of our Help Desk</figcaption></figure><p>Now, let’s login with our default username and password. The default email for peppermint is “admin@admin.com”. The default password is “1234”. You can change both once you login. (As you should).</p><p>Once logged in you can begin configuring your Help Desk as desired and share it with the world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2BWePAQolRXjmMe2BBWHw.png" /></figure><p>Congratulations! You just launched a peppermint help desk on a docker container using AWS!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*lXvZ_5nWEdBrT4kmW0LOww.jpeg" /></figure><p>If you found this article helpful helpful, then you might also like a previous article I wrote about <a href="https://medium.com/faun/a-rollercoaster-ride-through-kubernetes-security-for-devops-ninjas-f7f47585ea16">security considerations for Kubernetes deployments in production.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3bcbb36d2ba2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Rollercoaster Ride Through Kubernetes Security for DevOps Ninjas]]></title>
            <link>https://faun.pub/a-rollercoaster-ride-through-kubernetes-security-for-devops-ninjas-f7f47585ea16?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7f47585ea16</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Fri, 05 May 2023 02:53:38 GMT</pubDate>
            <atom:updated>2023-05-09T15:12:46.768Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Security Considerations for Kubernetes Deployments in Production</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Vhoz4nopBEfswgiNXPDr4A.png" /></figure><p>As Kubernetes continues to gain popularity, more organizations are adopting it for their production environments. However, with the benefits of Kubernetes come security challenges that must be addressed to ensure the safety and integrity of your production deployment.</p><p>As a Kubernetes DevOps administrator, it’s important to understand the security considerations and best practices that will help you to secure your Kubernetes deployment.</p><p>This article will cover some of the key security considerations that you need to take into account when deploying Kubernetes in production, including authentication and authorization, network security, and RBAC (Role-Based Access Control).</p><p>By implementing these best practices, you can help to protect your Kubernetes deployment and ensure that your organization can benefit from the full potential of this powerful container orchestration platform.</p><h3><strong>Securing your Cluster API</strong></h3><p>As a DevOps engineer, securing your Kubernetes Cluster API should be a top priority. There are several factors to consider when it comes to securing your Kubernetes Cluster API, such as authentication and authorization, MFA, network policies, and RBAC (Role-Based Access Control).</p><p>Firstly, authentication and authorization should be implemented to ensure that only authorized users have access to the Kubernetes Cluster API.</p><p>Secondly, network policies should be enforced to control access to the API from outside the cluster. This includes ingress rules that only allow certain port traffic from specific IP addresses of your development team members.</p><p>Finally, utilizing RBAC (Role-Based Access Control) should be configured to grant or restrict access to specific resources within the cluster.</p><p>These measures will help to prevent unauthorized access and ensure the confidentiality, integrity, and availability of your Kubernetes Cluster API.</p><h3>Secure the Machines Operating your Cluster</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KLJqPNeYGPwaac7e7IRZQw.jpeg" /></figure><p>When it comes to deploying a Kubernetes cluster, a role-based approach is critical to ensuring that your developers and operators have the appropriate level of access to the cluster.</p><p>A role-based approach allows you to define roles for each user or group, with specific permissions and restrictions that determine what they can and cannot do within the cluster.</p><p>For instance, a developer may only need read-only access to your production code, whereas an operator will also need write access. While it’s tempting to use a “one-size fits all” approach to access, the larger your DevOps team becomes the more opportunities for someone to inadvertently screw up your production environment. Yikes!</p><p>It’s important to consider several factors when implementing a role-based approach, such as defining roles and permissions, managing access control, and auditing activity within the cluster.</p><p>By properly defining roles and permissions, you can ensure that users only have access to the resources they need to perform their tasks.</p><p>Managing access control is also crucial to ensure that users are not granted unnecessary access to sensitive data or resources. Finally, auditing activity within the cluster is important to maintain visibility into who is accessing the cluster and what actions they are performing.</p><p>By taking these factors into account, you can implement a role-based approach for your Kubernetes deployment that helps to maintain the security and integrity of your cluster.</p><h3>Proper Code Pushes via a Secured CI/CD Pipeline</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cZW2_-SOKc13TdxzmxBpRQ.png" /></figure><p>The ONLY way you should be pushing code into production is through a thorough a secured CI/CD pipeline to ensure your code is tested, vetted, and that your cluster is protected against intrusion.</p><p>This also means that your CI/CD pipeline needs to be properly secured and that each integration of code <strong><em>should be</em></strong> examined and tested by another developer.</p><p>A proper CI/CD pipeline process typically includes:</p><ol><li>Pulling Down Production (or latest) code</li><li>Branching from Production Code to implement new features or bug fixes.</li><li>Building and Testing pushes in a production-level development environment.</li><li>Pushing code for review and testing from another developer in a production-grade development environment.</li></ol><p>All of this is done prior to a final review for pushing to production and selecting the right deployment strategy.</p><p>The pipeline should be designed with security, scalability, and reliability in mind, with proper access control mechanisms, load balancing, and robust error handling and rollback mechanisms.</p><p>The pipeline should also be monitored and optimized regularly to ensure that it continues to meet the needs of the production environment.</p><p>By following a well-designed and well-maintained CI/CD pipeline process, DevOps teams can achieve faster, more efficient software delivery with fewer errors and greater confidence in the quality of their software.</p><p>Without protecting access to your CI/CD pipeline whether that’s Jenkins, a GitHub private repository, etc. preferably via access tokens, you are providing another vulnerability for unvetted code to be pushed to your production cluster.</p><h3>Installing Vulnerability detection on your Kubernetes Cluster</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WI5UocY3nGUC8Pg6oJw1kg.jpeg" /></figure><p>Vulnerability detection is a critical aspect of securing your Kubernetes cluster, as it allows you to identify and remediate security risks before they can be exploited by attackers.</p><p>One effective way to implement vulnerability detection on your Kubernetes cluster is by using daemons sets, which are Kubernetes objects that ensure that a specific pod is running on every node in the cluster.</p><p>By deploying a daemon set that runs a vulnerability scanner, you can scan each node in your cluster for vulnerabilities and get a comprehensive report of potential security issues.</p><p>One example of a vulnerability scanner that can be deployed as a daemon set on Kubernetes is Clair, an open-source vulnerability scanner that is designed to scan container images for known vulnerabilities. Clair can be integrated with Kubernetes via a Helm chart, which provides an easy way to deploy and configure the scanner as a daemon set.</p><p>Once you have deployed a vulnerability scanner as a daemon set on your Kubernetes cluster, you can set up alerts and notifications to inform you of any new vulnerabilities that are detected.</p><p>You can also integrate the scanner with your CI/CD pipeline to automatically scan new container images as they are built and deployed to your production environment.</p><p>Monitoring solution on the cluster <br>(open source ELK stack on cluster)<br>When a problem happens monitoring is already happening.</p><p>Testing every process you have.</p><h3><strong>Cluster Fail Over Protection</strong></h3><p>One way to implement failover protection on your Kubernetes cluster is by using a tool like Kubernetes HAProxy Controller, which is a load balancer that automatically configures HAProxy to balance traffic across the nodes in your cluster.</p><p>HAProxy Controller can also detect and recover from failures in your cluster by automatically re-routing traffic to healthy nodes.</p><p>Failover protection is a critical aspect of ensuring the availability and reliability of your Kubernetes cluster.</p><p>When implementing failover protection, there are several factors to consider, such as redundancy, load balancing, and disaster recovery. Redundancy involves ensuring that critical components of your cluster, such as master nodes, are duplicated to prevent single points of failure.</p><p>Load balancing is important to distribute traffic across multiple nodes to prevent any one node from becoming overwhelmed. Finally, disaster recovery is crucial to ensure that your cluster can recover quickly from any unforeseen events or outages.</p><p>Another tool that can be used for failover protection is <em>etcd</em>, a distributed key-value store that is used by Kubernetes to store critical configuration data.</p><p>By setting up a highly-available <em>etcd</em> cluster with redundancy and disaster recovery mechanisms, you can ensure that your Kubernetes cluster remains stable and reliable even in the event of node failures.</p><p>While there are certainly more aspects I could go into, by following these 5 simple considerations, you will have a great start at protecting your deployments from misfortune.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*1DLAUFyH3QL4JcMK-pxA8Q.jpeg" /></figure><p>If you found this security article on Kubernetes helpful helpful. You might also like a previous article I wrote about <a href="https://medium.com/aws-in-plain-english/kubernetes-deployments-for-micro-service-applications-fa3236592c9f">Kubernetes Deployments for Micro-service Applications.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*uGCWAIQ3xyD0a1Hr.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7f47585ea16" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/a-rollercoaster-ride-through-kubernetes-security-for-devops-ninjas-f7f47585ea16">A Rollercoaster Ride Through Kubernetes Security for DevOps Ninjas</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraforming the Clouds]]></title>
            <link>https://aws.plainenglish.io/terraforming-the-clouds-fc4977347e9?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc4977347e9</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Sat, 15 Apr 2023 02:22:35 GMT</pubDate>
            <atom:updated>2023-04-15T20:23:35.465Z</atom:updated>
            <content:encoded><![CDATA[<p>How to develop and deploy via a CI/CD Pipeline in Terraform Cloud using a GitHub repository on AWS</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Th9-Si2HCwnMhOKYN5HRIw.png" /></figure><h3>Use-Case</h3><p><em>A fast-growing online retail company is looking to scale its operations and cater to an ever-increasing customer base. The company’s current infrastructure is struggling to accommodate the surge in traffic, resulting in slow page load times, frequent outages, and suboptimal user experience. These issues are negatively impacting customer satisfaction, conversion rates, and the company’s overall reputation in the market. To address these challenges and ensure a seamless online shopping experience, the company needs a highly available, scalable, and secure infrastructure that can handle fluctuating traffic patterns and support the rapid deployment of new features and enhancements.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pd4GcPhNSBRm3sfrpEBFSg.png" /></figure><p><em>The proposed two-tier AWS architecture, with its custom VPC, separate public and private subnets, and appropriate route tables, provides the necessary foundation for the company’s growing needs. The web server tier, featuring auto-scaled EC2 instances, ensures optimal performance and high availability, even during peak traffic periods.</em></p><p><em>The RDS tier provides a secure and efficient management of the company’s crucial customer and transaction data, while the Application Load Balancer intelligently routes incoming traffic, ensuring smooth and responsive customer interactions.</em></p><p><em>Additionally, using Terraform Cloud for continuous integration and deployment, the company can rapidly roll out updates and improvements to stay ahead of the competition. By implementing this architecture, the online retail company can significantly enhance its infrastructure capabilities, providing an exceptional shopping experience to its customers and driving long-term growth and success.</em></p><p>Let’s get started!</p><h3>Prerequisites</h3><ul><li>Development environment set up with AWS CLI and Terraform installed (I will be using Cloud9 with an S3 Bucket).</li><li>Basic knowledge of AWS services (especially features within EC2, VPC, and S3).</li><li>Networking concepts (subnetting, route tables, and gateways).</li><li>Web server administration experience.</li><li>Security best practices (IAM roles and policies).</li><li>Terraform best practices.</li><li>An AWS and Terraform Cloud Accounts (Both are free for basic services)</li></ul><h3>Phase 1 — Setting up Terraform Cloud as our Back-End to Host our CI/CD Pipeline</h3><p>If you don’t have a Terraform Cloud account yet, you can sign up for one for free by visiting the Terraform Cloud <a href="https://app.terraform.io/signup/account">sign-up page </a>by providing your email address, a username, and a password.</p><p>Once you register, check your inbox for a verification email and click on the link provided to verify your account.</p><p>Once your account is created, head back to the Terraform cloud login page and create an organization.</p><p>Provide a unique name for your organization and click “Create organization”.</p><p>In your newly created organization, click on the “New Workspace” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eemx1HzLv96KiMnc1pKCJA.png" /></figure><p>Choose the “CLI-driven workflow” option, as we will be using Cloud9 CLI to interact with Terraform Cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xyY4ICKzdpSaZ5-Gnq4wfw.png" /></figure><p>Enter a name for your workspace and click “Create workspace”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s_R5mMwOLUT152RI6jPvIw.png" /></figure><p>In your workspace, navigate to the “Variables” tab. Here, you will add your AWS access key, secret access key, and region as environment variables. Click on “Add Variable” and create the following variables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZIME9vUyaUJr-dSAi2AAJQ.png" /></figure><p>I am going to add three variables, which will allow my Terraform CI/CD pipeline in terraform cloud to interact with AWS resources in my account:</p><ol><li>My AWS_ACCESS_KEY_ID, Value: &lt;your_aws_access_key&gt;</li></ol><p>2. My AWS_SECRET_ACCESS_KEY, Value: &lt;your_aws_secret_access_key&gt;</p><p>3. My AWS_DEFAULT_REGION, Value: &lt;your_aws_region&gt;,</p><p>Make sure to mark the AWS access and secret keys as sensitive to ensure they are encrypted and not exposed in logs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bp-tjvgZ5tfdYU7P_SClag.png" /></figure><p>Next, generate a Terraform Cloud API token: Click on your avatar in the top-right corner then go to “User settings” and “Tokens”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rAu36HnUiMVscMCQc7qEgQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bpDU65nSw-nO6oZzS29caw.png" /></figure><p>Create a new API token by providing a description and clicking “Generate API token”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NHhSXZxqH7SbZ35CNlK7aA.png" /></figure><p>Copy the generated token and store it securely, as you will not be able to see it again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FdXVscFrkSBdQvQYeSmQ7w.png" /></figure><p>Next, head over to your IDE in Cloud9 via your AWS Account. From the CLI, ensure that Terraform is installed and updated on your Cloud9 instance.</p><pre>sudo yum update -y &amp;&amp; sudo yum install terraform -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*LTZVg1K3wAfgmAEq0d6iQw.png" /></figure><p>Next, configure the Terraform CLI with your API token: On your Cloud9 CLI, run the following command to log in to your Terraform Cloud account:</p><pre>terraform login</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/537/1*P9pilJUJeGPMETg5pMMasQ.png" /></figure><p>When prompted, paste your Terraform Cloud API token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*UQ4Z-P3iZEbPvDLN4_f8BA.png" /></figure><p>Next, initialize your terraform configuration directory on Cloud9 by creating a <em>‘backend.tf’</em> file with the following content:</p><pre>terraform {<br>  backend &quot;remote&quot; {<br>    organization = &quot;&lt;your_organization_name&gt;&quot;<br>    workspaces {<br>      name = &quot;&lt;your_workspace_name&gt;&quot;<br>    }<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*Tzk1QzlN53CXmViN6iQh6g.png" /></figure><p>We have successfully configured a CI/CD Pipeline for our Terraform files in our Cloud9 IDE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*DjZ-KsupN7nbnBY6M2PhtA.png" /></figure><p>We can verify that our pipeline is working by running a terraform plan or apply command.</p><p>Even though we have not configured any infrastructure to run, terraform Cloud will show a record of this.</p><p>This tells us any future terraform configuration files that we apply will be caught by Terraform Cloud:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t1Ghv9aoXloxbMn56Yh_aA.png" /></figure><h3>Phase 2 — Our Terraform Configuration Files</h3><p>I have broken up this two-tier architecture into a number of more manageable files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*_-w4CchgsNmd7S9DP562RQ.png" /></figure><p>Here is <a href="https://github.com/hogtai/Public_Repo/tree/main/Terraform/Week22">a link</a> to my public GitHub repo with all the files in them. You are welcome to clone the repo yourself to your local IDE.</p><p>Additionally, to maintain the re-usability of your terraform code, you should avoid hard-coding any specific items like IP addresses or specific names to your .tf configuration files.</p><p>Instead, establish variables in a separate <em>variables.tf</em> file. If this will be your case, make sure you code your variables to your specific values.</p><p>One final thing to keep in mind would be to <em>nest</em> your resources using <a href="https://www.terraform.io/docs/language/modules/index.html">parent and child modules</a>. However for the sake of simplicity, I have placed all my variables into one single <em>variables.tf </em>file and all my configuration files into separate <em>.tf</em> files in a single directory.</p><h3>Phase 3 — Launching our Cloud Resources with Terraform</h3><p>Once our terraform configuration files are established and our variables are set, we will run the following commands and in this order to ensure that:</p><ol><li>Terraform has any needed provider dependencies downloaded to our local instance.</li><li>All of our .tf files are optimized for human-readability.</li></ol><p>3. All of our .tf files are validated and ready for deployment.</p><p>4. We can visualize our resource deployment prior to launch</p><p>5. Terraform launches our cloud resources for our two-tier architecture.</p><pre>terraform init // Installs any provider resources in our .tf files into our present working directory<br>terraform fmt // Formats our .tf files to be human-readable<br>terraform validate // Verifies that our .tf files contain no errors in syntax, logic, or arugments<br>terraform plan // Displays what resources will be deployed, changed, or destroyed<br>terraform apply // Deploys our resources</pre><p>After running ‘<em>terraform plan</em>’, you should see the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*yhkIeS8BKLOj3Q_8Ywc3PA.png" /></figure><p>Once you run ‘<em>terraform apply -auto-approve’</em>, you will see terraform starting to launch your resources. This could take several minutes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*FOLYpO21zksJx-h12bRhrw.png" /></figure><p>As you can see, it took almost 7 minutes for terraform to launch my resources:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/1*xxuF6fOTtHIfOxJvVJJhrQ.png" /><figcaption>In my directory, I also had an s3.tf from a previous project that was already provisioned, which is why I received an “operation failed”. However, all other resources were launched successfully.</figcaption></figure><h3>Phase 4— Verifying Results</h3><p>We should be able to test our deployment by visiting the DNS name of our Application Load Balancer for our Web-Tier:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xp7RFaSdgne2L4Ffn47mLg.png" /></figure><p>Last test is to test the connectivity between our web-tier WordPress instances with our backend RDS Database.</p><p>First, we can check to verify that our RDS Database can not be accessed publicly. We can visit our RDS instance profile in our AWS console to verify this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b75kf7ETVSYqBHkWqRzQtg.png" /></figure><p>Finally, from within our WordPress instance we need to ensure that our Web-Tier can read and write to our Database-Tier instance.</p><p>Start by SSH’ing into one of our WordPress instances. My AMI thru Bitnami had different default values for the <em>username </em>for default SSH login. I had to locate it via the system logs for my instance in the EC2 instance menu.</p><p>If you are having difficulty logging into your EC2 WordPress instance, this might be why. Check your logs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rRzwtsF9rxjO9graIi-7lQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hAX4EvrWOi5vKxl5KoXZ6g.png" /></figure><pre>ssh -i /path/to/your/key_pair.pem user@&lt;ec2-instance-ip&gt;</pre><p>Since our RDS database is Aurora MySQL compatible, we can use the MySQL CLI client to test the connectivity. First, install the MySQL client on your EC2 instance:</p><pre>sudo yum install -y mysql<br>sudo yum update<br>sudo yum install -y mysql-client</pre><p>Now, we can test the connectivity to your RDS database using the MySQL client. Our RDS endpoint value can be found via our RDS database instance via the AWS console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZkpqWp-Wp_hWinZ2g2dm-A.png" /></figure><p>The values for our username and password were set in our <em>variables.tf </em>file prior to running terraform apply.</p><pre>mysql -h &lt;rds-endpoint&gt; -u &lt;db-master-username&gt; -p</pre><p>After running the above command, you will be prompted for the default password</p><p>If you’re able to connect successfully, you’ll see the MySQL prompt, indicating that your EC2 instance can connect to the RDS database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lrgRL0ifykcpDcesNVQqEw.png" /></figure><p>Nice work! You’ve just successfully automated the deployment of a two-tier, highly available cloud infrastructure using Terraform!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/0*6iVgMqLJdb0H-JM_.jpeg" /></figure><p>If you found this terraform walk-thru helpful. You might also like a previous article I wrote about a<a href="https://medium.com/devops-dev/taming-the-traffic-tsunami-autoscaling-using-terraform-1be82e367175">utoscaling using Terraform.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc4977347e9" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/terraforming-the-clouds-fc4977347e9">Terraforming the Clouds</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taming the Traffic Tsunami: Autoscaling using Terraform]]></title>
            <link>https://blog.devops.dev/taming-the-traffic-tsunami-autoscaling-using-terraform-1be82e367175?source=rss-efc3aac42050------2</link>
            <guid isPermaLink="false">https://medium.com/p/1be82e367175</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Tait Hoglund]]></dc:creator>
            <pubDate>Thu, 06 Apr 2023 03:09:21 GMT</pubDate>
            <atom:updated>2023-04-14T01:46:02.975Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Effortlessly Scale Your E-Commerce Platform with Auto Scaling Groups and Terraform on AWS — A Comprehensive Guide for DevOps Engineers</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XHto4lE3aXAfNtU0cZf5KQ.png" /></figure><h3>Use-Case</h3><p><em>A retail e-commerce business is seeking to manage the increased traffic flow during a promotional period. The primary objective is to maintain consistent website accessibility and responsiveness for customers, even when traffic spikes occur.</em></p><p><em>To address this challenge, you can use Terraform to implement an Auto Scaling group covering two subnets within the default VPC, guaranteeing both high availability and fault tolerance.</em></p><p><em>As a result, the Auto Scaling group will dynamically adjust its capacity depending on the incoming traffic, ensuring the website continually provides an optimal customer experience.</em></p><p>In this tutorial, I am going to walk you through how to use Terraform to deploy and manage this type of cloud infrastructure to ensure high availability and servicing of your company website when traffic is unpredictable.</p><p>A couple of disclaimers: I will be building this infrastructure in multiple Terraform files. Many of the steps and commands will be repeated throughout each step. I designed this for folks building their first 2-tiered architecture in Terraform in mind.</p><p>Ideally, to make this build repeatable, I would substitute out hardcoded values for variables in a variables.tf file, in order to protect sensitive information and make our Terraform reusable. However, that is beyond the scope of this article.</p><p>Let’s get started!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*aLczrsb0s6LDUMaS5uKOwA.png" /></figure><h3>Prerequisites:</h3><ol><li>Development environment set up with AWS CLI and Terraform installed (I will be using Cloud9 with an S3 Bucket).</li><li>Basic knowledge of AWS services (especially features within EC2, VPC, and S3).</li><li>Networking concepts (subnetting, route tables, and gateways).</li><li>Web server administration experience.</li><li>Security best practices (IAM roles and policies).</li><li>Terraform best practices.</li></ol><h3>Phase 1 — Creating our S3 Bucket to Host our Terraform Files</h3><p>Let’s start by setting up the back-end for our terraform configuration files in Amazon S3:</p><pre>resource &quot;aws_s3_bucket&quot; &quot;project21_s3_bucket_84135&quot; {<br>  bucket = &quot;project21-s3-bucket-84135&quot; // Create an S3 bucket named &quot;project21-s3-bucket-84135&quot;<br>}<br><br>resource &quot;aws_s3_bucket_versioning&quot; &quot;project21_s3_bucket_84135&quot; {<br>  bucket = aws_s3_bucket.project21_s3_bucket_84135.id // Enable versioning on the S3 bucket created above<br>  <br>  versioning_configuration {<br>    status = &quot;Enabled&quot; // Set versioning status to &quot;Enabled&quot;<br>  }<br>}<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;project21_s3_bucket_84135_access_block&quot; {<br>  bucket = aws_s3_bucket.project21_s3_bucket_84135.id // Block public access to the S3 bucket created above<br><br>  block_public_acls   = true<br>  block_public_policy = true<br>  ignore_public_acls  = true<br>  restrict_public_buckets = true<br>}<br><br>resource &quot;aws_kms_key&quot; &quot;s3_bucket_key&quot; {<br>  description             = &quot;S3 bucket key for project21_s3_bucket_84135&quot; // Create an AWS KMS key for the S3 bucket created above<br>  deletion_window_in_days = 7<br>  key_usage               = &quot;ENCRYPT_DECRYPT&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*iwromDgEbuIZDU9ixu6i4g.png" /></figure><p>Next, we run the following commands deploy our back-end S3 bucket using Terraform commands</p><pre>terraform init // Installs any provider resources in our .tf files into our present working directory<br>terraform fmt // Formats our .tf files to be human-readable<br>terraform validate // Verifies that our .tf files contain no errors in syntax, logic, or arugments<br>terraform plan // Displays what resources will be deployed, changed, or destroyed<br>terraform apply // Deploys our resources</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*rPSz3FDG5Tb8kl66tchAkQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*7z63KJrcF5BvgIRyfn1QMA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*Y6T94deDVQ4dll5i7cgvRg.png" /></figure><p>Now that our S3 bucket is created. We can test by copying our s3.tf file from our local environment into our s3 bucket using the following command:</p><pre>aws s3 cp s3.tf s3://project21-s3-bucket-84135/s3.tf // Copies our s3.tf file from our local directory into our  S3 bucket</pre><p>We will get validation in our CLI and we can also verify by visiting our S3 Bucket via our AWS Console.</p><p>This S3 bucket is where we will continue to place our terraform files as we build them.</p><p>Using an S3 bucket with versioning allows us to revert back to previous versions of our terraform if needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iZEvA0gClWrHbFIoD1uBMQ.png" /></figure><p>As we progress forward in creating our .tf files, we will want to also define any variables in a separate ‘variables.tf’ file. This ensures that we can easily redeploy our terraform by only needing to re-define our variables in one spot.</p><p>As we progress, I will show you both the screenshots of when I first built the architecture, however the code blocks of each .tf resource file will contain the variable names that will also be defined in the variable.tf file.</p><p>We will continue to build out and add additional variables to our variables.tf file as we progress through our build.</p><h3>Phase 2: Creating our VPC with attached Internet Gateway</h3><p>Next, we will create our <em>vpc.tf </em>which will contain the settings for our VPC</p><pre># Set up the AWS provider and the region we&#39;ll be using<br>provider &quot;aws&quot; {<br>  region = var.aws_region<br>}<br><br># Define a new VPC resource<br>resource &quot;aws_vpc&quot; &quot;project_21_vpc&quot; {<br><br># Specify the CIDR block for the VPC<br>cidr_block       = var.vpc_cidr_block<br>  <br># Use the default tenancy setting for the VPC<br>instance_tenancy = var.vpc_instance_tenancy<br><br># Assign a name tag to the VPC<br> tags = {<br>    Name = var.vpc_name_tag<br>  }<br>}<br><br># Create an Internet Gateway resource and attach it to the VPC<br>resource &quot;aws_internet_gateway&quot; &quot;project_21_internet_gateway&quot; {<br><br># Use the ID of the VPC we just created<br>  vpc_id = aws_vpc.project_21_vpc.id<br><br>  # Assign a name tag to the Internet Gateway<br>  tags = {<br>    Name = var.internet_gateway_name_tag<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*Dp46xB7kpPmmyf_pkYKYvg.png" /></figure><p>Additionally, we add the variables defined in our <em>vpc.tf</em> file to a new file called “<em>variables.tf</em>”</p><pre>variable &quot;aws_region&quot; {<br>  description = &quot;The AWS region to deploy resources in&quot;<br>  default     = &quot;us-east-2&quot;<br>}<br><br>variable &quot;vpc_cidr_block&quot; {<br>  description = &quot;The CIDR block for the VPC&quot;<br>  default     = &quot;10.10.0.0/16&quot;<br>}<br><br>variable &quot;vpc_instance_tenancy&quot; {<br>  description = &quot;The instance tenancy for the VPC&quot;<br>  default     = &quot;default&quot;<br>}<br><br>variable &quot;vpc_name_tag&quot; {<br>  description = &quot;The name tag for the VPC&quot;<br>  default     = &quot;Project_21-Terraform&quot;<br>}<br><br>variable &quot;internet_gateway_name_tag&quot; {<br>  description = &quot;The name tag for the Internet Gateway&quot;<br>  default     = &quot;project_21_internet_gateway&quot;<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence. We technically don’t need to run <em>terraform init </em>since we know our infrastructure is only using AWS and terraform providers, but it’s a good habit to practice especially in multi-cloud environments.</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once we confirm our deployment by running <em>terraform apply</em>, we receive another verification via our CLI that our VPC and Internet Gateway were provisioned successfully:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*O80owOSv08X4VYh6z_AjUA.png" /></figure><h3>Phase 3 — Creating Two Public Subnets and Two Private Subnets for High Availability</h3><p>Next, we will create four total subnets-two public, and two private in a <em>subnet.tf</em> file. These subnets will be launched within the VPC we just created.</p><pre># Two Public &amp;&amp; Two Private Subnets<br>resource &quot;aws_subnet&quot; &quot;public_subnet1_project21&quot; {<br>  vpc_id            = aws_vpc.project_21_vpc.id<br>  cidr_block        = var.public_subnet1_cidr_block<br>  availability_zone = var.availability_zone_a<br><br>  tags = {<br>    Name = var.public_subnet1_name_tag<br>  }<br>}<br><br>resource &quot;aws_subnet&quot; &quot;public_subnet2_project21&quot; {<br>  vpc_id            = aws_vpc.project_21_vpc.id<br>  cidr_block        = var.public_subnet2_cidr_block<br>  availability_zone = var.availability_zone_b<br><br>  tags = {<br>    Name = var.public_subnet2_name_tag<br>  }<br>}<br><br>resource &quot;aws_subnet&quot; &quot;private_subnet1_project21&quot; {<br>  vpc_id            = aws_vpc.project_21_vpc.id<br>  cidr_block        = var.private_subnet1_cidr_block<br>  availability_zone = var.availability_zone_a<br><br>  tags = {<br>    Name = var.private_subnet1_name_tag<br>  }<br>}<br><br>resource &quot;aws_subnet&quot; &quot;private_subnet2_project21&quot; {<br>  vpc_id            = aws_vpc.project_21_vpc.id<br>  cidr_block        = var.private_subnet2_cidr_block<br>  availability_zone = var.availability_zone_b<br><br>  tags = {<br>    Name = var.private_subnet2_name_tag<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*IKR_pwXACIh3rO_I7Zierg.png" /></figure><p>Additionally, we add these as new variables to our “<em>variables.tf</em>” file:</p><pre># New variables for subnet.tf<br>variable &quot;public_subnet1_cidr_block&quot; {<br>  description = &quot;The CIDR block for public subnet 1&quot;<br>  default     = &quot;10.10.1.0/24&quot;<br>}<br><br>variable &quot;public_subnet2_cidr_block&quot; {<br>  description = &quot;The CIDR block for public subnet 2&quot;<br>  default     = &quot;10.10.2.0/24&quot;<br>}<br><br>variable &quot;private_subnet1_cidr_block&quot; {<br>  description = &quot;The CIDR block for private subnet 1&quot;<br>  default     = &quot;10.10.3.0/24&quot;<br>}<br><br>variable &quot;private_subnet2_cidr_block&quot; {<br>  description = &quot;The CIDR block for private subnet 2&quot;<br>  default     = &quot;10.10.4.0/24&quot;<br>}<br><br>variable &quot;availability_zone_a&quot; {<br>  description = &quot;The first availability zone&quot;<br>  default     = &quot;us-east-2a&quot;<br>}<br><br>variable &quot;availability_zone_b&quot; {<br>  description = &quot;The second availability zone&quot;<br>  default     = &quot;us-east-2b&quot;<br>}<br><br>variable &quot;public_subnet1_name_tag&quot; {<br>  description = &quot;The name tag for public subnet 1&quot;<br>  default     = &quot;public_subnet1_project21&quot;<br>}<br><br>variable &quot;public_subnet2_name_tag&quot; {<br>  description = &quot;The name tag for public subnet 2&quot;<br>  default     = &quot;public_subnet2_project21&quot;<br>}<br><br>variable &quot;private_subnet1_name_tag&quot; {<br>  description = &quot;The name tag for private subnet 1&quot;<br>  default     = &quot;private_subnet1_project21&quot;<br>}<br><br>variable &quot;private_subnet2_name_tag&quot; {<br>  description = &quot;The name tag for private subnet 2&quot;<br>  default     = &quot;private_subnet2_project21&quot;<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence.</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once we confirm our deployment by running <em>terraform apply</em>, we receive another verification via our CLI that our four subnets were provisioned successfully:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*6uzc1a5oI61JVpnP7-aaGg.png" /></figure><h3>Phase 4— Creating Two NAT Gateways — One in each Public Subnet</h3><p>Next, we will create two NAT gateways in a <em>natgateways.tf</em> file. Each NAT gateway will be attached to each of our public subnets.</p><p>NAT gateways will allow our EC2 instances outbound access to the internet to download needed updates while also not exposing them directly to inbound traffic that could compromise their security.</p><pre>resource &quot;aws_eip&quot; &quot;nat_eip1&quot; {<br>  vpc = true<br><br>  tags = {<br>    Name = var.nat_eip1_name_tag<br>  }<br>}<br><br>resource &quot;aws_nat_gateway&quot; &quot;public_subnet1_nat_gateway&quot; {<br>  allocation_id = aws_eip.nat_eip1.id<br>  subnet_id     = aws_subnet.public_subnet1_project21.id<br><br>  tags = {<br>    Name = var.nat_gateway1_name_tag<br>  }<br>}<br><br>resource &quot;aws_eip&quot; &quot;nat_eip2&quot; {<br>  vpc = true<br><br>  tags = {<br>    Name = var.nat_eip2_name_tag<br>  }<br>}<br><br>resource &quot;aws_nat_gateway&quot; &quot;public_subnet2_nat_gateway&quot; {<br>  allocation_id = aws_eip.nat_eip2.id<br>  subnet_id     = aws_subnet.public_subnet2_project21.id<br><br>  tags = {<br>    Name = var.nat_gateway2_name_tag<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*HaYjjreVgTy5hhMjVfvdAw.png" /></figure><p>Additionally, we add these as new variables to our “<em>variables.tf</em>” file:</p><pre>variable &quot;nat_eip1_name_tag&quot; {<br>  description = &quot;The name tag for the NAT Elastic IP 1&quot;<br>  default     = &quot;public_subnet1_NAT_eip&quot;<br>}<br><br>variable &quot;nat_gateway1_name_tag&quot; {<br>  description = &quot;The name tag for the NAT Gateway 1&quot;<br>  default     = &quot;public_subnet1_NAT_gateway&quot;<br>}<br><br>variable &quot;nat_eip2_name_tag&quot; {<br>  description = &quot;The name tag for the NAT Elastic IP 2&quot;<br>  default     = &quot;public_subnet2_NAT_eip&quot;<br>}<br><br>variable &quot;nat_gateway2_name_tag&quot; {<br>  description = &quot;The name tag for the NAT Gateway 2&quot;<br>  default     = &quot;public_subnet2_NAT_gateway&quot;<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence.</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that our two NAT gateways were provisioned and attached to both public subnets successfully:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*n6JmVvEqv3Xm8U_-5ywxwQ.png" /></figure><h3>Phase 5— Adding a Public Route Table to VPC</h3><p>Next, we will create a public route table in a <em>public-route-table.tf</em> file, which will be assigned to our VPC:</p><pre>resource &quot;aws_route_table&quot; &quot;project21_vpc_public_route_table&quot; {<br>  vpc_id = aws_vpc.project_21_vpc.id<br><br>  tags = {<br>    Name = var.public_route_table_name_tag<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*juQx9s_5FVb_tPBBCUoN8A.png" /></figure><p>Additionally, we add this variable to our “<em>variables.tf</em>” file:</p><pre>variable &quot;public_route_table_name_tag&quot; {<br>  description = &quot;The name tag for the public route table&quot;<br>  default     = &quot;project21_vpc_public_route_table&quot;<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that our public route table was deployed</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*pLRSQFXJrtAGZkCI92r5AQ.png" /></figure><h3>Phase 6— Configuring our Public Route Table</h3><p>Next, we need to configure route table in our <em>public-route-table-config.tf</em> file.</p><p>This tells the route to direct any traffic destined for an address outside the VPC to the internet gateway. By creating this route, resources in the VPC’s public subnet can communicate with resources outside the VPC.</p><pre>resource &quot;aws_route&quot; &quot;public_route_table_internet_gateway&quot; {<br>  route_table_id         = aws_route_table.project21_vpc_public_route_table.id<br>  destination_cidr_block = &quot;0.0.0.0/0&quot;<br>  gateway_id             = aws_internet_gateway.project_21_internet_gateway.id<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*JE1J0Ct04Gpz2OwCeRDUCg.png" /></figure><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that we have configured our public route table to project specifications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*6RRiKWjGJnp7iZQTvru1Xw.png" /></figure><h3>Phase 7— Associate our Public Subnets to the Public Route Table</h3><p>Next, we will create an association with our Public Subnets in a <em>public-subnet-route-table-config.tf</em> file.</p><p>This means that any resources launched in the two public subnets will have access to the internet via the public route table, which is configured with our internet gateway.</p><pre>resource &quot;aws_route_table_association&quot; &quot;public_subnet_association1&quot; {<br>  subnet_id      = aws_subnet.public_subnet1_project21.id<br>  route_table_id = aws_route_table.project21_vpc_public_route_table.id<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;public_subnet_association2&quot; {<br>  subnet_id      = aws_subnet.public_subnet2_project21.id<br>  route_table_id = aws_route_table.project21_vpc_public_route_table.id<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*LgzM4665r0Hq0Fs1Vtmvpw.png" /></figure><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that our public subnets have been associated with our public route table</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*cYbJpe8ABLZKKB9r_OAvjQ.png" /></figure><h3>Phase 8— Creating Two Private Route Tables</h3><p>Next, we will create two private route tables in a <em>private-route-tables.tf</em> file. It’s typically more appropriate to create separate private route tables for each of private subnets and associate each private route table with a respective NAT Gateway.</p><pre>resource &quot;aws_route_table&quot; &quot;project21_vpc_private_route_table1&quot; {<br>  vpc_id = aws_vpc.project_21_vpc.id<br><br>  tags = {<br>    Name = var.private_route_table1_name_tag<br>  }<br>}<br><br>resource &quot;aws_route_table&quot; &quot;project21_vpc_private_route_table2&quot; {<br>  vpc_id = aws_vpc.project_21_vpc.id<br><br>  tags = {<br>    Name = var.private_route_table2_name_tag<br>  }<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;private_subnet1_association&quot; {<br>  subnet_id      = aws_subnet.private_subnet1_project21.id<br>  route_table_id = aws_route_table.project21_vpc_private_route_table1.id<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;private_subnet2_association&quot; {<br>  subnet_id      = aws_subnet.private_subnet2_project21.id<br>  route_table_id = aws_route_table.project21_vpc_private_route_table2.id<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*LzfZKiILYyNVLQeIavU9zw.png" /></figure><p>Additionally, we need to add these variables to our “<em>variables.tf</em>” file:</p><pre>variable &quot;private_route_table1_name_tag&quot; {<br>  description = &quot;The name tag for the first private route table&quot;<br>  default     = &quot;project21_vpc_private_route_table1&quot;<br>}<br><br>variable &quot;private_route_table2_name_tag&quot; {<br>  description = &quot;The name tag for the second private route table&quot;<br>  default     = &quot;project21_vpc_private_route_table2&quot;<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that our private route tables were created and that each was associated with a different private subnet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*XWPaTQewd4GU31Y_jFTRdA.png" /></figure><h3>Phase 9— Creating our Web-Tier Security Group</h3><p>Next, we will create our web-tier security group in a <em>sg.tf</em> file.</p><p>Our <em>web tier security group</em> is defined to allow inbound traffic on ports 80 (HTTP), 443 (HTTPS), and 22 (SSH) from any IP address (0.0.0.0/0).</p><p>The security group also allows all outbound traffic to any IP address and created within the VPC that we created earlier.</p><pre>resource &quot;aws_security_group&quot; &quot;project21_webtier_sg&quot; {<br>  name        = var.sg_name<br>  description = var.sg_description<br>  vpc_id      = aws_vpc.project_21_vpc.id<br><br>  dynamic &quot;ingress&quot; {<br>    for_each = var.sg_ingress_ports<br>    content {<br>      from_port = ingress.value.from_port<br>      to_port   = ingress.value.to_port<br>      protocol  = ingress.value.protocol<br>      cidr_blocks = ingress.value.protocol == &quot;tcp&quot; &amp;&amp; ingress.value.from_port == 22 ? var.sg_ssh_cidr_blocks : [&quot;0.0.0.0/0&quot;]<br>    }<br>  }<br><br>  egress {<br>    from_port   = var.sg_egress_rule.from_port<br>    to_port     = var.sg_egress_rule.to_port<br>    protocol    = var.sg_egress_rule.protocol<br>    cidr_blocks = var.sg_egress_rule.cidr_blocks<br>  }<br><br>  tags = var.sg_tags<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*6HTeXjDpsapuJ2_faOI5sw.png" /></figure><p>Additionally, we need to add these variables to our “<em>variables.tf</em>” file:</p><pre>variable &quot;sg_name&quot; {<br>  default = &quot;project21_webtier_sg&quot;<br>}<br><br>variable &quot;sg_description&quot; {<br>  default = &quot;Allows access to webserver&quot;<br>}<br><br>variable &quot;sg_ingress_ports&quot; {<br>  type = list(object({<br>    from_port = number<br>    to_port   = number<br>    protocol  = string<br>  }))<br>  default = [<br>    {<br>      from_port = 80<br>      to_port   = 80<br>      protocol  = &quot;tcp&quot;<br>    },<br>    {<br>      from_port = 443<br>      to_port   = 443<br>      protocol  = &quot;tcp&quot;<br>    },<br>    {<br>      from_port = 22<br>      to_port   = 22<br>      protocol  = &quot;tcp&quot;<br>    }<br>  ]<br>}<br><br>variable &quot;sg_ssh_cidr_blocks&quot; {<br>  type    = list(string)<br>  default = [&quot;0.0.0.0/0&quot;]<br>}<br><br>variable &quot;sg_egress_rule&quot; {<br>  type = object({<br>    from_port   = number<br>    to_port     = number<br>    protocol    = string<br>    cidr_blocks = list(string)<br>  })<br>  default = {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>}<br><br>variable &quot;sg_tags&quot; {<br>  type = map(string)<br>  default = {<br>    Name = &quot;project21_webtier_sg&quot;<br>  }<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we will receive another verification via our CLI that our security group is ready for deployment</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*r6mb7YTh7yzAOpNUpJ5viw.png" /></figure><h3>Phase 10— Creating our Web-Tier Launch Template</h3><p>Next, we will create our web-tier launch template in a <em>launchtemplate.tf</em> file. This template is what our auto scaling group will use to launch our EC2 instances when traffic spikes to our website.</p><pre>resource &quot;aws_launch_template&quot; &quot;WebServerTemplate&quot; {<br>  name        = var.launch_template_name<br>  description = var.launch_template_description<br><br>  update_default_version = true<br><br>  image_id      = var.launch_template_image_id<br>  instance_type = var.launch_template_instance_type<br><br>  key_name = var.launch_template_key_name<br><br>  network_interfaces {<br>    associate_public_ip_address = var.launch_template_network_interface.associate_public_ip_address<br>    security_groups             = [aws_security_group.project21_webtier_sg.id]<br>  }<br><br>  user_data = base64encode(var.launch_template_user_data)<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*bdb1Gydrr6u81a4QJz0YJQ.png" /></figure><p>Additionally, we need to add these variables to our “<em>variables.tf</em>” file:</p><pre>variable &quot;launch_template_name&quot; {<br>  description = &quot;The name of the launch template&quot;<br>  default     = &quot;WebServerTemplate&quot;<br>}<br><br>variable &quot;launch_template_description&quot; {<br>  description = &quot;The description of the launch template&quot;<br>  default     = &quot;prod webserver&quot;<br>}<br><br>variable &quot;launch_template_image_id&quot; {<br>  description = &quot;The AMI ID for the launch template&quot;<br>  default     = &quot;ami-02f97949d306b597a&quot;<br>}<br><br>variable &quot;launch_template_instance_type&quot; {<br>  description = &quot;The instance type for the launch template&quot;<br>  default     = &quot;t2.micro&quot;<br>}<br><br>variable &quot;launch_template_key_name&quot; {<br>  description = &quot;The key pair name for the launch template&quot;<br>  default     = &quot;Project18-KeyPair&quot;<br>}<br><br>variable &quot;launch_template_network_interface&quot; {<br>  description = &quot;The network interface configuration for the launch template&quot;<br>  default = {<br>    associate_public_ip_address = true<br>  }<br>}<br><br>variable &quot;launch_template_user_data&quot; {<br>  description = &quot;The user data script for the launch template&quot;<br>  default     = &lt;&lt;-EOT<br>    #!/bin/bash<br>    yum update -y<br>    yum install httpd -y<br>    systemctl start httpd<br>    systemctl enable httpd<br>    amazon-linux-extras install opel -y<br>    yum install stress -y<br>  EOT<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that the launch template for our web tier was created successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*CO9rWvcBUZsqJMp4-gebbw.png" /></figure><h3>Phase 11 — Creating our Target Group</h3><p>Next, we will create a target group for our web tier in a <em>webtier-targetgroup.tf</em> file.</p><p>A target group is used to route requests to one or more registered targets, such as EC2 instances in our case. Target groups are a required component of Load Balancers to define the destination for traffic that the load balancer receives.</p><p>When you create a target group, you associate it with a listener to route traffic from the load balancer to the targets.</p><pre>resource &quot;aws_lb_target_group&quot; &quot;web_target_group&quot; {<br>  name     = &quot;web-target-group&quot;<br>  port     = 80<br>  protocol = &quot;HTTP&quot;<br>  vpc_id   = aws_vpc.project_21_vpc.id<br><br>  health_check {<br>    enabled             = true<br>    interval            = 30<br>    path                = &quot;/&quot;<br>    timeout             = 5<br>    healthy_threshold   = 2<br>    unhealthy_threshold = 2<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*0QPwKmLH4u1RgyNv1f5MXw.png" /></figure><p>Additionally, we need to add these variables to our “<em>variables.tf</em>” file:</p><pre>variable &quot;target_group_name&quot; {<br>  description = &quot;The name of the target group&quot;<br>  default     = &quot;web-target-group&quot;<br>}<br><br>variable &quot;target_group_port&quot; {<br>  description = &quot;The port on which the target group listens&quot;<br>  default     = 80<br>}<br><br>variable &quot;target_group_protocol&quot; {<br>  description = &quot;The protocol used by the target group&quot;<br>  default     = &quot;HTTP&quot;<br>}<br><br>variable &quot;health_check_enabled&quot; {<br>  description = &quot;Indicates whether health checks are enabled for the target group&quot;<br>  default     = true<br>}<br><br>variable &quot;health_check_interval&quot; {<br>  description = &quot;The approximate amount of time, in seconds, between health checks of an individual target&quot;<br>  default     = 30<br>}<br><br>variable &quot;health_check_path&quot; {<br>  description = &quot;The destination for the health check request&quot;<br>  default     = &quot;/&quot;<br>}<br><br>variable &quot;health_check_timeout&quot; {<br>  description = &quot;The amount of time, in seconds, during which no response means a failed health check&quot;<br>  default     = 5<br>}<br><br>variable &quot;healthy_threshold&quot; {<br>  description = &quot;The number of consecutive health checks successes required before considering an unhealthy target healthy&quot;<br>  default     = 2<br>}<br><br>variable &quot;unhealthy_threshold&quot; {<br>  description = &quot;The number of consecutive health check failures required before considering a target unhealthy&quot;<br>  default     = 2<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that the target group for our web tier was created successfully</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*Syosj_sixZN6cEPsE6uZxA.png" /></figure><h3>Step 12 — Creating our Web Tier Auto Scaling Group and Load Balancer</h3><p>Next, we will create our Load Balancer and our Auto Scaling Group for our web tier in a <em>a lb.tf </em>file<em> and auto-scaling-group-web.tf </em>file.</p><p>An Application Load Balancer to distribute incoming traffic to multiple targets. This load balancer is associated with our <em>web tier security group</em> that we created earlier and is attached to two our <em>two public subnets</em>.</p><p>An Auto Scaling Group that launches and terminates EC2 instances based on the demand for our application. In our case, our website. Auto scaling groups make it easy to ramp up with demand and tear down when traffic is slow.<br>Here is our <em>lb.tf</em> file:</p><pre>resource &quot;aws_lb&quot; &quot;WebServerLoadBalancer&quot; {<br>  name               = var.load_balancer_name<br>  internal           = var.load_balancer_internal<br>  load_balancer_type = var.load_balancer_type<br>  ip_address_type    = var.ip_address_type<br>  security_groups    = [aws_security_group.project21_webtier_sg.id]<br>  subnets            = [aws_subnet.public_subnet1_project21.id, aws_subnet.public_subnet2_project21.id]<br><br>  depends_on = [aws_security_group.project21_webtier_sg]<br>}<br><br>resource &quot;aws_lb_listener&quot; &quot;webserver_lb_listener&quot; {<br>  load_balancer_arn = aws_lb.WebServerLoadBalancer.arn<br>  port              = var.listener_port<br>  protocol          = var.listener_protocol<br><br>  default_action {<br>    type             = var.default_action_type<br>    target_group_arn = aws_lb_target_group.web_target_group.arn<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*ifwDZoeIjc6v35xHITNrYQ.png" /></figure><p>And here is our <em>auto-scaling-group-web.tf</em> file:</p><pre>resource &quot;aws_autoscaling_group&quot; &quot;web_tier_autoscaling_group&quot; {<br>  name_prefix               = var.asg_name_prefix<br>  desired_capacity          = var.asg_desired_capacity<br>  min_size                  = var.asg_min_size<br>  max_size                  = var.asg_max_size<br>  health_check_type         = var.asg_health_check_type<br>  health_check_grace_period = var.asg_health_check_grace_period<br>  vpc_zone_identifier       = [aws_subnet.public_subnet1_project21.id, aws_subnet.public_subnet2_project21.id]<br><br>  launch_template {<br>    id      = aws_launch_template.WebServerTemplate.id<br>    version = &quot;$Latest&quot;<br>  }<br><br>  target_group_arns = [aws_lb_target_group.web_target_group.arn]<br><br>  lifecycle {<br>    create_before_destroy = true<br>  }<br><br>  depends_on = [<br>    aws_subnet.public_subnet1_project21,<br>    aws_subnet.public_subnet2_project21,<br>    aws_subnet.private_subnet1_project21,<br>    aws_subnet.private_subnet2_project21,<br>    aws_lb_target_group.web_target_group,<br>    aws_launch_template.WebServerTemplate,<br>  ]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/797/1*uGo-r6Dt4pmVBvEVlblxjg.png" /></figure><p>Additionally, we need to add these variables to our “<em>variables.tf</em>” file for our Load Balancer:</p><pre>variable &quot;load_balancer_name&quot; {<br>  type        = string<br>  description = &quot;Name of the load balancer&quot;<br>  default     = &quot;WebServerLoadBalancer&quot;<br>}<br><br>variable &quot;load_balancer_internal&quot; {<br>  type        = bool<br>  description = &quot;Whether the load balancer is internal or not&quot;<br>  default     = false<br>}<br><br>variable &quot;load_balancer_type&quot; {<br>  type        = string<br>  description = &quot;Type of load balancer&quot;<br>  default     = &quot;application&quot;<br>}<br><br>variable &quot;ip_address_type&quot; {<br>  type        = string<br>  description = &quot;IP address type for the load balancer&quot;<br>  default     = &quot;ipv4&quot;<br>}<br><br>variable &quot;listener_port&quot; {<br>  type        = number<br>  description = &quot;Port for the load balancer listener&quot;<br>  default     = 80<br>}<br><br>variable &quot;listener_protocol&quot; {<br>  type        = string<br>  description = &quot;Protocol for the load balancer listener&quot;<br>  default     = &quot;HTTP&quot;<br>}<br><br>variable &quot;default_action_type&quot; {<br>  type        = string<br>  description = &quot;Default action type for the load balancer listener&quot;<br>  default     = &quot;forward&quot;<br>}</pre><p>And we need to add these variables to our “<em>variables.tf</em>” file for our Auto Scaling Group:</p><pre>variable &quot;asg_name_prefix&quot; {<br>  description = &quot;Name prefix for the auto-scaling group&quot;<br>  default     = &quot;web-tier-auto-scaling-group&quot;<br>}<br><br>variable &quot;asg_desired_capacity&quot; {<br>  description = &quot;The desired capacity of the auto-scaling group&quot;<br>  default     = 2<br>}<br><br>variable &quot;asg_min_size&quot; {<br>  description = &quot;The minimum size of the auto-scaling group&quot;<br>  default     = 2<br>}<br><br>variable &quot;asg_max_size&quot; {<br>  description = &quot;The maximum size of the auto-scaling group&quot;<br>  default     = 5<br>}<br><br>variable &quot;asg_health_check_type&quot; {<br>  description = &quot;The health check type for the auto-scaling group&quot;<br>  default     = &quot;ELB&quot;<br>}<br><br>variable &quot;asg_health_check_grace_period&quot; {<br>  description = &quot;The health check grace period for the auto-scaling group&quot;<br>  default     = 300<br>}</pre><p>Once again we repeat the same process in the previous step by running the same terraform commands in sequence</p><pre>terraform init<br>terraform fmt<br>terraform validate<br>terraform plan<br>terraform apply</pre><p>Once again, we receive another verification via our CLI that our auto scaling group and application load balancer for our web tier was created successfully.</p><p>Now that we’ve built everything piece by piece, we can test a full launch by running <em>terraform destroy </em>to tear down our whole infrastructure and running <em>terraform apply</em> one more time to ensure that all of our .tf files will work in concert.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*al9cte_-DLQrpLizxOtp6A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*DZg_05vwSeL_SNz05mgOKQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*TPbLVq6qTwrYZEvjUqsNIg.png" /></figure><h3>Step 13 — Confirm Results and Testing our Web-Servers</h3><p>Now that our terraform has successfully provisioned our web-tier, we should verify some results:</p><ul><li>Verify that we can only access our Web Server via our Application Load Balancer public DNS name</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q0YyHiQlC-FsLCpZcq91eQ.png" /></figure><ul><li>Verify that our Auto Scaling Group works. We can do this by terminating one or multiple of our EC2 instances and see if our Auto Scaling Group provisions additional ones:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TYqT1Q7ksvhSUDRr4HedzQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1013/1*mth10PSJgb-U12UzCrIjnQ.png" /></figure><ul><li>Verify that our Web Servers are working.</li><li>Verify that our Web Servers cannot be accessed via their IP addresses, which ensures our security groups are configured correctly.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ndG99rxwrhK_Hha_IH7lhA.png" /><figcaption>Our Web Server cannot be accessed via it’s public IP — Check!</figcaption></figure><p>Since this article is already long. I will create a part two later where I will show you how to build out the Load Balancer, Auto Scaling Group, and Security Group for the application tier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*-IYKSRKPg_ObcE3S6I0zIA.jpeg" /><figcaption>Truer words have never been spoken. Especially for this project.</figcaption></figure><p>Hey you made it to the end. How in the world did you do that?! This article was so long.</p><p>I assume that means that you found my content helpful. If that is you, then you might also like a previous article I wrote about <a href="https://medium.com/aws-in-plain-english/kubernetes-deployments-for-micro-service-applications-fa3236592c9f">Kubernetes Deployments for Micro Service Applications.</a></p><p>Additionally, I would appreciate if you would leave a couple of ‘claps’ or a comment below. It helps this type of content reach more DevOps folks.</p><p>Until next time, Onward!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1be82e367175" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/taming-the-traffic-tsunami-autoscaling-using-terraform-1be82e367175">Taming the Traffic Tsunami: Autoscaling using Terraform</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>