<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Brandi McCall on Medium]]></title>
        <description><![CDATA[Stories by Brandi McCall on Medium]]></description>
        <link>https://medium.com/@bm54cloud?source=rss-2e9072e523ba------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*DVY2BBTZ0iF9yZGMKqSbhw.jpeg</url>
            <title>Stories by Brandi McCall on Medium</title>
            <link>https://medium.com/@bm54cloud?source=rss-2e9072e523ba------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:33:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bm54cloud/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building Secure Systems: The Role and Implementation of Security Technical Implementation…]]></title>
            <link>https://medium.com/defense-unicorns/building-secure-systems-the-role-and-implementation-of-security-technical-implementation-a6a9a30f409b?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6a9a30f409b</guid>
            <category><![CDATA[department-of-defense]]></category>
            <category><![CDATA[compliance]]></category>
            <category><![CDATA[stig]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Tue, 03 Sep 2024 19:20:09 GMT</pubDate>
            <atom:updated>2024-09-03T19:20:09.475Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building Secure Systems: The Role and Implementation of Security Technical Implementation Guidelines</h3><p>Have you heard of a STIG? No? Well buckle up Chuck because for the next 3–5 minutes we’re going to go on a STIG-tastical journey!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*0LpNKJx4L9wDappUZAs7fg.png" /></figure><h3>What is a STIG?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ml1l5aGFfL_UgTr9tm4Yw.png" /></figure><p>The Stig is a character from the British motoring television show <a href="https://en.wikipedia.org/wiki/Top_Gear_(2002_TV_series)">Top Gear</a>. He has many interesting qualities, just a few of which are listed below:</p><ul><li>He gets terrible eczema on his helmet</li><li>If you tune your radio to 88.4 you can actually hear his thoughts</li><li>He never blinks</li><li>His voice can only be heard by cats</li><li>His skin has the texture of a dolphin</li><li>He will charge you if you attempt to remove his helmet</li><li>He knows two facts about ducks and both of them are wrong</li></ul><p>The list goes on and on…</p><h3>What is a STIG…really?</h3><p>In contrast to “The STIG,” “STIGs” are Security Technical Implementation Guidelines consisting of cybersecurity best practices for securing hardware and software. They were developed by the Defense Information Systems Agency (DISA) and aim to make technical systems used by the Department of Defense (DoD) as secure as possible, given the sensitive and/or classified information they may contain. STIGs exist for all kinds of hardware and software, like operating systems, switches, and cloud environments, just to name a few.</p><h3>Why would I want my system to be STIG compliant?</h3><p>STIG compliance is often mandatory for U.S. Department of Defense contracts and to achieve Authorization to Operate (ATO). Given the sensitive and classified nature of the information on DoD systems, adherence to STIGs is essential. By demonstrating STIG compliance, we not only meet contractual obligations but also build trust with our customers, partners, and stakeholders, showcasing our commitment to robust security practices.</p><h3>How do you obtain a copy of a STIG?</h3><p>STIGs can be found by visiting <a href="https://public.cyber.mil/stigs/">https://public.cyber.mil/stigs/</a> and then searching for the relevant STIG in the <strong>Document Library</strong>. Clicking the hyperlink will download the STIG to your local device.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PJx2nN63r-L_Mhxk" /></figure><h3>What if the hardware or software I want to STIG doesn’t have one available?</h3><p>The <a href="https://github.com/ComplianceAsCode/content">Compliance as Code</a> project is a great alternative! This open-source project began in 2011 as a joint effort between various U.S. government agencies and commercial operating system vendors. Their goal was to develop security data streams that could be ingested and used to scan a system. Over time, the data streams grew from government-specific security profiles to include commercial security profiles like <a href="https://www.cisecurity.org/cis-benchmarks">CIS benchmarks</a>, payment card industry data security standards, and more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_EU0IbE7zT_ZsHqMNasx5g.png" /></figure><h3>Cool…now what?</h3><p>Now that you have a data stream (STIG or CIS Benchmark from the Compliance as Code project) to run against your system, you need a tool to ingest the data stream and hopefully run an automated compliance scan and remediation script. Pick your poison: <a href="https://www.open-scap.org/">OpenSCAP</a>, <a href="https://www.tenable.com/data-sheets/assured-compliance-assessment-solution-acas-powered-by-tenable">ACAS</a>, <a href="https://public.cyber.mil/stigs/scap/">Scap Compliance Checker</a> (SCC), etc. OpenSCAP offers both scanning and remediation of a system against an ingested data stream, resulting in a colorful html report that clearly shows compliance and remediation steps to become even more compliant. If you choose a different tool, good luck!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bi37EfZcubxDD2Hhp93R_Q.png" /></figure><h3>How do you use OpenSCAP?</h3><p>Glad you asked! Check out my detailed tutorial <a href="https://medium.com/defense-unicorns/stig-scanning-with-openscap-675c7292d7cb">here</a> or a BRIEF explanation below (if you actually want to test this, follow the full tutorial). Steps are slightly different for running a Compliance as Code data stream versus a DISA STIG data stream, so below are the <em>general</em> steps for running a DISA STIG. The Compliance as Code data stream is demonstrated in the linked tutorial.</p><ol><li>Install OpenSCAP</li><li>Download the DISA STIG from <a href="https://public.cyber.mil/stigs/downloads">https://public.cyber.mil/stigs/downloads</a></li><li>Run oscap info against the STIG data stream to see available profiles</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GiXou_89EXrPn4ebWVX00A.png" /></figure><p>4. Run oscap xccdf eval command to evaluate your system against a specific profile and STIG data stream</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cK1IKWszpvsti38V" /></figure><p>5. View the stig-report.html to see compliance and areas for improvement</p><p>6. Create a remediation script if none exists</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Cq1kueqTjrdKQhco" /></figure><p>7. Run the remediation script</p><p>8. Run the oscap command again to evaluate your remediated system and view the new html report to see improved compliance</p><h3>Can we automate it?</h3><p>You betcha! OpenSCAP can be incorporated into custom <a href="https://developer.hashicorp.com/packer/tutorials/docker-get-started/docker-get-started-build-image">Packer</a> base images as well as CI/CD pipelines. <a href="https://candrews.integralblue.com/2023/09/scap-security-and-compliance-scanning-of-docker-images-in-github-actions-and-gitlab-ci/">This</a> tutorial includes some examples of incorporating OpenSCAP in GitLab and GitHub pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*oUpsIVpLfclDM1qE.jpg" /></figure><h3>That Was a Lot of Information</h3><p>It sure was. The truth is, STIGs are somewhat complicated and there are more components to it than what’s discussed above, but this is a decent primer. They are rarely “fun” to implement BUT, they are a necessary evil that you should become an expert on if working in the DoD space.</p><h3>I love STIGs so much! Where can I find more?</h3><p>Here are some resources:</p><p><a href="https://public.cyber.mil/stigs/">DISA STIGs</a></p><p><a href="https://www.cisecurity.org/cis-benchmarks">CIS Benchmarks</a></p><p><a href="https://github.com/ComplianceAsCode/content">Compliance as Code</a></p><p><a href="https://csrc.nist.gov/projects/security-content-automation-protocol">SCAP</a></p><p><a href="https://www.open-scap.org/tools/">OpenSCAP</a></p><p><a href="https://static.open-scap.org/openscap-1.3/oscap_user_manual.html">OpenSCAP User Manual</a></p><p>Tutorials:</p><p><a href="https://medium.com/defense-unicorns/stig-scanning-with-openscap-675c7292d7cb">STIG Scanning with OpenSCAP</a></p><p><a href="https://medium.com/defense-unicorns/automate-stig-scanning-with-openscap-and-cloudinit-b9426f515336">Automatic STIG Scanning with OpenSCAP and CloudInit</a></p><p><a href="https://medium.com/defense-unicorns/stig-scanning-a-remote-machine-from-local-21f6f3f9c27e">STIG Scanning a Remote Machine from Local</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*C5CEuOlIBYdYA13_.jpg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6a9a30f409b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/building-secure-systems-the-role-and-implementation-of-security-technical-implementation-a6a9a30f409b">Building Secure Systems: The Role and Implementation of Security Technical Implementation…</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy and Update Zarf Packages in an Air Gap]]></title>
            <link>https://medium.com/@bm54cloud/deploy-and-update-zarf-packages-in-an-air-gap-b2e3ec43abf7?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2e3ec43abf7</guid>
            <category><![CDATA[airgap]]></category>
            <category><![CDATA[airgapped]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[zarf]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Mon, 15 Apr 2024 16:07:43 GMT</pubDate>
            <atom:updated>2024-04-15T16:07:43.670Z</atom:updated>
            <content:encoded><![CDATA[<h4>This tutorial gives an example of using Zarf to deploy a Podinfo package into an air gapped Kubernetes cluster, and then upgrading that Podinfo package to a newer version.</h4><h3><strong>Prerequisites:</strong></h3><ul><li>Internet connected computer</li><li>Air gapped computer (see options below)</li><li>Zarf CLI installed</li><li>Familiar with Zarf CLI</li><li>Familiar with kubectl</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ypROLnkeZ584Bp4RQQxa6A.png" /></figure><h3>Introduction</h3><p>Have you ever needed to deploy or upgrade an application on an air gapped Kubernetes cluster? <a href="https://docs.zarf.dev/docs/zarf-overview">Zarf</a> is an open source tool that helps you package up parts of the internet and deploy those parts into an air gapped environment. If you’re not familiar with Zarf, checkout my previous tutorials <a href="https://medium.com/defense-unicorns/deploying-dos-games-on-a-kubernetes-cluster-with-zarf-b254c604a55d">here</a>, <a href="https://medium.com/defense-unicorns/using-zarf-to-deploy-a-podinfo-flux-application-in-a-kubernetes-cluster-22dc6c02510c">here</a>, and <a href="https://medium.com/defense-unicorns/zarf-deployment-with-plg-log-monitoring-bb61a04e72b7">here</a>. This tutorial will assume you have some Zarf knowledge and are familiar with basic Linux and kubectl commands.</p><h3>Goal</h3><blockquote>Use Zarf to deploy an air gapped Kubernetes cluster and Podinfo application</blockquote><p>The goal of this tutorial is to first use Zarf to deploy a k3s cluster. Zarf has the ability to deploy a cluster for you, if you are on a Linux machine. If you are on any other type of machine, you will first need to provision your own Kubernetes cluster using something like MiniKube, k3d, or some other Kubernetes tool of your choice.</p><p>First, we will deploy the Zarf init package (learn about it <a href="https://docs.zarf.dev/docs/create-a-zarf-package/zarf-init-package">here</a>) and this is where you will be asked (on a Linux machine) if you want to deploy a k3s cluster. If you have already deployed your own Kubernetes cluster and are on a non-Linux machine, you will still need to run zarf init to get some other components deployed (zarf-injector, zarf-seed-registry, and more).</p><p>Next, we will use <a href="https://github.com/defenseunicorns/zarf/tree/main/examples/podinfo-flux">this</a> Podinfo example repository to create a Zarf package of Podinfo. Zarf requires the creation of a ZarfPackageConfig object via a customized zarf.yaml. In the above-mentioned repository, the zarf.yaml has already been created for you. We will use zarf package create to create the Zarf Podinfo package, then we will take the Zarf init package and the Zarf Podinfo package, place them on a USB, connect that USB to an air gapped Linux machine, and deploy both the Zarf init package and the Zarf Podinfo package on the air gapped machine.</p><h3>Air Gap Setup</h3><p>There are several different options for setting up an air gapped environment. My development computer is a MacBook Pro, but I have a home lab that consists of this <a href="https://simplynuc.com/product/nuc13anki7-full/?gad_source=1&amp;gclid=Cj0KCQiAoeGuBhCBARIsAGfKY7was0AUjk1pkDq-vsLS30wJ-lt3Q75sqq3mQtUX8XAhkHkCswGLNfQaAiyNEALw_wcB">Intel Nuc 13 Pro</a>. I love the Nuc because it doesn’t take up much space and it allows me to have access to a Linux machine, which I test on consistently. Usually I SSH over wifi into the Nuc when I want to use it. For this project, I was able to quickly air gap the Nuc by first connecting its own monitor (I used a spare travel monitor) and keyboard (also a spare I had lying around). Then I followed <a href="https://www.intel.com/content/www/us/en/support/articles/000031666/intel-nuc.html#:~:text=Wireless%20and%20Bluetooth%20can%20be,Bluetooth%20by%20unchecking%20the%20boxes.">these commands</a> to turn off the wifi. Note that the method of turning off your internet connection will vary depending on the specific brand and model of equipment you have. After turning off the wifi, I confirmed no internet connectivity by running ping 8.8.8.8 which failed, as expected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MJGckLxv24nJESJ76Oq0Vg.png" /></figure><p>If you don’t have a home lab lying around, you could deploy a Kubernetes cluster to your computer, download the <a href="https://github.com/defenseunicorns/zarf/releases/tag/v0.32.3">Zarf CLI and Zarf init packages</a> for your computer’s architecture, create the Zarf Podinfo package and have it locally, then turn off your wifi and confirm your computer is not connected to the internet via a cable somewhere.</p><p>You could also follow <a href="https://medium.com/defense-unicorns/manual-air-gap-software-delivery-1fe6da7a7d0e">this tutorial</a> to spin up an air gapped VM. The key here is you need to have the <a href="https://docs.zarf.dev/docs/getting-started/#installing-zarf">Zarf CLI</a> installed on both your internet connected machine and your machine that will be air gapped. You also need to have the Zarf init package and the Zarf Podinfo package downloaded to a USB or some other medium device before you air gap your machine.</p><h3>Internet Connected Environment</h3><h4>Create Zarf Podinfo Package</h4><p>This section requires that the Zarf CLI is installed on your machine. If you’re not sure, run zarf version to see if it is installed. If you get back a version tag, you’re good to go. If the zarf command is not recognized, follow <a href="https://docs.zarf.dev/getting-started/install/">these steps</a> to install the Zarf CLI.</p><p>Clone the <a href="https://github.com/defenseunicorns/zarf">Zarf repo</a> to your local machine, then navigate to the <em>/examples/podinfo-flux</em> directory. Note that in this directory the zarf.yaml is already configured for you, so you can simply run zarf package create to create the Zarf Podinfo package.</p><pre>git clone git@github.com:defenseunicorns/zarf.git<br>cd zarf/examples/podinfo-flux/<br>zarf package create</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i-EOhWzPV6Q4ztWb6IY-tw.png" /></figure><p>The Zarf Podinfo package is simply a tarball file with all of the components in it required to deploy Podinfo onto your cluster. Note that the default file architecture appears to be amd64. If you will be working on an arm64 machine, you can add <em>architecture: arm64</em> to the metadata of the zarf.yaml or use the -a flag with the zarf package create command to specify the architecture (ex. zarf package create -a amd64).</p><h4>Get Zarf Init Package</h4><p>Our plan is to use the Zarf init package to deploy our Kubernetes cluster for us on the air gapped Nuc, so we need to obtain the appropriate Zarf init package for our architecture. You can find the Zarf init packages the same place you find Zarf <a href="https://github.com/defenseunicorns/zarf/releases">releases</a>. My air gapped machine already has Zarf v0.32.1 installed, so I am going to grab the corresponding Zarf-init package.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4qAvpIcLLusgGKl1otALw.png" /></figure><p>By clicking the link, you can download the package to your Downloads folder. I now want to move both the Zarf Podinfo package and the Zarf init package to my USB drive. On a Mac, USBs are mounted in /Volumes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A6hBIBOTl1vuXIL797xB3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qd_f9wztszLmhxfG5lcE3g.png" /></figure><p>Confirm the Zarf Podinfo package and the Zarf init package are now on your USB drive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DHc9fi7gyJ3K7PK4pQOgSg.png" /></figure><p>Safely eject your USB drive.</p><h3>Air Gapped Environment</h3><p>Follow one of the steps discussed previously to air gap your machine. I will be using my air gapped NUC, a Linux machine for the rest of the tutorial. Plug your USB into your Linux server and mount the device. You mount the USB device by first finding the name of it with sudo fdisk -l | grep sd .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TSKTs3DYqMyXnWnRocew0A.png" /></figure><p>Use lsblk to see if the USB is mounted or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*J9ndHutl2DKBHySbgIQ9SQ.png" /></figure><p>The USB is not currently mounted, so first let’s make a directory where we want to mount the device. Then use the sudo mount command to mount the USB to the directory you made.</p><pre>sudo mkdir /mnt/usb<br>sudo mount /dev/sda1 /mnt/usb/</pre><p>Confirm the mount with lsblk | grep sda1 .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PEHOgfzkfN4WeSPAPt9qFg.png" /></figure><h4>Deploy the Zarf Init Package<a href="https://docs.zarf.dev/docs/create-a-zarf-package/zarf-packages#using-the-init-package">​</a></h4><p>To initialize your cluster, you need to run the command zarf init preferably from the directory where your Zarf init package is located. This command will search for a file with the specific naming convention: zarf-init-{ARCHITECTURE}-{VERSION}.tar.zst. The architecture must match that of the cluster you are deploying to. If you are deploying to a cluster with a different architecture, you will need to specify the name of the architecture you are deploying on with the -a flag. Both my Zarf init package and Zarf Podinfo package are amd64, the appropriate architecture for my Linux machine, so I do not need to use the -a flag.</p><p>Using Zarf init to deploy a k3s cluster requires <strong>root</strong> access (not just <em>sudo</em>). To deploy the Zarf init package, use the following commands:</p><pre>sudo su<br>zarf init</pre><p>When prompted “Deploy the k3s component?” type y and then hit Enter (if you are not on a Linux machine, you will not see this prompt).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Fxbf5_QjDPxi-eJpFWEKg.png" /></figure><p>Once the Zarf init package has finished deploying, you can see all of your pods with kubectl get pods -A .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cdi3wC_1_IOJPJXfgnH2Fg.png" /></figure><h4>Deploy the Zarf Podinfo Package</h4><p>In the directory where your Zarf Podinfo package lives, run the following command:</p><pre>zarf package deploy &lt;zarf_podinfo_package_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9a_O5lB_TQCmKAnUaEe1PA.png" /></figure><p>Once the deployment is complete, run kubectl get pods -A again to see that your Podinfo pods deployed correctly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*heZv_fySZfIshFWnkZNsJQ.png" /></figure><p>That’s it! You’ve just deployed the Zarf init package and Zarf Podinfo package in your air gapped cluster.</p><h3>Bonus: Use Zarf to Upgrade an Application on an Air Gapped Kubernetes Cluster</h3><p>Let’s say you have Podinfo running on your air gapped cluster but you need to change the version. How can you do that?</p><p>The process is very similar to what we’ve already done. You would need to go back to an internet connected device, upgrade your Zarf yaml files with the different Podinfo version, then build a new Zarf Podinfo package. You would place that new Zarf Podinfo package on your USB device, then carry that USB to your air gapped machine and deploy the new package just like before. When deploying the new Zarf Podinfo package, Zarf will perform a rolling deployment, where it takes down a few pods at a time, updates them, then restarts new pods with the updates, repeating this process until all of the pods have been updated.</p><p>To get started, check the version of Podinfo that is currently deployed with:</p><pre>kubectl get pods -n podinfo -o jsonpath=&#39;{range .items[*]}{&quot;\n&quot;}{.metadata.name}{&quot;:\t&quot;}{range .spec.containers[*]}{.image}{&quot;, &quot;}{end}{end}&#39; | sort</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WxUKRQkTL4Eh8ukyDzgdPQ.png" /></figure><p>The current version of Podinfo deployed on my air gapped machine is v6.5.4. At the time of this writing, there is a newer version of 6.6.0, so we will upgrade to that. The new Zarf Podinfo package must be built in an internet connected environment, so <strong>head back over to your internet connected machine</strong> for the next few steps.</p><p>In the zarf.yaml file, change the image used to ghcr.io/stefanprodan/podinfo:6.6.0 .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U3S5df8Edf5_sTSlk7lrTA.png" /></figure><p>In the podinfo-source.yaml file, change the tag to 6.6.0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zsL9cEfDENzuaklSjKfZzA.png" /></figure><p>Now use zarf package create to create a new Zarf Podinfo package with the updated version. Remove the old zarf-package-podinfo-flux-amd64.tar.zst file from your USB since the new Zarf package will have the same file name. Move the new Zarf Podinfo package to your USB and insert the USB to your air gapped machine.</p><p>Back on your air gapped machine, navigate to the directory where your new Zarf package is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kvZDhzc6BMAG6L8anIItXQ.png" /></figure><p>Because zarf init has already been run on the air gapped machine, you do not have to run the command again. Use zarf package deploy to deploy the new package.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-TdiBe_NBDrFA9V_HWQmKg.png" /></figure><p>Once the package has deployed, you can check that your new pods are running with kubectl get pods -A . Note that the age of the Podinfo pods are measured in seconds, so those are our new pods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZJX_7hxDxUMvCU-jQL3Xxg.png" /></figure><p>Run the following command to confirm that we’re using the newer version of Podinfo:</p><pre>kubectl get pods -n podinfo -o jsonpath=&#39;{range .items[*]}{&quot;\n&quot;}{.metadata.name}{&quot;:\t&quot;}{range .spec.containers[*]}{.image}{&quot;, &quot;}{end}{end}&#39; | sort</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p5jX1S6vwm_F80uLhw7BdA.png" /></figure><p>Successfully upgrade to Podinfo v6.6.0!</p><h3>Conclusion</h3><p>Hopefully this tutorial helped you better understand how to use Zarf to deploy and upgrade packages in an air gapped environment. To clean up everything we’ve done here, simply run the following command:</p><pre>zarf destroy --confirm --remove-components</pre><p>You can also delete the Zarf init package and Zarf Podinfo package from your machines if desired. To learn more about Zarf and its capabilities, visit the official <a href="https://docs.zarf.dev/">Zarf documentation</a>. As always, I appreciate you taking the time to read this, and look forward to us all learning together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2e3ec43abf7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running Chatbot UI Locally with a LeapfrogAI Backend and API]]></title>
            <link>https://medium.com/defense-unicorns/running-chatbot-ui-locally-with-a-leapfrogai-backend-and-api-72a50420e174?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/72a50420e174</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Sun, 05 Nov 2023 02:29:35 GMT</pubDate>
            <atom:updated>2024-06-28T21:06:48.769Z</atom:updated>
            <content:encoded><![CDATA[<h4>Learn how to use the open-source tool LeapfrogAI to run a local instance of Chatbot UI</h4><blockquote>Note: The repository used in this tutorial has been archived and is now Read Only. Specifically, the ctransformers backend is no longer available. See <a href="https://github.com/defenseunicorns/leapfrogai">LeapfrogAI’s GitHub</a> site for currently available backends. If you have any questions or would like more info on LeapfrogAI, feel free to open an issue on GitHub. We love demoing this tool!</blockquote><h3>Prerequisites:</h3><ul><li>Git installed</li><li>Python3 installed</li><li>npm installed</li></ul><h3>Objectives:</h3><ul><li>Discuss LeapfrogAI and the benefits of using it</li><li>Setup a local LeapfrogAI backend</li><li>Run the LeapfrogAI API</li><li>Start and interact with a Chatbot UI</li></ul><p><strong><em>Disclaimer: </em></strong><em>LeapfrogAI and its repos are rapidly evolving. This tutorial was written on November 4, 2023 and was working with the repos at that time. Depending on repo updates, the steps below may vary as LeapfrogAI evolves.</em></p><h3>What is LeapfrogAI?</h3><p>When you first hear “LeapfrogAI,” what comes to mind? It’s the Leapfrog kid toys right? My first thought too. In fact, my kiddo had many of these gadgets at a young age, and they were all incredibly fun and educational. Let me introduce you to another frog:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pqyxfx9lRdMwMpYKpvn3ww.png" /></figure><p><a href="https://github.com/defenseunicorns/leapfrogai">LeapfrogAI</a> is an open-source artificial intelligence tool designed by <a href="https://www.defenseunicorns.com/">Defense Unicorns</a> whose primary goal is to get AI tools into resource-constrained environments, like <a href="https://en.wikipedia.org/wiki/Air_gap_(networking)">air gap environments</a>. This means that environments like those classified Department of Defense (DoD) ones can have access to incredible AI tools, even without internet connectivity. This is a game changer for the DoD and allows somewhat ancient government systems to have access to cutting edge technology, whether they’re on a submarine or in a SCIF. Or, in the case of this tutorial, LeapfrogAI allows you to run the open-source tool Chatbot UI locally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jMsSzM-7zNBdj4vE2pBbiw.png" /></figure><h3>Why use LeapfrogAI?</h3><p>You’ve probably heard of or even used ChatGPT, which uses the OpenAI API. OpenAI is trained on large data sets that include confidential and sensitive information, raising the concern for privacy and security (i.e. data leaks). In fact, some companies have banned their employees from using OpenAI for fear that proprietary or sensitive information may get ingested and subsequently leaked.</p><p>Enter LeapfrogAI. LeapfrogAI allows you to host your own large language model (LLM) and retain control over your data. LeapfrogAI’s API closely matches OpenAI’s API thus tools that are built around OpenAI can be used with LeapfrogAI as the backend. This allows you to specify the language model you want to use and fine tune it to your needs. With the freedom to choose your own model, you can easily train a model, switch it out to a different model, or build your own user interface around it, all while maintaining control over your data.</p><p>In the next few steps, we will go over how to use LeapfrogAI to set up a local backend with the <a href="https://huggingface.co/TheBloke/SynthIA-7B-v2.0-GGUF">SynthIA-7B-v2.0-GGUF</a> language model. We will then set up the LeapfrogAI API, and start up a Chatbot UI to show how we can use OpenAI-like tools to point to our own endpoint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YUsVNaDRu5c-VG95nifk6w.png" /></figure><h3>Setup Local LeapfrogAI Backend</h3><p>For this tutorial, I wanted to set up everything locally as a proof of concept for using your own endpoint with OpenAI technology. I am running this tutorial on a MacBook Pro with M2 processor and the Sonoma 14.0 OS. I have not tested the following steps on a Linux or Windows machine, thus there might be slight differences with deployment.</p><p>To set up the LeapfrogAI backend, we start with the <a href="https://github.com/defenseunicorns/leapfrogai-backend-ctransformers">leapfrogai-backend-ctransformers</a> repo. The <a href="https://github.com/marella/ctransformers">C Transformers</a> library provides a way to use Generalized Graph-based Machine Learning (GGML) models within Python programs. Move into the directory you want to run the project in, and clone the repo to your local machine.</p><pre>git clone https://github.com/defenseunicorns/leapfrogai-backend-ctransformers</pre><p>Change into the repo directory and run the following commands:</p><pre>mkdir .model/<br>wget https://huggingface.co/TheBloke/Synthia-7B-v2.0-GGUF/resolve/main/synthia-7b-v2.0.Q4_K_M.gguf<br>mv synthia-7b-v2.0.Q4_K_M.gguf .model/synthia-7b-v2.0.Q4_K_M.gguf</pre><p>The above commands will copy the SynthIA-7B-v2.0-GGUF model to a local .model directory. Next we want to set up a Python virtual environment to put our project dependencies in. If you have Python3 and get the error “command not found” when running Python commands, simply set the following alias or change the python commands to python3 .</p><pre>alias python=python3</pre><p>Now run the following commands:</p><pre>python -m venv .venv<br>source .venv/bin/activate<br>python -m pip install -r requirements-dev.txt</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Bky8tOgdOZOS6hcwFyadg.png" /></figure><p>The above commands create and activate a Python virtual environment, allowing you to install package requirements without affecting your system-wide Python installation. This ensures the correct versions of libraries are installed only within the virtual environment, avoiding interference with other Python projects you may have on the same computer. To start the model backend, run the following command:</p><pre>python main.py</pre><p>If you get an error, see if it says “Address already in use.” I got this error on my first attempt at running python main.py and used <a href="https://www.btaz.com/mac-os-x/find-the-process-listening-to-port-on-mac-os-x/#:~:text=Using%20the%20lsof%20command%20we,%3E%E2%80%9D%20with%20our%20port%20number.&amp;text=In%20the%20output%20above%20the,PID)%20is%20%E2%80%9C63851%E2%80%9D.">these</a> commands to discover that my Multipass application was running on port 50051 (the default LeapfrogAI port). After I killed Multipass, and ran the command again, I was able to get the backend working.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wr3PWIKsATvoLrGEE0FbJA.png" /><figcaption>Errors</figcaption></figure><p>If you did not get an error and your terminal looks something like this, you are ready to move on!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FyCxTUcEZCb1Sc_ky34xaA.png" /><figcaption>Good to go!</figcaption></figure><h3>Run LeapfrogAI API</h3><p>LeapfrogAI allows us to deploy an API that closely matches the <a href="https://platform.openai.com/docs/api-reference">OpenAI API</a>. For this tutorial we will use the <a href="https://github.com/defenseunicorns/leapfrogai_api_server">leapfrogai_api_server</a> repo. Open a new terminal window and clone the repo to your local machine.</p><pre>git clone https://github.com/defenseunicorns/leapfrogai_api_server</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GGXRYPLndowd97bebKLWDA.png" /></figure><p>Change into the repo directory and run the following command:</p><pre>go run main.go</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2TTHimvhTaIYEI_ocvpqYQ.png" /></figure><p>Once this is running, you should see something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ICPHLeM2R6dMkn5hzqbiNQ.png" /></figure><h3>Start the Chatbot UI</h3><p>To make LeapfrogAI easier to interact with, we want to connect it to a user interface. Chatbot UI is an open-source chat user interface for AI models. While LeapfrogAI is working on their own user interface (found <a href="https://github.com/defenseunicorns/leapfrog-ui">here</a>), it is still a work in progress, thus we will use the more mature Chatbot UI.</p><p>To get started, clone the <a href="https://github.com/mckaywrigley/chatbot-ui">Chatbot UI</a> repo to your local directory and change into it. Note that this is the Defense Unicorns version of Chatbot UI.</p><pre>git clone https://github.com/defenseunicorns/chatbot-ui</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xADCnaD0I95L2Vl5JSEpNQ.png" /></figure><p>Create a .env.local file and add the following environment variables to it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7yeyDczPnKssiIMt2I_njA.png" /></figure><pre>OPENAI_API_KEY=fakekey<br>OPENAI_API_HOST=http://localhost:8080/openai<br>DEFAULT_MODEL=ctransformers</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eGitFIbIeOP5YX0jM_--Nw.png" /></figure><p>Install dependencies by running:</p><pre>npm i</pre><p>Run the Chatbot UI with the following command:</p><pre>npm run dev</pre><p>Open your web browser and navigate to localhost:3000. You should be met with the Chatbot UI. Use the drop down arrow under <strong>Model</strong> to select <strong>ctransformers</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hxOzzAujFCSojdu_3_hVuQ.png" /></figure><p>Now you’re ready to ask Chatbot a question. The time to respond is dependent on your local machine’s performance. As you send requests in the Chatbot UI, you should see the LeapfrogAI API and the ctransformers backend responding in their terminal windows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TOgX6Hrl_L-YaWtehv4k6g.png" /><figcaption>ctransformers terminal</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*peigeKtpgr323BAOrQJq8w.png" /><figcaption>LeapfrogAI API terminal</figcaption></figure><p>Now pick a question to ask the Chatbot. I asked Chatbot a silly question about a unicorn novel. Check out the response below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*15wpNSNRfuPG4IupFZB9bA.png" /></figure><h3>Conclusion and Clean Up</h3><p>Hopefully you found this tutorial useful and can see how using LeapfrogAI allows you to run large language models locally. Depending on your local computer’s resources, you could replace the SynthIA-7B-v2.0-GGUF model with a different model that may be more accurate, or have a different personality. The beauty of LeapfrogAI is in its flexibility, so feel free to experiment!</p><p>If you want to cleanup everything we’ve done here, simple delete the three repos you cloned to your local machine. Thank you so much for following along, and keep watching for more DevOps tutorials!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72a50420e174" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/running-chatbot-ui-locally-with-a-leapfrogai-backend-and-api-72a50420e174">Running Chatbot UI Locally with a LeapfrogAI Backend and API</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[STIG Scanning a Remote Machine from Local]]></title>
            <link>https://medium.com/defense-unicorns/stig-scanning-a-remote-machine-from-local-21f6f3f9c27e?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/21f6f3f9c27e</guid>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[stig]]></category>
            <category><![CDATA[openscap]]></category>
            <category><![CDATA[department-of-defense]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Thu, 19 Oct 2023 17:14:32 GMT</pubDate>
            <atom:updated>2023-10-19T17:14:32.525Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using OpenSCAP’s oscap-ssh Utility</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6RCxWVa8PvCYetXedH7iRw.png" /></figure><p><em>In this tutorial, we will launch an Ubuntu 20.04 Multipass VM with a CloudInit file that creates a </em><em>scap_auditor user and installs OpenSCAP. We will then perform a STIG scan on the VM from our local machine using OpenSCAP’s </em><em>oscap-ssh utility.</em></p><h3>Prerequisites:</h3><ul><li>Multipass installed</li><li>Unzip installed</li></ul><h3>Objectives:</h3><ul><li>Discuss OpenSCAP’s oscap-ssh utility</li><li>Launch a Multipass VM and pass a CloudInit file</li><li>Discuss how to run an Ubuntu 20.04 STIG scan on a remote machine from your local machine</li></ul><h3>What is OpenSCAP’s oscap-ssh Tool?</h3><p><a href="https://static.open-scap.org/openscap-1.2/oscap_user_manual.html#_introduction">OpenSCAP</a> is an open-source implementation of the Security Content Automation Protocol (SCAP), which is a suite of security standards that was developed by the National Institute of Standards and Technology (NIST). One capability of OpenSCAP is to perform security compliance checks of IT systems by running a scan that compares the defined security configuration of a system to a set of security standards, such as the Security Technical Implementation Guide (STIG) for that system. In this tutorial, we will deploy Ubuntu 20.04 on a Multipass VM and run the available Ubuntu 20.04 STIG (security guide) against the OS.</p><p>Typically you install OpenSCAP and the STIG security guide on the machine you want to scan, then use the oscap xccdf eval command to evaluate the STIG against the machine. There may be an instance, however, where you don’t have physical access to the machine you want to scan, but you could SSH into it. OpenSCAP provides a way to scan a remote machine from your local machine via the oscap-ssh utility. This command is run on your local machine, where it SSHs into the remote machine, scans the machine per a provided security guide and profile, creates a report of the scan results, and secure copies the report back to your local machine.</p><h3>Launch Multipass VM</h3><p>In this section we will use <a href="https://multipass.run/docs/create-an-instance">Multipass</a> to quickly spin up a VM with the Ubuntu 20.04 image. At the time of this writing, the STIGs for Ubuntu 22.04 and 23.04 have not yet been released, thus the reason for using 20.04. In addition to spinning up the VM, we will pass in a <a href="https://cloudinit.readthedocs.io/en/latest/reference/examples.html">CloudInit</a> file on initial boot. Let’s get started!</p><p>On your local machine, change to the directory you want to work in and create a cloud-init.yaml file with the following contents:</p><pre>#cloud-config<br>users:<br>  - default<br>  - name: scap_auditor<br>    groups: sudo, admin<br>    shell: /bin/bash<br>    lock_passwd: true<br>    sudo: ALL=(ALL) NOPASSWD:ALL<br>    ssh_authorized_keys: <br>    - ssh-rsa &lt;public_key&gt;<br>    <br>package_update: true<br>package_upgrade: true    <br>packages:<br>  - libopenscap8</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mgSASmNRznWDO4w4IWEtOw.png" /></figure><p>When creating a CloudInit file, you must have #cloud-config at the start to identify it as CloudInit user data. The above file will create a default and scap_auditor user. For the scap-auditor user (you can name this user whatever you want), replace &lt;public_key&gt; with the public SSH key of a key pair you hold. If you do not have a key pair, or want to create a new one, follow <a href="https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent">these</a> instructions to create a new one and add it to your ssh-agent and Apple Keychain if using a Mac. If you still have questions, check out the “Create SSH Key for User” section in my previous tutorial <a href="https://medium.com/defense-unicorns/automate-stig-scanning-with-openscap-and-cloudinit-b9426f515336">here</a>. Alternatively, you can follow the CloudInit documentation to create a user password instead of an SSH key.</p><p>The rest of the file above will update and upgrade packages, then install the libopenscap8 package. The libopenscap8 package installs OpenSCAP. After saving the file in your local directory, <strong>open a new terminal window</strong> and run the below command to spin up the VM. Note that you can change the name to whatever you want, I just used STIG for its simplicity. focal is the name used to identify that you want Ubuntu 20.04 as the operating system.</p><pre>#Launch Multipass VM with Ubuntu 20.04<br>multipass launch focal --name STIG --cloud-init cloud-init.yaml</pre><p>After the Multipass VM finishes spinning up, launch a shell:</p><pre>multipass shell &lt;VM_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UY6BYbEuSfAjJRHGyZyVjw.png" /></figure><h3>Scanning a Remote Machine with oscap-ssh</h3><p>The oscap-ssh utility allows you to perform SCAP evaluations on a remote machine via SSH from your local. After performing the compliance check, oscap-ssh will automagically create a report.html of the findings and secure copy it to your local machine. The oscap-ssh tool is built into Ubuntu 23.04, but since we are using Ubuntu 20.04, we will have to install it in a later step. Instructions on how to use oscap-ssh can be found in the OpenSCAP <a href="http://static.open-scap.org/openscap-1.2/oscap_user_manual.html#_scanning_remote_machines_using_oscap_ssh">user manual</a> under section 5.1:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z9fqSw_o4TszpY7zR7SCPA.png" /></figure><p>The instructions are brief and can be a bit misleading. Lucky for you, I’ve already gone through the pain of not having the above command work, and figuring out what exactly they want you to do.</p><p>First, let’s confirm that OpenSCAP correctly installed on the VM from the CloudInit file:</p><pre>which oscap</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2dBxaA0p9LtgoYTzSORZxA.png" /></figure><p>Now let’s confirm oscap was installed in our $PATH. It looks like it was installed in /usr/bin . To check our current $PATH, run:</p><pre>echo $PATH</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*esIVOVtFabWqjWePg7NxGA.png" /></figure><p>Perfect, we should be good to continue. The official documentation states <em>“The tool uses SSH connection to copy the SCAP content to a remote machine, then it runs an evaluation of the target system and downloads the results back.”</em> This means we need the SCAP content on our local machine, so that the tool can copy it to the remote machine and run the scan. The SCAP content the documentation is referring to is the SCAP security guide. For Ubuntu 16.04 and later, use the following command <strong>on your local machine</strong> (not the VM):</p><pre>sudo wget https://github.com/ComplianceAsCode/content/releases/download/v0.1.69/scap-security-guide-0.1.69.zip</pre><p>The SCAP security guide will install as a zip file. Find the file in your current directory and unzip it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y_PCSUM4gKWuoBn1Aa1Sew.png" /></figure><p>After unzipping the file, cd into the scap-security-guide-0.1.69 and you will find the security guide for Ubuntu 20.04.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tuboLqjQMoSCLKuNZP2nOg.png" /></figure><p>When running a scan with OpenSCAP, you need to include in the command which security profile you want to run the scan as. Each security policy has a variety of profiles that contain different rules and values to adhere to different security baselines, thus tailoring the security configurations to specific use cases. You can see the available profiles of the ssg-ubuntu2004-ds-1.2.xml package by using the following command:</p><pre>oscap info ssg-ubuntu2004-ds-1.2.xml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PbnXotiQtV1S_bacSA5eWA.png" /></figure><p>Well that’s no fun. The documentation told me I don’t need OpenSCAP (oscap) installed on my local machine but I can’t view the security guide profiles without it. Now what?</p><p>A few workarounds here:</p><ul><li>You have oscap installed on the VM, so you could use the same wget command above to also install the security guides on the VM, then run oscap info ssg-ubuntu2004-ds-1.2.xml to see the profiles.</li><li>Or, you could go ahead and install OpenSCAP on your local machine so that you can actually read the contents of the security guide.</li><li>Or, best case scenario, you’ve done this before and you already know the name of the profile you want to use.</li></ul><p>Fortunately for us, the latter is the case. I’ve done this a few times before, so trust me when I say the STIG profile for the Ubuntu 20.04 security guide is xccdf_org.ssgproject.content_profile_stig . Here’s a screen shot from one of my previous tutorials to show the profiles (in this tutorial, I had oscap and the security guides installed on the same machine, thus I was able to access the profiles):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TnNpma3pSPJ9ooXnB0jTRg.png" /></figure><p>The oscap-ssh documentation continues, <em>“The remote machine needs to have OpenSCAP installed.”</em> We did that earlier via our CloudInit file when we spun up the VM, and confirmed it was installed and in our $PATH.</p><p>Ok, at this point, we have the Ubuntu 20.04 security guide installed on our local machine, OpenSCAP installed on the remote machine, and we know the name of the security guide and profile we want to use to run our scan. Let’s move on to installing the oscap-ssh utility.</p><h3>Install oscap-ssh</h3><p>The oscap-ssh tool is built into Ubuntu 23.04, but since we are using Ubuntu 20.04, we will have to install it from <a href="https://github.com/OpenSCAP/openscap/blob/main/utils/oscap-ssh">this</a> script. On your <strong>local machine</strong>, back out of the scap-security-guide-0.1.69 directory we were in and create a new file called oscap-ssh with the contents from this file:</p><p><a href="https://github.com/OpenSCAP/openscap/blob/main/utils/oscap-ssh">https://github.com/OpenSCAP/openscap/blob/main/utils/oscap-ssh</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H1j34VpmQz3CjOpFYGrRWA.png" /></figure><p>Make the file executable:</p><pre>sudo chmod u+x oscap-ssh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xqEIX7VPJspG26BJot-hWg.png" /></figure><h3>Scan the Remote VM</h3><p>Let’s do a quick recap:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DQIpcBmzBbKSC6MdseILwQ.png" /></figure><p>We are now ready to run our oscap-ssh command from <strong>your local machine</strong>! The structure of the command goes like this:</p><pre>./oscap-ssh user@host &lt;SSH_port&gt; xccdf eval --profile &lt;ssg_profile&gt; \<br>--report report.html &lt;path_to_security_guide&gt;/ssg-ubuntu2004-ds-1.2.xml</pre><p>Running ./ executes the oscap-ssh script in your current directory. The user will be the scap_auditor user that we created earlier because remember, we have the private and public SSH keys for that user locally and we added them to the ssh-agent. The host IP is the IP of our Multipass VM. We can find the IP by running multipass list or multipass info &lt;VM_name&gt; from your local machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C60HAb3Ylt_hCHcIwuAz8w.png" /></figure><p>The SSH_port will be 22 and xccdf eval tells oscap-ssh to evaluate the XCCDF content specified in the given --profile. The --report flag will create a report of the scan results and export those results in an easy to read HTML file report.html . We will replace &lt;path_to_security_guide with the path of the security guide located on our local machine, so oscap-ssh knows where to copy the security guide from to the remote machine. Putting all of this together, we get something like the below command. Make sure you are running the command from your local directory that has the oscap-ssh script!</p><pre>./oscap-ssh scap_auditor@192.168.64.23 22 xccdf eval --profile \<br>xccdf_org.ssgproject.content_profile_stig --report \<br>report.html /Users/brandi/Development/Cyber/STIG/scap-security-guide-0.1.69/ssg-ubuntu2004-ds-1.2.xml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fRqSaXG5Lw3oJy8A9FqwAA.png" /></figure><p>You will see a long list of rules evaluated and whether they passed, failed, or are not applicable. Once the scan finishes, you should see something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fV-I60i1ItdCWbCf1Pn6MQ.png" /></figure><p>Run ls to see that your report.html file was created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uWd0QWkb0euTZhdrdrHdQQ.png" /></figure><p>Open it up and …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dl9hLxK85tUkkHuPA6KEFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LyBAN8pGW2vcCq8zgiz0Dw.png" /></figure><h3>Conclusion and Clean Up</h3><p>Hopefully this tutorial has helped you better understand how you can use OpenSCAP’s oscap-ssh tool to perform a STIG scan on a remote machine from your local one. Remediation of failed rules can be done using OpenSCAP as well, and this will likely be covered in a future tutorial.</p><p>If you no longer need the tools we installed or the report.html file on your local computer, you can delete them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mqVwnOPO7Uvo-JKmNABJTw.png" /></figure><p>To clean up the Multipass VM, run the following:</p><pre>multipass delete &lt;name_of_VM&gt;<br>multipass purge<br>multipass list #Confirm VM has been fully removed</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*egfiECtSXdxDgliS6eIeKQ.png" /></figure><p>Thank you so much for following along! I hope that you learned something and will continue to watch for more DevOps content!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21f6f3f9c27e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/stig-scanning-a-remote-machine-from-local-21f6f3f9c27e">STIG Scanning a Remote Machine from Local</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automate STIG Scanning with OpenSCAP and CloudInit]]></title>
            <link>https://medium.com/defense-unicorns/automate-stig-scanning-with-openscap-and-cloudinit-b9426f515336?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9426f515336</guid>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[openscap]]></category>
            <category><![CDATA[stig]]></category>
            <category><![CDATA[cloud-init]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Thu, 12 Oct 2023 15:13:39 GMT</pubDate>
            <atom:updated>2023-10-12T15:25:20.903Z</atom:updated>
            <content:encoded><![CDATA[<h4>In this tutorial we create a CloudInit file to automate STIG scanning with OpenSCAP at server creation time</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hbje1aqEIHbHpPozJEWSmQ.png" /></figure><h3>Prerequisites:</h3><ul><li><a href="https://multipass.run/docs/how-to-guides">Multipass</a> installed (click link for installation guide)</li></ul><h3>Objectives:</h3><ul><li>Create new SSH key for new user</li><li>Launch Multipass VM with CloudInit file that creates a new user and automates Ubuntu 20.04 STIG scanning via OpenSCAP</li><li>SSH into Multipass VM using new user and new SSH key</li><li>Confirm Ubuntu STIG scan was completed and view report</li></ul><h3>Create SSH Key for User</h3><p>This tutorial is an extension of my work done <a href="https://medium.com/@bm54cloud/stig-scanning-with-openscap-675c7292d7cb">here</a> where I discussed how to manually run a STIG scan using OpenSCAP. In this tutorial, we will use CloudInit to automate STIG scanning with OpenSCAP upon server creation.</p><p>The goal here is to create a user that will automatically run the Ubuntu 20.04 STIG via OpenSCAP upon server creation. To do this, we can use a <a href="https://cloudinit.readthedocs.io/en/latest/reference/examples.html">CloudInit</a> YAML file. The CloudInit file located <a href="https://github.com/bm54cloud/STIG/blob/main/cloud-init.yaml">here</a> is used to create a user named scap_auditor . The scap_auditor user has sudo permissions and is assigned an SSH key. You may already have a personal SSH key on your local machine, but we want to create a separate RSA-type SSH key specifically for the scap_auditor user. To do this, run the following command on your local machine:</p><pre>ssh-keygen -t rsa -C &quot;scap_auditor@ATAR_IP&quot;</pre><p>The -t flag indicates what type of SSH key to generate (I chose RSA). The -C flag stands for “Comment” and allows us to create a custom comment at the end of the key that says “scap_auditor@ATAR_IP.” This comment is not necessary, but since I have more than one SSH key, it helps me to remember that this is my scap_auditor user SSH key. You can put whatever you want in this comment section.</p><p>After running the above command, you will be asked to give a file name in which to save the new SSH key. If you just press Enter and don’t indicate a file name, the key will be saved as the default “id_rsa” in your ~/.ssh folder. I wanted to remember that this key is specifically for my scap_auditor user, so I chose to save the key in a file called “scap-key.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IjoBjfhety0yNX4iKZvcJg.png" /></figure><p>When asked to enter a “passphrase” you can simply press Enter twice if you do not want to use a passphrase. If you do want to use a passphrase, type it out and press Enter, then type it out again to confirm it. Make sure you take note of the passphrase somewhere, as you will not be able to recover it later. After entering the passphrase, your key pair will be created and can be found in ~/.ssh .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pRFrrjwpUbI8TkbBtPqMcg.png" /></figure><p>Your SSH keys consist of a pair (one private key and one public key). The public key ends with the .pub extension, and the corresponding private key has the same name, without an extension.</p><h3>Launch Multipass VM</h3><p>For this section, copy the below CloudInit file to your local directory. The file can also be found in my repo <a href="https://github.com/bm54cloud/STIG/blob/main/cloud-init.yaml">here</a>.</p><pre>#cloud-config<br>users:<br>  - default<br>  - name: scap_auditor<br>    groups: sudo, admin<br>    shell: /bin/bash<br>    lock_passwd: true<br>    sudo: ALL=(ALL) NOPASSWD:ALL<br>    ssh_authorized_keys: <br>    - ssh-rsa &lt;SSH_public_key&gt;<br>    <br>package_update: true<br>package_upgrade: true    <br>packages:<br>  - unzip<br>  - libopenscap8<br>  - ssg-debderived<br><br>runcmd:<br>  - cd /usr/share/xml/scap/ssg/content<br>  - wget https://github.com/ComplianceAsCode/content/releases/download/v0.1.69/scap-security-guide-0.1.69.zip<br>  - unzip scap-security-guide-0.1.69.zip<br>  - cd scap-security-guide-0.1.69/<br>  - oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_stig --report report.html ssg-ubuntu2004-ds-1.2.xml<br><br># Secure copy results file from remote to local<br># scp ubuntu@10.1.29.82:/usr/share/xml/scap/ssg/content/scap-security-guide-0.1.69/report.html ~/Development</pre><p>You will need to replace &lt;SSH_public_key&gt;under ssh_authorized_keys with the public SSH key you created in the previous section. To get the actual key, vim or cat your &lt;key.pub&gt; file, copy it, and paste it into the CloudInit file. Make sure you are pasting your public key (not private) to the CloudInit file. This CloudInit file creates a default user, then a user called scap_auditor (you can name this user whatever you want) with sudo permissions. Beyond the user creation, the rest of the file automates my previous STIG scanning work done <a href="https://medium.com/@bm54cloud/stig-scanning-with-openscap-675c7292d7cb">here</a>. Specifically, upon creation of the Ubuntu 20.04 server, a scap_auditor user will be created, the listed packages will be installed, and then the user will perform all of the commands under runcmd which will result in the Ubuntu 20.04 STIG scan being run via OpenSCAP. The STIG scan results results will then be output to a report.html file.</p><p><a href="https://cloudinit.readthedocs.io/en/latest/reference/examples.html">Multipass</a> is pretty simple to use and allows you to quickly spin up VMs for development and testing work. This came in particularly useful recently when my work’s remote servers were down. I was able to spin up a Multipass VM within minutes to continue testing the work I was doing. For this tutorial, I wanted to spin up a VM with the Ubuntu 20.04 OS. To do this, I first visited the Multipass documentation <a href="https://multipass.run/docs/get-started-with-multipass-linux">here</a> to find the custom alias for Ubuntu 20.04 (see screenshot below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bfFd5gVTnneUSSwhhO1ApA.png" /></figure><p>After finding the alias of the OS you want to use, pass it into the below command to spin up the Multipass VM:</p><pre>multipass launch focal --name STIG --cloud-init cloud-init.yaml</pre><p>Use --name to give your VM a name, and --cloud-init to indicate the name of your CloudInit YAML file. By passing in the CloudInit file at VM creation, all of the commands in the file will be performed on creation of the server. If you’re familiar with creating an AWS EC2 instance, this is similar to putting a script in the User Data section.</p><h3>Find IPv4 of VM</h3><p>At this point, you could simply run multipass shell &lt;name_of_VM&gt; to shell into your newly created VM. However, we want to test that we can SSH into the server with the scap_auditor user we created on initial boot. First we need to find the IPv4 address of the VM we just created, and can do that with the following command:</p><pre>multipass info &lt;name_of_VM&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IiM-fN9_6nUQeiOcM_n9Uw.png" /></figure><h3>SSH into Multipass VM</h3><p>Change directory to where your private SSH key is located.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PHfoMWb4KMkh-pHMYF8R_w.png" /></figure><p>Use the -i flag (-i = identity) to pass in the name of the private key file you want to use. Remember, your public key will have the .pub extension and your corresponding private key will be the same key name with no extension. In the above screenshot, scap-key is my private key file and scap-key.pub is my public key file. Replace &lt;VM_IPv4_address&gt; in the below command with the IPv4 address you found in the last section.</p><pre>ssh scap_auditor@&lt;VM_IPv4_address&gt; -i scap-key</pre><p>If prompted “Are you sure you want to continue connecting” type “Yes” and press Enter to continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6OglL0JbR8IRQLLD4CvBQw.png" /></figure><p>Notice I am now SSH’d in as <strong>scap_auditor</strong>.</p><h3>Confirm Ubuntu STIG Completed</h3><p>If you followed along with my previous tutorial, you may remember that when a STIG is run with OpenSCAP, the results are output to a report.html file in located at /usr/share/xml/scap/ssg/content/scap-security-guide-0.1.69/ . Let’s see if our report was created upon server creation.</p><pre>cd /usr/share/xml/scap/ssg/content/scap-security-guide-0.1.69/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8FWngm739H-3kv8LGevexw.png" /></figure><p>We can see that a report.html file does exist where we expect it to. Let’s secure copy it from the remote server to our local machine so we can open it up and see the report. Open a new terminal window on your local machine where you are not SSH’d into the remote server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*-PbXfZZuAibsyJ9ZHqTjpA.png" /></figure><p>Note that in the below command I will copy the file to a Development folder located in my home directory. A few things to note here if you get a “Permission denied (publickey)” error. I you have more than one SSH key (like myself), you need to use the -i flag to indicate which key should be used. Replace &lt;VM_IPv4_address&gt; with the IP address of the Multipass VM you found earlier.</p><pre>scp -i scap-key scap_auditor@&lt;VM_IPv4_address&gt;:/usr/share/xml/scap/ssg/content/scap-security-guide-0.1.69/report.html ~/Development</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ESJq3HnkV1yyHlGhG0gIPA.png" /></figure><p>Find the file in your local directory and open it to view the results of your OpenSCAP STIG scan.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*auTzh7jV-lWN6HA-xte5Mg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J65vBkqCMvtWHDIHQ9Fb6A.png" /></figure><h3>Conclusion and Clean Up</h3><p>There you have it! STIG scanning with OpenSCAP has been automated by passing in a specific CloudInit file upon server creation. If you no longer need the report.html file, you can delete it from your local directory. To clean up the Multipass VM, run the following:</p><pre>exit<br>multipass delete &lt;name_of_VM&gt;<br>multipass purge<br><br># Confirm VM has been deleted<br>multipass list</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cdo4hY0046ZWWUjeMot19g.png" /></figure><p>Thank you so much for following along! Between this tutorial and my last one, hopefully STIG scanning is a little less scary. Keep watching for more tutorials, as I’ll continue to teach as I learn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9426f515336" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/automate-stig-scanning-with-openscap-and-cloudinit-b9426f515336">Automate STIG Scanning with OpenSCAP and CloudInit</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[STIG Scanning with OpenSCAP]]></title>
            <link>https://medium.com/defense-unicorns/stig-scanning-with-openscap-675c7292d7cb?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/675c7292d7cb</guid>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[ubuntu-20-04]]></category>
            <category><![CDATA[stig]]></category>
            <category><![CDATA[openscap]]></category>
            <category><![CDATA[disa]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Tue, 03 Oct 2023 19:12:22 GMT</pubDate>
            <atom:updated>2024-08-22T23:51:39.793Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Learn how to run a STIG for Ubuntu 20.04 using OpenSCAP</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PEN50NeFaFli6_EMY7ojcQ.png" /></figure><h3>Prerequisites:</h3><ul><li>Server with Ubuntu 20.04 installed</li></ul><h3>Objectives:</h3><ul><li>Discuss STIG compliance and its purpose</li><li>Discuss SCAP tools and their justification</li><li>Install OpenSCAP on Ubuntu server</li><li>Install SCAP security guides</li><li>Use OpenSCAP to evaluate the Ubuntu 20.04 STIG</li><li>View the STIG scan report</li><li>Discuss what to do with the results</li></ul><h3>What is a STIG?</h3><p>If you’ve ever worked with the U.S. Department of Defense (DoD) then you may be familiar with or have heard of the term “STIG.” As you can imagine, DoD computer systems contain confidential information that must be protected from vulnerabilities. One layer of protection is to require that any piece of hardware or software that connects to or is a part of the DoD IT network and systems be STIG compliant.</p><p>“STIG” stands for <a href="https://public.cyber.mil/stigs/">Security Technical Implementation Guide</a>, and consists of hardware and software configuration standards whose goals are to make systems as secure as possible. STIGs were developed by the Defense Information Systems Agency (DISA), and compliance with STIGs are mandatory for all DoD agencies and any organizations or contractors using DoD information networks. This ensures that the network and anything utilizing it meets a minimum level of security, based on security best practices and compliance standards.</p><p>In short, a STIG is basically a list of cybersecurity requirements that must be met to ensure basic security measures. For example, a STIG may require secure SSH configuration, and may provide a list of SSH requirements like:</p><ul><li>A specific SSH protocol version</li><li>Strong encryption algorithm</li><li>Key-based authentication (instead of password-based authentication)</li><li>Disabling of root login</li><li>Idle timeout period</li><li>Restrict SSH access to authorized users only</li></ul><p>The list can go on and on but in the end, it supports the best practice configuration. <strong>Remember, the STIG is simply a <em>guide</em>.</strong> If your system were to be 100% compliant with the guide, it likely would not be a functional system. After running a STIG scan, you must take into account the pros and cons of correcting each “Fail,” and determine a balance of risk acceptance for functionality and security.</p><h3>What is SCAP?</h3><p>Imagine you have created a system that has an Ubuntu OS and eight applications running on it. You are creating this system for the DoD, so the entirety of it must be as STIG compliant as possible. It would take weeks to months to have engineers manually check configuration compliance of the OS and all of the applications. On top of the initial check, let’s say you are required to check compliance on a monthly basis, as new vulnerabilities may be identified frequently. How in the world can you keep up with and meet these requirements?</p><p>Enter <a href="https://csrc.nist.gov/projects/security-content-automation-protocol">Security Content Automation Protocol</a> (SCAP) tools. SCAP is a method that helps organizations automate STIG compliance scans. Whereas STIGs are a set of guidelines that provide detailed instructions for securing computer systems and software, SCAP is a protocol that provides a standardized approach to security automation, enabling the automation of repetitive security-related tasks, including vulnerability scanning, configuration assessment, and compliance checks.</p><h3>What is OpenSCAP?</h3><p><a href="https://www.open-scap.org/">OpenSCAP</a> is one of many tools that leverage SCAP standards to automate the assessment and validation of STIG compliance. OpenSCAP is open-source and has a vibrant community committed to its maintenance. In this tutorial, we will install OpenSCAP on our Ubuntu server and use it to run the Ubuntu 20.04 STIG.</p><h3>Install OpenSCAP on Ubuntu Server</h3><p>For this tutorial, I am using a clean server that only has a minimal Ubuntu 20.04 OS installed (no GUI), thus everything will be run through the CLI. To get started, SSH into your server, then update and upgrade packages.</p><pre># SSH into Server<br>ssh ubuntu@server_IP<br><br># Update and upgrade packages<br>sudo apt update &amp;&amp; sudo apt upgrade</pre><p>Next, we can install OpenSCAP with the following command:</p><pre># Install OpenSCAP<br>sudo apt install libopenscap8</pre><p>(Optional) Confirm that OpenSCAP was installed:</p><pre># Confirm installation and location of OpenSCAP<br>which oscap<br><br># Confirm OpenSCAP version<br>oscap -V</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1gOI5ZPqnAklNHP25bP82A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OiOLCwm--BLcorBTLeo7BQ.png" /></figure><h3>Install SCAP Security Guides</h3><p>In addition to OpenSCAP, we need to install the <a href="https://www.open-scap.org/security-policies/scap-security-guide/#:~:text=SCAP%20Security%20Guide%20is%20a,provides%20the%20best%2Dpractice%20solutions.">SCAP Security Guide</a> (SSG) for our Debian-based system. SSGs are security policies that transform STIGs (and other security guidelines) into a machine readable format (XCCDF) that can be used by OpenSCAP to scan your system for vulnerabilities and compliance. We can download the latest SSG packages from the ComplianceAsCode project using the following command:</p><pre># Download the latest Scap Security Guide<br>mkdir /usr/share/xml/scap/ssg/content<br>cd /usr/share/xml/scap/ssg/content<br>sudo wget https://github.com/ComplianceAsCode/content/releases/download/v0.1.69/scap-security-guide-0.1.69.zip</pre><p><em>The </em><a href="https://github.com/ComplianceAsCode/content"><em>ComplianceAsCode</em></a><em> project is an open-source project that creates security policy content and offers Ansible playbooks and Bash scripts to automate compliance solutions, aiming to simplify the process of maintaining compliance with security standards. To find the official DISA STIG, navigate to </em><a href="https://public.cyber.mil/stigs/downloads/"><em>https://public.cyber.mil/stigs/downloads/</em></a><em> and search for “Canonical Ubuntu 20.04 LTS STIG — Ver 1, Rel 9.” I will use the official DISA STIG in a future post.</em></p><p>The latest SSG is contained in a zip file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nO1lqdiRLPnM8NNbZKA0Sw.png" /></figure><p>If you do not have unzip installed, use the following command:</p><pre># Install unzip on Ubuntu server<br>sudo apt install unzip</pre><p>Unzip the SSG file, then change into the SSG directory and look around:</p><pre># Unzip Scap Security Guide<br>sudo unzip scap-security-guide-0.1.69.zip<br>cd scap-secuirty-guide-0.1.69/<br>ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1BBCBTXxnLTU9KleyTdlmA.png" /></figure><p>You should now see the SSG security policies for Ubuntu 20.04.</p><h3>Display Profile Options</h3><p>Each security policy has a variety of profiles that contain different rules and values to adhere to different security baselines, thus tailoring the security configurations to specific use cases. You can see the available profiles of the <strong>ssg-ubuntu2004-ds-1.2.xml</strong> package by using the following command:</p><pre># Display a list of available Profiles<br>oscap info ssg-ubuntu2004-ds-1.2.xml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ufiDtI2TfMqM3uKX9ErSMw.png" /></figure><p>Find the profile with profile_stig in it.</p><h3>Use OpenSCAP to Evaluate the Security Policy and Profile</h3><p>The purpose of OpenSCAP is to evaluate a local system for vulnerabilities and a standard configuration based on a security policy and profile. The basic syntax of scanning using a SCAP source data stream (ex. XCCDF file) starts with the oscap xccdf eval command and an explanation of the command and its flags can be found <a href="https://static.open-scap.org/openscap-1.3/oscap_user_manual.html#_displaying_information_about_scap_content">here</a> in section 4.1.</p><p>For our purposes, we will scan our Ubuntu system using the xccdf_org.ssgproject.content_profile_stig profile and the ssg-ubuntu2004-ds-1.2.xml security policy, so our command will look like this:</p><pre># Evaluate a STIG Profile and write XCCDF results into a report.html file<br>sudo oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_stig \<br>--report report.html ssg-ubuntu2004-ds-1.2.xml</pre><p>Here’s a breakdown of the above command:</p><p>sudo oscap xccdf eval: Indicates to evaluate the compliance of the system against the rules specified in the XCCDF file.</p><p>--profile xccdf_org.ssgproject.content_profile_stig: Specifies to use the xccdf_org.ssgproject.content_profile_stig profile, which represents the Ubuntu 20.04 STIG configuration.</p><p>--report report.html: Indicates to save the compliance evaluation results of the scan in HTML format in a new document called report.html.</p><p>ssg-ubuntu2004-ds-1.2.xml: Indicates the XCCDF content file against which the system&#39;s compliance is being evaluated.</p><p>After the evaluation has completed, you will see a long list of rules and whether your system passed or failed those rules. Your output should look something similar to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZWhdTNrxePNQyaVpyRC0Mw.png" /></figure><p>If you run ls you will see that the report.html file has been created. This file contains all of the output of the evaluation in HTML format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dBlfk3BLDownaxn6l3W_Rw.png" /></figure><h3>View STIG Scan Results</h3><p>To view the results, you can simply vim the report.html file, but it looks very messy. A better way is to secure copy the file from the remote server to your local machine, then view the HTML document in a web browser. To secure copy a file from a remote server to your local machine, the basic syntax looks like this:</p><pre>scp &lt;USER&gt;@&lt;HOST&gt;:source destination</pre><p>If you have multiple SSH keys, you can specify which key to use with the -i flag. For more details on how to do this, see <a href="https://www.techrepublic.com/article/how-to-use-secure-copy-with-ssh-key-authentication/">this</a> tutorial. If the SSH key is not specified, the default will be used. If you get a “permission denied/public key” error, see if <a href="https://saturncloud.io/blog/troubleshooting-scp-permission-denied-publickey-on-ec2-when-using-r-flag-on-directories/#:~:text=SCP%20relies%20on%20SSH%20(Secure,unable%20to%20authenticate%20your%20session.">this</a> tutorial helps you resolve it <em>(even though that tutorial is not exactly what we’re doing here, I found its troubleshooting steps helpful).</em></p><p>To copy the report file from the remote server, first open a new terminal window on your local machine. I am going to copy the report.html file from the remote server to my local ~/Development directory, so my scp command looks like this:</p><pre># Exmaple secure copy from remote to local<br>scp ubuntu@10.1.29.82:/usr/share/xml/scap/ssg/content/scap-security-guide-0.1.69/report.html ~/Development</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eql3VtNBRRHH9L_T-QXRAQ.png" /></figure><p>Notice I run this command from my local machine (not the window where I am SSH’d into the Ubuntu server). If I ls I can see that the report.html file now lives locally in my Development directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5yHcOmdmv3D5jvGvlXLLZw.png" /></figure><p>To view the report, simply double click it in your file directory (Finder on a Mac, File Explorer on Microsoft) and it will open the HTML in your default web browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*HOkYfj0lLnCOng_Nnbcn3A.png" /></figure><p>The web browser version will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JKfxKIhOtSDeDunuN5pwtg.png" /></figure><p>The report gives you a score of compliance (only 58% for this server) and if you scroll down, you will see each configuration that failed and the severity of the failure. More details can be seen by clicking on the specific test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zf398geMvkTsKpaI9GYiYw.png" /></figure><h3>We Have Results, Now What?</h3><p>When building an IT system for the DoD, part of the approval process involves obtaining an Authority to Operate or ATO. This can be a difficult, timely process but is required before your IT system can be used by the DoD. Basically, the DoD wants to ensure your system is secure enough before it is incorporated into their system. As discussed previously, part of ensuring security is identifying vulnerabilities and risk acceptance through STIG scanning. Another part is mitigating as much risk as possible.</p><p>The <a href="https://csrc.nist.gov/pubs/sp/800/37/r2/final">Risk Management Framework</a> (RMF) is a publication by the National Institute of Standards and Technology (NIST) that offers guidance on how to manage information security and privacy risks effectively, providing a systematic approach for organizations to identify, assess, respond to, and monitor risks associated with their information systems and assets. Basically, once you’ve identified the risks and vulnerabilities of your system via your STIG scans, the RMF offers guidance on how to mitigate those risks. By adhering to the RMF objectives, the overall IT attack surface of a system can be reduced.</p><h3>Conclusion</h3><p>Thank you so much for following along with me on this STIG journey! Remember, this is a learning process for both me and you, and as I learn I will continue to try and teach others. If you have more questions on STIG scanning, I found these two resources to be very helpful:</p><h4><strong>OpenSCAP User Manual</strong></h4><p><a href="https://static.open-scap.org/openscap-1.3/oscap_user_manual.html#_introduction">https://static.open-scap.org/openscap-1.3/oscap_user_manual.html#_introduction</a></p><h4>Master Degree Thesis: Security assessment and threat response through SCAP</h4><p>chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/<a href="https://webthesis.biblio.polito.it/22850/1/tesi.pdf">https://webthesis.biblio.polito.it/22850/1/tesi.pdf</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=675c7292d7cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/stig-scanning-with-openscap-675c7292d7cb">STIG Scanning with OpenSCAP</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zarf Deployment with PLG Log Monitoring]]></title>
            <link>https://medium.com/defense-unicorns/zarf-deployment-with-plg-log-monitoring-bb61a04e72b7?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb61a04e72b7</guid>
            <category><![CDATA[promtail]]></category>
            <category><![CDATA[grafana-loki]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[zarf]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Mon, 24 Jul 2023 16:16:11 GMT</pubDate>
            <atom:updated>2023-07-25T19:26:50.477Z</atom:updated>
            <content:encoded><![CDATA[<h4>Deploy a WordPress blog using Zarf and take advantage of Zarf’s integrated log monitoring functionality</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mA3iyz-vjccu46mPQdwcUA.png" /></figure><p><em>In this tutorial, we will use the open source tool Zarf to deploy a sample WordPress blog into a Kubernetes cluster. Zarf offers optional deployment of the PLG (Promtail/Loki/Grafana) log monitoring stack. We will briefly dive into each component of the PLG stack and some of the commonly used monitoring features.</em></p><h3>Prerequisites:</h3><ul><li><a href="https://kubernetes.io/docs/tasks/tools/">kubectl</a> installed</li><li><a href="https://k3d.io/v5.5.1/">k3d</a> installed</li><li><a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">Git</a> installed</li><li><a href="https://docs.zarf.dev/docs/the-zarf-cli/#:~:text=You%20can%20get%20the%20Zarf,of%20the%20Getting%20Started%20guide.">Zarf CLI</a> installed</li></ul><h3>Objectives:</h3><ul><li>Explore Promtail, Loki, and Grafana</li><li>Spin up a k3d cluster</li><li>Use Zarf to deploy a sample WordPress blog into the cluster</li><li>Deploy the Zarf init package</li><li>Create the Zarf WordPress package</li><li>Deploy the Zarf WordPress package</li><li>Connect to the WordPress blog</li><li>Log into and explore Grafana</li></ul><h3>The PLG Stack</h3><p>The PLG stack consists of Promtail, Loki, and Grafana and is a popular log monitoring tool. Promtail pulls data logs and sends them to Loki, where they are indexed and grouped via labels. Grafana dashboards can then be used to query and display the logs.</p><h4>What is Promtail?</h4><p><a href="https://grafana.com/docs/loki/latest/clients/promtail/#:~:text=Promtail%20is%20an%20agent%20which,Discovers%20targets">Promtail</a> takes local log files or systemd journal logs and ships them to a private Loki instance. Promtail is first deployed to a Kubernetes cluster as a daemon set, where it performs service discovery on each node. After discovering services that emit log lines to files, it attaches labels to the log streams, then batches the target logs and ships them to Loki.</p><h4>What is Loki?</h4><p><a href="https://grafana.com/docs/loki/latest/#:~:text=Grafana%20Loki%20is%20a%20set,(just%20like%20Prometheus%20labels).">Loki</a> is a Grafana tool that aggregates logs efficiently via labels. After Promtail discovers and labels log streams, it ships them to Loki where they are indexed based on their labels. These logs can then be queried. Loki breaks up large queries into smaller requests, dispatching them in parallel (parallelization). This decreases response time, allowing a user to query large amounts of data without compromising speed.</p><h4>What is Grafana?</h4><p><a href="https://grafana.com/">Grafana</a> is an open-source tool that can be used to visualize and analyze data, allowing you to create custom dashboards for easier interpretation. After connecting your database to Grafana, you can choose from one of the many prebuilt dashboards, or you can create your own to include metrics, logs, and tracings specific to your needs. Grafana can integrate with hundreds of popular plugins like Prometheus, Loki, and CloudWatch, just to name a few. You can set alerts for your data as well as easily generate PDF reports from your dashboard to be shared with others (ex. daily performance logs). Below is an example dashboard monitoring server requests, memory/CPU, and more. The various graphs and colors make interpretation of the data easier to visualize and understand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CNEQOi6mT7wbbk64kLlh0Q.png" /><figcaption>Grafana sandbox dashboard</figcaption></figure><p>Many well-known and high profile companies use Grafana as part of their data monitoring stack, and case studies can be found <a href="https://grafana.com/success/">here</a>. Below are just a few advantages of using Grafana to optimize data interpretation:</p><ul><li>Provide real-time information on site performance</li><li>Decrease mean time to repair by identifying issues early</li><li>Measure performance to drive improvements</li><li>Create a consistent observability experience for developers</li><li>Provide fast feedback to developers</li><li>Analyze security logs at scale to detect malicious activity</li></ul><p>In this tutorial, we will deploy a sample WordPress blog into a Kubernetes cluster using Zarf. To learn more about Zarf, visit my previous blog posts <a href="https://medium.com/defense-unicorns/deploying-dos-games-on-a-kubernetes-cluster-with-zarf-b254c604a55d">here</a> and <a href="https://medium.com/defense-unicorns/using-zarf-to-deploy-a-podinfo-flux-application-in-a-kubernetes-cluster-22dc6c02510c">here</a>, or the official documentation <a href="https://docs.zarf.dev/docs/zarf-overview">here</a>. Although Zarf can do a lot of really cool things, in its simplest form, it packages an application’s source code into a single tarball file, allowing that file to be easily deployed into a cluster. In the next few steps, we’ll spin up a cluster and deploy a sample WordPress blog with Zarf. Zarf gives you the option to deploy the PLG monitoring stack, which we will do. Then we’ll log into Grafana and checkout the Loki dashboard Zarf has deployed.</p><p>It is important to note that while Grafana is a visualization tool, the Loki dashboard deployed with Zarf is specifically looking at <em>logs</em>, not the state of your cluster. With Loki, we can query logs, but not CPU utilization, server requests, or other metrics, so the dashboard that we will see later in this tutorial will look a bit different than the vibrant one above.</p><h3>Spin Up a k3d Cluster</h3><p>If you already have some other Kubernetes cluster running, you are welcome to use it. If you don’t, <a href="https://k3d.io/v5.5.1/">k3d</a> is a good option for local testing. To create a k3d cluster, first make sure k3d is installed. If you are using a Mac with Homebrew, simply run brew install k3d . If you are using a different machine, follow the k3d installation instructions <a href="https://k3d.io/v5.5.1/">here</a>. Once k3d is installed, use the following command to spin up a cluster:</p><pre>k3d cluster create &lt;cluster_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3-SctZb2DVGGemxz4HrZw.png" /></figure><p>The cluster should now be up and running but you can double check with the kubectl cluster-info command or look at your Docker Desktop if you have it installed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f6JKugr-2l1LWv3yc4nwJw.png" /></figure><h3>Confirm the Zarf CLI is Installed</h3><p>If you’ve used Zarf before, confirm the version with:</p><pre>zarf version</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*fUx56LkCNB5m5IC6TH0fdQ.png" /></figure><p>If you don’t have the Zarf CLI installed yet, follow <a href="https://docs.zarf.dev/docs/the-zarf-cli/">these</a> instructions.</p><h3>Clone the Zarf Repository to Your Local Machine</h3><p>The Zarf repository has several application examples that can be used for learning and local testing. We will use the sample WordPress blog located in the repository at zarf/examples/wordpress . Clone the entire Zarf repository to your local machine:</p><pre>git clone https://github.com/defenseunicorns/zarf.git</pre><p>To learn more about how Zarf works and how to compose a zarf.yaml file, visit my previous blog articles on Zarf or the official <a href="https://docs.zarf.dev/docs/zarf-overview">Zarf</a> documentation.</p><h3>Deploy the Zarf Init Package</h3><p>Zarf requires a special <a href="https://docs.zarf.dev/docs/create-a-zarf-package/zarf-init-package">‘init’ package</a> whose source code is found at the root of the Zarf repository. The Zarf init package installs several core components needed for Zarf to function, as well as some optional ones. For this tutorial, it’s important to note that logging is one of the optional features you will be prompted to install. When logging is enabled, Zarf installs the Promtail/Loki/Grafana monitoring stack into the cluster. This is important because our focus here is to take a deeper look at this log monitoring tool that is built into Zarf.</p><p>To install the Zarf init package, first make sure you are in the root of the Zarf repository:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iBKfBGQQZTtOLxGaWDeUuA.png" /></figure><p>Then run the following command:</p><pre>zarf init</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rNCruS5fK6HMN-HryCFcOw.png" /></figure><p>You will be given a few prompts:</p><ul><li>Do you want to download this init package? Answer “y”</li><li>Deploy this Zarf package? (y/N)Answer “y”</li><li>Deploy the logging component?(y/N)Answer “y”</li><li>Deploy the git-server component?(y/N)Answer “y”</li></ul><p>The Zarf init package will take a few minutes to deploy. Once it is completed, you will see something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TNgLpXxyITyUGZk2-fs1FQ.png" /></figure><p>Save these Usernames and Passwords somewhere as you’ll need them later to log into Grafana.</p><h3>Create the Zarf WordPress Package</h3><p>Now that our Zarf init package has been deployed, we’re ready to package up the code for our sample WordPress blog. Use cdto change your directory to examples/wordpress then ls to confirm the files in your directory:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eiVvvbOe9bGxxjHTHFRp0g.png" /></figure><p>With the next command we’ll use Zarf to package up our WordPress blog source code into a single tarball file:</p><pre>zarf package create . </pre><p>You will be given a few prompts:</p><ul><li>Create this Zarf package? (y/N)</li><li>Please provide a value for &quot;Maximum Package Size&quot; (0)</li></ul><p>Answer y to the first prompt. For the second, simply hit Enter and this will override the max package size prompt, as it is not necessary for this tutorial. Once the application package is created, you will see this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z9n1ekLDeJnQ_8xhUqqxfw.png" /></figure><p>Check your local files again and you will see the tarball has been created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZxSsH0wSYA1M2OipWmkfMg.png" /></figure><h3>Deploy the Zarf WordPress Package</h3><p>Now that our tarball package is created, we want to deploy it to our Kubernetes cluster. We’re going to use <a href="https://k9scli.io/">K9s</a> to watch this deployment. Zarf has the K9s tool built into it, so open a new terminal window and run:</p><pre>zarf tools monitor</pre><p>This opens the K9s tool for your current cluster, and you can see all of the pods that are currently deployed. Notice that the Zarf init package deployed several pods (agent-hook, zarf-loki-stack, etc) into a “zarf” namespace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8A8lvZF00nKALmM7FAQO2Q.png" /></figure><p>Place your two terminal windows side by side so you can see events in real-time as commands are run.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*693lyC265XXaU88gYAIeXA.png" /></figure><p>Now we’re ready to deploy the Zarf WordPress package (our tarball file). Use the following command to deploy the package:</p><pre>zarf package deploy</pre><p>You will be prompted Choose or type the package file [tab for suggestions] . Press the tab button and you should see the tarball that we created earlier. After confirming the correct tarball, press Enter .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CafOKLEZEP6ny2zFEgYiNA.png" /></figure><p>You will then have some additional prompts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o-yKsM-SrPX6-fRzPHOSzg.png" /></figure><p>Answer Yes to the first prompt, then enter values for the requested environment variables. These prompts are specific to the WordPress blog that’s being deployed, so if you were deploying a different application, you would not encounter them. For this example, I’m using a simple Username and Password that will be deleted once this tutorial is complete, but in the real word, you would want to use a more secure password. Once the deployment has finished, you will see this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UEg01xxlwYWBSuIlW4cpEQ.png" /></figure><p>If you were watching the K9s window, you should have seen the WordPress pods being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2d8jRtsnfTJyuHP5MZ7nqQ.png" /></figure><h3>Connect to WordPress Blog</h3><p>Let’s connect to the sample WordPress blog we just deployed on our cluster. Zarf makes it easy and gives you the command to use:</p><pre>zarf connect wordpress-blog</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2a26PbO48TFTgmvPQXnbQw.png" /></figure><p>This opens a port-forward tunnel from your local machine, allowing you to access the WordPress blog on your cluster. Your default web browser will automatically open to the WordPress site:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DTOLV1B8abGlcUCC9ucFdQ.png" /></figure><p>Ok that was cool! Use ctrl + c to end the port-forward. What if I wanted to log into the backend of the WordPress site with the Username and Password environment variables I set? Run the following command:</p><pre>zarf connect wordpress-admin</pre><p>This opens another port-forward and your default browser, navigating you to the WordPress admin login page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SX4jnb7zn70WzTg3cP3QYQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nds3xoT5goVa9egWffjhYg.png" /></figure><p>Use the Username and Password environment variables you set to log in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OZg5KmcNsAMs8d8cQq651w.png" /></figure><p>Boom! Okay, enough of that. In the terminal, close the port with ctrl + c . Let’s move on to the point of this tutorial: log monitoring!</p><h3>Log Into and Explore Grafana</h3><p>When we deployed the Zarf init package earlier, we selected Y when prompted to install logging. Zarf installs the PLG stack, which includes Grafana. After installing the Zarf init package, you are given the Zarf command to connect to logging as well as the Username and Password. Remember I told you to save them? Go find them now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*TNgLpXxyITyUGZk2-fs1FQ.png" /></figure><p>To keep yourself organized, <strong>open a new terminal window</strong> and run the following command (you’ll see why you want it in a separate window):</p><pre>zarf connect logging</pre><p>The above command opens a port-forward tunnel and will open your default web browser to the Grafana login page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UX_79Vo1qnSn2hpjJKRU0Q.png" /><figcaption>Port-forward (see why you want it in its own window)</figcaption></figure><p>Use the “Logging” Username and Password you saved to log into the Grafana website.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z06snFAguimTrRw1UIrPFA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PfHnYUegrk8yqNQrtr26LA.png" /><figcaption>Grafana login page</figcaption></figure><p>Your initial login should be met with a dashboard similar to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*klA0eDnSprmXdWSBQIgH1g.png" /></figure><p>This is the Loki dashboard that Zarf has chosen for monitoring your data. In the Loki dashboard, logs are aggregated and indexed based on their labels that were assigned by Promtail.</p><p>Notice that the dashboard automatically opened to a query based on <strong>“{namespace=”zarf”}”</strong>. The histogram shows the quantity of new logs per timestamp. In the picture below, you can see there was a spike in log activity between the 07:07:30 timestamp and the 07:08:00 timestamp. The bars of the histogram are color-coded to indicate <em>unknown, warning, info, error, </em>or <em>debug</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F5TLD60RHGRGdwtDOHFWeA.png" /></figure><p>Under the histogram there are toggle buttons that allow you to optimize the output of your logs. For example, you toggle on/off if you want the timestamp to appear. To see a description of the <em>None, Exact, Numbers, </em>and <em>Signature</em> settings, simply hover over the word.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETYAoYbm6BcDXLt7vNswzA.png" /></figure><p>The Loki dashboard will display the most recent 1,000 logs. If you want to see more, scroll to the bottom of the displayed logs and click <strong>Older logs</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xnS-WIeFUgaKLgWn3icDbQ.png" /></figure><p>Grafana has made it easy to query logs without knowing a query language. You can search logs via labels by clicking the <strong>Log browser</strong> button on the top left, then selecting a label from the given options. We were previously looking at the logs in the <em>zarf</em> namespace. To look at logs in a different namespace, select the <em>namespace</em> label and then select from the three namespaces that are found in your cluster (I chose the <em>wordpress</em> namespace). Once you’ve selected your namespace, click <strong>Show logs</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jtwehN6ExatIrJu48wHhWA.png" /></figure><p>You are now viewing logs for the <em>wordpress</em> namespace and notice how the histogram has changed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dw6aD3q2okZ97RjVLyDhwg.png" /></figure><p>You can evaluate an individual log by clicking on it. This will expand a window and give you detailed information about that log. The log below contains a warning message, and once expanded you can see the complete message that a datasource is outdated.</p><p>To narrow down your query, you can use piping to further filter your results. Grafana log query syntax explanation can be found <a href="https://grafana.com/docs/loki/latest/logql/log_queries/">here</a>. In the log browser search bar, use the pipe character to add a filter. Maybe we only want to see the logs that have the word “error” in them. To do this, we would pipe for “error” by typing |= “error”then click <strong>Run query</strong>. Notice that now all of the logs listed are then ones that include the word “error.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uQzb5WqrgR9DEfpOAai5qA.png" /></figure><p>If you try to run a query filter and get an error like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3YaVU01XAyrLKuN2DkfPCA.png" /></figure><p>copy your query filter, then start a new query (click <strong>Log browser</strong> and select what labels you want to query), then paste the filter expression in the log browser search bar instead of directly typing it out. Then click the <strong>Run query</strong> or refresh button. There is a known bug in Grafana at the time of this writing and is discussed <a href="https://github.com/grafana/grafana/issues/54817">here</a> and <a href="https://github.com/grafana/grafana/issues/54535">here</a>.</p><h3>Conclusion and Cleanup</h3><p>In this tutorial we used Zarf to deploy a sample WordPress blog into a Kubernetes cluster, then logged into Grafana and reviewed some of the monitoring capabilities of the Loki dashboard. Zarf has made monitoring of your logs easier by directing your logging to the Loki dashboard, instead of you having to build a dashboard from scratch. Grafana does allow you to create custom dashboards if needed, and videos of how to do this can be found on the Grafana <a href="https://grafana.com/videos/?plcmt=footer">website</a>.</p><p>If you’re ready to clean up everything we created, use the following command to tear down the resources that were created with Zarf:</p><pre>zarf destroy --confirm --remove-components</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JMMPv63PZgzJkrgloapquA.png" /></figure><p>If you run zarf package list you should no longer see your <em>init</em> or <em>wordpress </em>packages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*furHeMS7pHKmSQm-vUk3kQ.png" /></figure><p>To remove the k3d cluster you created, run the following command:</p><pre>k3d cluster delete &lt;cluster_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*QiWDat-CX3K9-7IyLai1Iw.png" /></figure><p>The last thing to do is remove the tarball file from your local directory:</p><pre>rm &lt;zarf_tarball_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OZbO9eh2SL_pBMPJDl6KIw.png" /></figure><p>Hopefully this tutorial was helpful in demonstrating how Zarf can be used to deploy packages as well as Zarf’s built in log monitoring capability. Thank you so much for following along and keep watching for more DevOps tutorials!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb61a04e72b7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/zarf-deployment-with-plg-log-monitoring-bb61a04e72b7">Zarf Deployment with PLG Log Monitoring</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Zarf to Deploy a Podinfo-Flux Application in a Kubernetes Cluster]]></title>
            <link>https://medium.com/defense-unicorns/using-zarf-to-deploy-a-podinfo-flux-application-in-a-kubernetes-cluster-22dc6c02510c?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/22dc6c02510c</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[zarf]]></category>
            <category><![CDATA[airgap]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[flux]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Tue, 11 Jul 2023 13:52:45 GMT</pubDate>
            <atom:updated>2023-07-11T13:52:45.868Z</atom:updated>
            <content:encoded><![CDATA[<p><em>In this tutorial, we will use the open-source tool Zarf to deploy a Podinfo with Flux application to a Kubernetes cluster.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WtanPG7VmxzboEn3pVfzWg.png" /></figure><h3>Prerequisites:</h3><ul><li>Basic Kubernetes knowledge</li><li><a href="https://kubernetes.io/docs/tasks/tools/">kubectl</a> installed</li><li><a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">git</a> installed</li></ul><h3>Objectives:</h3><ul><li>Discuss Zarf and its benefits</li><li>Install Zarf CLI</li><li>Discuss the flow of Zarf in an internet-connected environment</li><li>Clone the Zarf repository</li><li>Install the Zarf ‘init’ package</li><li>Create a Zarf package containing a Podinfo-Flux application</li><li>Deploy the Zarf Podinfo-Flux package to a Kubernetes cluster</li><li>Test connectivity to pods with kubectl port-forward</li><li>Zarf connect to Gitea</li></ul><h3>What is Zarf?</h3><p><a href="https://docs.zarf.dev/docs/zarf-overview">Zarf</a> is an open-source tool created to support easier deployment of software packages, particularly in air-gap or disconnected environments. Although Zarf was designed to solve the challenge of deploying software with little or no internet connectivity, it can also be used to deploy software in connected environments. The benefits of using Zarf, especially in highly-regulated government and non-government industries, are that it simplifies the creation of a Kubernetes cluster if one does not already exist, enables cluster logging, and automates <a href="https://www.cisa.gov/sbom">SBOM</a> creation.</p><h3>Spin up a Kubernetes Cluster</h3><p><em>If you already have a Kubernetes cluster running, skip to the “Install Zarf” section.</em></p><h4>Linux OS</h4><p>Zarf has the ability to create a k3s Kubernetes cluster on a Linux machine. If you look at the <a href="https://github.com/defenseunicorns/zarf/blob/main/packages/distros/k3s/common/zarf.yaml#L13">source code</a> for the Zarf cluster creation, you will see that the k3s component is specifically for a Linux operating system, and it requires root access. If you are running a Linux machine and run the command zarf init you will be prompted to deploy the k3s component (if you are not on a Linux machine, you will not be prompted). When asked “Deploy the k3s component? (y/N)” type “Yes”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*v6O3NBdQ-SL4bdKIpvVkCw.png" /></figure><p>This will create a k3s Kubernetes cluster for you. For more details on this process, see the official Zarf documentation <a href="https://docs.zarf.dev/docs/zarf-tutorials/creating-a-k8s-cluster-with-zarf">here</a>.</p><h4>Non-Linux OS</h4><p>For a non-Linux machine without a cluster running, use the Kubernetes distribution of your choice to spin up a simple cluster. If you have k3d installed, you can spin up a cluster with the following command:</p><pre>k3d cluster create &lt;cluster_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOXyKTerTsPNDi4R93AsBQ.png" /></figure><p>Confirm your cluster creation using the following command:</p><pre>kubectl config get-contexts</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nU6_RdOnhFdQyjd3uS_B7A.png" /></figure><h3>Install Zarf CLI</h3><p><em>If the Zarf CLI is already installed, skip to the next section.</em></p><p>Zarf CLI installation instructions can be found <a href="https://docs.zarf.dev/docs/getting-started">here</a> and if you’re working on a Mac with <a href="https://brew.sh/">Homebrew</a> installed, you can use the following commands:</p><pre>brew tap defenseunicorns/tap<br>brew install zarf</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wq5lan8tFSXRRGex.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*89_nnHXn6zK52ohS.png" /></figure><p>By installing the Zarf CLI, we can now use zarf commands. Once installation is complete, check the version with:</p><pre>zarf version</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*ZMA0kVeeQ-8XnAol.png" /></figure><h3>Zarf Flow in Connected Environments</h3><p>Zarf can be used in internet-connected environments and disconnected environments, and the flow of commands is slightly different for each. This tutorial is deploying a software application in an internet-connected environment, so the steps of deployment will be one less than if we were working with an air-gap environment. In this tutorial, we will use Zarf to deploy the podinfo-flux example found at <a href="https://github.com/defenseunicorns/zarf/tree/main/examples/podinfo-flux">zarf/examples/podinfo-flux</a>. Our flow will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sCHHf0dFcZgxs11gnwoMZw.png" /></figure><h3>Clone the Zarf Repository</h3><p>To work with Zarf, you need access to the Zarf ‘init’ package, which can be found at the root of the Zarf repository. Use the following command to clone the Zarf repository to your local machine:</p><pre>git clone https://github.com/defenseunicorns/zarf</pre><p>Take note of the <strong>zarf.yaml</strong> file (we’ll look at this closer in the next section).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JHmNX7ZtakNxgXtHsB3jEw.png" /></figure><h3>Install the Zarf Init Package</h3><p>One of the first steps to using Zarf is installing the <strong>Zarf ‘init’</strong> package. For detailed information about the Zarf ‘init’ package, visit the official documentation <a href="https://docs.zarf.dev/docs/create-a-zarf-package/zarf-init-package">here</a>. The Zarf ‘init’ package is defined in the <strong>zarf.yaml file </strong>found at the root of the Zarf repository we cloned in the previous section. Let’s take a look at the zarf.yaml file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*64Ipbom2UGZVFrvc7bT6VQ.png" /></figure><p>This zarf.yaml file defines the seven core components that make up the Zarf ‘init’ package:</p><ul><li>k3s</li><li>zarf-injector</li><li>zarf-seed-registry</li><li>zarf-registry</li><li>zarf-agent</li><li>logging</li><li>git-server</li></ul><p>Note that each component imports from a path of the Zarf GitHub repo. Feel free to explore the repo and the <a href="https://github.com/defenseunicorns/zarf/tree/main/packages"><strong>packages</strong></a> directory to see the various zarf.yaml files that define the components. You don’t necessarily need to know exactly what each of these components does to use Zarf. However, if you’re interested to learn more about each component, visit the official Zarf ‘init’ documentation <a href="https://docs.zarf.dev/docs/create-a-zarf-package/zarf-init-package">here</a>.</p><p>Now that we know a little but about the Zarf ‘init’ package, let’s install it. First, make sure you are in the root of the Zarf repository you cloned:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*ucO9p7Hl6Fm-mw2AjRXahA.png" /></figure><p>Then run the following command:</p><pre>zarf init</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Spwe9q61gpuwurpFeINUjA.png" /></figure><p>You will be given a few prompts. The first will be “Deploy this Zarf package?” Type “Yes”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*nT9ln-XN71X3KKKcH7RSMg.png" /></figure><p>The next will be “Deploy the logging component?” This is optional so you can choose “Yes” or “No”, but I’ve selected “No”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c500-qJIi-IrPBRjxQxIRQ.png" /></figure><p>The third will be “Deploy the git-server component?” I’ve selected “Yes” for this, as it adds <a href="https://about.gitea.com/">Gitea</a> into the cluster, allowing for GitOps source control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-jC6CZClgefsDCIhFRO7jQ.png" /></figure><p>Once the Zarf ‘init’ package has completed deployment, you will see something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_NwHvANBRRVL7OWugfrDyw.png" /></figure><p>Save this Username and Password information somewhere like a notepad or temporary file. We may use them later. To check that the Zarf ‘init’ package was successfully installed, run the following command:</p><pre>zarf package list</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ab14ZbMHHTkyuujG1E6zqA.png" /></figure><p>Great! The init package is there, so now we can move on to creating our Podinfo-Flux application package.</p><h3>Create Podinfo-Flux Application Package with Zarf</h3><p>The overall purpose of this tutorial is to demonstrate how to use Zarf to deploy a package on a Kubernetes cluster. The package can be anything, but the Zarf documentation has an entire folder of <a href="https://github.com/defenseunicorns/zarf/tree/main/examples"><strong>example</strong></a> applications to work with. One of those examples is the <a href="https://github.com/defenseunicorns/zarf/tree/main/examples/podinfo-flux"><strong>podinfo-flux</strong></a> application, and this is the one we will be deploying.</p><p><a href="https://artifacthub.io/packages/helm/podinfo/podinfo">Podinfo</a> is a small web application written in Go that is often used for end-to-end testing. <a href="https://fluxcd.io/#:~:text=Flux%20is%20a%20set%20of,is%20a%20CNCF%20Graduated%20project.">Flux</a> is a Cloud Native Computing Foundation (CNCF) graduated project that allows you to incorporate GitOps into your Kubernetes cluster. Basically, Flux reconciles the state of your cluster to the desired state defined in a Git repository. Flux monitors the repository, so when changes are made to the source code in the repo, Flux detects these changes, and reconciles the state of the cluster accordingly. Regardless of what Podinfo and Flux do, the point is we will deploy them with Zarf.</p><p>On your local machine, navigate to:</p><pre>zarf/examples/podinfo-flux</pre><p>Take a look at the files comprising the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*jSn0uBXXPLvukZCJnbjOaw.png" /></figure><p>The <strong>flux-install.yaml</strong> file consists of an installation manifest generated by Flux, and should not be edited.</p><p>The <strong>podinfo-kustomization.yaml </strong>file instructs Flux to apply customizations from the ./kustomize directory of the Podinfo Git repository to the Podinfo namespace every 5 minutes.</p><p>The <strong>podinfo-source.yaml</strong> file defines the source of the Git repo for Flux to watch and reconcile against (https://github.com/stefanprodan/podinfo.git).</p><p>The <strong>zarf.yaml </strong>file tells the cluster to first deploy Flux via the flux-install.yaml manifest, then deploy Podinfo via the podinfo-source.yaml and podinfo-kustomization.yaml files.</p><p>Now that we’ve reviewed the source code for our package, let’s create it. Use the following command:</p><pre>zarf package create .</pre><p>The above command takes all of your source code files in the current working directory (don’t forget the period), and bundles them into a single tarball file, known as the Zarf package.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K3ZBBF15z6noU4R5kO4qAA.png" /></figure><p>You will be prompted “Create this Zarf package?” Answer “Yes”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*TM-LMDeDacquzry0Ty0zXQ.png" /></figure><p>You will then be asked “Please provide a value for Maximum Package Size”. This prompt is useful in a situation where your tarball package size matters (ex. you have limited disk space). For this tutorial, our package will be small so you can simply press <strong>Enter</strong> and it will bypass this question. You should see the Flux and Podinfo components loading.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hbxXSm14PlpexAhcOsX83Q.png" /></figure><p>Now if you look at your files in your current directory again, you will see a new tarball file has been created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gMMAZS-nb7MebZRkeU6AEA.png" /></figure><h3>Deploy Podinfo-Flux Application Package with Zarf</h3><p>We’re in the home stretch guys, but first let’s do a quick recap. We cloned the Zarf repo to your local directory. We installed the Zarf ‘init’ package. We then created a Podinfo-Flux application package with Zarf, and we are now ready to deploy our application to our Kubernetes cluster. Use the following command to deploy:</p><pre>zarf package deploy</pre><p>Upon running this command, you will be prompted to “Choose or type the package file”. If you only have a single tarball file in the current directory, you can simply hit the <strong>Tab</strong> button and the file’s name will pop in. Then you can press <strong>Enter</strong>. Alternatively, you can manually type the name of your Zarf package tarball file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SS5XC6ZWHObu8zAbmr6Pzw.png" /></figure><p>You will then be prompted “Deploy this Zarf package?” Answer “Yes”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vLk4vbiT7X_6kJinm5jDkg.png" /></figure><p>You know your package is deployed when you see the green “Zarf deployment complete”. If you have <a href="https://k9scli.io/topics/install/">K9s</a> installed, you could also open a new terminal window and see your Flux and Podinfo components there. If you don’t have K9s installed, open a new terminal window and run the following command for Zarf to open K9s for you:</p><pre>zarf tools monitor</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QR9Zp4FqG30gv_RT8DC3dw.png" /></figure><p>In the K9s window, you can see Flux and Podinfo pods running in your cluster. Back in your original terminal, use the following command to see all Zarf packages currently installed on your Kubernetes cluster:</p><pre>zarf package list</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0BqopivVeEgZeY9kp_l7Hw.png" /></figure><h3>Interact with Podinfo</h3><p>At this point, we appear to have used Zarf to successfully deploy Podinfo and Flux on our Kubernetes cluster. If you looked at the source code before deployment, you may have noticed that a few namespaces were created when our application was deployed. You are likely in the <strong>default </strong>namespace currently, so if you were to run kubectl get pods you would not see any pods running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*Z_t9c3Pn4vKsFcxXStiyjQ.png" /></figure><p>To see all of the pods in all of the namespaces, use:</p><pre>kubectl get pods -A</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1CP9HFiHOE71-vqXId38JA.png" /></figure><p>To interact with our Podinfo pods, let’s change to the podinfo namespace:</p><pre>kubectl config set-context --current --namespace=podinfo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vYtQrDhrtIQTjsQhk4u2aw.png" /></figure><p>Then run kubectl get pods again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*JUUM3W8CpVw2wwy2LSosuA.png" /></figure><p>Now we can see our two Podinfo pods up and running. Let’s see if we can access one of the pods from our local machine using a port-forward. If you’re not familiar with kubectl port-forward , check out <a href="https://www.middlewareinventory.com/blog/kubectl-port-forward/">this</a> article.</p><pre>kubectl port-forward pod/&lt;pod_name&gt; &lt;local_port&gt;:9898</pre><p>Replace &lt;pod_name&gt; with the name of one of your Podinfo pods, and &lt;local_port&gt; with a port of your choosing. I chose 3005.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MQJhBvBgPuAbrMmP43oyPQ.png" /></figure><p>With the port-forward command, we are mapping our local port 3005 to the default Podinfo port of 9898. We can find the default Podinfo port in the Podinfo source code <a href="https://github.com/stefanprodan/podinfo/blob/master/kustomize/service.yaml">here</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ZdCircWomUVhfkfcOXlTg.png" /></figure><p>Now that our port-forward tunnel is running, open a browser of your choosing and go to <strong>http://localhost:3005 </strong>where you should be met with the Podinfo landing page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J1h0q4nz9C3vzTpKf2Su2g.png" /></figure><p>If you are met with an adorable clapping purple sea creature, congratulations! If you’re met with an error, make sure your web browser didn’t automatically reroute you to HTTPS instead of HTTP.</p><p>After you’ve had your full share of clapping from the purple guy, go back to your terminal and stop the port-forward with ctrl + c .</p><h3>Zarf Connect to Gitea</h3><p>At the beginning of the tutorial, in the “Install the Zarf Init Package” section, we answered “Yes” to the “Deploy the git-server component?” prompt. Zarf uses the Gitea git-server component as a GitOps-compatible source control service. To log into the Gitea server for the Zarf package you created, run the following command:</p><pre>zarf connect git</pre><p>This will open a port-forward on your local machine:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xQt6UDkTFPH3m9Lu9Vx59Q.png" /></figure><p>As well as a new browser window to the Gitea login screen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a9ioKhOT5eq2kAgfzp1ZQw.png" /></figure><p>Go find the Usernames and Passwords we saved earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iW__f3VrbkjZqw0xu1XIDA.png" /></figure><p>Back on the Gitea website, click the <strong>Sign In</strong> button in the top right, and use the Username and Password for the “Git” application to log in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zOIUT_LSph6CgVXaGAMUPA.png" /></figure><p>After logging in, you should see a version-control interface, similar to GitHub. We won’t dive further into this, but hopefully you can see how Zarf has some optional built-in tools that make CI/CD easier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3LFt2HPI04JxLPsx3XtHYw.png" /></figure><h3>Conclusion and CleanUp</h3><p>In this tutorial, we used the open-source tool Zarf to deploy a Podinfo-Flux application to a Kubernetes cluster. Although this was done in an internet-connected environment, the beauty of Zarf is that with just a few additional steps, you can use it to deploy application packages in secure air-gap or disconnected environments.</p><p>Zarf makes it easy to destroy everything you built with a single command. If you want to watch resources being destroyed, either pull up your K9s terminal or open a new terminal and run zarf tools monitor again to use the K9s component built into Zarf. Then run:</p><pre>zarf destroy --confirm --remove-components</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dOBXfa1n88p7UutXNtHsHw.png" /></figure><p>Now just sit back and watch Zarf tear everything down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FID_U-bkKvHKpEDLLADNdA.png" /></figure><p>Note that my podinfo and flux-system namespaces have disappeared. If you run kubectl get pods -A you will find that your Podinfo, Flux and Zarf pods have all been destroyed and are no longer running. If you run zarf package list you will see that you no longer have any Zarf packages deployed on your cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GvPmp4uisa1d0vvTpOdEGA.png" /></figure><p>Use the following command to delete the k3d cluster you created:</p><pre>k3d cluster delete &lt;cluster_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*b8nSv1jvQwjlQa_Z39i4hg.png" /></figure><p>The last thing to clean up is the tarball file from your local directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*85YfphRoAk5Y-2rkbyp2hQ.png" /></figure><p>Run the following command:</p><pre>rm &lt;tarball_file_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Tc74BzqnpURl0LMYgI6gHg.png" /></figure><p>That’s all the clean up you should have to do. I hope you found this tutorial useful and easy to follow. Thank you so much for reading and keep watching for more DevOps tutorials!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22dc6c02510c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/defense-unicorns/using-zarf-to-deploy-a-podinfo-flux-application-in-a-kubernetes-cluster-22dc6c02510c">Using Zarf to Deploy a Podinfo-Flux Application in a Kubernetes Cluster</a> was originally published in <a href="https://medium.com/defense-unicorns">Defense Unicorns</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Secrets]]></title>
            <link>https://medium.com/@bm54cloud/kubernetes-secrets-77a798f412aa?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/77a798f412aa</guid>
            <category><![CDATA[secrets]]></category>
            <category><![CDATA[k3d]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Mon, 05 Jun 2023 22:43:33 GMT</pubDate>
            <atom:updated>2023-06-05T22:43:33.843Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to Consume Environment Variable Secrets within a Pod</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d1z8ru8RHWQvcy9tYeBC2w.png" /></figure><h3>Prerequisites:</h3><ul><li>Basic Kubernetes knowledge</li><li>kubectl installed</li><li>A running k3d cluster with at least 2 worker nodes</li></ul><h3>Objectives:</h3><ul><li>Discuss Kubernetes Secrets</li><li>Create literal secrets in source files</li><li>Create a Secret object using the kubectl utility</li><li>Observe how Kubernetes encodes Secrets</li><li>Deploy a PersistentVolume and PersistentVolumeClaim to the cluster</li><li>Create a mySQL Deployment to consume the Secrets</li><li>Confirm Secrets were consumed by the Pod</li></ul><h3>Kubernetes Secrets</h3><p>Kubernetes allows you to incorporate secrets like passwords, tokens, and keys into clusters via a <a href="https://kubernetes.io/docs/concepts/configuration/secret/">Secret</a> object. It’s important to note that Secrets are <strong>not</strong> encrypted by default but they are Base64 encoded. To make Secrets more secure, they should be encrypted before being applied to a Kubernetes cluster. There is much more to securing Secrets than what can be discussed here, but the purpose of this tutorial is to show how a Secret can by consumed by a Pod.</p><p>Secrets can be created and stored in a manifest separate from the Pods that use them. This allows you to keep your confidential data and your application code separate. Secret objects are declared and consumed similarly to <a href="https://kubernetes.io/docs/concepts/configuration/configmap/">ConfigMaps</a>, but the latter should never be used for confidential data.</p><p>Pods can consume Secrets via the following methods:</p><ul><li>As a file or files within in a volume mounted to a container</li><li>As environment variables within the container</li><li>By the kubelet when it pulls images required for the Pod</li></ul><p>In this tutorial, we will create a Secret using the kubectl utility, then consume that Secret by a Pod. Confirm you have a k3d Kubernetes cluster with at least 2 worker nodes running, then move on to the next section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*Qdt-Uk9soukfC3EklQa9iQ.png" /><figcaption>3-node k3d cluster</figcaption></figure><h3>Store Secrets in Files</h3><p>When using kubectl to manage your Secrets, you can either pass raw data into a command, or store the secrets in files, and pass those files in the command. Placing your Secrets in plain text files is <strong>not</strong> the most secure format. We will create our Secrets in source files, then pass those file paths in the kubectl command. The Secrets we create will consist of a sample username and password.</p><p>We will now create a username Secret and password Secret. The username we want to store is “<strong>admin</strong>” and the password is “<strong>1f2d1e2e67df</strong>”. Use the following commands to write the secrets into new txt files:</p><pre>echo -n &#39;admin&#39; &gt; ./username.txt<br>echo -n &#39;1f2d1e2e67df&#39; &gt; ./password.txt</pre><p>The above commands take our “admin” username and “1f2d1e2e67df” password, and sends the encoded strings to new files within the current directory. You can view the new files with the ls command (disregard my other files).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*qyapqWBQPCpG3vOGfm1PZQ.png" /></figure><p>If we look at the contents of those files, we will see this for the username.txt file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/102/1*JWxKWuEmQ7rezdPJduCt_A.png" /></figure><p>And this for the password.txt file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/184/1*vHsU5ASSdze0nnkHy8IwOA.png" /></figure><h3>Create the Secrets with kubectl</h3><p>We can now create our Secret objects using the following command:</p><pre>kubectl create secret generic db-user \<br>    --from-file=username=./username.txt \<br>    --from-file=password=./password.txt</pre><p>The above command uses kubectl create secret to create a generic Secret with the name “db-user”. It creates the Secret by reading the contents of the files “username.txt” and “password.txt” from the local directory and storing them as key-value pairs within the Secret. The key for the “username.txt” file is “username” and the key for the “password.txt” file is “password”. The value for each key is the contents of the files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*nYgJcDnqdc70Kj_RfFQm1g.png" /></figure><p>It looks like the Secret was created, but let’s verify that with this command:</p><pre>kubectl get secrets</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*kLoJqlUu8L2g0XFAEJvaYA.png" /></figure><p>Perfect, the Secret was in fact created. Note the “2” under the “Data” column. This 2 denotes that there are 2 secrets (a username and password). Typically when you create a Kubernetes object, you can get some information using the kubectl get all command. Secrets, however, are intentionally not included in this output. To view more details of the Secret, use the following command:</p><pre>kubectl describe secret db-user</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*bkwoZOJ21SjVAFpryDnQVQ.png" /></figure><p>When creating a Secret in Kubernetes, it gets Base64 encoded in the background. To confirm the Secrets have been Base64 encoded, use the following command:</p><pre>kubectl get secret -o yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*4kN65C196EfuU-aXr9jaVw.png" /></figure><p>You can see the password value is MWYyZDFlMmU2N2RmCg==, which is 1f2d1e2e67df (our literal password) Base64 encoded. If you weren’t sure, you could paste the password into a web browser decoder like <a href="https://www.base64decode.org/">base64decode.org</a> and click the “decode” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7yJm9gF63zhcOTE0L1ml6g.png" /></figure><p>Likewise, YWRtaW4K is our literal username of admin Base64 encoded.</p><h3>Deploy a PersistentVolume and PersistentVolumeClaim to the Cluster</h3><p>MySQL databases typically use persistent volumes to store data, but that storage must be requested via a persistent volume claim. Persistent volumes and their claims must be created before a workload (in this case mySQL) attempts to consume it. Create a new file called storage.yaml and paste the following code:</p><pre>apiVersion: v1<br>kind: PersistentVolume<br>metadata:<br>  name: mysql-pv-volume<br>  labels:<br>    type: local<br>spec:<br>  storageClassName: manual<br>  capacity:<br>    storage: 20Gi<br>  accessModes:<br>    - ReadWriteOnce<br>  hostPath:<br>    path: &quot;/mnt/data&quot;<br>---<br>apiVersion: v1<br>kind: PersistentVolumeClaim<br>metadata:<br>  name: mysql-pv-claim<br>spec:<br>  storageClassName: manual<br>  accessModes:<br>    - ReadWriteOnce<br>  resources:<br>    requests:<br>      storage: 20Gi</pre><p>Apply the PersistentVolume and PersistentVolumeClaim to the cluster with the following command:</p><pre>kubectl apply -f storage.yaml</pre><p>We can check that these were deployed with kubectl get pv and kubectl get pvc . Note that the persistent volume has a status of “BOUND” meaning the PVC found a volume that met its requirements, and the PV and PVC are now bound. Now that the PV is bound, it is available for use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nVGw3ZAqSIUK83TP6tzzlA.png" /></figure><h3>Create a mySQL Deployment to Consume Secrets</h3><p>We can now create a Pod on the cluster to consume the Secret. Create a new file called deployment.yaml and copy the following code into it:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: mysql<br>spec:<br>  selector:<br>    matchLabels:<br>      app: mysql<br>  template:<br>    metadata:<br>      labels:<br>        app: mysql<br>    spec:<br>      containers:<br>      - name: mysql<br>        image: mysql:latest<br>        env:<br>          - name: MYSQL_USER<br>            valueFrom:<br>              secretKeyRef:<br>                name: db-user<br>                key: username<br>          - name: MYSQL_ROOT_PASSWORD<br>            valueFrom:<br>              secretKeyRef:<br>                name: db-user<br>                key: password<br>        ports:<br>        - containerPort: 3306<br>        volumeMounts:<br>          - name: mysql-persistent-storage<br>            mountPath: /var/lib/mysql<br>      volumes:<br>      - name: mysql-persistent-storage<br>        emptyDir: {}</pre><p>Here’s an explanation of what this code is doing:</p><ul><li>apiVersion: apps/v1 specifies the API version for the Deployment.</li><li>kind: Deployment indicates a Deployment resource.</li><li>metadata contains information about the Deployment (ex. name).</li><li>spec describes the desired state of the Deployment.</li><li>selector specifies the label selector used to identify the Pods managed by this Deployment. The Pods should have the label app: mysql.</li><li>template defines the Pod template for creating new Pods.</li><li>metadata within template specifies metadata for the Pods.</li><li>labels within metadata sets the labels for the Pods. The label app: mysql is used to match the selector defined in the Deployment.</li><li>spec within template defines the specification for the Pods.</li><li>containers lists the containers running in the Pod. In this case, there is one container named mysql with the mysql:latest image.</li><li>env specifies the environment variables for the container.</li><li>MYSQL_USER environment variable is set with the value from a Secret named db-user using secretKeyRef. The value comes from the key username in the Secret.</li><li>MYSQL_ROOT_PASSWORD environment variable is set with the value from the same Secret (db-user) using secretKeyRef. The value comes from the key password in the Secret.</li><li>ports defines the container ports that should be exposed. In this case, port 3306 is exposed, which is the default for mySQL.</li><li>volumeMounts specifies the volumes to be mounted inside the container. In this case, a volume named mysql-persistent-storage is mounted at the path /var/lib/mysql.</li><li>volumes defines the volumes available for the Pod. In this case, there is one volume named mysql-persistent-storage of type emptyDir, which is an empty volume that is tied to the Pod&#39;s lifetime.</li></ul><p>Overall, this manifest deploys a MySQL database as a Deployment with a single container. The container has environment variables populated from a Secret, exposes port 3306, and mounts an emptyDir volume for persistent storage. Save the deployment.yaml file and apply it to the cluster using:</p><pre>kubectl apply -f deployment.yaml</pre><p>Give the container a few minutes to create and then check the Pods:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*SENeaZE1oDwEogMLaxzyJw.png" /></figure><h3>Confirm Secrets Were Consumed by the Pod</h3><p>Remember, this is a tutorial on Kubernetes Secrets and how they get consumed by a Pod. It is not a tutorial on how to use mySQL, because I simply have not used it much. Nonetheless, we can confirm that our username and password Secrets were in-fact consumed by the mySQL Pod. In our deployment.yaml file, we used our Secrets to define the MYSQL_USER and MYSQL_ROOT_PASSWORD environment variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4sCXazOSjsaRsM3V9W3cdQ.png" /></figure><p>To see if the secrets got passed to the Pod, we can simply look at the environment variables within the Pod. Find the Pod name with:</p><pre>kubectl get pods</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/1*EMSpFmMwiaqK2xigeM3ZgQ.png" /></figure><p>Now let’s enter the container and look for its environment variables:</p><pre>kubectl exec -it mysql-7b7989ffb-q84l7 -- env</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0uCk5fHTJ1mad9DxBL5DEA.png" /></figure><p>We can see that the Secrets did in-fact get consumed by the Pod!</p><h3>Conclusion and Clean Up</h3><p>In this tutorial we discussed Kubernetes Secrets and how to create them from files. We referenced the Secrets in our deployment.yaml file, thereby allowing our mySQL Pod to consume them. We also reviewed steps on how to confirm that our Secrets were consumed.</p><p>To clean up what we just created, use the following command to delete the cluster and everything in it:</p><pre>k3d cluster delete &lt;cluster_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MqE34g1exzwj4bzyP8Q9qw.png" /></figure><p>Thank you for reading and keep watching for more DevOps tutorials!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /><figcaption>‘</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77a798f412aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy an HAProxy Ingress Controller on a k3d Kubernetes Cluster]]></title>
            <link>https://medium.com/@bm54cloud/deploy-an-haproxy-ingress-controller-on-a-k3d-kubernetes-cluster-3c88007eea36?source=rss-2e9072e523ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c88007eea36</guid>
            <category><![CDATA[ingresscontroller]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[haproxy]]></category>
            <category><![CDATA[ingress]]></category>
            <category><![CDATA[k3d]]></category>
            <dc:creator><![CDATA[Brandi McCall]]></dc:creator>
            <pubDate>Sun, 04 Jun 2023 17:49:07 GMT</pubDate>
            <atom:updated>2023-06-06T16:47:36.358Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This tutorial will discuss the Kubernetes object Ingress and how to deploy an HAProxy Ingress controller on a k3d cluster. We will deploy a simple web application to test Ingress traffic routing, as defined in our Ingress rules, then test reachability using kubectl port-forward.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*91dpwIWqRvK88bHZFzfH9Q.png" /></figure><h3>Prerequisites:</h3><ul><li>Basic Kubernetes knowledge</li><li>Basic Helm chart knowledge</li><li><a href="https://helm.sh/docs/intro/install/">Helm</a> installed</li><li><a href="https://k3d.io/v5.5.1/">k3d</a> installed</li><li><a href="https://docs.docker.com/engine/install/">Docker</a> installed</li><li><a href="https://kubernetes.io/docs/tasks/tools/">kubectl</a> installed</li><li>GitHub account and basic Git knowledge</li></ul><h3>Objectives:</h3><ul><li>Discuss the concepts of Ingress and Ingress controllers</li><li>Discuss k3d clusters and their default Traefik controller</li><li>Deploy a k3d cluster without an Ingress controller</li><li>Deploy an HAProxy Ingress controller on the cluster via a Helm chart</li><li>Deploy a simple coffee and tea web application</li><li>Route application traffic through the HAProxy Ingress controller</li><li>Test reachability of the application and confirm appropriate routing</li></ul><h3>Getting Started</h3><p>The files for this project can be found in a GitHub repository located <a href="https://github.com/defenseunicorns/sparkle-academy/tree/main/zarf-101/Kubernetes_Objects/Ingress/miniproject_solution">here</a>. Clone the repo to a new directory on your local computer, navigate to sparkle-academy/zarf-101/Kubernetes_Objects/Ingress/miniproject<br>_solution, then use the tree command to confirm you have the files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*fxnAxe-_fvYMF7wC1AxPXw.png" /></figure><h3>What is Ingress?</h3><p>In Kubernetes, Ingress refers to an API object that provides a way to manage external access to Services running within a Kubernetes cluster. Ingress works as an <a href="https://en.wikipedia.org/wiki/OSI_model">OSI Layer 7</a> load balancer to route incoming HTTP and HTTPS requests based on hostnames, paths, or other criteria, then directs that traffic according to defined rules. This figure from the official Kubernetes <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Ingress</a> documentation shows how Ingress can route traffic to different Services, depending on the path being requested:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p46LLJD-brBlkWdor_PZww.png" /></figure><p>Ingress acts as a gateway or entry point for incoming traffic, allowing external clients to reach the services deployed in the cluster. It allows for the integration of load balancing, SSL/TLS termination, and URL rewriting. By using Ingress, multiple services can be exposed through a single external IP address, simplifying the management and routing of network traffic to different applications within the cluster.</p><h3>What is an Ingress Controller?</h3><p>For the Ingress object to work, the cluster running it must have an <a href="https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/">Ingress controller</a>. The Ingress controller runs on the control plane of a cluster and the controller is typically not automatically started with the cluster. We will see an exception to this in the next section, when we talk about a k3d cluster. There are many different options of Ingress controllers to choose from and some can be found <a href="https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/">here</a>.</p><p>The Ingress controller is responsible for fulfilling the Ingress resource configuration in a cluster. It interprets and implements the rules defined in the Ingress resource. It acts as a reverse proxy and load balancer, managing the incoming traffic and routing it to the appropriate services based on the rules defined in the Ingress resource. The Ingress Controller runs as a separate Pod or Deployment in the cluster and typically integrates with an external load balancer or implements its own load balancing algorithms. It continuously monitors the Ingress resources and updates the configuration of the underlying load balancer accordingly, ensuring that the incoming traffic is correctly directed to the desired Services.</p><h3>k3d Clusters and the Traefik Controller</h3><p>K3d is a lightweight wrapper that runs k3s, a minimal Kubernetes distribution. K3s utilizes Docker containers, thus both k3d and Docker must be installed on your local machine to start a k3d cluster. K3d deploys <a href="https://doc.traefik.io/traefik/providers/kubernetes-ingress/">Traefik</a> as its default Ingress controller. We will see in the next section how the Traefik controller can be disabled upon k3d cluster creation.</p><h3>Create a k3d Cluster Without Traefik</h3><p>To create a k3d cluster without the default Traefik Ingress controller, we can use an argument in our YAML manifest to disable it. Open the k3d_no_traefik_cluster.yaml file from the project repo:</p><pre>apiVersion: k3d.io/v1alpha4<br>kind: Simple<br>metadata:<br>  name: k3d-no-traefik<br>servers: 1<br>agents: 2<br>options:<br>  k3s:<br>    extraArgs:<br>      - arg: --disable=traefik@server:0<br>        nodeFilters:<br>          - server:*</pre><p>Here’s an explanation of the code:</p><ul><li>apiVersion: k3d.io/v1alpha4: Specifies the API version for the k3d cluster configuration.</li><li>kind: Simple: Declares a simple k3d cluster to be created.</li><li>metadata: Contains metadata information about the cluster.</li><li>name: k3d-no-traefik: Specifies the name of the cluster as &quot;k3d-no-traefik&quot;.</li><li>servers: 1: Specifies to create one server node in the cluster.</li><li>agents: 2: Specifies to create two agent nodes in the cluster.</li><li>options: Provides additional options for configuring the cluster.</li><li>k3s: Contains options specific to the k3s Kubernetes distribution.</li><li>extraArgs: Specifies additional arguments to be passed to the k3s server during startup.</li><li>- arg: --disable=traefik@server:0: Disables the Traefik load balancer in the cluster. It adds the --disable=traefik@server:0 argument to the k3s server startup configuration.</li><li>nodeFilters: Specifies node filters for the argument. In this case, it applies the argument only to server nodes (server:*).</li></ul><p>To create the cluster, use the following command:</p><pre>k3d cluster create --config k3d_no_traefik_cluster.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ukMAoUfkx-CZh1LjDVn5NQ.png" /><figcaption>k3d cluster creation</figcaption></figure><p>Once the cluster has finished creating, Kubernetes automatically switches you to that cluster. Using the k3d cluster list command, we see that the k3d-no-traefik cluster was created with one server and two agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nyc33wvIDx2Ba9Yyx-QtQQ.png" /></figure><h3>Deploy an HAProxy Ingress Controller via Helm</h3><p>For Ingress to work, you must have an Ingress controller. Since we disabled k3d’s default controller, we need to deploy a new controller to our cluster. For this tutorial, we will deploy the HAProxy Ingress controller via a Helm chart. Detailed information and installation instructions of the HAProxy Ingress controller can be found <a href="https://haproxy-ingress.github.io/docs/getting-started/">here</a>. The first step to deploying the HAProxy Helm chart is adding the chart to your local Helm repo:</p><pre>helm repo add haproxytech <a href="https://haproxytech.github.io/helm-charts">https://haproxytech.github.io/helm-charts</a></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ubur7j0Fe7PrcaLRdL9d2A.png" /></figure><p>It is good practice to run the following command after adding a chart:</p><pre>helm repo update</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*C6aFLuAP4ZxD2ZuPKrQBAg.png" /></figure><p>Now that the Helm chart has been added to our local Helm repo, we can install the chart on our k3d cluster using the following command:</p><pre>helm upgrade -i haproxy haproxytech/kubernetes-ingress</pre><p>This command upgrades or installs the HAProxy Helm chart from the repo “haproxytech/kubernetes-ingress”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Ozo7HIujXhhD83-rJJEZQ.png" /></figure><p>Note that the output from this command is much longer than the screenshot above. Use the kubectl get all command to see everything that was created when the HAProxy Helm chart was installed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tiFIWWfiQY0zXA9kSs-dQA.png" /></figure><p>You should have two pods running, an HAProxy Ingress service, a Deployment, and a ReplicaSet.</p><h3>Deploy a Coffee and Tea Application</h3><p>Remember when we discussed Ingress earlier, we talked about how it routes HTTP and HTTPS traffic depending on hostnames, paths, or other criteria. To see this in action, we need to deploy some sort of application on our cluster that has Ingress rules defined. Our application code can be found in the <strong>objects</strong> directory of the project’s GithHub <a href="https://github.com/defenseunicorns/sparkle-academy/tree/main/zarf-101/Kubernetes_Objects/Ingress/miniproject_solution">repo</a>. Take a minute to look at those seven files and here is a brief description of each:</p><ul><li>ing-coffee-cm.yaml: ConfigMap for nginx-coffee that has custom index.html data to say “Welcome to nginx-coffee!”</li><li>ing-coffee-deployment.yaml: Deployment for nginx-coffee that defines 3 replicas, with Pods based on an NGINX image and port 80 exposed. It uses a volume to mount the nginx-coffee ConfigMap, so users will see the custom nginx-coffee index.html data when connecting to the NGINX server.</li><li>ing-coffee-svc.yaml: Deploys a Service that exposes port 80 and maps it to targetPort 80, thus the Service will forward incoming traffic to the Pods listening on port 80. Note the selector here uses the same labels as the coffee Deployment, so traffic received on this Service will be routed to Pods that have the “app: nginx” and “drink: coffee” labels.</li><li>ing-rules-haproxy.yaml: Deploys an Ingress resource and defines Ingress traffic routing rules. Basically, if the user is looking for a path with “/coffee”, they will be routed to the coffee Pods. If the user is looking for a path with “/tea”, they will be routed to the tea Pods.</li><li>ing-tea-cm.yaml: ConfigMap for nginx-tea that has custom index.html data to say “Welcome to nginx-tea!”</li><li>ing-tea-deployment.yaml: Deployment for nginx-tea that defines 3 replicas, with Pods based on an NGINX image and port 80 exposed. It uses a volume to mount the nginx-tea ConfigMap, so users will see the custom nginx-tea index.html data when connecting to the NGINX server.</li><li>ing-tea-svc.yaml: Deploys a Service that exposes port 80 and maps it to targetPort 80, thus the Service will forward incoming traffic to the Pods listening on port 80. Note the selector here uses the same labels as the tea Deployment, so traffic received on this Service will be routed to Pods that have the “app: nginx” and “drink:tea” labels.</li></ul><p>Let’s talk about ing-rules-haproxy.yaml a little bit more. To use Ingress, you must have an Ingress resource and an Ingress controller (I know…that’s the 3rd time I’ve said that). We already deployed our HAProxy Ingress controller. In the ing-rules-haproxy.yaml manifest, we are deploying our Ingress resource and defining the rules of how it should route traffic. Here’s a simple visual:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LonbfkXOBJ3oQI4iJK-R7A.png" /></figure><p>When users access “website”, they can access either the “coffee” path or the “tea” path. If a user is trying to reach website/coffee, the HAProxy Ingress controller will route that traffic to the coffee Service, which is linked to the coffee Pods. The HAProxy knows to route the traffic this way because we defined it in our Ingress resource rules:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*unqDVVeRrwG_XSq6vSlxcw.png" /><figcaption>Rules defined in ing-rules-haproxy.yaml</figcaption></figure><p>Once reaching the coffee Pods, the user will be greeted with the custom index-html data “Welcome to nginx-coffee!” that was defined in the coffee ConfigMap. If the user is trying to reach the website/tea path, the traffic will be sent to the tea Service, then routed to the tea Pods.</p><p>Now that we know what these configuration files will do, let’s deploy them to our cluster. Make sure you are in the directory that has the <strong>objects</strong> folder, then use the following command to deploy all of the objects manifest files at once:</p><pre>kubectl apply -f objects/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*YbpdQ8XHphFP0hC305uUVw.png" /></figure><p>Use the kubectl get all command to confirm everything is running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VL8ijaFKGXwWhtDsBgbFFQ.png" /></figure><h3>Route Traffic Through the HAProxy Ingress Controller and Test Reachability</h3><p>At this point, we have set up our Ingress, Ingress controller, and we have three coffee and three tea Pods running. Now we want to route traffic through the HAProxy and test that our Ingress controller is routing correctly. We can test this by setting up a <a href="https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/">port-forward</a>.</p><p>A <strong>port-forward</strong> allows you to access your Kubernetes cluster from your localhost without exposing your cluster to the general public. Port-forward is typically used for testing and debugging purposes, not for production workloads. To test our set-up, we need to direct traffic to the haproxy-kubernetes-ingress Service we created earlier. This Service was created when we installed the HAProxy Helm chart. Use the following command:</p><pre>kubectl port-forward services/haproxy-kubernetes-ingress 8082:80</pre><p>This will forward traffic from port 8082 on our local machine (we randomly chose this port) to port 80, which is the port that we exposed earlier on our Pods and Services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Uo7wz2HJCqIKPqdbgvxpA.png" /></figure><p>We can now try to access the coffee and tea Services in two different ways:</p><h4>Method 1</h4><p>Open a new terminal window and curl http://localhost:8082/tea. You should see the custom index.html data that says “Welcome to nginx-tea!”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*1j2rUT6YaZHkI1Qi2AlK7w.png" /></figure><p>Use curl http://localhost:8082/coffee to test the coffee path:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dO6HRNLNS_8-mBU2BvFNww.png" /></figure><h4>Method 2</h4><p>Open your web browser and paste http://localhost:8082/tea into it. You should be met with the custom tea NGINX landing page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QTrVZnu7RW7AhohQY7rT1Q.png" /></figure><p>Use http://localhost:8082/coffee in your web browser to see the custom coffee NGINX landing page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uRVZX4HUNuuvgXCr5RBiOw.png" /></figure><p>It looks like our Ingress and Ingress controller are routing traffic appropriately!</p><h3>Conclusion and Clean Up</h3><p>Hopefully this tutorial gave you a better understanding of what Ingress is and the components needed (Ingress controller) to make it work. The key here is you need an Ingress resource (object) that defines your Ingress rules, an Ingress controller that interprets and enforces those rules, and some sort of application that the rules and traffic are applied to.</p><p>To clean up the resources and cluster we just created, we can simply delete the k3d cluster using the following command:</p><pre>k3d cluster delete k3d-no-traefik</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3d6iniFJNCSf0b8lEwJKeA.png" /></figure><p>By deleting the cluster this will also stop the port-forward connection. If you want to stop your port-forward connection without deleting the cluster, use ctrl + c.</p><p>Thank you so much for following along with my learning journey, and keep watching for more DevOps tutorials!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKS7crh9SK4db60TlMcUSw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c88007eea36" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>