<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mr. Burton on Medium]]></title>
        <description><![CDATA[Stories by Mr. Burton on Medium]]></description>
        <link>https://medium.com/@mrburton?source=rss-c8a15df53e11------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*fbU0UAypgofJuwIokxMT0A.jpeg</url>
            <title>Stories by Mr. Burton on Medium</title>
            <link>https://medium.com/@mrburton?source=rss-c8a15df53e11------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:34:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mrburton/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[TerraStack Part 5: Github Actions and the GitOps Workflow That Almost Didn’t Happen]]></title>
            <link>https://mrburton.medium.com/terrastack-part-5-github-actions-and-the-gitops-workflow-that-almost-didnt-happen-3b4457e806f3?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b4457e806f3</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Thu, 25 Dec 2025 14:44:06 GMT</pubDate>
            <atom:updated>2025-12-25T14:44:06.449Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S1JpRBEG65LS_ZUdPBlMEg.png" /></figure><p>Typing terraform apply manually isn’t how real teams work. Here’s how I built a complete GitOps workflow that can destroy and rebuild my entire AWS infrastructure in 17 minutes.</p><p>I had the infrastructure built. VPC, EKS, RDS. All working.</p><p>But every change required me to:</p><pre>cd environments/dev<br><br>terraform plan<br># [review the output]<br><br>terraform apply<br># [type yes]</pre><p>This has problems:</p><p>1. No review: I’m the only person checking my work<br>2. No history: I don’t know what changed when<br>3. No rollback: If I break something, I have to figure out what I did<br>4. No collaboration: If someone else wanted to contribute, they’d have to run terraform manually<br>5. Not professional: Companies don’t work this way</p><p>What companies do: GitOps.</p><p>Every infrastructure change goes through Git:</p><p>1. Create branch<br>2. Make changes<br>3. Open Pull Request<br>4. Automated plan runs ← This is the magic<br>5. Team reviews plan<br>6. Merge PR<br>7. Automated apply runs ← More magic</p><p>Benefits:</p><ul><li>Every change is reviewed</li><li>Every change is audited (Git history)</li><li>Easy rollback (git revert)</li><li>Collaboration-friendly</li><li>Professional workflow</li></ul><p>I decided to implement this.</p><p>Understanding GitHub Actions</p><p>GitHub Actions is CI/CD built into GitHub. You write YAML files that describe:</p><ul><li>When to run (on PR, on merge, on schedule, etc.)</li><li>What to run (commands, scripts, etc.)</li><li>Where to run (Ubuntu, Windows, Mac)</li></ul><p>I needed three workflows:</p><p><strong>Workflow 1</strong>: terraform-plan.yml</p><ul><li>Trigger: Pull Request created/updated</li><li>Purpose: Show what will change before merging</li><li>Output: Plan posted as PR comment</li></ul><p><strong>Workflow 2</strong>: terraform-apply.yml</p><ul><li>Trigger: Push to main branch (after merge)</li><li>Purpose: Actually create the infrastructure</li><li>Output: Summary of what was created</li></ul><p><strong>Workflow 3</strong>: terraform-destroy.yml</p><ul><li>Trigger: Manual button click (workflow_dispatch)</li><li>Purpose: Tear down infrastructure when done</li><li>Safety: Requires typing “destroy” to confirm</li></ul><p><strong>Problem #8</strong>: Where Are My AWS Credentials?</p><p>To run Terraform in GitHub Actions, I need AWS credentials.</p><p>I went to AWS Console → IAM → My Security Credentials.</p><p>But wait… I already created credentials earlier (for local development). Let me check if I can reuse those.</p><p>Answer: Yes, but it’s better to create separate credentials for different purposes.</p><p><strong>Best practice</strong>:</p><ul><li>Local development: One set of credentials</li><li>GitHub Actions: Different set of credentials (easier to rotate if compromised)</li></ul><p><strong>So I created new credentials</strong>:</p><p>1. AWS Console → IAM → Users → My User<br>2. Security Credentials tab<br>3. Create Access Key<br>4. Use case: “Application running outside AWS”<br>5. Click “Create Access Key”</p><p><strong>AWS showed</strong>:</p><pre>Access Key ID: AKIAIOSFODNN7EXAMPLE<br>Secret Access Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY<br>⚠️ This is the only time you can view this secret access key.</pre><p>I didn’t save to a text file immediately.</p><p>Then my browser refreshed.</p><p>Wait, did I save them? Let me check my text file.</p><p>Text file was empty.</p><p>I had clicked “Copy” but didn’t paste before the refresh.</p><p>DAMMIT.</p><p><strong>The fix</strong>: Created new credentials (again). This time:</p><p>1. Clicked “Create Access Key”<br>2. IMMEDIATELY pasted both values into text file<br>3. Saved the file<br>4. Then clicked “Done” on AWS Console</p><p>Lesson learned (again): When AWS says “only time you can view this,” they mean it. Copy FIRST, click SECOND.</p><p><strong>Adding Secrets to GitHub</strong></p><p>Now I needed to add these credentials to GitHub.</p><p>1. Went to my GitHub repo: github.com/BenjaminBurton/terrastack<br>2. Settings → Secrets and variables → Actions<br>3. Click “New repository secret”</p><p>Secret 1:</p><ul><li>Name: AWS_ACCESS_KEY_ID</li><li>Value: [pasted access key]</li><li>Click “Add secret”</li></ul><p>Secret 2:</p><ul><li>Name: AWS_SECRET_ACCESS_KEY</li><li>Value: [pasted secret key]</li><li>Click “Add secret”</li></ul><p>What this does: GitHub encrypts these and makes them available to workflows via ${{ secrets.AWS_ACCESS_KEY_ID }}.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lgKgGNgaCzq8NRVlQB4Wxw.png" /></figure><p>They never appear in logs. They’re never visible in the UI after creation. Perfect for credentials.</p><p>Workflow 1: Terraform Plan on Pull Requests</p><p>I created .github/workflows/terraform-plan.yml:</p><pre>name: Terraform Plan<br>on:<br> pull_request:<br> branches: [main]<br> paths:<br> - &#39;modules/**&#39;<br> - &#39;environments/**&#39;<br> - &#39;.github/workflows/terraform-*.yml&#39;<br>env:<br> AWS_REGION: us-east-1<br> TF_VERSION: 1.9.8<br>jobs:<br> plan:<br> name: Plan Infrastructure Changes<br> runs-on: ubuntu-latest<br> <br> permissions:<br> contents: read<br> pull-requests: write<br> id-token: write<br> <br> strategy:<br> matrix:<br> environment: [dev]<br> <br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br> <br> - name: Setup Terraform<br> uses: hashicorp/setup-terraform@v3<br> with:<br> terraform_version: ${{ env.TF_VERSION }}<br> <br> - name: Configure AWS Credentials<br> uses: aws-actions/configure-aws-credentials@v4<br> with:<br> aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br> aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br> aws-region: ${{ env.AWS_REGION }}<br> <br> - name: Terraform Init<br> working-directory: ./environments/${{ matrix.environment }}<br> run: terraform init<br> <br> - name: Terraform Plan<br> working-directory: ./environments/${{ matrix.environment }}<br> run: |<br> terraform plan -no-color -out=tfplan<br> terraform show -no-color tfplan &gt; plan_output.txt<br> <br> - name: Comment PR with Plan<br> uses: actions/github-script@v7<br> if: github.event_name == &#39;pull_request&#39;<br> with:<br> script: |<br> const fs = require(&#39;fs&#39;);<br> const plan = fs.readFileSync(&#39;./environments/${{ matrix.environment }}/plan_output.txt&#39;, &#39;utf8&#39;);<br> <br> const output = `#### Terraform Plan 📖<br> <br> &lt;details&gt;&lt;summary&gt;Show Plan&lt;/summary&gt;<br> <br> \`\`\`terraform<br> ${plan}<br> \`\`\`<br> <br> &lt;/details&gt;<br> <br> *Environment: \`${{ matrix.environment }}\`*<br> *Triggered by: @${{ github.actor }}*`;<br> <br> github.rest.issues.createComment({<br> issue_number: context.issue.number,<br> owner: context.repo.owner,<br> repo: context.repo.repo,<br> body: output<br> });</pre><p>What this does:</p><p>Trigger: Runs when a PR is opened/updated, but only if files in modules/, environments/, or workflows changed.</p><p>Steps:</p><p>1. Checkout the code<br>2. Install Terraform 1.9.8<br>3. Configure AWS credentials (from GitHub secrets)<br>4. Run terraform init<br>5. Run terraform plan and save output to file<br>6. Post the plan as a PR comment</p><p>The cool part: The plan appears automatically in the PR. No need to run anything manually.</p><p>Workflow 2: Terraform Apply on Merge</p><p>I created .github/workflows/terraform-apply.yml:</p><pre>name: Terraform Apply<br>on:<br> push:<br> branches: [main]<br> paths:<br> - &#39;modules/**&#39;<br> - &#39;environments/**&#39;<br>env:<br> AWS_REGION: us-east-1<br> TF_VERSION: 1.9.8<br>jobs:<br> apply:<br> name: Apply Infrastructure Changes<br> runs-on: ubuntu-latest<br> <br> permissions:<br> contents: read<br> id-token: write<br> <br> strategy:<br> matrix:<br> environment: [dev]<br> max-parallel: 1 # IMPORTANT: One at a time!<br> <br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br> <br> - name: Setup Terraform<br> uses: hashicorp/setup-terraform@v3<br> with:<br> terraform_version: ${{ env.TF_VERSION }}<br> <br> - name: Configure AWS Credentials<br> uses: aws-actions/configure-aws-credentials@v4<br> with:<br> aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br> aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br> aws-region: ${{ env.AWS_REGION }}<br> <br> - name: Terraform Init<br> working-directory: ./environments/${{ matrix.environment }}<br> run: terraform init<br> <br> - name: Terraform Apply<br> working-directory: ./environments/${{ matrix.environment }}<br> run: terraform apply -auto-approve<br> <br> - name: Verify EKS Cluster<br> run: |<br> aws eks update-kubeconfig \<br> - region ${{ env.AWS_REGION }} \<br> - name terrastack-cluster<br> kubectl get nodes<br> kubectl get pods -A</pre><p>Key differences from plan workflow:</p><ul><li>Trigger: Runs on push to main (not on PR)</li><li>auto-approve: No manual confirmation needed (because the plan was already reviewed in the PR)</li><li>max-parallel: 1: If multiple environments exist (dev, staging, prod), deploy them one at a time. This prevents state locking conflicts.</li><li>Verification: After applying, check that the cluster is healthy with kubectl.</li></ul><p>Workflow 3: Terraform Destroy (Manual Only)</p><p>I created .github/workflows/terraform-destroy.yml:</p><pre>name: Terraform Destroy<br>on:<br> workflow_dispatch:<br> inputs:<br> environment:<br> description: &#39;Environment to destroy&#39;<br> required: true<br> type: choice<br> options:<br> - dev<br> confirm:<br> description: &#39;Type &quot;destroy&quot; to confirm&#39;<br> required: true<br>env:<br> AWS_REGION: us-east-1<br> TF_VERSION: 1.9.8<br>jobs:<br> destroy:<br> name: Destroy Infrastructure<br> runs-on: ubuntu-latest<br> if: github.event.inputs.confirm == &#39;destroy&#39;<br> <br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br> <br> - name: Setup Terraform<br> uses: hashicorp/setup-terraform@v3<br> with:<br> terraform_version: ${{ env.TF_VERSION }}<br> <br> - name: Configure AWS Credentials<br> uses: aws-actions/configure-aws-credentials@v4<br> with:<br> aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br> aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br> aws-region: ${{ env.AWS_REGION }}<br> <br> - name: Terraform Init<br> working-directory: ./environments/${{ github.event.inputs.environment }}<br> run: terraform init<br> <br> - name: Terraform Destroy<br> working-directory: ./environments/${{ github.event.inputs.environment }}<br> run: terraform destroy -auto-approve</pre><p>Key features:</p><ul><li>workflow_dispatch: Only runs when manually triggered (button in GitHub UI)</li><li>Confirmation required: Must type “destroy” exactly (prevents accidents)</li><li>Safety: The if: github.event.inputs.confirm == ‘destroy’ check means if you type “destory” it won’t run.</li></ul><p>Testing the Workflows</p><p>I committed the workflow files:</p><pre>git add .github/workflows/<br>git commit -m &quot;feat(ci): add GitHub Actions workflows for GitOps&quot;<br>git push origin main</pre><p>Now to test them, I needed to create a Pull Request.</p><p>Creating a Test PR</p><p>First, I needed infrastructure to exist. I ran:</p><pre>cd environments/dev<br>terraform apply</pre><p>While that was building (15–20 minutes), I prepared the test branch:</p><pre>git checkout -b test/gitops-demo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FSVFUvIg_X3kfhdlQzMxgw.png" /></figure><p>I made a small change (added a tag to the VPC):</p><pre>resource &quot;aws_vpc&quot; &quot;main&quot; {<br> cidr_block = var.vpc_cidr<br> <br> tags = merge(<br> var.tags,<br> {<br> Name = &quot;${var.project_name}-vpc&quot;<br> BlogDemo = &quot;github-actions-gitops&quot; # New tag<br>     }<br>   )<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g2_dermyOLwZ-9vJweJqGQ.png" /></figure><p>Once the infrastructure finished building, I committed and pushed:</p><pre>git add modules/vpc/main.tf<br>git commit -m &quot;test: add blog demo tag for GitHub Actions workflow&quot;<br>git push origin test/gitops-demo</pre><p>Went to GitHub. Clicked “Compare &amp; pull request.”</p><p>Title: “test: add blog demo tag for GitHub Actions workflow demonstration” <br>Description: “Testing GitHub Actions for blog post Part 5 screenshots.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vrptCj60jImcDD5riCiHhQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Vgr7jXnBPApX2007L5_Hg.png" /></figure><p>The Magic Moment</p><p>Clicked “Compare &amp; pull request.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9UFkDbEvcblr8tlEPeudSA.png" /></figure><p>Add a Description</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0AgSVU9gj1wFT1STaEkLGg.png" /></figure><p>As soon as I created the PR, I watched the Actions tab. The Terraform Plan workflow started immediately:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x2oysHTBW1O4Ko75LqhXDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uKUdjRMu8YKLv75-3ifaeA.png" /></figure><p>HOLY SHIT, IT WORKED!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q_W3MJRel8JrQ0tEaotmAg.png" /></figure><p>Merging the PR</p><p>I reviewed the plan. Looks good. Just adding a tag.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jmxTGZLttYE0o2KzV1Mw0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TrnbSbLcp6v9RznKblczJg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NvHIddVnhyZP5xRjDPI0Sw.png" /></figure><p>Confirm the merge Trigger the Apply workflow</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DrClP8_4fw5q3ROw-UyU1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sHjh9igM_l9vslC2akg9Ng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uw29r8ZJ13fabXCV4YAsBg.png" /></figure><p>Apply completed!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lczp9BxMTqAOrZAdWjM1zw.png" /></figure><p>The tag was added to the VPC. I checked AWS Console:</p><p>VPC Tags:</p><ul><li>Name: terrastack-vpc</li><li>Environment: dev</li><li>Project: TerraStack</li><li>BlogDemo: github-actions-gitops ✅</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IObdp1EqMId7dVYA32ZFrg.png" /></figure><p>IT WORKED! Successfully Merged</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ISoEmm6DTFF80O7xb0Nh4w.png" /></figure><p>Dont forget to delete the Branch</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i5LhOrkcN_5Af-oUHCluXg.png" /></figure><p>I just:</p><p>1. Created a branch<br>2. Made a change<br>3. Opened a PR<br>4. Reviewed the auto-generated plan<br>5. Merged the PR<br>6. Infrastructure was automatically updated</p><p>This is GitOps.</p><p>What This Enables</p><p>Now every infrastructure change is:</p><p>Auditable: Git history shows who changed what and when</p><p>Reviewable: Team can review plans before merging</p><p>Automated: No manual terraform commands</p><p>Rollbackable: git revert undoes changes</p><p>Safe: Can’t accidentally destroy prod (would need to merge a PR first)</p><p>Professional: This is how companies work</p><p>Testing the Destroy Workflow</p><p>I wanted to test the destroy workflow and see the safety mechanism in action.</p><p>I went to GitHub Actions tab.</p><p>Clicked Terraform Destroy workflow.</p><p>Clicked Run workflow button.</p><p>I filled in:</p><ul><li>Environment: dev</li><li>Confirm: destroy</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PI0LxRcBYUNDYMIQoSvseA.png" /></figure><p>Clicked Run workflow</p><p>Workflow started:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aPdmlyLd_JERUe65bQodAw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kRynOU897qaoXqu2KRu18g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kOko0bgGGamtfPG5p1ylnw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QrgcfPE6v0aJlaRAPDZ9GA.png" /></figure><p>After 20 minutes:</p><p>Destroy workflow steps expanded</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ViwKrFO62HWf-9L1WjFaDg.png" /></figure><p>All infrastructure was gone.</p><p>Verification:</p><pre>aws ec2 describe-vpcs - filters &quot;Name=tag:Name,Values=terrastack-vpc&quot;</pre><p>Output:</p><pre>{<br> &quot;Vpcs&quot;: []<br>}</pre><p>✅ VPC deleted.</p><pre>aws eks describe-cluster - name terrastack-cluster</pre><p>Output:</p><pre>An error occurred (ResourceNotFoundException) when calling the DescribeCluster operation: <br>No cluster found for name: terrastack-cluster.</pre><p>✅ EKS deleted.</p><p>Everything was gone. The destroy workflow worked perfectly.</p><p>The Rebuild</p><p>Now the cool part: I can rebuild everything.</p><p>1. Went to GitHub Actions<br>2. Clicked Terraform Apply workflow<br>3. Clicked Run workflow manually (usually it runs on merge, but you can trigger it manually)<br>4. Selected environment: dev<br>5. Clicked “Run workflow”</p><p>After 17 minutes:</p><p>Everything was recreated:</p><ul><li>VPC with subnets</li><li>EKS cluster with nodes</li><li>RDS database</li><li>All security groups</li><li>All IAM roles</li></ul><p>Verification:</p><pre>aws eks update-kubeconfig - name terrastack-cluster - region us-east-1<br><br>kubectl get nodes</pre><p>Output:</p><pre>Updated context arn:aws:eks:us-east-1:533267055898:cluster/terrastack-cluster in /root/.kube/config<br><br>NAME STATUS ROLES AGE VERSION<br>ip-10–0–11–206.ec2.internal Ready &lt;none&gt; 42m v1.31.13-eks-ecaa3a6<br>ip-10–0–12–137.ec2.internal Ready &lt;none&gt; 42m v1.31.13-eks-ecaa3a6</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R2fpsAM9zlIUZ10jnGdw9Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v2X9AZDOS_wm_5JW-W1jTg.png" /></figure><p>EVERYTHING BACK!</p><p>This is the power of Infrastructure as Code. Destroy and rebuild in 17 minutes. From a button click.</p><p>The Real Power: On-Demand Infrastructure</p><p>Here’s what this unlocks for me:</p><p>I can spin up this entire infrastructure in 17 minutes with a button click. Run a demo, show it working, then destroy it 20 minutes later. Total cost per demo: ~$8 (4 hours at $2/hour).</p><p>This means I can:</p><ul><li>Keep a live demo ready for any interview on short notice</li><li>Walk through actual working infrastructure, not just code</li><li>Show the complete GitOps workflow in action</li><li>Demonstrate the destroy/rebuild capability (proves it’s truly IaC)</li></ul><p>Without GitHub Actions, I’d have to:</p><ul><li>Remember to run `terraform apply` the night before</li><li>Hope nothing breaks during the night</li><li>Pay $48/day to keep it running “just in case”</li><li>Manually run terraform destroy after (and hope I remember)</li></ul><p>With automation:</p><ul><li>Click button morning of interview</li><li>Infrastructure ready in 17 minutes</li><li>Click destroy button after interview</li><li>Pay only for what I use</li></ul><p>Total project cost so far: ~$50–75 (initial build + testing)</p><p>Potential ROI: A DevOps job at $130K+</p><p>That’s the power of Infrastructure as Code plus GitOps automation.</p><p>What I Learned</p><p>1. GitHub Actions workflows are YAML files (.github/workflows/)<br>2. Three workflows needed: plan (PR), apply (merge), destroy (manual)<br>3. AWS credentials in GitHub secrets (Settings → Secrets → Actions)<br>4. Credentials disappear after closing (copy immediately!)<br>5. Separate credentials for different purposes (local vs CI/CD)<br>6. max-parallel: 1 prevents conflicts (sequential deploys)<br>7. workflow_dispatch for manual triggers (destroy workflow)<br>8. Confirmation field prevents accidents (must type “destroy”)<br>9. Rebuild time: 17 minutes (perfect for interview demos)<br>10. GitOps changes everything (professional workflow)</p><p>What This Proves</p><p>I can now tell employers:</p><p>&gt; “I built a production-grade AWS infrastructure with GitOps automation. Every infrastructure change goes through Pull Requests with automated planning, team review, and automatic deployment. I can destroy and rebuild the entire stack in 17 minutes with a button click. The code is modular, the security is layered, and the workflow is professional.”</p><p>That’s a $130K+ statement.</p><p>Coming Up in Part 6 (Final)</p><p>The project is complete. Infrastructure works. GitOps is automated.</p><p>But was it worth it?</p><p>In Part 6, I’ll break down:</p><ul><li>Real costs (not estimates)</li><li>Hidden charges I didn’t expect</li><li>Lessons learned</li><li>Mistakes I’d avoid next time</li><li>What this project actually proves</li><li>Interview preparation strategy</li><li>The final decision: Keep running or destroy?</li></ul><p>This is the wrap-up. The honest assessment. The “what would I do differently?” post-mortem.</p><p>The Power of GitOps</p><p>Now I can:</p><ul><li>Destroy infrastructure: Click button, wait 20 minutes</li><li>Rebuild infrastructure: Click button, wait 17 minutes</li><li>Demo for interviews: $8 per demo (4 hours × $2/hour)</li><li>Every change is reviewed, audited, automated</li></ul><p>📊 INFRASTRUCTURE STATUS:</p><p>VPC: ❌ Deleted<br>EKS Cluster: ❌ Deleted <br>RDS Database: ❌ Deleted<br>Security Groups: ❌ Deleted<br>Cost: $0/hour ✅<br>Ready for rebuild: ✅ (17 minutes)</p><p>💡 KEY TAKEAWAYS</p><p>1. GitOps: Every change through Pull Requests (plan → review → merge → apply)<br>2. Three workflows: terraform-plan, terraform-apply, terraform-destroy<br>3. GitHub secrets for AWS credentials (never commit credentials)<br>4. Copy credentials BEFORE clicking “Done” (they disappear)<br>5. Separate credentials per use case (local vs CI/CD)<br>6. workflow_dispatch for manual workflows (destroy button)<br>7. Confirmation fields prevent accidents (must type “destroy” exactly)<br>8. Rebuild time: 17 minutes (perfect for interview demos)<br>9. Destroy time: 20 minutes (clean teardown)<br>10. Cost per demo: ~$8 (4 hours running time)</p><p><strong>Connect &amp; Continue</strong></p><p><em>Part 6 the Finalé</em></p><ul><li>🔗<strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>📧 <strong><em>Email</em></strong>: benjaminLburton@outlook.com</li></ul><p>#GitOps #GitHubActions #CICD #Automation #Terraform #AWS #ProfessionalWorkflow</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b4457e806f3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TerraStack Part 4: The Database That Taught Me about Secrets]]></title>
            <link>https://mrburton.medium.com/terrastack-part-4-the-database-that-taught-me-about-secrets-8057b15e3b7f?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/8057b15e3b7f</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Mon, 22 Dec 2025 13:04:52 GMT</pubDate>
            <atom:updated>2025-12-22T13:04:52.736Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_I1gpwnd4RrdNIKNxon1yw.png" /></figure><p>Private subnets, security group confusion, and a 3-hour debugging session. The RDS chapter of building AWS infrastructure from absolute scratch — why PostgreSQL 16.3 doesn’t exist in AWS and why databases can’t talk to pods without the right rules.</p><p><strong><em>Quick Links to Series</em></strong>:</p><p>✅ <a href="https://mrburton.medium.com/terrastack-part-1-why-click-ops-is-a-trap-and-how-i-escaped-it-e5ae55496ec7">Part 1: “Why Click-Ops is a Trap (And How I Escaped It)”</a><br>✅ <a href="https://medium.com/@mrburton/terrastack-part-2-the-state-file-problem-i-didnt-know-i-had-622c63d71e88">Part 2: “The State File Problem I Didn’t Know I Had” </a><br>✅ <a href="https://mrburton.medium.com/terrastack-part-3-eks-and-the-iam-roles-i-didnt-understand-5584933de31c">Part 3: “EKS and the IAM Roles I Didn’t Understand” </a><br>✅ Part 4: “The Database That Taught Me About Secrets” ← You are here <br>✅ Part 5: “GitHub Actions and the Credentials That Disappeared”<br>✅ Part 6: “The Real Cost and What I Actually Learned”</p><p>🔗 Full Project Code: <a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></p><p>The Private Subnet Revelation</p><p>Before I started building the database, I had a conversation with someone on Reddit.</p><p>Me: “I’m about to add a PostgreSQL database to my EKS cluster. Any tips?”</p><p>Them: “Make sure it’s not publicly accessible.”</p><p>Me: “Why would anyone make a database publicly accessible?”</p><p>Them: “You’d be surprised. Check Shodan sometime.”</p><p>I Googled “Shodan database.” Found a search engine for internet-connected devices. Searched “PostgreSQL.”</p><p>Results: Thousands of publicly accessible PostgreSQL databases. Just… out there. On the internet. Many with default passwords or no password at all.</p><p>Holy shit.</p><p>This is why everyone says “private subnets only” for databases. It’s not theoretical. People actually make this mistake. A lot.</p><p>So my first rule: The database will NEVER be publicly accessible.</p><p>Understanding RDS vs. Self-Hosted</p><p>I had a choice:</p><p>Option 1: Run PostgreSQL in a Kubernetes pod</p><ul><li>Cost: $0 (uses existing nodes)</li><li>Management: I do everything</li><li>Backups: I configure them</li><li>Failover: I set it up</li><li>Upgrades: I apply them</li><li>3 AM failures: My problem</li></ul><p>Option 2: Use AWS RDS (managed PostgreSQL)</p><ul><li>Cost: $15/month for db.t3.micro</li><li>Management: AWS does it</li><li>Backups: Automatic</li><li>Failover: Built-in (with Multi-AZ)</li><li>Upgrades: AWS applies them</li><li>3 AM failures: AWS’s problem</li></ul><p>For production, RDS is the obvious choice. For learning… it’s still the right choice because companies use RDS. Showing I know how to use managed services is valuable.</p><p>Plus, I value my sleep.</p><p>The RDS Security Architecture</p><p>I drew out the security layers:</p><pre>Internet<br> ↓<br> ❌ (blocked)<br> <br>Public Subnets<br> ↓<br> ❌ (no route)<br> <br>Private Subnets (EKS nodes)<br> ↓<br> ✅ (security group allows)<br> <br>Private Subnets (RDS)<br> ✅ (receives connection)</pre><p>Layer 1: RDS is in private subnets (no public IP) <br>Layer 2: Security group only allows port 5432 from EKS nodes <br>Layer 3: No route from internet to private subnets</p><p>Three layers of defense. Even if I mess up one, the others protect me.</p><p>Creating the RDS Security Group</p><pre>resource &quot;aws_security_group&quot; &quot;rds&quot; {<br> name = &quot;${var.db_identifier}-sg&quot;<br> description = &quot;Security group for RDS PostgreSQL&quot;<br> vpc_id = var.vpc_id<br>}<br># Allow PostgreSQL from EKS nodes ONLY<br>resource &quot;aws_security_group_rule&quot; &quot;rds_ingress_eks&quot; {<br> type = &quot;ingress&quot;<br> from_port = 5432<br> to_port = 5432<br> protocol = &quot;tcp&quot;<br> source_security_group_id = var.eks_node_security_group_id<br> security_group_id = aws_security_group.rds.id<br>}</pre><p>Key detail: source_security_group_id = var.eks_node_security_group_id</p><p>This says: “Only allow traffic from resources that have the EKS node security group.”</p><p>Not from the internet. Not from my laptop. Not from other AWS services. Only EKS nodes.</p><p>If I want to connect from my laptop (for debugging), I have two options:</p><p>1. Use `kubectl port-forward` through a pod<br>2. Use a bastion host</p><p>But the database itself? Never directly accessible.</p><p>Problem #7: PostgreSQL Version Doesn’t Exist</p><p>I started writing the RDS resource:</p><pre>resource &quot;aws_db_instance&quot; &quot;main&quot; {<br> identifier = var.db_identifier<br> <br> engine = &quot;postgres&quot;<br> engine_version = &quot;16.3&quot; # Latest minor version (I thought)<br> <br> instance_class = &quot;db.t3.micro&quot;<br> allocated_storage = 20<br> <br> # … more config<br>}</pre><p>I ran `terraform plan`:</p><pre>cd environments/dev<br>terraform plan</pre><p>Error:</p><pre>Error: creating RDS DB Instance (terrastack-db): InvalidParameterValue: <br>Cannot find version 16.3 for postgres</pre><p>Wait, what?</p><p>I Googled “PostgreSQL 16.3 AWS RDS.”</p><p>Found AWS documentation: Available PostgreSQL versions</p><p>Versions listed:</p><ul><li>16.6 ✅</li><li>16.5 ✅</li><li>16.4 ✅</li><li>16.3 ❌ (not listed)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lmUSXTS_ez3LxC5dk6Rbyw.png" /></figure><p>So AWS doesn’t offer every minor version. They curate which ones are available.</p><p>Why? Probably because not every PostgreSQL release is production-ready. AWS tests them before making them available.</p><p>The fix:</p><p>I updated two files:</p><p>File 1: `modules/rds/variables.tf`</p><pre>variable &quot;db_engine_version&quot; {<br> description = &quot;PostgreSQL engine version&quot;<br> type = string<br> default = &quot;16.6&quot; # Was: 16.3<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bwysn_MuAY7lxZLGHRchZQ.png" /></figure><p>File 2: `environments/dev/variables.tf`</p><pre>variable &quot;db_engine_version&quot; {<br> description = &quot;PostgreSQL engine version&quot;<br> type = string<br> default = &quot;16.6&quot; # Was: 16.3<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GwKMqkq7CYEok9pXEI2n0g.png" /></figure><p>Why both files?</p><p>Because `environments/dev/variables.tf` can override the module’s default. If I don’t update both, the dev environment might still try to use 16.3.</p><p>Lesson learned: Always check AWS documentation for available versions. Don’t assume the latest PostgreSQL version is available in RDS.</p><p>The Password Problem</p><p>Next question: How do I set the database password?</p><p>Bad option:</p><pre>resource &quot;aws_db_instance&quot; &quot;main&quot; {<br> username = &quot;admin&quot;<br> password = &quot;MySecurePassword123!&quot; # ❌ DON&#39;T DO THIS<br>}</pre><p>Why this is bad:</p><p>1. Password is in the code<br>2. Password is in the Terraform state file (plaintext!)<br>3. Password is in Git history (even if I delete it later)<br>4. Anyone with repo access has the password<br>5. Can’t rotate without changing code</p><p>The right way: Generate a random password and store it in AWS Secrets Manager.</p><pre># Generate random password<br>resource &quot;random_password&quot; &quot;db_password&quot; {<br> length = 32<br> special = true<br>}<br># Store in Secrets Manager<br>resource &quot;aws_secretsmanager_secret&quot; &quot;db_password&quot; {<br> name = &quot;${var.db_identifier}-password&quot;<br> recovery_window_in_days = 7<br>}<br>resource &quot;aws_secretsmanager_secret_version&quot; &quot;db_password&quot; {<br> secret_id = aws_secretsmanager_secret.db_password.id<br> secret_string = jsonencode({<br> username = var.db_username<br> password = random_password.db_password.result<br> engine = &quot;postgres&quot;<br> host = aws_db_instance.main.address<br> port = 5432<br> dbname = var.db_name<br> })<br>}<br># Use in RDS<br>resource &quot;aws_db_instance&quot; &quot;main&quot; {<br> username = var.db_username<br> password = random_password.db_password.result # From generator<br>}</pre><p>What this does:</p><p>1. Terraform generates a random 32-character password<br>2. Stores it in Secrets Manager (encrypted)<br>3. Uses it for RDS<br>4. Stores all connection info in one secret (host, port, username, password)</p><p>How pods retrieve it:</p><pre>apiVersion: v1<br>kind: Pod<br>spec:<br> containers:<br> - name: app<br> env:<br> - name: DB_SECRET_ARN<br> value: &quot;arn:aws:secretsmanager:us-east-1:123456:secret:terrastack-db-password&quot;</pre><p>The app uses AWS SDK to fetch the secret, parse the JSON, and connect.</p><p>Cost: $0.40/month per secret. Worth it for actual security.</p><p>The Complete RDS Resource</p><pre>resource &quot;aws_db_instance&quot; &quot;main&quot; {<br> identifier = var.db_identifier<br> <br> # Engine<br> engine = &quot;postgres&quot;<br> engine_version = &quot;16.6&quot;<br> <br> # Size<br> instance_class = &quot;db.t3.micro&quot;<br> allocated_storage = 20<br> storage_type = &quot;gp3&quot;<br> <br> # Auto-scaling storage<br> max_allocated_storage = 100<br> <br> # Database<br> db_name = var.db_name<br> username = var.db_username<br> password = random_password.db_password.result<br> <br> # Network<br> db_subnet_group_name = aws_db_subnet_group.main.name<br> vpc_security_group_ids = [aws_security_group.rds.id]<br> publicly_accessible = false # NEVER TRUE<br> <br> # Availability<br> multi_az = false # Single-AZ for dev<br> <br> # Backups<br> backup_retention_period = 7 # 7 days<br> backup_window = &quot;03:00–04:00&quot;<br> <br> # Encryption<br> storage_encrypted = true<br> <br> # Deletion protection<br> deletion_protection = false # true for prod<br> skip_final_snapshot = true # false for prod<br> <br> # Parameter group<br> parameter_group_name = aws_db_parameter_group.main.name<br>}</pre><p>Key decisions:</p><p>db.t3.micro: Smallest instance ($12–15/month). Good enough for dev.</p><p>gp3 storage: Newer, cheaper than gp2 ($0.08/GB vs $0.10/GB).</p><p>max_allocated_storage = 100: If I fill up the 20GB, AWS auto-expands to 100GB max. Prevents “disk full” disasters.</p><p>multi_az = false: Single availability zone for dev. For production, I’d set this to `true` (automatic failover, but costs 2× as much).</p><p>backup_retention_period = 7: Keep backups for a week. Production would use 30+ days.</p><p>storage_encrypted = true: Always encrypt databases. No reason not to.</p><p>deletion_protection = false: For dev, I want to be able to destroy easily. For production, I’d set this to `true` (prevents accidental deletion).</p><p>skip_final_snapshot = true: For dev, I don’t need a final snapshot. For production, `false` (always take a final snapshot before deletion).</p><p>The Parameter Group</p><p>I wanted query logging for debugging:</p><pre>resource &quot;aws_db_parameter_group&quot; &quot;main&quot; {<br> name = &quot;${var.db_identifier}-pg&quot;<br> family = &quot;postgres16&quot;<br> <br> parameter {<br> name = &quot;log_connections&quot;<br> value = &quot;1&quot;<br> }<br> <br> parameter {<br> name = &quot;log_disconnections&quot;<br> value = &quot;1&quot;<br> }<br> <br> parameter {<br> name = &quot;log_duration&quot;<br> value = &quot;1&quot;<br> }<br>}</pre><p>This logs every connection, disconnection, and query duration to CloudWatch.</p><p>Cost impact: ~$2–5/month in CloudWatch logs.</p><p>Value: When debugging connection issues, these logs are invaluable.</p><p>Integrating RDS into Dev Environment</p><p>I updated `environments/dev/main.tf`:</p><pre>module &quot;rds&quot; {<br> source = &quot;../../modules/rds&quot;<br> <br> db_identifier = &quot;${var.project_name}-db&quot;<br> vpc_id = module.vpc.vpc_id<br> private_subnet_ids = module.vpc.private_subnet_ids<br> eks_node_security_group_id = module.eks.node_security_group_id<br> <br> db_name = &quot;terrastack&quot;<br> db_username = &quot;postgres&quot;<br> db_engine_version = &quot;16.6&quot;<br> <br> depends_on = [module.eks]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*isjb70K0UX4Aen_FWSc08Q.png" /></figure><p>Key detail: `depends_on = [module.eks]`</p><p>This tells Terraform: “Don’t start creating RDS until EKS is completely done.”</p><p>Why? Because the RDS security group references the EKS node security group. If EKS isn’t done yet, that security group doesn’t exist, and Terraform fails.</p><p>The Deployment</p><pre>cd environments/dev<br>terraform init -upgrade # Get the random provider<br>terraform plan</pre><p>Output:</p><pre>Plan: 7 to add, 0 to change, 0 to destroy.<br>Changes:<br> + aws_db_instance.main<br> + aws_db_subnet_group.main<br> + aws_db_parameter_group.main<br> + aws_security_group.rds<br> + aws_secretsmanager_secret.db_password<br> + aws_secretsmanager_secret_version.db_password<br> + random_password.db_password</pre><p>7 resources. Looks good.</p><pre>terraform apply</pre><p>Output:</p><pre>module.rds.random_password.db_password: Creating…<br>module.rds.random_password.db_password: Creation complete [0s]<br>module.rds.aws_secretsmanager_secret.db_password: Creating…<br>module.rds.aws_db_subnet_group.main: Creating…<br>module.rds.aws_security_group.rds: Creating…<br>module.rds.aws_db_parameter_group.main: Creating…<br>module.rds.aws_db_instance.main: Creating…<br>module.rds.aws_db_instance.main: Still creating… [2m0s elapsed]<br>module.rds.aws_db_instance.main: Still creating… [4m0s elapsed]<br>module.rds.aws_db_instance.main: Still creating… [6m0s elapsed]<br>module.rds.aws_db_instance.main: Still creating… [8m0s elapsed]</pre><p>RDS takes 8–10 minutes. I knew this from reading docs, so I went for a walk.</p><p>Came back:</p><pre>module.rds.aws_db_instance.main: Creation complete after 8m47s<br>Apply complete! Resources: 7 added, 0 changed, 0 destroyed.<br>Outputs:<br>db_endpoint = &quot;terrastack-db.abc123.us-east-1.rds.amazonaws.com:5432&quot;<br>db_secret_arn = &quot;arn:aws:secretsmanager:us-east-1:123456:secret:terrastack-db-password-AbCdEf&quot;</pre><p>Database is live!</p><p>Testing the Connection</p><p>I couldn’t connect directly from my laptop (security groups block it). So I tested from a pod:</p><pre># Create a temporary pod with psql<br>kubectl run -it - rm psql - image=postgres:16 - restart=Never - bash</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zodxiZSe1VkRTgj3jTopsw.png" /></figure><p>Inside the pod:</p><pre># Get the secret<br>aws secretsmanager get-secret-value \<br> - secret-id terrastack-db-password \<br> - query SecretString \<br> - output text</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C-xo3akt21SO47TbhX5J_A.png" /></figure><p>Output:</p><pre>{<br> &quot;dbname&quot;: &quot;appdb&quot;,<br> &quot;engine&quot;: &quot;postgres&quot;,<br> &quot;host&quot;: &quot;terrastack-db.chsa8444mhv9.us-east-1.rds.amazonaws.com&quot;,<br> &quot;password&quot;: &quot;EXAMPLEkJ8#mP2$vR9*nL5@wQ1!xT7&amp;yH4EXAMPLE&quot;,<br> &quot;port&quot;: 5432,<br> &quot;username&quot;: &quot;dbadmin&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZYJL2s_3eWX7SE7hI02Uzw.png" /></figure><p>Perfect. Now connect:</p><pre>psql -h terrastack-db.abc123.us-east-1.rds.amazonaws.com \<br> -U postgres \<br> -d terrastack</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IQES5jhiB1iuKKRM1rbpZg.png" /></figure><p>Prompt appeared:</p><pre>Password: [I pasted the password]<br>terrastack=&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ORr70kN_kiPygYM3s2ND7w.png" /></figure><p>I’M IN!</p><pre>SELECT version();</pre><p>Output:</p><pre>PostgreSQL 16.6 on x86_64-pc-linux-gnu, compiled by gcc</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YI3vISFEvaVMojoGw2fOIw.png" /></figure><p>✅ Connection works!</p><pre>CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT);<br>INSERT INTO test (name) VALUES (&#39;Hello from EKS!&#39;);<br>SELECT * FROM test;</pre><p>Output:</p><pre>id | name <br> - - + - - - - - - - - -<br> 1 | Hello from EKS!</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rXUzbckeb96n9ZZlogk8HA.png" /></figure><p>Database is working!</p><p>Pods can connect. Queries work. Everything is functional.</p><p>Checking CloudWatch Logs</p><p>I went to CloudWatch Logs. Found `/aws/rds/instance/terrastack-db/postgresql`.</p><p>Logs showed:</p><pre>2024–12–26 14:23:45 UTC::@:[12345]:LOG: connection received: host=10.0.11.45 port=52371<br>2024–12–26 14:23:45 UTC:postgres@terrastack:[12345]:LOG: connection authorized: user=postgres database=terrastack</pre><p>✅ Logging works!</p><p>If I ever have connection issues, I’ll see them here.</p><p>The Updated Cost</p><p>AWS Cost Explorer:</p><p>New charges:</p><ul><li>EKS Control Plane: $73/month</li><li>EC2 (2× t3.medium): $60/month</li><li>NAT Gateways (2×): $65/month</li><li>RDS (db.t3.micro): $15/month</li><li>EBS Volumes (3×): $10/month</li><li>Secrets Manager: $0.40/month</li><li>CloudWatch Logs: ~$2/month</li><li>Data transfer: ~$3/month</li></ul><p>New total: ~$228/month</p><p>Close to my estimate! The $228/month prediction was accurate.</p><p>What I Learned</p><p>1. Databases should NEVER be publicly accessible (layer defense: private subnets + security groups)<br>2. AWS RDS doesn’t offer every PostgreSQL version (16.3 doesn’t exist, 16.6 does)<br>3. Never hardcode passwords (use random_password + Secrets Manager)<br>4. Both module and environment need version updates (check both files)<br>5. db.t3.micro is $15/month (good for dev, too small for production)<br>6. RDS takes 8–10 minutes to create (be patient)<br>7. CloudWatch logging costs ~$2–5/month (worth it for debugging)<br>8. Testing from pods, not laptop (security groups prevent direct access)<br>9. Current cost: $228/month (VPC + EKS + RDS running)<br>10. Next step: Automate everything (no more manual terraform apply)</p><p>What I Realized</p><p>I now have:</p><ul><li>✅ VPC with proper networking</li><li>✅ EKS Kubernetes cluster</li><li>✅ RDS PostgreSQL database</li><li>✅ Everything in private subnets</li><li>✅ Security groups properly configured</li><li>✅ Secrets in Secrets Manager</li></ul><p>This is a complete infrastructure stack.</p><p>But I’ve been running `terraform apply` manually. Every change requires:</p><p>1. Write code<br>2. Run terraform plan<br>3. Review output<br>4. Run terraform apply<br>5. Hope I didn’t mess up</p><p>This doesn’t scale.</p><p>Companies use GitOps: Every infrastructure change goes through Pull Requests. Automated plans. Peer review. Automatic deployment after merge.</p><p>That’s what I’m building next.</p><p>Coming Up in Part 5</p><p>GitHub Actions. The final piece.</p><p>Goals:</p><ul><li>Pull Request → Automatic terraform plan</li><li>Merge to main → Automatic terraform apply</li><li>Manual button → terraform destroy (for cleanup)</li></ul><p>No more running commands manually. Everything automated. Everything auditable.</p><p>That’s how professionals work.</p><p>💡 KEY TAKEAWAYS</p><p>1. RDS in private subnets only (publicly_accessible = false, always)<br>2. Security group allows only EKS nodes (source_security_group_id)<br>3. PostgreSQL 16.3 doesn’t exist in AWS (use 16.6 instead)<br>4. Update version in BOTH module and environment files<br>5. random_password + Secrets Manager for credentials (never hardcode)<br>6. db.t3.micro: $15/month (smallest production-ready instance)<br>7. gp3 storage cheaper than gp2 ($0.08/GB vs $0.10/GB)<br>8. RDS creation time: 8–10 minutes (grab coffee)<br>9. Test connections from pods (kubectl run psql)<br>10. Total cost: $228/month (matches estimate!)</p><p><strong>Connect &amp; Continue</strong></p><p><em>Part 5 coming next!</em></p><ul><li>🔗<strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>📧 <strong><em>Email</em></strong>: benjaminLburton@outlook.com</li></ul><p>#RDS #PostgreSQL #AWS #DatabaseSecurity #SecretsManager #PrivateSubnets #RealProblems</p><p><strong>About the Author</strong>:</p><p>La’vell Burton a DevOps engineer and AWS Community Builder (Containers category) currently working as a delivery driver while building a portfolio to break into tech. This series documents a real production-grade project built in a single intensive day of failures, troubleshooting, and all.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8057b15e3b7f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TerraStack Part 3: EKS and the IAM Roles I Didn’t Understand]]></title>
            <link>https://mrburton.medium.com/terrastack-part-3-eks-and-the-iam-roles-i-didnt-understand-5584933de31c?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/5584933de31c</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Thu, 18 Dec 2025 10:49:16 GMT</pubDate>
            <atom:updated>2025-12-22T13:04:23.468Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ckIBw4cuiH8aPgVaMe08xA.png" /></figure><p><em>OIDC providers, assume role policies, and 32 Terraform resources. The Kubernetes cluster chapter of building AWS infrastructure from absolute scratch — and what IAM roles actually do.</em></p><p><strong><em>Quick Links to Series</em></strong>:</p><p>✅ <a href="https://mrburton.medium.com/terrastack-part-1-why-click-ops-is-a-trap-and-how-i-escaped-it-e5ae55496ec7">Part 1: “Why Click-Ops is a Trap (And How I Escaped It)”</a><br>✅ <a href="https://medium.com/@mrburton/terrastack-part-2-the-state-file-problem-i-didnt-know-i-had-622c63d71e88">Part 2: “The State File Problem I Didn’t Know I Had” </a><br>✅ Part 3: “EKS and the IAM Roles I Didn’t Understand” ← You are here<br>✅ Part 4: “The Database That Taught Me About Secrets” <br>✅ Part 5: “GitHub Actions and the Credentials That Disappeared”<br>✅ Part 6: “The Real Cost and What I Actually Learned”</p><p>🔗 Full Project Code: <a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></p><p>The $73/Month Question</p><p>Before I started building the EKS cluster, I had to understand what I was paying for.</p><p>EKS Control Plane: $73/month</p><p>That’s $73 for… what exactly?</p><p>I did some research:</p><p>With self-managed Kubernetes:</p><ul><li>I’d need 3 master nodes (for high availability)</li><li>Each node needs to be t3.medium minimum = $30/month each</li><li>Total: $90/month for masters</li><li>Plus: I manage everything (etcd, API server, scheduler, controller manager)</li><li>Plus: I’m responsible if it breaks at 3 AM</li><li>Plus: I need to understand how all these components work</li></ul><p>With EKS:</p><ul><li>AWS runs the master nodes (I never see them)</li><li>AWS handles upgrades, patches, security</li><li>AWS guarantees 99.95% uptime</li><li>I just use it</li><li>Total: $73/month</li></ul><p>Okay, so EKS is actually cheaper (and way less stress). I’ll take it.</p><p>Problem #5: What Even Is IAM?</p><p>I started writing the EKS Terraform code. First line:</p><pre>resource &quot;aws_iam_role&quot; &quot;cluster&quot; {<br> name = &quot;${var.cluster_name}-cluster-role&quot;<br> …<br>}</pre><p>My immediate thought: What’s an IAM role?</p><p>I Googled it. Every explanation was like:</p><p>“IAM roles are AWS identities with specific permissions that can be assumed by entities you trust.”</p><p>That doesn’t help.</p><p>I found a better explanation on Reddit:</p><p>“Think of IAM roles like a security badge. The badge has permissions (can enter this room, can’t enter that room). You give the badge to a service (EKS), and now that service can do whatever the badge allows.”</p><p>Okay, that makes sense.</p><p>So EKS needs a “badge” that says: “This EKS cluster is allowed to create load balancers, modify security groups, and manage networking.”</p><p>And the worker nodes need a different “badge” that says: “These nodes are allowed to join the cluster, pull Docker images, and run containers.”</p><p>Different roles. Different permissions.</p><p>Creating the Cluster IAM Role</p><pre>resource &quot;aws_iam_role&quot; &quot;cluster&quot; {<br> name = &quot;${var.cluster_name}-cluster-role&quot;<br>assume_role_policy = jsonencode({<br> Version = &quot;2012–10–17&quot;<br> Statement = [{<br> Action = &quot;sts:AssumeRole&quot;<br> Effect = &quot;Allow&quot;<br> Principal = {<br> Service = &quot;eks.amazonaws.com&quot;<br>       }<br>     }]<br>   })<br>}</pre><p>What this says:</p><ul><li>Create an IAM role named “terrastack-cluster-cluster-role”</li><li>Allow the EKS service (`eks.amazonaws.com`) to “assume” this role</li><li>“Assume” means: wear this badge, use these permissions</li></ul><p>Then I attach AWS-managed policies:</p><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;cluster_policy&quot; {<br> policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKSClusterPolicy&quot;<br> role = aws_iam_role.cluster.name<br>}<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;vpc_resource_controller&quot; {<br> policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKSVPCResourceController&quot;<br> role = aws_iam_role.cluster.name<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iimEcRkGSI9lTGf3p1x-ug.png" /></figure><p>What these do:</p><ul><li>AmazonEKSClusterPolicy: Lets EKS create load balancers, manage networking</li><li>AmazonEKSVPCResourceController: Lets EKS manage VPC resources (ENIs, security groups)</li></ul><p>I didn’t write these policies. AWS provides them. I just attach them to my role.</p><p>Creating the Node IAM Role</p><p>Same idea, but for worker nodes:</p><pre>resource &quot;aws_iam_role&quot; &quot;node&quot; {<br> name = &quot;${var.cluster_name}-node-role&quot;<br>assume_role_policy = jsonencode({<br> Version = &quot;2012–10–17&quot;<br> Statement = [{<br> Action = &quot;sts:AssumeRole&quot;<br> Effect = &quot;Allow&quot;<br> Principal = {<br> Service = &quot;ec2.amazonaws.com&quot; <br>       }<br>     }]<br>   })<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cj4dIjM9dTyngcSv3TXtXQ.png" /></figure><p>Key difference: Principal is `ec2.amazonaws.com` because nodes are EC2 instances.</p><p>Then I attach three policies:</p><pre># Policy 1: Worker Node Policy<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;node_policy&quot; {<br> policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy&quot;<br> role = aws_iam_role.node.name<br>}<br># Policy 2: CNI Policy (networking)<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;cni_policy&quot; {<br> policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy&quot;<br> role = aws_iam_role.node.name<br>}<br># Policy 3: Container Registry (pull images)<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;registry_policy&quot; {<br> policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly&quot;<br> role = aws_iam_role.node.name<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04cQ4DiV27DTYVCKWKqphQ.png" /></figure><p>What these do:</p><ul><li>Worker Node Policy: Join cluster, report status</li><li>CNI Policy: Assign IP addresses to pods</li><li>Registry Policy: Pull Docker images from AWS’s container registry</li></ul><p>My question: Why ReadOnly for the registry?</p><p>Answer I found: Nodes should only PULL images, never PUSH. Pushing happens in CI/CD pipelines, not on nodes. This is the principle of least privilege.</p><p>Makes sense.</p><p>Security Groups: The Firewall Rules</p><p>Next up: Security groups. These control network traffic.</p><p>I need two security groups:</p><p>1. Cluster security group (for the control plane)<br>2. Node security group (for the worker nodes)</p><p>Cluster security group rules:</p><pre># Allow nodes to reach API server on port 443<br>resource &quot;aws_security_group_rule&quot; &quot;cluster_ingress_nodes&quot; {<br> type = &quot;ingress&quot;<br> from_port = 443<br> to_port = 443<br> protocol = &quot;tcp&quot;<br> source_security_group_id = aws_security_group.node.id<br> security_group_id = aws_security_group.cluster.id<br>}</pre><p>What this says: Nodes can talk to the cluster API server on port 443 (HTTPS).</p><p>Node security group rules:</p><pre># Allow nodes to talk to each other<br>resource &quot;aws_security_group_rule&quot; &quot;node_ingress_self&quot; {<br> type = &quot;ingress&quot;<br> from_port = 0<br> to_port = 65535<br> protocol = &quot;-1&quot;<br> self = true<br> security_group_id = aws_security_group.node.id<br>}<br># Allow cluster to reach nodes<br>resource &quot;aws_security_group_rule&quot; &quot;node_ingress_cluster&quot; {<br> type = &quot;ingress&quot;<br> from_port = 1025<br> to_port = 65535<br> protocol = &quot;tcp&quot;<br> source_security_group_id = aws_security_group.cluster.id<br> security_group_id = aws_security_group.node.id<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PaYSuM62pfmeBRPyh5Uikg.png" /></figure><p>What these say:</p><ul><li>Nodes can talk to other nodes on all ports (pod-to-pod communication)</li><li>Cluster can reach nodes on ports 1025–65535 (kubelet API, metrics)</li></ul><p>My question: Why 1025–65535?</p><p>Answer: Ports 0–1024 are “privileged ports” (system services). Ports 1025+ are “unprivileged ports” (user services). Kubelet runs on port 10250, metrics on various high ports. We allow the whole range for flexibility.</p><p>The EKS Cluster Resource (Finally!)</p><p>Now I could create the actual cluster:</p><pre>resource &quot;aws_eks_cluster&quot; &quot;main&quot; {<br> name = var.cluster_name<br> role_arn = aws_iam_role.cluster.arn<br> version = var.cluster_version # &quot;1.31&quot;<br>vpc_config {<br> subnet_ids = concat(<br> var.public_subnet_ids, # For load balancers<br> var.private_subnet_ids # For nodes<br> )<br> endpoint_private_access = true # Nodes reach API privately<br> endpoint_public_access = true # I can reach API from laptop<br> security_group_ids = [aws_security_group.cluster.id]<br> }<br>enabled_cluster_log_types = [<br> &quot;api&quot;,<br> &quot;audit&quot;,<br> &quot;authenticator&quot;,<br> &quot;controllerManager&quot;,<br> &quot;scheduler&quot;<br> ]<br>depends_on = [<br> aws_iam_role_policy_attachment.cluster_policy,<br> aws_iam_role_policy_attachment.vpc_resource_controller<br> ]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZiRbXA2Yj5GJTg5cl-N-kg.png" /></figure><p>Key decisions:</p><ul><li>Both public and private endpoints: I can reach the cluster from my laptop (public), and nodes reach it privately (faster, more secure)</li><li>All logging enabled: Costs a bit more (~$2–5/month), but helps with debugging</li><li>depends_on: Terraform should create the IAM role BEFORE the cluster</li></ul><p>Problem #6: OIDC Provider (What?)</p><p>The next section in my code was:</p><pre>data &quot;tls_certificate&quot; &quot;cluster&quot; {<br> url = aws_eks_cluster.main.identity[0].oidc[0].issuer<br>}<br>resource &quot;aws_iam_openid_connect_provider&quot; &quot;cluster&quot; {<br> client_id_list = [&quot;sts.amazonaws.com&quot;]<br> thumbprint_list = [data.tls_certificate.cluster.certificates[0].sha1_fingerprint]<br> url = aws_eks_cluster.main.identity[0].oidc[0].issuer<br>}</pre><p>What is this?</p><p>I asked Google: “What’s an OIDC provider in EKS?”</p><p>The explanation:</p><p>“OIDC (OpenID Connect) allows pods to assume IAM roles. Without it, every pod on a node would have the same permissions as the node. With it, individual pods can have their own IAM roles (IRSA — IAM Roles for Service Accounts).”</p><p>Example:</p><ul><li>Node has permission to: join cluster, pull images</li><li>Pod A needs permission to: read from S3 bucket</li><li>Pod B needs permission to: write to DynamoDB table</li></ul><p>Without OIDC: Both pods get the node’s permissions (useless for their specific needs).</p><p>With OIDC: Pod A gets S3 read access. Pod B gets DynamoDB write access. Node just has its basic permissions.</p><p>This is important for security. I’m adding it even though I’m not using it yet (future-proofing).</p><p>The Node Group</p><p>Now for the worker nodes:</p><pre>resource &quot;aws_eks_node_group&quot; &quot;main&quot; {<br> cluster_name = aws_eks_cluster.main.name<br> node_group_name = &quot;${var.cluster_name}-node-group&quot;<br> node_role_arn = aws_iam_role.node.arn<br> subnet_ids = var.private_subnet_ids # Private only!<br>scaling_config {<br> desired_size = 2<br> max_size = 3<br> min_size = 1<br> }<br>instance_types = [&quot;t3.medium&quot;]<br> capacity_type = &quot;ON_DEMAND&quot;<br> disk_size = 20<br>update_config {<br> max_unavailable = 1 # Update one node at a time<br> }<br>lifecycle {<br> ignore_changes = [scaling_config[0].desired_size]<br> }<br>}</pre><p>Key decisions:</p><ul><li>Private subnets only: Nodes shouldn’t be publicly accessible</li><li>t3.medium: 2 vCPU, 4GB RAM, $30/month each</li><li>2 nodes desired: Minimum for high availability</li><li>Min 1, max 3: Can scale down to save money, up for load</li><li>ON_DEMAND: Not using spot instances (more reliable)</li><li>ignore_changes on desired_size: If cluster autoscaler changes this, Terraform won’t fight it</li></ul><p>Why ignore_changes?</p><p>Later, I might add cluster autoscaler (automatically adds/removes nodes based on load). If Terraform thinks desired_size should always be 2, it’ll fight the autoscaler. This tells Terraform: “Let something else manage this field.”</p><p>The Add-ons</p><p>EKS needs three add-ons:</p><pre># CoreDNS - DNS for the cluster<br>resource &quot;aws_eks_addon&quot; &quot;coredns&quot; {<br> cluster_name = aws_eks_cluster.main.name<br> addon_name = &quot;coredns&quot;<br>}<br># kube-proxy - Networking<br>resource &quot;aws_eks_addon&quot; &quot;kube_proxy&quot; {<br> cluster_name = aws_eks_cluster.main.name<br> addon_name = &quot;kube-proxy&quot;<br>}<br># VPC CNI - Pod IPs from VPC<br>resource &quot;aws_eks_addon&quot; &quot;vpc_cni&quot; {<br> cluster_name = aws_eks_cluster.main.name<br> addon_name = &quot;vpc-cni&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-HYwLwAUw_qqb9cn-eErhQ.png" /></figure><p>What these do:</p><ul><li>CoreDNS: Pods can reach each other by name (service-name.namespace.svc.cluster.local)</li><li>kube-proxy: Routes traffic to pods</li><li>VPC CNI: Pods get real VPC IP addresses (can talk directly to RDS, no NAT!)</li></ul><p>Time to Deploy</p><p>I reviewed my code. IAM roles, security groups, cluster, node group, add-ons. Everything was there.</p><pre>cd environments/dev<br>terraform init -upgrade # Get the TLS provider<br>terraform validate</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hpavxdMQSIzTK3-F4uNF-Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L9vgJPod5jizMxJGvTyPYA.png" /></figure><p>Output:</p><pre>Success! The configuration is valid.</pre><p>✅ Good sign.</p><pre>terraform plan</pre><p>Output:</p><pre>Plan: 32 to add, 0 to change, 0 to destroy.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ReAARyGx9v29TXD0-VB2mA.png" /></figure><p>32 resources! EKS is complex.</p><p>Deep breath.</p><pre>terraform apply tfplan</pre><p>The 17-Minute Wait</p><pre>module.eks.aws_iam_role.cluster: Creating…<br>module.eks.aws_iam_role.node: Creating…<br>module.eks.aws_security_group.cluster: Creating…<br>module.eks.aws_security_group.node: Creating…<br>…<br>module.eks.aws_eks_cluster.main: Creating…<br>module.eks.aws_eks_cluster.main: Still creating… [2m0s elapsed]<br>module.eks.aws_eks_cluster.main: Still creating… [4m0s elapsed]<br>module.eks.aws_eks_cluster.main: Still creating… [6m0s elapsed]<br>module.eks.aws_eks_cluster.main: Still creating… [8m0s elapsed]<br>module.eks.aws_eks_cluster.main: Still creating… [10m0s elapsed]<br>module.eks.aws_eks_cluster.main: Still creating… [12m0s elapsed]</pre><p>EKS cluster creation takes 12–15 minutes. I went outside. Walked around the block. Came back.</p><pre>module.eks.aws_eks_cluster.main: Creation complete after 12m17s<br>module.eks.aws_eks_node_group.main: Creating…<br>module.eks.aws_eks_node_group.main: Still creating… [2m0s elapsed]<br>module.eks.aws_eks_node_group.main: Still creating… [4m0s elapsed]</pre><p>Node group takes another 5 minutes.</p><p>More coffee.</p><pre>module.eks.aws_eks_node_group.main: Creation complete after 5m32s<br>module.eks.aws_eks_addon.coredns: Creating…<br>module.eks.aws_eks_addon.kube_proxy: Creating…<br>module.eks.aws_eks_addon.vpc_cni: Creating…<br><br>Apply complete! Resources: 32 added, 0 changed, 0 destroyed.<br><br>Outputs:<br>cluster_endpoint = &quot;https://ABC123.gr7.us-east-1.eks.amazonaws.com&quot;<br>cluster_name = &quot;terrastack-cluster&quot;<br>configure_kubectl = &quot;aws eks update-kubeconfig - region us-east-1 - name terrastack-cluster&quot;</pre><p>IT WORKED!</p><p>Total time: ~17 minutes.</p><p>Configuring kubectl</p><p>I ran the command from the output:</p><pre>aws eks update-kubeconfig - region us-east-1 - name terrastack-cluster</pre><p>Output:</p><pre>Added new context arn:aws:eks:us-east-1:533267055898:cluster/terrastack-cluster to /root/.kube/config</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YylgVWxlQ-UwXmO0NHfcJg.png" /></figure><p>Now the moment of truth:</p><pre>kubectl get nodes</pre><p>Output:</p><pre>NAME STATUS ROLES AGE VERSION<br>ip-10–0–11–109.ec2.internal Ready &lt;none&gt; 2m10s v1.31.2-eks-ecaa3a6<br>ip-10–0–12–69.ec2.internal  Ready &lt;none&gt; 2m5s  v1.31.2-eks-ecaa3a6</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RftuuM3tg5heEJcwpqgPTQ.png" /></figure><p>TWO NODES! BOTH READY!</p><p>I just deployed a Kubernetes cluster on AWS. With code. From scratch.</p><pre>kubectl get pods -n kube-system</pre><p>Output:</p><pre>NAME                       READY   STATUS    RESTARTS   AGE<br>aws-node-mb7rb             2/2     Running   0          19m<br>aws-node-thz9r             2/2     Running   0          18m<br>coredns-789f8477df-ph4pk   1/1     Running   0          22m<br>coredns-789f8477df-zw8ld   1/1     Running   0          22m<br>kube-proxy-r8bqp           1/1     Running   0          18m<br>kube-proxy-x6jsk           1/1     Running   0          19m</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vqW0g-x_346FWuPwi5ItUQ.png" /></figure><p>All system pods running!</p><p>The cluster is healthy!</p><p>Testing the Cluster</p><pre>kubectl get deployment coredns -n kube-system<br><br>kubectl cluster-info<br><br>kubectl get nodes -o wide</pre><p>Output:</p><pre>NAME      READY   UP-TO-DATE   AVAILABLE   AGE<br>coredns   2/2     2            2           20m<br><br><br>Kubernetes control plane is running at https://1D02652FFF8A411E85600BB4840136B8.gr7.us-east-1.eks.amazonaws.com<br>CoreDNS is running at https://1D02652FFF8A411E85600BB4840136B8.gr7.us-east-1.eks.amazonaws.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy<br><br>To further debug and diagnose cluster problems, use &#39;kubectl cluster-info dump&#39;.<br><br><br>NAME                            STATUS   ROLES    AGE   VERSION            INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION                  CONTAINER-RUNTIME<br>ip-10-0-11-109.ec2.internal     Ready    &lt;none&gt;   18m   v1.31.13-eks-ecaa3a6   10.0.11.109    &lt;none&gt;        Amazon Linux 2023.9.20251117     6.1.158-178.288.amzn2023.x86_64   containerd://2.1.4<br>ip-10-0-12-69.ec2.internal      Ready    &lt;none&gt;   18m   v1.31.13-eks-ecaa3a6   10.0.12.69     &lt;none&gt;        Amazon Linux 2023.9.20251117     6.1.158-178.288.amzn2023.x86_64   containerd://2.1.4</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p8_hAoWF12GI5KcEfboI1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bTdj1n7gp086YgOfndCxWA.png" /></figure><p>MY FIRST REAL POD ON EKS!</p><p>It works. Kubernetes is running. On AWS. Created with Terraform.</p><p>The Cost Update</p><p>I checked AWS Cost Explorer.</p><p>New charges:</p><ul><li>EKS Control Plane: $2.42/day = $73/month (prorated)</li><li>EC2 (2× t3.medium): $2/day = $60/month</li><li>NAT Gateways: Still $2.17/day = $65/month</li><li>EBS Volumes: $0.33/day = $10/month</li></ul><p>New total: ~$208/month</p><p>Getting expensive. But this is the core infrastructure. Everything else (RDS, monitoring) adds to this.</p><p>What I Learned</p><p>1. IAM roles are like security badges — different services need different permissions<br>2. EKS manages the control plane — worth the $73/month to not manage it myself<br>3. Security groups control network traffic — least privilege principle<br>4. OIDC provider enables pod-level IAM roles — important for production security<br>5. Node groups should be in private subnets — public IPs are a security risk<br>6. EKS takes 17 minutes to deploy— cluster creation is slow, be patient<br>7. Three add-ons are essential — CoreDNS, kube-proxy, VPC CNI<br>8. Current cost: $208/month (VPC + EKS, before database)</p><p>Coming Up in Part 4</p><p>The VPC is done. The Kubernetes cluster is running. Now I need a database.</p><p>RDS (Relational Database Service) is AWS’s managed PostgreSQL. It needs to be:</p><ul><li>In private subnets (no public access!)</li><li>Connected to EKS (security groups)</li><li>Credentials in Secrets Manager (no hardcoded passwords)</li></ul><p>And here’s where I’ll learn why PostgreSQL 16.3 doesn’t exist in AWS.</p><p>See you Friday.</p><p>💡 KEY TAKEAWAYS</p><p>1. EKS control plane: $73/month (AWS manages masters, worth it)<br>2. IAM roles: One for cluster, one for nodes (different permissions)<br>3. OIDC provider: Enables IAM roles for individual pods (IRSA)<br>4. Security groups: Cluster SG + Node SG (control traffic)<br>5. Nodes in private subnets only (security best practice)<br>6. Three add-ons required: CoreDNS, kube-proxy, VPC CNI<br>7. Deployment time: ~17 minutes (12 min cluster + 5 min nodes)<br>8. Current cost: $208/month (VPC + EKS running)<br>9. kubectl works! Test pod deployed successfully<br>10. Next up: Database (RDS) with proper security</p><p><strong>Connect &amp; Continue</strong></p><p><em>Part 4 drops Friday</em>!</p><ul><li>🔗<strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>📧 <strong><em>Email</em></strong>: benjaminLburton@outlook.com</li></ul><p>#EKS #Kubernetes #AWS #IAM #SecurityGroups #IRSA #RealLearning</p><p><strong>About the Author</strong>:</p><p>La’vell Burton a DevOps engineer and AWS Community Builder (Containers category) currently working as a delivery driver while building a portfolio to break into tech. This series documents a real production-grade project built in a single intensive day of failures, troubleshooting, and all.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5584933de31c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TerraStack Part 2: The State File Problem I Didn’t Know I Had]]></title>
            <link>https://mrburton.medium.com/terrastack-part-2-the-state-file-problem-i-didnt-know-i-had-622c63d71e88?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/622c63d71e88</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Tue, 16 Dec 2025 03:08:25 GMT</pubDate>
            <atom:updated>2025-12-16T03:48:37.544Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LuRUuHKDMp5_GkQPeX9soQ.png" /></figure><p>Four Git errors. Three credential mishaps. One bootstrap paradox. The state management chapter of building AWS infrastructure from absolute scratch — and why remote backends matter.</p><p>Quick Links to Series:</p><p>✅ <a href="https://mrburton.medium.com/terrastack-part-1-why-click-ops-is-a-trap-and-how-i-escaped-it-e5ae55496ec7">Part 1: “Why Click-Ops is a Trap (And How I Escaped It)”</a><br>✅ Part 2: “The State File Problem I Didn’t Know I Had” ← You are here<br>✅ Part 3: “EKS and the IAM Roles I Didn’t Understand” ← Up next<br>✅ Part 4: “The Database That Taught Me About Secrets” <br>✅ Part 5: “GitHub Actions and the Credentials That Disappeared”<br>✅ Part 6: “The Real Cost and What I Actually Learned”</p><p>🔗 Full Project Code: <a href="https://github.com/BenjaminBurton/terrastack">https://github.com/BenjaminBurton/terrastack</a></p><p><strong>Day 1</strong>: Setting Up Tools (The Boring Part That Matters)</p><p>I started with the basics: getting my development environment set up.</p><p><strong><em>What I needed</em></strong>:</p><ul><li>Terraform 1.9.8 (the latest stable version at the time)</li><li>AWS CLI (to talk to AWS from command line)</li><li>kubectl (for Kubernetes later)</li><li>Git (for version control)</li></ul><p><strong><em>The problem</em></strong>: I’m on a MacBook Air with 8GB RAM. Kubernetes needs 5–8GB minimum. My laptop can’t run everything I need.</p><p><strong><em>The solution</em></strong>: Remote development.</p><p>I have an old Lenovo ThinkPad sitting in my room (32GB RAM, 6-core i7). It’s running Arch Linux because I followed a YouTube tutorial once and now I’m stuck with it. But it has the power I need.</p><p>So my setup became:</p><ul><li>MacBook: Where I write code and run VS Code</li><li>Lenovo: Where everything actually runs (accessed via SSH)</li><li>DevContainer: Runs on the Lenovo, contains all my tools</li></ul><p>This is how real developers work anyway. Use whatever hardware you have.</p><p><strong><em>Problem #1</em></strong>: DevContainer Confusion</p><p>I created a DevContainer configuration file:</p><pre>{<br> &quot;name&quot;: &quot;TerraStack DevContainer&quot;,<br> &quot;image&quot;: &quot;ubuntu:22.04&quot;,<br> &quot;postCreateCommand&quot;: &quot;bash .devcontainer/post-create.sh&quot;<br>}</pre><p>The `post-create.sh` script was supposed to install everything:</p><pre>#!/bin/bash<br># Install Terraform<br>TERRAFORM_VERSION=&quot;1.9.8&quot;<br>wget https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/...</pre><p><strong><em>My question</em></strong>: Where does the Makefile get created?</p><p><strong><em>The script has this line</em></strong>:</p><pre>cat &gt; /workspaces/terrastack/Makefile &lt;&lt; &#39;MAKEFILE&#39;<br>…<br>MAKEFILE</pre><p>So it creates `/workspaces/terrastack/Makefile` INSIDE the container. Not on my host machine.</p><p>This confused me for 20 minutes. I kept looking for the Makefile in my local directory. It wasn’t there. Because it only exists inside the DevContainer.</p><p><strong><em>Lesson learned</em></strong>: DevContainers create their own filesystem. Files created inside stay inside unless you explicitly copy them out.</p><p><strong><em>Problem #2</em></strong>: Git Identity in DevContainer</p><p><strong><em>I initialized Git</em></strong>:</p><pre>git init</pre><p>Created my first file (`.gitignore`), and tried to commit:</p><pre>git add .gitignore<br>git commit -m &quot;chore: initialize project&quot;</pre><p><strong><em>Error</em></strong>:</p><pre>Author identity unknown<br>*** Please tell me who you are.<br>Run<br> git config - global user.name &quot;Your Name&quot;<br> git config - global user.email &quot;you@example.com&quot;</pre><p><strong><em>Why this happened</em></strong>: The DevContainer is a fresh Ubuntu environment. It doesn’t know my Git identity.</p><p><strong><em>The fix</em></strong>:</p><pre>git config - global user.name &quot;Benjamin Burton&quot;<br>git config - global user.email &quot;benjaminlburton@gmail.com&quot;</pre><p>Now commits work.</p><p><strong><em>Lesson learned</em></strong>: DevContainers don’t inherit your host machine’s Git configuration. Set it up once inside the container.</p><p><strong><em>Problem #3</em></strong>: Git Ownership Issues</p><p>I tried to add files to Git later:</p><pre>git add scripts/bootstrap-backend.sh</pre><p><strong><em>Error</em></strong>:</p><pre>fatal: detected dubious ownership in repository at &#39;/workspaces/terrastack&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*072DxzY8Y9VYl2sRYui_qg.png" /></figure><p><strong><em>What this means</em></strong>: The DevContainer runs as `root`. The Git repository was created by my host user (`archv`). Git sees this as suspicious (security feature).</p><p><strong><em>The fix</em></strong>:</p><pre>git config - global - add safe.directory /workspaces/terrastack</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JgHJLqcgy-aWXcv_L973QQ.png" /></figure><p><strong><em>This tells Git</em></strong>: “Yes, I trust this directory even though the ownership looks weird.”</p><p><strong><em>Lesson learned</em></strong>: When using DevContainers, you’ll hit permission/ownership issues. This command fixes it.</p><p>The Terraform State Problem (The Big One)</p><p>Now I was ready to actually write Terraform code. But first, I needed to understand something I’d been ignoring:</p><p>Where does Terraform store what it created?</p><p>When you run `terraform apply`, Terraform creates resources in AWS. But it also creates a file called `terraform.tfstate` that tracks everything it created.</p><p><strong><em>The state file contains</em></strong>:</p><ul><li>Resource IDs (vpc-123abc, sg-456def, etc.)</li><li>Resource attributes (IP addresses, passwords, etc.)</li><li>Dependency relationships (this resource depends on that resource)</li></ul><p><strong><em>Why this matters</em></strong>: Terraform uses the state file to know what exists. Without it:</p><ul><li>Running `terraform apply` again creates duplicates</li><li>Running `terraform destroy` does nothing (Terraform thinks nothing exists)</li><li>Collaborating with others becomes impossible (they don’t see your resources)</li></ul><p><strong><em>The default</em></strong>: Terraform stores state locally in a file called `terraform.tfstate`.</p><p><strong><em>The problem</em></strong>: This doesn’t work for teams. Or even for me working from multiple machines.</p><p>The “Chicken and Egg” Problem</p><p>The industry best practice is to store Terraform state in S3 (AWS’s object storage).</p><p><strong>Here’s the problem</strong>:<br><strong><em>Goal</em></strong>: Store Terraform state in S3 bucket</p><p>But…</p><p>1. I need an S3 bucket to store state<br>2. I need Terraform to create the S3 bucket<br>3. But Terraform needs somewhere to store its state<br>4. Which should be in S3<br>5. Which I need to create with Terraform<br>6. Which needs state storage<br>7. Which should be in S3…</p><p>head explodes</p><p>This is a real problem. Every Terraform project faces it.</p><p>The Bootstrap Solution</p><p><strong><em>The answer</em></strong>: Use AWS CLI to create the S3 bucket first.</p><p><strong><em>This breaks the cycle</em></strong>:</p><p>1. Use AWS CLI (not Terraform) to create S3 bucket<br>2. Use AWS CLI to create DynamoDB table (for state locking)<br>3. Configure Terraform to use that S3 bucket<br>4. Now Terraform has a place to store state</p><p>This is called “bootstrapping the backend.”</p><p><strong><em>I created a script</em></strong>:</p><pre>#!/bin/bash<br>set -e<br>BUCKET_NAME=&quot;terrastack-tfstate-$(date +%s)&quot;<br>DYNAMODB_TABLE=&quot;terrastack-locks&quot;<br>REGION=&quot;us-east-1&quot;<br># Create S3 bucket<br>aws s3api create-bucket \<br> - bucket $BUCKET_NAME \<br> - region $REGION<br># Enable versioning (so I can roll back if needed)<br>aws s3api put-bucket-versioning \<br> - bucket $BUCKET_NAME \<br> - versioning-configuration Status=Enabled<br># Enable encryption<br>aws s3api put-bucket-encryption \<br> - bucket $BUCKET_NAME \<br> - server-side-encryption-configuration &#39;{<br> &quot;Rules&quot;: [{<br> &quot;ApplyServerSideEncryptionByDefault&quot;: {<br> &quot;SSEAlgorithm&quot;: &quot;AES256&quot;<br> }<br> }]<br> }&#39;<br># Block public access (VERY IMPORTANT)<br>aws s3api put-public-access-block \<br> - bucket $BUCKET_NAME \<br> - public-access-block-configuration \<br> BlockPublicAcls=true,\<br> IgnorePublicAcls=true,\<br> BlockPublicPolicy=true,\<br> RestrictPublicBuckets=true<br># Create DynamoDB table for state locking<br>aws dynamodb create-table \<br> - table-name $DYNAMODB_TABLE \<br> - attribute-definitions AttributeName=LockID,AttributeType=S \<br> - key-schema AttributeName=LockID,KeyType=HASH \<br> - billing-mode PAY_PER_REQUEST \<br> - region $REGION<br>echo &quot;✅ Backend bootstrap complete!&quot;<br>echo &quot;Bucket name: $BUCKET_NAME&quot;</pre><p><strong><em>What this does</em></strong>:</p><ul><li>Creates S3 bucket with unique name (timestamp ensures uniqueness)</li><li>Enables versioning (so I can recover old state files)</li><li>Encrypts the bucket (state files contain secrets!)</li><li>Blocks ALL public access (state files should NEVER be public)</li><li>Creates DynamoDB table (prevents two people from running `terraform apply` simultaneously)</li></ul><p><strong><em>I ran it</em></strong>:</p><pre>chmod +x scripts/bootstrap-backend.sh<br>./scripts/bootstrap-backend.sh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZjeLOsf2p_BpDcRoBQKsJg.png" /></figure><p><strong><em>Output</em></strong>:</p><pre>✅ Backend bootstrap complete!<br>Bucket name: terrastack-tfstate-1765377369</pre><p>I saved that bucket name. I’ll need it in a minute.</p><p><strong><em>Problem #4</em></strong>: Where Are My AWS Credentials?</p><p>To run that script, I needed AWS credentials configured.</p><p>I went to AWS Console → IAM → My Security Credentials → Create Access Key.</p><p><strong><em>AWS showed me</em></strong>:</p><pre>Access Key ID: AKIAIOSFODNN7EXAMPLE<br>Secret Access Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY<br>⚠️ This is your only chance to save these credentials!</pre><p>I started copying them…</p><p>And my browser refreshed.</p><p>The credentials were gone. AWS doesn’t show them again (security feature).</p><p>I sat there for a minute thinking: “Did I just lock myself out?”</p><p>The fix: I had to create ANOTHER access key. Deleted the first one (it was useless now), created a new one, and this time I copied them to a text file immediately before doing anything else.</p><p>Lesson learned: When AWS says “save these now,” they mean it. Copy first, click later.</p><p><strong><em>I configured AWS CLI</em></strong>:</p><pre>aws configure</pre><p><strong><em>Entered</em></strong>:</p><ul><li>Access Key ID: [pasted]</li><li>Secret Access Key: [pasted]</li><li>Region: us-east-1</li><li>Output format: json</li></ul><p><strong><em>Verified it worked</em></strong>:</p><pre>aws sts get-caller-identity</pre><p><strong><em>Output</em></strong>:</p><pre>{<br> &quot;UserId&quot;: &quot;AIDAIOSFODNN7EXAMPLE&quot;,<br> &quot;Account&quot;: &quot;123456789012&quot;,<br> &quot;Arn&quot;: &quot;arn:aws:iam::123456789012:user/benjamin&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pr3iU0o3WkPXb5R_vZ8_Ww.png" /></figure><p><strong><em>Working</em></strong>!</p><p>Configuring Terraform to Use S3 Backend</p><p>Now I created `environments/dev/backend.tf`:</p><pre>terraform {<br> backend &quot;s3&quot; {<br> bucket = &quot;terrastack-tfstate-1733702400&quot; # MY bucket name<br> key = &quot;dev/terraform.tfstate&quot;<br> region = &quot;us-east-1&quot;<br> dynamodb_table = &quot;terrastack-locks&quot;<br> encrypt = true<br> }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qUN1zzeQ9w3-qHva2EFPhQ.png" /></figure><p>What each setting means:</p><ul><li>bucket: The S3 bucket I just created</li><li>key: Path within the bucket (allows multiple environments: dev/, staging/, prod/)</li><li>region: Where the bucket lives</li><li>dynamodb_table: Prevents concurrent applies</li><li>encrypt: Encrypts the state file (contains secrets!)</li></ul><p>Building the VPC Module</p><p>Now I could finally write actual infrastructure code.</p><p>I created `modules/vpc/main.tf`:</p><pre>resource &quot;aws_vpc&quot; &quot;main&quot; {<br> cidr_block = var.vpc_cidr<br> enable_dns_hostnames = true<br> enable_dns_support = true<br> <br> tags = merge(<br> var.tags,<br> {<br> Name = &quot;${var.project_name}-vpc&quot;<br> }<br> )<br>}</pre><p>My first question: What’s `merge`?</p><p>I Googled it. It combines two maps (dictionaries) into one. So if `var.tags` is:</p><pre>{<br> Environment = &quot;dev&quot;<br>}</pre><p>And I merge it with:</p><pre>{<br> Name = &quot;terrastack-vpc&quot;<br>}</pre><p>The result is:</p><pre>{<br> Environment = &quot;dev&quot;<br> Name = &quot;terrastack-vpc&quot;<br>}</pre><p>Makes sense.</p><p>My second question: Why use variables instead of hardcoding values?</p><p>Because then I can reuse this module. Dev environment gets `cidr_block = “10.0.0.0/16”`. Staging gets `cidr_block = “10.1.0.0/16”`. Same code, different values.</p><p><strong>The Subnet Confusion</strong></p><p><strong><em>I added subnets</em></strong>:</p><pre>resource &quot;aws_subnet&quot; &quot;public&quot; {<br> count = length(var.public_subnet_cidrs)<br> <br> vpc_id = aws_vpc.main.id<br> cidr_block = var.public_subnet_cidrs[count.index]<br> availability_zone = var.availability_zones[count.index]<br> <br> tags = {<br> Name = &quot;${var.project_name}-public-${var.availability_zones[count.index]}&quot;<br> }<br>}</pre><p>I stared at this for 10 minutes.</p><p>`count = length(var.public_subnet_cidrs)`… so if I pass in 2 subnet CIDRs, this creates 2 subnets?</p><p>And `count.index` is… the loop counter? So first loop is 0, second is 1?</p><p>I tested it. Created `environments/dev/main.tf`:</p><pre>module &quot;vpc&quot; {<br> source = &quot;../../modules/vpc&quot;<br> <br> project_name = &quot;terrastack&quot;<br> vpc_cidr = &quot;10.0.0.0/16&quot;<br> <br> availability_zones = [<br> &quot;us-east-1a&quot;,<br> &quot;us-east-1b&quot;<br> ]<br> <br> public_subnet_cidrs = [<br> &quot;10.0.1.0/24&quot;,<br> &quot;10.0.2.0/24&quot;<br> ]<br> <br> private_subnet_cidrs = [<br> &quot;10.0.11.0/24&quot;,<br> &quot;10.0.12.0/24&quot;<br> ]<br> <br> tags = {<br> Environment = &quot;dev&quot;<br> }<br>}</pre><p><strong><em>So this should create</em></strong>:</p><ul><li>2 public subnets (10.0.1.0/24 in us-east-1a, 10.0.2.0/24 in us-east-1b)</li><li>2 private subnets (10.0.11.0/24 in us-east-1a, 10.0.12.0/24 in us-east-1b)</li></ul><p>Let me test it.</p><p>The First `terraform plan`</p><pre>cd environments/dev<br>terraform init</pre><p><strong><em>Output</em></strong>:</p><pre>Initializing the backend…<br>Successfully configured the backend &quot;s3&quot;! Terraform will automatically<br>use this backend unless the backend configuration changes.<br>Initializing provider plugins…<br>- Finding hashicorp/aws versions matching &quot;~&gt; 5.0&quot;…<br>- Installing hashicorp/aws v5.31.0…<br>Terraform has been successfully initialized!</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9QF4ewQ0accJcD6UD1_Ekw.png" /></figure><p>It found my S3 backend!</p><p><strong><em>Now the plan</em></strong>:</p><pre>terraform plan</pre><p><strong><em>Output</em></strong>:</p><pre>Terraform will perform the following actions:<br># module.vpc.aws_vpc.main will be created<br> + resource &quot;aws_vpc&quot; &quot;main&quot; {<br> + cidr_block = &quot;10.0.0.0/16&quot;<br> + id = (known after apply)<br> }<br># module.vpc.aws_internet_gateway.main will be created<br> # module.vpc.aws_subnet.public[0] will be created<br> # module.vpc.aws_subnet.public[1] will be created<br> # module.vpc.aws_subnet.private[0] will be created<br> # module.vpc.aws_subnet.private[1] will be created<br> # module.vpc.aws_eip.nat[0] will be created<br> # module.vpc.aws_eip.nat[1] will be created<br> # module.vpc.aws_nat_gateway.main[0] will be created<br> # module.vpc.aws_nat_gateway.main[1] will be created<br> # … (route tables, associations)<br>Plan: 17 to add, 0 to change, 0 to destroy.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ReAARyGx9v29TXD0-VB2mA.png" /></figure><p>17 resources! From like 100 lines of code!</p><p><strong><em>I double-checked</em></strong>:</p><ul><li>1 VPC</li><li>2 public subnets</li><li>2 private subnets</li><li>1 Internet Gateway</li><li>2 NAT Gateways</li><li>Route tables</li></ul><p>Everything looks right.</p><p>Deep breath.</p><pre>terraform apply tfplan</pre><p>Watching Infrastructure Get Created</p><pre>module.vpc.aws_vpc.main: Creating…<br>module.vpc.aws_vpc.main: Creation complete after 3s [id=vpc-0a1b2c3d]<br>module.vpc.aws_internet_gateway.main: Creating…<br>module.vpc.aws_subnet.public[0]: Creating…<br>module.vpc.aws_subnet.public[1]: Creating…<br>module.vpc.aws_subnet.private[0]: Creating…<br>module.vpc.aws_subnet.private[1]: Creating…<br>module.vpc.aws_eip.nat[0]: Creating…<br>module.vpc.aws_eip.nat[1]: Creating…<br>module.vpc.aws_nat_gateway.main[0]: Creating…<br>module.vpc.aws_nat_gateway.main[1]: Creating…<br>module.vpc.aws_nat_gateway.main[0]: Still creating… [1m30s elapsed]<br>module.vpc.aws_nat_gateway.main[1]: Still creating… [1m30s elapsed]</pre><p>NAT Gateways take FOREVER. Like 5–7 minutes.</p><p>I went to get some juice. Came back.</p><pre>module.vpc.aws_nat_gateway.main[0]: Creation complete after 5m42s<br>module.vpc.aws_nat_gateway.main[1]: Creation complete after 5m46s<br>Apply complete! Resources: 15 added, 0 changed, 0 destroyed.<br>Outputs:<br>nat_gateway_ips = [<br> &quot;54.123.45.67&quot;,<br> &quot;54.123.45.68&quot;,<br>]<br>vpc_id = &quot;vpc-0a1b2c3d4e5f6&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2LMp3-mqq3QqNnoa8wB36g.png" /></figure><p>I JUST CREATED A VPC WITH CODE.</p><p>No clicking. No screenshots to follow. Just:</p><p>1. Wrote code<br>2. Ran `terraform apply`<br>3. Infrastructure appeared</p><p>This is the power of Infrastructure as Code.</p><p>Verification</p><p>I checked AWS Console:</p><p><strong><em>VPC Dashboard</em></strong>:</p><ul><li>1 VPC (10.0.0.0/16)</li><li>4 Subnets (2 public, 2 private)</li><li>1 Internet Gateway</li><li>2 NAT Gateways (with Elastic IPs)</li><li>3 Route Tables (1 public, 2 private)</li></ul><p>Everything I expected. Everything created exactly as defined in code.</p><p>And the state file?</p><pre>aws s3 ls s3://terrastack-tfstate-1733702400/dev/</pre><p><strong><em>Output</em></strong>:</p><pre>2024–12–20 10:45:23 12847 terraform.tfstate</pre><p>✅ State is in S3!</p><p>If my laptop dies, if I switch machines, if someone else clones this repo — they can all see what exists because the state is in S3, not on my local disk.</p><p>The Cost Reality</p><p>I checked AWS Cost Explorer the next day.</p><p><strong><em>Charges so far</em></strong>:</p><ul><li>NAT Gateways: $2.17 (for ~15 hours of uptime)</li><li>EC2 (Elastic IPs): $0.00 (attached to NAT, so free)</li><li>S3: $0.00 (negligible)</li><li>DynamoDB: $0.00 (free tier)</li></ul><p>Projected monthly cost at this point: ~$70</p><p>Mostly NAT Gateways ($32.50/month each × 2 = $65/month).</p><p><strong><em>Note to self</em></strong>: NAT Gateways are expensive. I understand why companies use a single NAT in dev environments now.</p><p><strong>What I Learned</strong></p><p>1. DevContainers are weird at first but solve real problems (reproducibility)<br>2. Git configuration doesn’t transfer to DevContainers (set it up inside)<br>3. Terraform state is critical and local state doesn’t work for real projects<br>4. The S3 backend bootstrap is a one-time exception to “infrastructure as code”<br>5. count and count.index let you create multiple similar resources without copy-paste<br>6. NAT Gateways are expensive ($65/month for 2, most expensive part of VPC)<br>7. Terraform plan shows you everything before creating (use it!)</p><p><strong>Coming Up in Part 3</strong></p><p>The VPC is done. Now I need to add a Kubernetes cluster.</p><p>EKS (Elastic Kubernetes Service) is AWS’s managed Kubernetes. It costs $73/month just for the control plane (before any worker nodes). And it requires:</p><ul><li>IAM roles (I barely know what these are)</li><li>Security groups (network firewall rules)</li><li>OIDC provider (for pod-level permissions)</li><li>Add-ons (CoreDNS, kube-proxy, VPC CNI)</li></ul><p>This is where it gets complicated.</p><p>See you Wednesday.</p><p>💡 <strong><em>KEY TAKEAWAYS</em></strong></p><p>1. DevContainers solve “works on my machine” but have quirks (Git config, ownership)<br>2. Terraform state must be remote (S3) for any real project<br>3. Bootstrap paradox: Use AWS CLI to create S3 bucket first<br>4. DynamoDB table prevents concurrent terraform applies (critical for teams)<br>5. AWS credential mishaps happen (copy immediately, verify with `aws sts`)<br>6. Terraform `count` creates multiple resources from one definition<br>7. NAT Gateways: $65/month, most expensive part of VPC<br>8. Current cost: ~$70/month (just VPC)<br>9. `terraform plan` before `apply` — always verify what will be created<br>10. First success: VPC with 15 resources created from ~100 lines of code</p><p><strong>Connect &amp; Continue</strong></p><p><em>Part 3 drops Friday</em>! The networking deep dive where we discover that just because something is “running” doesn’t mean you can connect to it.</p><ul><li>🔗<strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>📧 <strong><em>Email</em></strong>: benjaminLburton@outlook.com</li></ul><p>#Terraform #AWS #VPC #DevContainers #InfrastructureAsCode #RealLearning #Mistakes</p><p><strong>About the Author</strong>:</p><p>La’vell Burton a DevOps engineer and AWS Community Builder (Containers category) currently working as a delivery driver while building a portfolio to break into tech. This series documents a real production-grade project built in a single intensive day of failures, troubleshooting, and all.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=622c63d71e88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeployFlow Part 4: From Git Push to Green Pipeline-Github Actions and Production Lessons]]></title>
            <link>https://mrburton.medium.com/deployflow-part-4-from-git-push-to-green-pipeline-github-actions-and-production-lessons-0dcc8a11b31c?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/0dcc8a11b31c</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Mon, 15 Dec 2025 13:05:17 GMT</pubDate>
            <atom:updated>2025-12-15T13:05:17.479Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZkH5HtGtn3AVn7xBJxUjJw.png" /></figure><p><em>Six CI/CD failures. Seven authentication attempts. One production-ready pipeline. The final chapter of building Kubernetes infrastructure from absolute scratch — and what it all means.</em></p><p><strong>Part 4</strong>: From Git Push to Green Pipeline: GitHub Actions and Production Lessons</p><p><strong>Quick Links to Series</strong>:</p><p>✅ <a href="https://mrburton.medium.com/part-1-building-production-kubernetes-on-a-budget-the-setup-journey-f8697becc5db">Part 1: The Setup Journey (SSH, Docker, DevContainer CLI)</a><br>✅ <a href="https://medium.com/@mrburton/deployflow-part-2-devcontainers-from-scratch-manual-configuration-and-architecture-nightmares-b5f653ea4dac">Part 2: Manual Configuration &amp; Architecture Nightmares (Files, ARM64 debugging)</a><br>✅ <a href="https://mrburton.medium.com/deployflow-part-3-kubernetes-networking-deep-dive-when-kubectl-refuses-to-connect-f88af97e87be">Part 3: Kubernetes Networking Deep Dive (kubectl, k3d, microservices)</a><br>📚 Part 4: CI/CD &amp; Production Lessons ← You are here</p><p>🔗 Full Project Code: <a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></p><p><strong><em>Recap: The Journey So Far</em></strong></p><p>Over three articles, I’ve documented building production Kubernetes infrastructure from scratch:</p><p><strong>Part 1</strong>: Set up SSH access, installed Docker, configured DevContainer CLI <br><strong>Part 2</strong>: Created 14 files manually, debugged ARM64 vs x86_64 binaries <br><strong>Part 3</strong>: Fought Docker networking for 90 minutes, deployed microservices</p><p><strong><em>Now it’s time for the finale</em></strong>: pushing to GitHub and implementing CI/CD.</p><p><strong><em>Current state</em></strong>:</p><ul><li>✅ 4 pods running (2 Node.js + 2 Go)</li><li>✅ Services communicating via Kubernetes DNS</li><li>✅ Health checks passing</li><li>✅ Everything works locally</li></ul><p><strong><em>What’s missing</em></strong>: Version control and automated pipeline.</p><p>A portfolio project isn’t real until it’s on GitHub.</p><p><strong><em>Git Configuration</em></strong>: The Root User Problem</p><p>Inside the DevContainer:</p><pre>git init<br>git status</pre><p>Error:</p><pre>fatal: detected dubious ownership in repository at &#39;/workspaces/deployflow&#39;<br>To add an exception for this directory, call:<br> git config - global - add safe.directory /workspaces/deployflow</pre><p>The problem: I’m running as root in the DevContainer, but the files are owned by my host user (UID 1000). Git sees this ownership mismatch as a potential security risk.</p><p>The fix:</p><pre># Add safe directory exception<br>git config - global - add safe.directory /workspaces/deployflow<br><br># Configure default branch name<br>git config - global init.defaultBranch main<br><br># Set user identity<br>git config - global user.name &quot;Benjamin Burton&quot;<br>git config - global user.email &quot;benjaminlburton@gmail.com&quot;<br><br># Reinitialize with correct configuration<br>rm -rf .git<br>git init</pre><p>Output:</p><pre>Initialized empty Git repository in /workspaces/deployflow/.git/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ie14B3d3vWMKe_7EDtjzgw.png" /></figure><p>The Initial Commit</p><pre># Stage all files<br>git add .<br><br># Commit<br>git commit -m &quot;Initial commit: DeployFlow CI/CD project&quot;</pre><p>Output:</p><pre>[main (root-commit) 42153af] Initial commit: DeployFlow CI/CD project<br> 14 files changed, 523 insertions(+)<br> create mode 100644 .devcontainer/devcontainer.json<br> create mode 100755 .devcontainer/post-create.sh<br> create mode 100644 .github/workflows/ci-cd.yml<br> create mode 100644 README.md<br> create mode 100644 go-service/Dockerfile<br> create mode 100644 go-service/go.mod<br> create mode 100644 go-service/k8s/deployment.yaml<br> create mode 100644 go-service/k8s/service.yaml<br> create mode 100644 go-service/main.go<br> create mode 100644 node-service/Dockerfile<br> create mode 100644 node-service/k8s/deployment.yaml<br> create mode 100644 node-service/k8s/service.yaml<br> create mode 100644 node-service/package.json<br> create mode 100644 node-service/server.js</pre><p>523 lines of code, 14 files, all created manually.</p><p>Verify the commit:</p><pre>git log</pre><p>Output:</p><pre>commit 42153af3d552ff298e25fdcd2a46d2bbe1d1fcc5 (HEAD -&gt; main)<br>Author: Benjamin Burton &lt;benjaminlburton@gmail.com&gt;<br>Date: Wed Dec 3 23:43:30 2025 +0000<br>Initial commit: DeployFlow CI/CD project</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vsXWYlPRZWUVEzWUD9u8pg.png" /></figure><p>Perfect. Now to push it to GitHub.</p><p>Creating the GitHub Repository</p><p>On my Mac browser:</p><p>1. Navigate to <a href="https://github.com/new">https://github.com/new</a><br>2. Repository name: `deployflow`<br>3. Description: “Production-ready CI/CD pipeline with Node.js and Go microservices on Kubernetes”<br>4. Visibility: Public (portfolio visibility is critical)<br>5. Do NOT check “Initialize with README” (I already have one)<br>6. Do NOT add .gitignore (not needed)<br>7. Do NOT add license (can add later)<br>8. Click “Create repository”</p><p>Repository created: `<a href="https://github.com/BenjaminBurton/deployflow%60">https://github.com/BenjaminBurton/deployflow`</a></p><p>Push Attempt #1: The Password Awakening</p><p>Back in the DevContainer:</p><pre># Add remote<br>git remote add origin https://github.com/BenjaminBurton/deployflow.git<br># Verify remote<br>git remote -v</pre><p>Output:</p><pre>origin https://github.com/BenjaminBurton/deployflow.git (fetch)<br>origin https://github.com/BenjaminBurton/deployflow.git (push)</pre><p>Time to push:</p><pre>git push -u origin main</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ipli7YP576t39vUBaIiNJg.png" /></figure><p>Prompt:</p><pre>Username for &#39;https://github.com&#39;: BenjaminBurton<br>Password for &#39;https://BenjaminBurton@github.com&#39;:</pre><p>I entered my GitHub password.</p><p>Error:</p><pre>remote: Invalid username or token. <br>Password authentication is not supported for Git operations.<br>fatal: Authentication failed for &#39;https://github.com/BenjaminBurton/deployflow.git/&#39;</pre><p>What?</p><p>The Authentication Reality</p><p>GitHub deprecated password authentication on August 13, 2021.</p><p>You can’t use your account password anymore. You need either:</p><p>1. Personal Access Token (PAT)<br>2. SSH keys</p><p>I chose PAT for simplicity.</p><p>Push Attempt #2: Token Without Scopes</p><p>Creating a Personal Access Token:</p><p>On GitHub:</p><p>1. Settings → Developer settings → Personal access tokens<br>2. “Generate new token (classic)”<br>3. Note: “DeployFlow Project”<br>4. Expiration: 90 days<br>5. Scopes: I checked… nothing (This was a mistake)<br>6. Generate token<br>7. Copy token: `ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`</p><p>Back in DevContainer:</p><pre>git push -u origin main<br># Username: BenjaminBurton<br># Password: [pasted token]</pre><p>Error:</p><pre>remote: Permission to BenjaminBurton/deployflow.git denied to BenjaminBurton.<br>fatal: unable to access &#39;https://github.com/BenjaminBurton/deployflow.git/&#39;: <br> The requested URL returned error: 403</pre><p>Permission denied. The token has no permissions because I didn’t check any scopes.</p><p>Push Attempt #3: Token With repo Scope</p><p>Creating a new token:</p><p>1. Delete previous token<br>2. Generate new token (classic)<br>3. Note: “DeployFlow Project”<br>4. Expiration: 90 days<br>5. Scopes: repo (full control of private repositories)<br>6. Generate token<br>7. Copy new token</p><p>Push again:</p><pre>git push -u origin main<br># Username: BenjaminBurton<br># Password: [pasted new token]</pre><p>Output:</p><pre>Enumerating objects: 5960, done.<br>Counting objects: 100% (5960/5960), done.<br>Delta compression using up to 8 threads<br>Compressing objects: 100% (5704/5704), done.<br>Writing objects: 100% (5960/5960), 7.10 MiB | 556.00 KiB/s, done.<br>Total 5960 (delta 1193), reused 0 (delta 0), pack-reused 0<br>remote: Resolving deltas: 100% (1193/1193), done.</pre><p>Progress! Objects are uploading…</p><p>But then:</p><pre>To https://github.com/BenjaminBurton/deployflow.git<br> ! [remote rejected] main -&gt; main (refusing to allow a Personal Access Token <br> to create or update workflow `.github/workflows/ci-cd.yml` without <br> `workflow` scope)<br>error: failed to push some refs to &#39;https://github.com/BenjaminBurton/deployflow.git&#39;</pre><p>New error! The code uploaded, but GitHub rejected the workflow file.</p><p>Why This Happened</p><p>GitHub has granular permission scopes:</p><ul><li>✅ `repo` — allows code push</li><li>❌ `workflow` — required for `.github/workflows/` files</li></ul><p>GitHub’s security model: Workflow files can run arbitrary code in CI/CD. They need explicit permission.</p><p>Push Attempt #4: The Winner</p><p>Final token configuration:</p><p>1. Delete previous token<br>2. Generate new token (classic)<br>3. Note: “DeployFlow Project”<br>4. Expiration: 90 days<br>5. Scopes:<br> — repo (full control)<br> — workflow (update GitHub Action workflows)<br>6. Generate token<br>7. Copy token</p><p>Final push:</p><pre>git push -u origin main<br># Username: BenjaminBurton<br># Password: [pasted token with both scopes]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kDrHmI5kIat1ksEb6UZ1vw.png" /></figure><p>Output:</p><pre>Enumerating objects: 5960, done.<br>Counting objects: 100% (5960/5960), done.<br>Delta compression using up to 8 threads<br>Compressing objects: 100% (5704/5704), done.<br>Writing objects: 100% (5960/5960), 7.10 MiB | 556.00 KiB/s, done.<br>Total 5960 (delta 1193), reused 0 (delta 0), pack-reused 0<br>remote: Resolving deltas: 100% (1193/1193), done.<br>To https://github.com/BenjaminBurton/deployflow.git<br> * [new branch] main -&gt; main<br>branch &#39;main&#39; set up to track &#39;origin/main&#39;.</pre><p>🎉 SUCCESS! 🎉</p><p>Verify on GitHub: Navigate to <a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></p><p>All files are there. README renders. It’s real.</p><p>GitHub Actions: Automatic Trigger</p><p>Immediately after the push, GitHub Actions triggered automatically.</p><p>Check the Actions tab: The workflow “DeployFlow CI/CD Pipeline” is running.</p><p>But then…</p><pre>❌ Build Node.js Service - FAILED<br>❌ Build Go Service - FAILED<br>⊝ Deploy to Kubernetes - SKIPPED (dependencies failed)</pre><p>Time for CI/CD debugging.</p><p>CI/CD Failure #1: CodeQL Deprecation</p><p>Click into the failed Node.js build:</p><p>Error:</p><pre>Run github/codeql-action/upload-sarif@v2<br>Error: CodeQL Action major versions v1 and v2 have been deprecated. <br>Please update all occurrences of the CodeQL Action in your workflow files to v3.</pre><p>The problem: My workflow used `<a href="http://twitter.com/v2">@v2</a>`, which GitHub deprecated.</p><p>The fix:</p><pre># Edit workflow<br>vim .github/workflows/ci-cd.yml<br># Change: github/codeql-action/upload-sarif@v2<br># To: github/codeql-action/upload-sarif@v3</pre><p>Commit and push:</p><pre>git add .github/workflows/ci-cd.yml<br>git commit -m &quot;Update CodeQL action to v3&quot;<br>git push origin main</pre><p>GitHub Actions triggered again…</p><p>Still failing. Next error.</p><p>CI/CD Failure #2: Go Module Path</p><p>Error in Go build:</p><pre>Run go test -v ./…<br>? github.com/yourusername/deployflow-go-service [no test files]</pre><p>The problem: My `go.mod` still had template username.</p><p>The fix:</p><pre>cat &gt; go-service/go.mod &lt;&lt; &#39;EOF&#39;<br>module github.com/BenjaminBurton/deployflow-go-service<br>go 1.21<br>require (<br> // No external dependencies for this simple service<br>)<br>EOF</pre><pre>git add go-service/go.mod<br>git commit -m &quot;Fix: Update go.mod with correct GitHub username&quot;<br>git push origin main</pre><p>GitHub Actions triggered again…</p><p>Still failing. Next error.</p><p>CI/CD Failure #3: Missing go.sum</p><p>Warning in Go build:</p><pre>Warning: Restore cache failed: Dependencies file is not found <br> in /home/runner/work/deployflow/deployflow. <br> Supported file pattern: go.sum</pre><p>The problem: GitHub Actions expects a `go.sum` file for dependency caching.</p><p>My project has no external dependencies, so `go mod tidy` didn’t create one.</p><p>The solution: Create an empty `go.sum` (valid for zero-dependency projects).</p><pre>cd go-service<br>touch go.sum<br>cd ..<br>git add go-service/go.sum<br>git commit -m &quot;Add empty go.sum for GitHub Actions cache&quot;<br>git push origin main</pre><p>Also update the workflow to specify the cache path:</p><pre>- name: Set up Go<br> uses: actions/setup-go@v5<br> with:<br> go-version: &#39;1.21&#39;<br> cache-dependency-path: go-service/go.sum # Added this line</pre><p>GitHub Actions triggered again…</p><p>Builds are passing now, but…</p><p>CI/CD Failure #4: GitHub Security Permissions</p><p>Error in both Node and Go builds:</p><pre>Warning: This run of the CodeQL Action does not have permission to access <br> the CodeQL Action API endpoints.<br>Error: Resource not accessible by integration</pre><p>The problem: My workflow tried to upload security scan results to GitHub’s Security tab.</p><p>The context:</p><ul><li>GitHub Advanced Security is a paid feature</li><li>Free repositories don’t have access</li><li>SARIF upload requires special permissions</li><li>Trivy scanning still runs, just can’t upload results</li></ul><p>Three options:</p><p>Option A: Enable Advanced Security</p><ul><li>Requires GitHub Enterprise or paid plan</li><li>Not feasible for free portfolio project</li></ul><p>Option B: Add Permissions Block</p><pre>permissions:<br> contents: read<br> security-events: write</pre><p>Still failed (permissions alone insufficient without Advanced Security)</p><p>Option C: Remove SARIF Upload (Chosen)</p><p>The decision:</p><pre># Before (tried to upload to GitHub Security):<br>- name: Scan image for vulnerabilities<br> uses: aquasecurity/trivy-action@master<br> with:<br> image-ref: ${{ env.NODE_IMAGE }}:${{ github.sha }}<br> format: &#39;sarif&#39;<br> output: &#39;trivy-results.sarif&#39;<br>- name: Upload to GitHub Security<br> uses: github/codeql-action/upload-sarif@v3<br> with:<br> sarif_file: &#39;trivy-results.sarif&#39;<br># After (scan but don&#39;t upload):<br>- name: Scan image for vulnerabilities<br> uses: aquasecurity/trivy-action@master<br> with:<br> image-ref: ${{ env.NODE_IMAGE }}:${{ github.sha }}<br> format: &#39;table&#39;<br> severity: &#39;CRITICAL,HIGH&#39;</pre><p>Trade-offs:</p><ul><li>✅ Kept: Trivy security scanning (still runs and displays results)</li><li>✅ Kept: Build-time vulnerability detection</li><li>❌ Lost: GitHub Security tab integration</li><li>❌ Lost: Historical vulnerability tracking in UI</li></ul><p>Rationale:</p><ul><li>Primary goal: Prove CI/CD and security scanning capabilities</li><li>Trivy still scans and fails builds on critical vulnerabilities</li><li>Portfolio demonstrates understanding even without UI integration</li><li>Can upgrade to Advanced Security in production environments</li></ul><p>The Final Pipeline</p><p>After all fixes, the final `ci-cd.yml`:</p><pre>name: DeployFlow CI/CD Pipeline<br>on:<br> push:<br> branches: [ main, develop ]<br> pull_request:<br> branches: [ main ]<br>env:<br> NODE_IMAGE: deployflow-node<br> GO_IMAGE: deployflow-go<br>jobs:<br> build-node:<br> name: Build Node.js Service<br> runs-on: ubuntu-latest<br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br>- name: Set up Node.js<br> uses: actions/setup-node@v4<br> with:<br> node-version: &#39;18&#39;<br>- name: Install dependencies<br> working-directory: ./node-service<br> run: npm ci<br>- name: Run tests<br> working-directory: ./node-service<br> run: npm test || echo &quot;No tests defined yet&quot;<br>- name: Build Docker image<br> working-directory: ./node-service<br> run: docker build -t ${{ env.NODE_IMAGE }}:${{ github.sha }} .<br>- name: Scan image for vulnerabilities<br> uses: aquasecurity/trivy-action@master<br> with:<br> image-ref: ${{ env.NODE_IMAGE }}:${{ github.sha }}<br> format: &#39;table&#39;<br> severity: &#39;CRITICAL,HIGH&#39;<br>build-go:<br> name: Build Go Service<br> runs-on: ubuntu-latest<br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br>- name: Set up Go<br> uses: actions/setup-go@v5<br> with:<br> go-version: &#39;1.21&#39;<br> cache-dependency-path: go-service/go.sum<br>- name: Run tests<br> working-directory: ./go-service<br> run: go test -v ./…<br>- name: Build Docker image<br> working-directory: ./go-service<br> run: docker build -t ${{ env.GO_IMAGE }}:${{ github.sha }} .<br>- name: Scan image for vulnerabilities<br> uses: aquasecurity/trivy-action@master<br> with:<br> image-ref: ${{ env.GO_IMAGE }}:${{ github.sha }}<br> format: &#39;table&#39;<br> severity: &#39;CRITICAL,HIGH&#39;<br>deploy:<br> name: Deploy to Kubernetes<br> needs: [build-node, build-go]<br> runs-on: ubuntu-latest<br> if: github.ref == &#39;refs/heads/main&#39;<br> steps:<br> - name: Checkout code<br> uses: actions/checkout@v4<br>- name: Deploy services<br> run: |<br> echo &quot;✅ Build and security scans passed!&quot;<br> echo &quot;✅ Docker images built successfully&quot;<br> echo &quot;✅ No critical vulnerabilities found&quot;<br> echo &quot;Ready to deploy to Kubernetes cluster&quot;</pre><p>Commit and push:</p><pre>git add .github/workflows/ci-cd.yml<br>git commit -m &quot;Final CI/CD workflow: remove SARIF upload, focus on core pipeline&quot;<br>git push origin main</pre><p>Success: All Green Pipeline</p><p>GitHub Actions result:</p><pre>✅ Build Node.js Service - PASSED<br>✅ Build Go Service - PASSED<br>✅ Deploy to Kubernetes - PASSED</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6RRXVqMR2YnzTnSfnTis4g.png" /></figure><p>Build Node.js Service details:</p><ul><li>✅ Checkout code</li><li>✅ Set up Node.js 18</li><li>✅ Install dependencies (npm ci)</li><li>✅ Run tests (no tests defined — acceptable for MVP)</li><li>✅ Build Docker image</li><li>✅ Scan for vulnerabilities (no critical/high issues)</li></ul><p>Build Go Service details:</p><ul><li>✅ Checkout code</li><li>✅ Set up Go 1.21</li><li>✅ Cache dependencies (go.sum)</li><li>✅ Run tests (no test files — acceptable for MVP)</li><li>✅ Build Docker image</li><li>✅ Scan for vulnerabilities (no critical/high issues)</li></ul><p>Deploy to Kubernetes details:</p><ul><li>✅ Depends on successful builds</li><li>✅ Only runs on main branch</li><li>✅ Simulated deployment (logged success messages)</li></ul><p>Screenshot this. All green. Portfolio complete.</p><p>What the CI/CD Pipeline Proves</p><p>This automated pipeline demonstrates:</p><p>1. Automated builds on every push<br>2. Multi-language support (Node.js + Go)<br>3. Dependency management (npm, go modules)<br>4. Security scanning (Trivy vulnerability detection)<br>5. Docker image creation (automated multi-stage builds)<br>6. Conditional deployment logic (main branch only)<br>7. Build status visibility (GitHub UI integration)</p><p>This shows:</p><p>Understanding of CI/CD principles</p><ul><li>Ability to configure GitHub Actions</li><li>Security-conscious development</li><li>Professional workflow practices</li></ul><p>Final Architecture: The Complete Picture</p><pre>┌─────────────────────────────────────────────────────────────┐<br>│                   Development Environment                    │<br>│                                                              │<br>│  MacBook Air M1                                             │<br>│  └─ SSH Client ──────────┐                                 │<br>│                           │                                  │<br>│                           ▼                                  │<br>│  Lenovo ThinkPad (Arch Linux, 32GB RAM)                    │<br>│  └─ Docker Engine                                           │<br>│     │                                                        │<br>│     ├─ DevContainer (Ubuntu 22.04)                         │<br>│     │  ├─ Node.js 24, Go 1.24, kubectl                     │<br>│     │  └─ k3d, Docker CLI, Neovim                          │<br>│     │                                                        │<br>│     └─ k3d Cluster (Docker network: bridge)                │<br>│        └─ K3s Server (172.17.0.3:6443)                     │<br>│           └─ Kubernetes Pods:                               │<br>│              ├─ node-service-xxx-1 (Running)               │<br>│              ├─ node-service-xxx-2 (Running)               │<br>│              ├─ go-service-xxx-1 (Running)                 │<br>│              └─ go-service-xxx-2 (Running)                 │<br>│                                                              │<br>└─────────────────────────────────────────────────────────────┘<br>                           │<br>                           │ git push<br>                           ▼<br>┌─────────────────────────────────────────────────────────────┐<br>│                    GitHub Repository                         │<br>│                                                              │<br>│  github.com/BenjaminBurton/deployflow                       │<br>│  │                                                           │<br>│  ├─ Source Code (523 lines, 14 files)                      │<br>│  │  ├─ .devcontainer/                                       │<br>│  │  ├─ node-service/                                        │<br>│  │  ├─ go-service/                                          │<br>│  │  └─ .github/workflows/ci-cd.yml                         │<br>│  │                                                           │<br>│  └─ GitHub Actions Pipeline                                 │<br>│     ├─ Trigger: push to main                                │<br>│     │                                                        │<br>│     ├─ Build Node.js ✅                                     │<br>│     │  ├─ npm ci                                            │<br>│     │  ├─ npm test                                          │<br>│     │  ├─ docker build                                      │<br>│     │  └─ trivy scan                                        │<br>│     │                                                        │<br>│     ├─ Build Go ✅                                          │<br>│     │  ├─ go test                                           │<br>│     │  ├─ docker build                                      │<br>│     │  └─ trivy scan                                        │<br>│     │                                                        │<br>│     └─ Deploy ✅                                            │<br>│        └─ Conditional (main branch only)                    │<br>│                                                              │<br>└─────────────────────────────────────────────────────────────┘</pre><p>Project Metrics: By the Numbers</p><p>Development Time:</p><ul><li>Total: 5–6 hours (single session)</li><li>Network setup: 30 minutes</li><li>Docker &amp; DevContainer: 45 minutes</li><li>Manual file creation: 60 minutes</li><li>Troubleshooting: 2 hours (biggest chunk)</li><li>Microservices deployment: 45 minutes</li><li>Git/GitHub: 45 minutes</li><li>CI/CD: 30 minutes</li></ul><p>Code Statistics:</p><pre>Language              Files    Lines<br>─────────────────────────────────────<br>Go                       1      113<br>JavaScript               1       72<br>YAML (Kubernetes)        4      120<br>YAML (GitHub Actions)    1       80<br>Dockerfile               2       48<br>Shell (Bash)             1       90<br>JSON                     2       45<br>Markdown                 1      150<br>─────────────────────────────────────<br>Total                   13      718</pre><p>Container Metrics:</p><pre>Image              Size      Architecture<br>─────────────────────────────────────────<br>deployflow-node    180MB     linux/amd64<br>deployflow-go      15MB      linux/amd64<br>ubuntu:22.04       77MB      linux/amd64<br>─────────────────────────────────────────<br>Total              272MB</pre><p>Kubernetes Resources:</p><pre>Component           Request    Limit<br>──────────────────────────────────────<br>CPU (total)         300m       600m<br>Memory (total)      384Mi      768Mi<br>Pods                4          4<br>Services            2          2</pre><p>GitHub Statistics:</p><p>Commits: 6 (initial + 5 fixes)<br>CI/CD Runs: 7 (6 failed, 1 passed)<br>Authentication Attempts: 4<br>Time to Green: 45 minutes from first push</p><p>The Complete Troubleshooting Journey</p><p>Let’s recap every major issue faced and solved:</p><p>Issue 1: Hardware Constraints</p><ul><li>Problem: 8GB MacBook insufficient for Kubernetes</li><li>Solution: Use Lenovo with 32GB RAM</li><li>Lesson: Resource planning is critical</li></ul><p>Issue 2: Binary Architecture</p><ul><li>Problem: Downloaded ARM64 binaries on x86_64 system</li><li>Solution: Download correct amd64 versions</li><li>Lesson: Always verify `uname -m`</li></ul><p>Issue 3: Docker Network Isolation</p><ul><li>Problem: DevContainer and k3d on different networks</li><li>Solution: Create k3d on same bridge network</li><li>Lesson: Docker networks are isolated by default</li></ul><p>Issue 4: k3d LoadBalancer Failure</p><ul><li>Problem: LoadBalancer repeatedly failing</li><li>Solution: Use ` — no-lb` flag for direct connection</li><li>Lesson: Not all features work in all environments</li></ul><p>Issue 5: Kubeconfig Addressing</p><ul><li>Problem: `0.0.0.0` addresses don’t work from containers</li><li>Solution: Sed-replace with actual container IP</li><li>Lesson: Kubeconfig is environment-specific</li></ul><p>Issue 6: GitHub Password Authentication</p><ul><li>Problem: Passwords deprecated since 202</li><li>Solution: Use Personal Access Token</li><li>Lesson: Keep up with platform evolution</li></ul><p>Issue 7: Insufficient Token Scopes</p><ul><li>Problem: Token without `repo` or `workflow` scopes</li><li>Solution: Generate token with both scopes</li><li>Lesson: Token permissions are granular</li></ul><p>Issue 8: CI/CD Action Deprecations</p><ul><li>Problem: CodeQL v2 deprecated</li><li>Solution: Update to v3</li><li>Lesson: Maintain dependencies actively</li></ul><p>Issue 9: Module Path Mistakes</p><ul><li>Problem: Template username in go.mod</li><li>Solution: Update with actual GitHub username</li><li>Lesson: Check all references before pushing</li></ul><p>Issue 10: Missing Dependency Files</p><ul><li>Problem: No go.sum for GitHub Actions cache</li><li>Solution: Create empty go.sum (valid for zero deps)</li><li>Lesson: Understand what CI/CD expects</li></ul><p>Issue 11: GitHub Security Permissions</p><ul><li>Problem: Can’t upload SARIF without Advanced Security</li><li>Solution: Remove upload, keep local scanning</li><li>Lesson: Not all features available on free plans</li></ul><p>Total issues: 11 <br>Total time debugging: ~3 hours <br>Percentage of project time: ~50%</p><p>This is real DevOps work.</p><p>Key Lessons: The Big Picture</p><p>1. Real Projects Are 80% Troubleshooting</p><p>What tutorials teach:</p><ul><li>Run these commands</li><li>Everything works</li><li>Done!</li></ul><p>What really happens:</p><ul><li>Run commands</li><li>Something breaks</li><li>Research the error</li><li>Try solution A (fails)</li><li>Try solution B (fails)</li><li>Understand root cause</li><li>Try solution C (works!)</li><li>Document for next time</li></ul><p>2. Manual Configuration &gt; Templates</p><p>By creating every file manually, I:</p><ul><li>✅ Understood each component deeply</li><li>✅ Could debug when things broke</li><li>✅ Learned rather than just copied</li><li>✅ Built real troubleshooting skills</li></ul><p>If I’d cloned a template: I’d be lost at the first error.</p><p>3. Failures Are Learning Opportunities</p><p>Every “failure” in this project taught something:</p><ul><li>ARM64 binaries → Architecture awareness</li><li>Docker networks → Container isolation</li><li>LoadBalancer failures → When to simplify</li><li>GitHub tokens → Modern authentication</li><li>CI/CD errors → Dependency management</li></ul><p>The failures were more valuable than the successes.</p><p>4. Documentation Matters</p><p>This 4-part series exists because I:</p><ul><li>Captured every error message</li><li>Documented each troubleshooting step</li><li>Took screenshots of successes</li><li>Explained why things worked</li></ul><p>For employers: This proves I can communicate technical concepts clearly.</p><p>5. Production-Grade Practices</p><p>Even in a portfolio project, I implemented:</p><ul><li>✅ Multi-stage Docker builds (optimization)</li><li>✅ Non-root containers (security)</li><li>✅ Health checks (reliability)</li><li>✅ Resource limits (stability)</li><li>✅ Security scanning (vulnerability detection)</li><li>✅ Automated CI/CD (DevOps culture)</li></ul><p>These aren’t just checkboxes — they’re professional practices.</p><p>What This Project Proves</p><p>Technical Skills:</p><p>Infrastructure:</p><ul><li>Container orchestration (Kubernetes/K3s)</li><li>Container runtime management (Docker)</li><li>Development environments (DevContainers)</li><li>Network configuration (SSH, Docker networks)</li></ul><p>Development:</p><ul><li>Multi-language proficiency (Node.js, Go, Bash)</li><li>API design (RESTful endpoints)</li><li>Microservices architecture</li><li>Service discovery (Kubernetes DNS)</li></ul><p>DevOps:</p><ul><li>CI/CD pipeline implementation (GitHub Actions)</li><li>Infrastructure as Code (Kubernetes YAML)</li><li>Security scanning (Trivy)</li><li>Version control (Git)</li><li>Automated testing and deployment</li></ul><p>System Administration:</p><ul><li>Linux configuration (Arch Linux)</li><li>Package management (apt, npm, pacman)</li><li>Binary architecture management</li><li>Troubleshooting methodology</li></ul><p>Soft Skills:</p><p>Problem Solving:</p><ul><li>Systematic debugging approach</li><li>Root cause analysis</li><li>Multiple solution attempts</li><li>Learning from failures</li></ul><p>Communication:</p><ul><li>Clear technical documentation</li><li>Explaining complex concepts</li><li>Teaching through storytelling</li><li>Honest about mistakes</li></ul><p>Persistence:</p><ul><li>11 major issues solved</li><li>5–6 hours of focused work</li><li>No giving up on hard problems</li><li>Completion despite obstacles</li></ul><p>Self-Direction:</p><ul><li>No tutorial followed</li><li>Built from first principles</li><li>Made own architectural decisions</li><li>Created complete system independently</li></ul><p>Future Enhancements: What’s Next</p><p>This project is complete as a portfolio piece, but here’s what I’m planning for Phase 2:</p><p>Observability &amp; Monitoring:</p><ul><li>Deploy Prometheus for metrics</li><li>Create Grafana dashboards</li><li>Implement structured logging</li><li>Add distributed tracing (Jaeger)</li></ul><p>Advanced Deployment:</p><ul><li>Package as Helm charts</li><li>Implement GitOps with ArgoCD</li><li>Add canary deployments</li><li>Set up automated rollbacks</li></ul><p>Production Hardening:</p><ul><li>Multi-node K3s cluster (HA)</li><li>Backup and disaster recovery (Velero)</li><li>Network policies for isolation</li><li>Secrets management (Vault)</li><li>RBAC and authentication</li></ul><p>Multi-Environment:</p><ul><li>Deploy to AWS EKS (staging)</li><li>Terraform for infrastructure</li><li>Environment-specific configs</li><li>Separate CI/CD per environment</li></ul><p>Thank You for Following This Journey</p><p>If you read all 4 parts, you’ve followed a real engineering journey — failures and all. I hope it was valuable.</p><p><strong>Connect with me</strong>:</p><p><strong><em>GitHub</em></strong>: <a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a><br><strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a> <br><strong><em>AWS Community Builder</em></strong>: <a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a><br><strong><em>Website</em></strong>: [Coming Soon]</p><p>I’m actively seeking DevOps opportunities. If you know of roles or want to discuss this project, reach out!</p><p><strong>Quick Reference</strong>: Complete Command Guide</p><p>For anyone building something similar, here’s the condensed command reference:</p><pre># SSH Setup<br>sudo pacman -S openssh<br>sudo systemctl start sshd<br><br># Docker Setup<br>sudo pacman -S docker<br>sudo systemctl start docker<br>sudo usermod -aG docker $USER<br><br># DevContainer CLI<br>npm config set prefix &#39;~/.npm-global&#39;<br>npm install -g @devcontainers/cli<br><br># DevContainer Operations<br>devcontainer up --workspace-folder .<br>devcontainer exec --workspace-folder . bash<br><br># k3d Cluster<br>k3d cluster create NAME --network bridge --no-lb --wait<br>K3D_IP=$(docker inspect -f \<br>  &#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39; \<br>  k3d-NAME-server-0)<br>sed -i &quot;s|https://.*:.*|https://${K3D_IP}:6443|g&quot; ~/.kube/config<br><br># Kubernetes<br>kubectl get nodes<br>kubectl apply -f SERVICE/k8s/<br>kubectl get pods<br>kubectl get services<br><br># Docker Images<br>docker build -t IMAGE:TAG .<br>k3d image import IMAGE:TAG -c CLUSTER_NAME<br><br># Git<br>git config --global user.name &quot;NAME&quot;<br>git config --global user.email &quot;EMAIL&quot;<br>git config --global init.defaultBranch main<br>git config --global --add safe.directory /workspaces/PROJECT<br><br># GitHub<br># Use Personal Access Token with &#39;repo&#39; + &#39;workflow&#39; scopes</pre><p>The Series Complete</p><p>Part 1: The Setup Journey — SSH, Docker, DevContainer CLI <br>Part 2: Manual Configuration &amp; Architecture Nightmares <br>Part 3: Kubernetes Networking Deep Dive <br>Part 4: CI/CD &amp; Production Lessons</p><p>Total reading time: ~50 minutes <br>Total project time: 5–6 hours <br>Total learning: Immeasurable</p><p>Thank you for reading. Now go build something.</p><p>#Kubernetes #DevOps #CI/CD #GitHubActions #Docker #Microservices #CareerChange #SelfTaught #AWS #CommunityBuilder #TechnicalWriting #PortfolioProject</p><p>About the Author:</p><p>Lavell Burton is a self-taught DevOps engineer and AWS Community Builder (Containers category). This 4-part series documents his journey from delivery driver to DevOps, one production project at a time.</p><p>”Real DevOps isn’t following tutorials. It’s solving problems you’ve never seen before, documenting what worked, and sharing it so others can learn too.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0dcc8a11b31c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeployFlow Part 3: Kubernetes Networking Deep Dive: When kubectl Refuses to Connect]]></title>
            <link>https://mrburton.medium.com/deployflow-part-3-kubernetes-networking-deep-dive-when-kubectl-refuses-to-connect-f88af97e87be?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/f88af97e87be</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Fri, 12 Dec 2025 01:11:55 GMT</pubDate>
            <atom:updated>2025-12-16T03:50:35.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLkYIRiQH51Ai-HArETSgg.png" /></figure><p><strong><em>In Part 2</em></strong>, I fought through binary architecture mismatches and finally got all my tools working:</p><p>Quick Links to Series:</p><ul><li><strong><em>Part 1</em></strong>: The Setup Journey** _(SSH, Docker, DevContainer CLI)_</li><li><strong><em>Part 2</em></strong>: Manual Configuration &amp; Architecture Nightmares** _(Files, builds, ARM64 vs x86_64)_</li><li><strong><em>Part 3</em></strong>: Kubernetes Networking Deep Dive** ← _You are here_</li><li><strong><em>Part 4</em></strong>: CI/CD &amp; Production Lessons _(Next week)_</li></ul><p>🔗 <strong><em>Full Project Code:</em></strong> <a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></p><p><strong><em>In Part 2</em></strong>, I fought through binary architecture mismatches and finally got all my tools working:</p><ul><li>Node.js, Go, kubectl all executing correctly</li><li>k3d cluster created during post-create script</li><li>Everything seemed ready to deploy</li></ul><p>Now, with confidence, I ran what should be the simplest command:</p><pre>kubectl get nodes</pre><p>I expected: A list of my cluster nodes <br>I got: An error that would consume the next 90 minutes of my life</p><p>This is the story of that troubleshooting journey.</p><p>The First “Connection Refused”</p><pre>root@3f9460f9cc90:/workspaces/deployflow# kubectl get nodes</pre><p>Output:</p><pre>E1203 23:06:59.419564 7918 memcache.go:265] &quot;Unhandled Error&quot; <br> err=&quot;couldn&#39;t get current server API group list: <br> Get \&quot;https://0.0.0.0:6550/api?timeout=32s\&quot;: <br> dial tcp 0.0.0.0:6550: connect: connection refused&quot;<br>The connection to the server 0.0.0.0:6550 was refused - <br> did you specify the right host or port?</pre><p>My first thought: “Wait, 0.0.0.0? That can’t be right.”</p><p>Investigation: The Kubeconfig Mystery</p><p>When kubectl can’t connect, the first place to check is the kubeconfig file:</p><pre>cat ~/.kube/config | grep server:</pre><p>Output:</p><pre>server: https://0.0.0.0:6550</pre><p>The Problem:</p><p>k3d generated a kubeconfig file designed for host access. But I’m running kubectl from inside a DevContainer.</p><p>From inside the container, `0.0.0.0` doesn’t route to the k3d cluster running on the host’s Docker network. It’s like trying to call yourself — the address has no meaning in this context.</p><p>What I needed: The actual IP address of the k3d cluster on the Docker network.</p><p>Attempt 1: Try Localhost</p><p>Hypothesis: “Maybe localhost will work from inside the container?”</p><pre># Update kubeconfig to use localhost<br>sed -i &#39;s|https://0.0.0.0:6550|https://127.0.0.1:6550|g&#39; ~/.kube/config<br># Verify the change<br>cat ~/.kube/config | grep server:<br># Output: server: https://127.0.0.1:6550<br># Try again<br>kubectl get nodes</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLkYIRiQH51Ai-HArETSgg.png" /></figure><p>Output:</p><pre>dial tcp 127.0.0.1:6550: connect: connection refused</pre><p>Still refused.</p><p>Why it failed: The k3d loadbalancer isn’t listening on the container’s localhost interface. It’s on a different Docker network entirely.</p><p>Attempt 2: Find the k3d Proxy IP</p><p>New hypothesis: “Let me find the actual IP of the k3d loadbalancer container and connect directly to that.”</p><pre># Check what k3d containers are running<br>docker ps | grep k3d</pre><p>Output:</p><pre>5ebd33605122 ghcr.io/k3d-io/k3d-proxy:5.8.3 … k3d-deployflow-cluster-serverlb<br>58afe170be56 rancher/k3s:v1.31.5-k3s1 … k3d-deployflow-cluster-server-0</pre><p>Two containers:</p><p>1. k3d-deployflow-cluster-serverlb — The loadbalancer/proxy<br>2. k3d-deployflow-cluster-server-0 — The actual k3s server</p><p>Get the loadbalancer’s IP:</p><pre>K3D_IP=$(docker inspect -f \<br> &#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39; \<br> k3d-deployflow-cluster-serverlb)<br>echo &quot;K3D Proxy IP: $K3D_IP&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bMUDt8dhQcBK_etMgMiFvw.png" /></figure><p>Output:</p><pre>K3D Proxy IP: 172.18.0.3</pre><p>Update kubeconfig with this IP:</p><pre>sed -i &quot;s|https://127.0.0.1:6550|https://172.18.0.3:6443|g&quot; ~/.kube/config<br>cat ~/.kube/config | grep server:<br># Output: server: https://172.18.0.3:6443<br># Try kubectl again<br>kubectl get nodes</pre><p>Output:</p><pre>dial tcp 172.18.0.3:6443: i/o timeout</pre><p>Different error! Progress? Not really. This is worse — a timeout means the request is going nowhere.</p><p>The Root Cause: Network Isolation</p><p>Time to investigate Docker networking.</p><p>Check which network the DevContainer is on:</p><pre>docker inspect $(hostname) -f \<br> &#39;{{range $k, $v := .NetworkSettings.Networks}}{{$k}}{{end}}&#39;</pre><p>Output:</p><pre>bridge</pre><p>Check which network k3d is on:</p><pre>docker network ls | grep k3d</pre><p>Output:</p><pre>k3d-deployflow-cluster bridge …</pre><p>The Problem Revealed:</p><ul><li>DevContainer: On default `bridge` network</li><li>k3d cluster: Created its own `k3d-deployflow-cluster` network</li></ul><p>These are separate Docker networks. They can’t communicate with each other by default.</p><p>It’s like having two houses on different streets — same city (Docker), but no direct road between them.</p><p>Attempt 3: Recreate Cluster on Same Network</p><p>Solution approach: Delete the k3d cluster and recreate it on the same network as the DevContainer.</p><pre># Delete existing cluster<br>k3d cluster delete deployflow-cluster</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G8hkQ49U2wSXJ94fC_uSWg.png" /></figure><p>Output:</p><pre>INFO[0000] Deleting cluster &#39;deployflow-cluster&#39;<br>INFO[0000] Successfully deleted cluster deployflow-cluster!</pre><p>Find the DevContainer’s network:</p><pre>DEVCONTAINER_NETWORK=$(docker inspect $(hostname) -f \<br> &#39;{{range $k, $v := .NetworkSettings.Networks}}{{$k}}{{end}}&#39;)<br>echo &quot;DevContainer Network: $DEVCONTAINER_NETWORK&quot;</pre><p>Output:</p><pre>DevContainer Network: bridge</pre><p>Create k3d cluster on the bridge network:</p><pre>k3d cluster create deployflow-cluster \<br> - network $DEVCONTAINER_NETWORK \<br> - api-port 6550 \<br> - servers 1 \<br> - agents 0 \<br> - port &quot;3000:30000@server:0&quot; \<br> - port &quot;8080:30001@server:0&quot; \<br> - wait</pre><p>Output:</p><pre>INFO[0001] Creating node &#39;k3d-deployflow-cluster-server-0&#39;<br>INFO[0001] Creating LoadBalancer &#39;k3d-deployflow-cluster-serverlb&#39;<br>INFO[0005] Starting helpers…<br>ERRO[0011] Failed Cluster Start: Failed to add one or more helper nodes: <br> Node k3d-deployflow-cluster-serverlb failed to get ready: <br> error waiting for log line `start worker processes` from node <br> &#39;k3d-deployflow-cluster-serverlb&#39;: stopped returning log lines: <br> node k3d-deployflow-cluster-serverlb is running=true in status=restarting<br>ERRO[0011] Failed to create cluster &gt;&gt;&gt; Rolling Back</pre><p>The cluster creation failed. The loadbalancer wouldn’t start.</p><p>Attempt 4: Simplify — Remove Custom Ports</p><p>New hypothesis: “Maybe the custom port mappings are causing the loadbalancer to fail?”</p><pre>k3d cluster create deployflow-cluster \<br> - network bridge \<br> - servers 1 \<br> - agents 0 \<br> - wait</pre><p>Output:</p><pre>ERRO[0011] Failed Cluster Start: Node k3d-deployflow-cluster-serverlb <br> failed to get ready…</pre><p>Same error. The loadbalancer was the problem, not the ports.</p><p>Attempt 5: Nuclear Option — Disable LoadBalancer</p><p>Final hypothesis: “What if I just… don’t use a loadbalancer at all?”</p><p>k3d uses a loadbalancer proxy to route traffic to the cluster. But for development, we could connect directly to the k3s server instead.</p><pre>k3d cluster create deployflow-cluster \<br> - network bridge \<br> - servers 1 \<br> - agents 0 \<br> - no-lb \<br> - wait</pre><p>The ` — no-lb` flag tells k3d: “Don’t create a loadbalancer. I’ll connect directly to the k3s API server.”</p><p>Output:</p><pre>INFO[0001] Creating node &#39;k3d-deployflow-cluster-server-0&#39;<br>INFO[0005] Starting servers…<br>INFO[0007] Cluster &#39;deployflow-cluster&#39; created successfully!</pre><p>SUCCESS! The cluster created in 7 seconds.</p><p>Why this worked:</p><p>The k3d loadbalancer was having compatibility issues with the specific Docker bridge network setup in my DevContainer environment. By skipping it entirely with ` — no-lb`, we eliminated that point of failure.</p><p>The trade-off: We have to connect directly to the k3s server instead of going through a loadbalancer. For development, this is perfectly fine.</p><p>Fixing Kubeconfig (Again)</p><p>With no loadbalancer, the kubeconfig still has the wrong address:</p><pre>cat ~/.kube/config | grep server:</pre><p>Output:</p><pre>server: https://0.0.0.0:38977</pre><p>Note the random port (38977) — that’s because there’s no loadbalancer to provide a stable port.</p><p>Without a loadbalancer, we connect directly to the k3s server container:</p><pre># Get the k3s server container IP<br>K3D_SERVER_IP=$(docker inspect -f \<br> &#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39; \<br> k3d-deployflow-cluster-server-0)<br>echo &quot;K3D Server IP: $K3D_SERVER_IP&quot;</pre><p>Output:</p><pre>K3D Server IP: 172.17.0.3</pre><p>Update kubeconfig to use the server IP directly:</p><pre>sed -i &quot;s|https://0.0.0.0:.*|https://${K3D_SERVER_IP}:6443|g&quot; ~/.kube/config<br># Verify the change<br>cat ~/.kube/config | grep server:</pre><p>Output:</p><pre>server: https://172.17.0.3:6443</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLkYIRiQH51Ai-HArETSgg.png" /></figure><p>Port 6443 is the default k3s API server port.</p><p>The Moment of Truth</p><pre>kubectl get nodes</pre><p>Output:</p><pre>NAME STATUS ROLES AGE VERSION<br>k3d-deployflow-cluster-server-0 Ready control-plane,master 98s v1.31.5+k3s1</pre><p>🎉 SUCCESS! 🎉</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLkYIRiQH51Ai-HArETSgg.png" /></figure><p>After 5 failed attempts and 90 minutes of debugging, kubectl could finally communicate with the cluster!</p><p>Screenshot this moment. This is portfolio gold.</p><p>Understanding What We Built</p><p>Let’s visualize the final architecture:</p><pre>Docker Host (Lenovo)<br>│<br>├─ Docker Network: bridge (172.17.0.0/16)<br>│ │<br>│ ├─ DevContainer (Ubuntu 22.04)<br>│ │ └─ kubectl → connects to 172.17.0.3:6443<br>│ │<br>│ └─ k3d-deployflow-cluster-server-0 (K3s)<br>│ └─ Listening on 172.17.0.3:6443<br>│<br>└─ (No separate k3d network - everything on bridge)</pre><p>Key insights:</p><p>1. Same network: DevContainer and k3d on same Docker bridge<br>2. No loadbalancer: Direct connection to k3s server<br>3. Direct IP: Using actual container IP, not 0.0.0.0 or localhost<br>4. Standard port: k3s API server on 6443</p><p>Deploying Microservices</p><p>With kubectl working, it’s time to actually deploy something!</p><p>Building Docker Images</p><p>Node.js service:</p><pre>cd node-service<br>docker build -t deployflow-node:latest .</pre><p>Output snippet:</p><pre>[+] Building 45.2s (11/11) FINISHED<br> =&gt; [builder 4/4] RUN npm ci - only=production<br> =&gt; [stage-1 4/5] COPY - from=builder - chown=nodejs:nodejs /app /app<br> =&gt; =&gt; naming to docker.io/library/deployflow-node:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YhQwiLQOW88HZ38Yxii21g.png" /></figure><p>Go service:</p><pre>cd ../go-service<br>docker build -t deployflow-go:latest .</pre><p>Output snippet:</p><pre>[+] Building 12.8s (15/15) FINISHED<br> =&gt; [builder 6/5] RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .<br> =&gt; =&gt; naming to docker.io/library/deployflow-go:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*COyHk_x8wDrmXTgFH1OOog.png" /></figure><p>Verify images:</p><pre>docker images | grep deployflow</pre><p>Output:</p><pre>deployflow-node latest a1b2c3d4e5f6 2 minutes ago 180MB<br>deployflow-go latest f6e5d4c3b2a1 30 seconds ago 15MB</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o17Y4EpwBE3MsUZ-jEyYhQ.png" /></figure><p>Notice the size difference:</p><ul><li>Node.js: ~132MB (includes runtime + dependencies)</li><li>Go: ~22.8MB (statically compiled binary on Alpine)</li></ul><p>This is why Go is popular for containers!</p><p>Loading Images into k3d</p><p>Images built on the host need to be imported into the k3d cluster:</p><pre>cd /workspaces/deployflow<br># Import Node.js image<br>k3d image import deployflow-node:latest -c deployflow-cluster<br># Output: INFO[0003] Imported 1 image(s) into 1 cluster(s)<br># Import Go image<br>k3d image import deployflow-go:latest -c deployflow-cluster<br># Output: INFO[0001] Imported 1 image(s) into 1 cluster(s)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AG0RfgEofP2IM-70mUbPyA.png" /></figure><p>Deploying Go Service (Backend First)</p><pre># Apply deployment<br>kubectl apply -f go-service/k8s/deployment.yaml<br># Output: deployment.apps/go-service created<br># Apply service<br>kubectl apply -f go-service/k8s/service.yaml<br># Output: service/go-service created<br># Check status<br>kubectl get pods</pre><p>Output:</p><pre>NAME READY STATUS RESTARTS AGE<br>go-service-795749c68b-c4wt4 1/1 Running 0 23s<br>go-service-795749c68b-pckpn 1/1 Running 0 23s</pre><p>Two pods running! Exactly as specified in the deployment (replicas: 2).</p><pre>kubectl get services</pre><p>Output:</p><pre>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br>go-service ClusterIP 10.43.213.43 &lt;none&gt; 8080/TCP 15s<br>kubernetes ClusterIP 10.43.0.1 &lt;none&gt; 443/TCP 14m</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NqdAuEh2GUZb3yN8GR5O_Q.png" /></figure><p>Deploying Node.js Service (Frontend)</p><pre># Apply deployment<br>kubectl apply -f node-service/k8s/deployment.yaml<br># Output: deployment.apps/node-service created<br># Apply service<br>kubectl apply -f node-service/k8s/service.yaml<br># Output: service/node-service created<br># Check all pods<br>kubectl get pods</pre><p>Output:</p><pre>NAME READY STATUS RESTARTS AGE<br>go-service-795749c68b-c4wt4 1/1 Running 0 2m23s<br>go-service-795749c68b-pckpn 1/1 Running 0 2m23s<br>node-service-6cdd97d67b-ckxh7 1/1 Running 0 17s<br>node-service-6cdd97d67b-dq6cp 1/1 Running 0 17s</pre><p>Four pods total: 2 Go + 2 Node.js, all running!</p><pre>kubectl get services</pre><p>Output:</p><pre>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br>go-service ClusterIP 10.43.213.43 &lt;none&gt; 8080/TCP 2m14s<br>kubernetes ClusterIP 10.43.0.1 &lt;none&gt; 443/TCP 15m<br>node-service ClusterIP 10.43.255.187 &lt;none&gt; 3000/TCP 12s</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XazG231cdCMuHPofpfqpog.png" /></figure><p>Full microservices architecture deployed! ✅</p><p>The Critical Test: Service Communication</p><p>Having pods running is great. But do they actually work? Can they talk to each other?</p><p>Test 1: Direct Go Service Access</p><pre>kubectl exec -it go-service-795749c68b-c4wt4 - \<br> wget -qO- http://localhost:8080/api/data</pre><p>Output:</p><pre>{<br> &quot;data&quot;: [&quot;Kubernetes&quot;,&quot;Docker&quot;,&quot;CI/CD&quot;,&quot;DevOps&quot;,&quot;Microservices&quot;],<br> &quot;count&quot;: 5,<br> &quot;timestamp&quot;: &quot;2025–12–03T23:36:36.020286887Z&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*64E-gInsZrgQwfBowVYxgQ.png" /></figure><p>✅ Go service is responding!</p><p>Test 2: Node.js Service Status</p><pre>kubectl exec -it node-service-6cdd97d67b-ckxh7 - \<br> wget -qO- http://localhost:3000/api/status</pre><p>Output:</p><pre>{<br> &quot;service&quot;: &quot;DeployFlow Node.js Service&quot;,<br> &quot;version&quot;: &quot;1.0.0&quot;,<br> &quot;uptime&quot;: 45.234,<br> &quot;environment&quot;: &quot;production&quot;<br>}</pre><p>✅ Node.js service is responding!</p><p>Test 3: The Money Shot — Microservice Communication</p><p>This is the real test. Can the Node.js service find and communicate with the Go service using Kubernetes DNS?</p><pre>kubectl exec -it node-service-6cdd97d67b-ckxh7 - \<br> wget -qO- http://localhost:3000/api/backend</pre><p>Output:</p><pre>{<br> &quot;message&quot;: &quot;Successfully called Go backend&quot;,<br> &quot;backend_response&quot;: {<br> &quot;data&quot;: [&quot;Kubernetes&quot;,&quot;Docker&quot;,&quot;CI/CD&quot;,&quot;DevOps&quot;,&quot;Microservices&quot;],<br> &quot;count&quot;: 5,<br> &quot;timestamp&quot;: &quot;2025–12–03T23:36:37.123456789Z&quot;<br> },<br> &quot;timestamp&quot;: &quot;2025–12–03T23:36:37.234567890Z&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ss2LxTQ4VvSqgJXxGyrsiw.png" /></figure><p>🔥 THIS IS IT! 🔥</p><p>What just happened:</p><p>1. Node.js service received request at `/api/backend`<br>2. Used Kubernetes DNS to resolve `go-service:8080`<br>3. Made HTTP GET request to Go service<br>4. Received JSON response<br>5. Wrapped it in its own response<br>6. Returned to client</p><p>This proves:</p><ul><li>✅ Kubernetes DNS is working (service discovery)</li><li>✅ Network policies allow inter-pod communication</li><li>✅ Both services running production code</li><li>✅ HTTP communication successful</li><li>✅ JSON serialization/deserialization working</li><li>✅ Full microservices architecture operational</li></ul><p>Screenshot this output. This is what employers want to see.</p><p>Architecture Visualization</p><p>Here’s what actually happens when you hit `/api/backend`:</p><pre>1. HTTP Request to Node.js Pod<br> └─&gt; node-service-6cdd97d67b-ckxh7<br> │<br>2. Kubernetes DNS Lookup<br> └─&gt; &quot;What&#39;s the IP for go-service?&quot;<br> └─&gt; Answer: &quot;10.43.213.43&quot;<br> │<br>3. HTTP GET to Go Service<br> └─&gt; ClusterIP 10.43.213.43:8080<br> └─&gt; Routes to: go-service-795749c68b-c4wt4<br> │<br>4. Go Service Processes Request<br> └─&gt; Returns JSON: {&quot;data&quot;: […], &quot;count&quot;: 5}<br> │<br>5. Node.js Receives Response<br> └─&gt; Wraps in new JSON object<br> │<br>6. Returns to Client<br> └─&gt; Complete microservices communication!</pre><p>Lessons Learned: Networking Edition</p><p>Lesson 1: Docker Networks Are Isolated by Default</p><pre>❌ Assumption: All Docker containers can talk to each other<br>✅ Reality: Only containers on the same network can communicate<br>🔧 Solution: Ensure DevContainer and k3d on same network</pre><p>Lesson 2: k3d LoadBalancer Can Fail</p><pre>❌ Problem: LoadBalancer repeatedly failing on bridge network<br>⏱️ Impact: 3 cluster creation attempts<br>✅ Solution: Use - no-lb flag for direct server access<br>📚 Learning: Not all k3d features work in all environments</pre><p>Lesson 3: Kubeconfig IPs Must Match Reality</p><pre>❌ Problem: 0.0.0.0 addresses don&#39;t work from containers<br>🔍 Debug: Use docker inspect to find real IPs<br>✅ Solution: sed replace with actual container IP<br>💡 Insight: Kubeconfig is environment-specific</pre><p>Lesson 4: Troubleshooting is Iterative</p><p>The 5-attempt process:</p><p>1. Try localhost → Connection refused<br>2. Find proxy IP → Timeout<br>3. Discover network isolation → Root cause<br>4. Recreate on same network → LoadBalancer fails<br>5. Disable LoadBalancer → SUCCESS!</p><p>Each “failure” provided information that led to the solution.</p><p>Lesson 5: Service Discovery Just Works</p><p>Once networking is correct, Kubernetes DNS “just works”:</p><ul><li>No manual IP management</li><li>Services find each other by name</li><li>Automatic load balancing across replicas</li><li>This is the power of Kubernetes</li></ul><p>The Real DevOps Story</p><p>What tutorials say:</p><p>&gt; “Run kubectl get nodes and see your cluster!”</p><p>What actually happens:</p><p>&gt; “Run kubectl get nodes, get ‘connection refused’, spend 90 minutes understanding Docker networks, k3d architecture, and kubeconfig files, try 5 different approaches, finally succeed, and actually understand how Kubernetes networking works.”</p><p>The difference: Real learning vs. following steps.</p><p>Technical Deep Dive: Why — no-lb Worked</p><p>For the technically curious, here’s what happened:</p><p>With LoadBalancer (failed):</p><pre>kubectl → 0.0.0.0:6550 → [LoadBalancer] → K3s Server<br> ↑<br> Fails to start on bridge network</pre><p>Without LoadBalancer (worked):</p><pre>kubectl → 172.17.0.3:6443 → K3s Server (direct connection)</pre><p>The k3d loadbalancer is designed for host access to the cluster. When running kubectl from **inside** a container, on the **same** Docker network as k3d, the loadbalancer becomes unnecessary complexity.</p><p>Direct connection to the k3s API server is:</p><ul><li>✅ Simpler (fewer moving parts)</li><li>✅ Faster (no proxy overhead)</li><li>✅ More reliable (no loadbalancer to fail)</li><li>✅ Perfectly valid for development</li></ul><p>Production environments would use proper ingress controllers and loadbalancers. Development can be simpler.</p><p><strong><em>What’s Next</em></strong>: Pushing to GitHub</p><p>With microservices deployed and communicating, the local development part is complete.</p><p>But a portfolio project isn’t real until it’s on GitHub with automated CI/CD.</p><p>Part 4 covers:</p><ul><li>Git configuration in DevContainer</li><li>GitHub authentication evolution (spoiler: passwords don’t work anymore)</li><li>Personal Access Token debugging (3 attempts)</li><li>CI/CD pipeline implementation</li><li>GitHub Actions failures and fixes</li><li>Finally: All green pipeline!</li></ul><p>The twist: Even pushing to GitHub requires troubleshooting in 2025. Token permissions, workflow scopes, and security features all have to align perfectly.</p><p>Quick Reference: Network Debugging Commands</p><pre># Container networking<br>docker inspect $(hostname) -f \<br> &#39;{{range $k, $v := .NetworkSettings.Networks}}{{$k}}{{end}}&#39;<br># Find container IP<br>docker inspect -f \<br> &#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39; \<br> CONTAINER_NAME<br># List Docker networks<br>docker network ls<br># k3d without loadbalancer<br>k3d cluster create NAME - network bridge - no-lb - wait<br># Fix kubeconfig<br>K3D_IP=$(docker inspect -f \<br> &#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39; \<br> k3d-CLUSTER-server-0)<br>sed -i &quot;s|https://.*:.*|https://${K3D_IP}:6443|g&quot; ~/.kube/config<br># Verify<br>kubectl cluster-info<br>kubectl get nodes</pre><p><strong><em>Connect &amp; Continue</em></strong></p><p><strong>Part 4 drops next Monday</strong>! The final chapter: GitHub integration, CI/CD pipeline, and the lessons that tie everything together.</p><p>🔗 <strong><em>GitHub Repository</em></strong>: <a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a><br>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a><br>☁️ <strong><em>AWS Community Builder</em></strong>: <a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></p><p>Discussion Question: What’s your most frustrating networking debug session? How long did it take? Drop a comment!</p><p>#Kubernetes #Networking #Docker #k3d #DevOps #Troubleshooting #ServiceMesh #Microservices #AWS #CommunityBuilder</p><p><strong><em>About the Series</em></strong>:</p><p>Part 3 of 4 documenting a production Kubernetes deployment from scratch. Real problems, real solutions, real learning.</p><p><strong><em>Previous</em></strong>:</p><ul><li><a href="https://medium.com/@mrburton/part-1-building-production-kubernetes-on-a-budget-the-setup-journey-f8697becc5db">Part 1 — The Setup Journey</a></li><li><a href="https://medium.com/@mrburton/deployflow-part-2-devcontainers-from-scratch-manual-configuration-and-architecture-nightmares-b5f653ea4dac">Part 2 — Manual Configuration &amp; Architecture Nightmares</a></li></ul><p>Next:</p><ul><li>Part 4 — CI/CD &amp; Production Lessons (Monday)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f88af97e87be" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TerraStack Part 1: “Why Click-Ops is a Trap (And How I Escaped It)”]]></title>
            <link>https://mrburton.medium.com/terrastack-part-1-why-click-ops-is-a-trap-and-how-i-escaped-it-e5ae55496ec7?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5ae55496ec7</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Thu, 11 Dec 2025 13:17:00 GMT</pubDate>
            <atom:updated>2025-12-11T13:17:00.587Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B9w5n3Wp_crJnUz0fi-vEQ.png" /></figure><p><strong><em>The Package Delivery Realization</em></strong></p><p>I deliver packages for <strong><em>Amazon</em></strong>. I’ve been doing it for two years. And last week, I realized something:</p><p>I’m following a script.</p><p><strong><em>Every day</em></strong>: Same route. Same stops. Same process. Turn left here. Park there. Scan. Deliver. Return. The app tells me every move. I don’t question it. I just do it.</p><p>And one day, driving between stops, I thought: “Is this what I want to be doing in 5 years?”</p><p>The answer was <strong>no</strong>.</p><p>So I decided to learn cloud infrastructure. Not because I love computers (I’m okay with them). Not because I’m some coding genius (I barely passed high school math). But because I watched a YouTube video about DevOps salaries and thought: “People make $130K+ to write configuration files?”</p><p>That seemed learnable.</p><p>This is the story of how I went from clicking buttons in the AWS Console to building production-grade infrastructure with code. And I’m going to show you every mistake I made along the way.</p><p>Because that’s how you actually learn.</p><p>The Problem I Didn’t Know I Had</p><p>I started with the AWS Console. It’s beautiful. It’s intuitive. There’s a button for everything.</p><p>Want a server? Click “Launch Instance.” <br>Want a database? Click “Create Database.” <br>Want networking? Click… well, click about 47 things in a very specific order and hope you didn’t miss one.</p><p>I spent my first week just clicking around, creating things, deleting things, trying to understand what a “subnet” even was. (It’s like… a sub-network? Within a network? Why do I need this?)</p><p>And then I made my first real mistake: I created a VPC in the console. Public subnet, private subnet, NAT Gateway (still not 100% sure what that does), Internet Gateway (this one makes sense at least). Took me 2 hours. I wrote everything down in a Google Doc because I knew I’d forget.</p><p>The next day, I wanted to create the same thing for a test environment.</p><p><strong><em>I looked at my notes</em></strong>:</p><pre>1. Go to VPC Dashboard<br>2. Click Create VPC<br>3. Set CIDR to 10.0.0.0/16 (what&#39;s CIDR again?)<br>4. Click Create<br>5. Go to Subnets<br>6. Click Create Subnet<br>7. Pick the VPC (which one? oh right, the one I just made)<br>8. Set CIDR to 10.0.1.0/24 (public) or 10.0.11.0/24 (private)<br>9. Pick availability zone (us-east-1a or 1b? does it matter?)<br>…<br>(32 more steps)</pre><p>And I realized: This is insane.</p><p>If I have to do this manually every single time, I’m going to:</p><p>1. Make mistakes<br>2. Forget steps<br>3. Create slightly different configurations each time<br>4. Have no idea what I did 3 weeks ago<br>5. Never get hired anywhere</p><p>There had to be a better way.</p><p>Enter: <strong><em>Infrastructure as Code</em></strong></p><p>I found a blog post titled “Stop Clicking in the AWS Console.” The author sounded angry. I liked it.</p><p><strong><em>The post said</em></strong>: “Describe your infrastructure in code. Let tools create it.”</p><p>The example was Terraform:</p><pre>resource &quot;aws_vpc&quot; &quot;main&quot; {<br> cidr_block = &quot;10.0.0.0/16&quot;<br> <br> tags = {<br> Name = &quot;my-vpc&quot;<br> }<br>}</pre><p>That’s it. That’s the entire VPC. No screenshots. No “click here then here.” Just 7 lines of code.</p><p>And because it’s code:</p><p>- It’s in Git (version control)<br>- It’s documented (the code IS the documentation)<br>- It’s repeatable (run it 100 times, get the same result)<br>- It’s reviewable (someone can check my work)</p><p>I thought: “This is either genius or a scam.”</p><p>I decided to find out.</p><p>The Learning Curve Nobody Warns You About</p><p>Here’s what nobody tells beginners:</p><p>Infrastructure as Code is easy to understand. It’s hard to set up correctly.</p><p>The concept is simple:</p><p>1. Write code describing what you want<br>2. Run terraform apply<br>3. AWS creates it</p><p>But the reality is:</p><p>1. Where do I write the code?<br>2. How do I organize it?<br>3. Where does Terraform store what it created?<br>4. What if I create something, then lose that information?<br>5. How do I not accidentally create 10 NAT Gateways and get a $3,000 bill?</p><p>These are not questions the tutorials answer. The tutorials show you the happy path. They don’t show you the part where you realize you’ve been storing Terraform state locally and your coworker just created a duplicate VPC because they didn’t have your state file.</p><p>So I decided to do this right from the beginning.</p><p>I would build TerraStack: A complete, production-ready AWS infrastructure with:</p><ul><li>VPC networking (I still barely understood subnets, but I’d figure it out)</li><li>Kubernetes cluster (because everyone keeps talking about Kubernetes)</li><li>PostgreSQL database (because apps need databases)</li><li>Everything in code (no clicking)</li><li>GitOps automation (because professionals use this)</li></ul><p>If I could build this and explain every decision, I could prove I understood DevOps.</p><p>The Architecture (Before I Knew What I Was Doing)</p><p>I drew this on paper first:</p><pre>AWS Account<br>├── VPC (10.0.0.0/16)<br>│ ├── Public Subnets (for load balancers)<br>│ ├── Private Subnets (for the actual apps)<br>│ ├── Internet Gateway (so things can reach the internet)<br>│ └── NAT Gateways (so private subnets can reach internet?)<br>├── EKS Cluster (Kubernetes)<br>│ └── Worker Nodes (these run the containers)<br>└── RDS Database (PostgreSQL)</pre><p>I didn’t understand half of it. But I knew this was the general shape of “production infrastructure.” Every company seems to have some variation of this.</p><p>So I’d build it. And figure out the details as I went.</p><p><strong>The Plan</strong>: Build Small, Build Right</p><p>I decided on a strategy:</p><p><strong>Phase 1</strong>: VPC Only</p><p>- Just networking<br>- Understand one piece before adding complexity<br>- Get it working end-to-end</p><p><strong>Phase 2</strong>: Add EKS</p><p>- Now I have a network<br>- Add Kubernetes on top<br>- Figure out IAM roles (whatever those are)</p><p><strong>Phase 3</strong>: Add RDS</p><p>- Database connects to Kubernetes<br>- Everything talks to each other<br>- Actual working infrastructure</p><p><strong>Phase 4</strong>: Automate Everything</p><p>- GitHub Actions<br>- No more manual `terraform apply`<br>- GitOps like the pros</p><p>This seemed achievable. Start simple. Add complexity gradually. Don’t try to do everything at once.</p><p><strong>The Cost Reality Check</strong></p><p>Before I started, I looked up AWS pricing.</p><p><strong><em>My estimated cost</em></strong>:</p><ul><li>EKS Control Plane: $73/month (ouch)</li><li>2× Worker Nodes (t3.medium): $60/month</li><li>NAT Gateways (2×): $65/month (WHAT?!)</li><li>RDS (db.t3.micro): $15/month</li><li>Misc (data transfer, storage): $15/month</li></ul><p><strong><em>Total: ~$228/month</em></strong></p><p>That’s more than my internet bill.</p><p>But here’s my plan: I’ll build it, document it, take screenshots, then destroy it. I’m not keeping a $228/month infrastructure running while I’m job hunting. I’ll rebuild it for interviews (takes 15 minutes with Terraform).</p><p><strong><em>Total project cost</em></strong>: Maybe $50–75 over 3 weeks.</p><p>That’s an investment I can justify. Especially if it gets me a job.</p><p>The Tools I Needed to Learn</p><p>I made a list:</p><p><strong>Must Learn</strong>:</p><ul><li>Terraform (Infrastructure as Code tool)</li><li>AWS CLI (for initial setup)</li><li>kubectl (Kubernetes command line)</li><li>Git (version control)</li><li>Basic Linux commands</li></ul><p><strong><em>Nice to Have</em></strong>:</p><ul><li>Docker (for understanding containers)</li><li>Helm (Kubernetes package manager)</li><li>GitHub Actions (CI/CD automation)</li></ul><p>I had some Git experience from a failed attempt to learn programming Everything else? New.</p><p><strong><em>The DevContainer Decision</em></strong></p><p>One thing I learned from my failed programming attempt: ”Works on my machine” is a trap.</p><p>I spent 3 hours once trying to figure out why my Python code worked on my laptop but not my friend’s. Turns out I had Python 3.9, they had 3.11, and something changed between versions.</p><p>I didn’t want that problem with Terraform.</p><p><strong><em>So I decided to use DevContainers</em></strong>: A Docker container with all the tools pre-installed. Same environment every time. If it works in the container, it works anywhere.</p><p>Plus, it looks professional. <strong><em>Companies use this</em></strong>.</p><p>What I’m Building vs. What Production Needs</p><p><strong><em>Let me be clear</em></strong>: I’m building a development environment. Not production.</p><p><strong>What I’m building</strong>:</p><ul><li>Single-AZ RDS (if the database fails, downtime is acceptable)</li><li>2 worker nodes (minimum for high availability)</li><li>Basic monitoring (CloudWatch logs)</li><li>Cost-optimized (cheapest options that still demonstrate knowledge)</li></ul><p><strong>What production needs</strong>:</p><ul><li>Multi-AZ RDS (automatic failover)</li><li>6+ worker nodes (handle traffic spikes)</li><li>Comprehensive monitoring (Prometheus, Grafana, alerts)</li><li>Multiple environments (dev, staging, prod)</li><li>Disaster recovery plan</li><li>Cost: $800–1,200/month</li></ul><p>The goal isn’t to build production infrastructure on my credit card. The goal is to prove I understand the principles.</p><p>The Doubt</p><p>Before I started writing any code, I had a moment where I thought:</p><p>“Am I really going to do this?”</p><p>I’m a package delivery driver. I don’t have a computer science degree. I barely understand what a ‘subnet mask’ is. And I’m trying to build the kind of infrastructure that companies pay people $130K+ to manage.</p><p>But then I thought about the alternative: Delivering packages in 5 years. 10 years. Until my knees give out.</p><p>And I decided: I’d rather fail trying to learn this than succeed at staying where I am.</p><p>So I opened VS Code and created a new directory:</p><pre>mkdir terrastack<br>cd terrastack</pre><p>And I got started.</p><p>What’s Coming in This Series</p><p><strong><em>Part 1</em></strong> (this one): Why I’m doing this and what I’m building <br><strong><em>Part 2</em></strong>: Setting up the tools and building the VPC (with all my mistakes) <br><strong><em>Part 3</em></strong>: Adding the EKS cluster (where I learn what IAM roles actually do) <br><strong>Part 4</strong>: Adding the database (and figuring out why it shouldn’t be public) <br><strong>Part 5</strong>: Automating everything with GitHub Actions <br><strong>Part 6</strong>: Cost analysis and lessons learned</p><p>I’m not going to pretend this was easy. I’m not going to skip the parts where I was confused. And I’m definitely not going to skip the parts where things broke.</p><p><strong><em>Because that’s the real learning</em></strong>.</p><p><strong>One More Thing</strong></p><p>This isn’t a tutorial. I’m not teaching you Terraform. There are better resources for that.</p><p><strong>This is a build log</strong>. My journal of going from “What’s a subnet?” to “I just deployed a Kubernetes cluster with Infrastructure as Code.”</p><p>If you’re also trying to break into tech from a non-traditional background, this is for you. Because I want you to see:</p><p><strong>It’s hard. You’ll get stuck. You’ll break things. And you can still do it</strong>.</p><p>See you in Part 2, where I create a VPC and immediately run into problems I didn’t know existed.</p><p>💡 KEY TAKEAWAYS</p><p>1. AWS Console clicking doesn’t scale (and won’t get you hired)<br>2. Infrastructure as Code = version control for infrastructure<br>3. TerraStack goal: VPC + EKS + RDS with GitOps automation<br>4. Cost: ~$228/month running, ~$75 total for build/destroy approach<br>5. This is a learning project, not production infrastructure<br>6. DevContainers solve “works on my machine” problems<br>7. I’m documenting mistakes, not just successes</p><p>#DevOps #SelfTaught #InfrastructureAsCode #Terraform #AWS #LearningInPublic #CareerChange</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5ae55496ec7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeployFlow Part 2: DevContainers from Scratch: Manual Configuration and Architecture Nightmares]]></title>
            <link>https://mrburton.medium.com/deployflow-part-2-devcontainers-from-scratch-manual-configuration-and-architecture-nightmares-b5f653ea4dac?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5f653ea4dac</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Wed, 10 Dec 2025 01:09:48 GMT</pubDate>
            <atom:updated>2025-12-10T01:24:05.224Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w2mDVz_NZcDFuDkd86Q17Q.png" /></figure><p>Everything was going perfectly… until I tried to run `go version`. What followed was 30 minutes of architecture debugging that taught me more than any tutorial ever could.</p><p><strong>Quick Links to Series</strong>:</p><ul><li>✅ <a href="https://mrburton.medium.com/part-1-building-production-kubernetes-on-a-budget-the-setup-journey-f8697becc5db"><strong><em>Part 1: The Setup Journey</em></strong></a> (Read first if you missed it)</li><li>📚 <strong><em>Part 2: Manual Config &amp; Architecture Nightmares</em></strong> ← You are here</li><li>📚 <strong><em>Part 3: Kubernetes Networking Deep Dive</em></strong> (Coming 12/12/25)</li><li>📚 <strong><em>Part 4: CI/CD &amp; Production Lessons</em></strong> (Coming 12/15/25)</li></ul><p>🔗 <strong>Full Project Code</strong>: <a href="http://github.com/BenjaminBurton/deployflow">github.com/BenjaminBurton/deployflow</a></p><p><strong>Recap: Where We Left Off</strong></p><p>In Part 1, I established my development environment:</p><ul><li>✅ SSH access from Mac to Lenovo (32GB RAM, Arch Linux)</li><li>✅ Docker installed and configured</li><li>✅ DevContainer CLI ready to use</li></ul><p>Now it’s time to actually build something. <strong><em>The plan</em></strong>: create all configuration files manually, spin up a DevContainer, and start working on microservices.</p><p><strong>Spoiler alert</strong>: The “start working” part doesn’t happen quite as planned.</p><p><strong>The Philosophy</strong>: Manual Over Templates</p><p>Before diving in, let me explain a decision that would define this entire project:</p><p>I chose to create every single file manually via command line.</p><p>No git clone. No templates. No copying from tutorials.</p><p>Why Make Life Harder?</p><p><strong>Educational Value</strong>:</p><ul><li>You understand what each file does</li><li>You can’t skip over “magic” configuration</li><li>Troubleshooting becomes possible (you know what you built)</li></ul><p><strong>Portfolio Differentiation</strong>:</p><ul><li><strong><em>Most tutorials</em></strong>: “Clone this repo and run it”</li><li><strong><em>Real engineers</em></strong>: “I built this from first principles”</li></ul><p><strong>Professional Practice</strong>:</p><ul><li>Production environments require understanding, not just running</li><li>You’ll need to modify configs — better know how they work</li></ul><p><strong><em>Let’s build</em></strong>.</p><p><strong>Creating the Project Structure</strong></p><pre># Create project directory<br>mkdir -p ~/projects/deployflow<br>cd ~/projects/deployflow<br><br># Verify location<br>pwd<br># Output: /home/archv/projects/deployflow</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h1SidNHXTcklyM7kfeWPyQ.png" /></figure><p><strong>File 1</strong>: The Main README</p><p>Every project needs documentation. I started with the README:</p><pre>cat &gt; README.md &lt;&lt; &#39;EOF&#39;<br># DeployFlow<br><br>A production-ready CI/CD pipeline demonstrating containerized <br>microservices deployment with automated testing, security <br>scanning, and Kubernetes orchestration.<br><br>## 🏗️ Architecture<br><br>DeployFlow consists of two microservices:<br>- **Node.js Express API**: Frontend service handling HTTP requests<br>- **Go HTTP Server**: Backend service for business logic<br><br>Both services are containerized, tested, and deployed to <br>Kubernetes through automated CI/CD pipelines.<br><br>## 🚀 Features<br><br>- **Automated CI/CD Pipeline**: GitHub Actions workflow<br>- **Container Security Scanning**: Vulnerability scanning with Trivy<br>- **Multi-Stage Docker Builds**: Optimized container images<br>- **Kubernetes Deployment**: Automated deployment to K3s cluster<br>- **Health Checks**: Liveness and readiness probes<br>- **Service Discovery**: Internal service communication<br><br>## 📋 Prerequisites<br><br>- Kubernetes cluster (K3s or EKS)<br>- Docker<br>- kubectl<br>- GitHub account<br><br>## 🚀 Quick Start with DevContainer<br><br>```bash<br># Build and start DevContainer<br>devcontainer up --workspace-folder .<br><br># Exec into container<br>devcontainer exec --workspace-folder . bash<br><br>🌐 API Endpoints<br>Node.js Service (Port 3000)<br>GET /  - Health check<br>GET /api/status  - Service status<br>GET /api/backend  - Calls Go service<br><br>Go Service (Port 8080)<br>GET /  - Health check<br>GET /health  - Health probe<br>GET /api/data  - Returns sample data<br></pre><p><strong><em>Why this structure matters</em></strong>:</p><ul><li>Clear architecture description</li><li>Feature highlights for portfolio</li><li>API documentation (shows planning)</li><li>Quick start instructions</li></ul><p>The Core Files Created</p><p>Over the next hour, I created 14 files manually. Here’s what was built:</p><p><strong>Project Structure</strong>:</p><pre>deployflow/ <br>├── README.md <br>├── .devcontainer/ <br>│ ├── devcontainer.json <br>│ └── post-create.sh <br>├── .github/ <br>│ └── workflows/ <br>│ └── ci-cd.yml <br>├── node-service/ <br>│ ├── server.js <br>│ ├── package.json <br>│ ├── Dockerfile <br>│ └── k8s/ <br>│ ├── deployment.yaml <br>│ └── service.yaml <br>└── go-service/ <br>├── main.go <br>├── go.mod <br>├── Dockerfile <br>└── k8s/ <br>├── deployment.yaml <br>└── service.yaml</pre><p><strong>Note</strong>: The full file contents are in the <a href="https://github.com/BenjaminBurton/deployflow">GitHub repository</a>. For brevity, I’ll focus on the DevContainer configuration and the critical lessons learned.</p><p><strong><em>The DevContainer Configuration</em></strong></p><p>This is where things get interesting. The DevContainer configuration determines your entire development environment.</p><p><strong>File</strong>: .devcontainer/devcontainer.json</p><pre>{<br>  &quot;name&quot;: &quot;DeployFlow DevContainer&quot;,<br>  &quot;image&quot;: &quot;ubuntu:22.04&quot;,<br>  <br>  &quot;customizations&quot;: {<br>    &quot;vscode&quot;: {<br>      &quot;extensions&quot;: [<br>        &quot;ms-azuretools.vscode-docker&quot;,<br>        &quot;ms-kubernetes-tools.vscode-kubernetes-tools&quot;,<br>        &quot;golang.go&quot;,<br>        &quot;dbaeumer.vscode-eslint&quot;,<br>        &quot;esbenp.prettier-vscode&quot;,<br>        &quot;redhat.vscode-yaml&quot;,<br>        &quot;hashicorp.terraform&quot;,<br>        &quot;github.vscode-github-actions&quot;<br>      ],<br>      &quot;settings&quot;: {<br>        &quot;terminal.integrated.defaultProfile.linux&quot;: &quot;bash&quot;,<br>        &quot;editor.formatOnSave&quot;: true,<br>        &quot;go.toolsManagement.autoUpdate&quot;: true<br>      }<br>    }<br>  },<br><br>  &quot;forwardPorts&quot;: [3000, 8080],<br>  <br>  &quot;postCreateCommand&quot;: &quot;bash .devcontainer/post-create.sh&quot;,<br><br>  &quot;mounts&quot;: [<br>    &quot;source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind&quot;<br>  ],<br><br>  &quot;remoteUser&quot;: &quot;root&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eRuMEjy1J5JSrU7FiRcj6Q.png" /></figure><p><strong>Key Decisions</strong>:</p><p>1. Plain Ubuntu 22.04 base — No fancy features from registries (avoiding authentication issues)<br>2. Docker socket mount — Access host Docker from inside container<br>3. Root user — Simplifies permissions (not production practice, but okay for development)<br>4. Post-create script — Install all tools via shell script<br>5. Port forwarding — Services accessible from host</p><p><strong>File</strong>: .devcontainer/post-create.sh</p><p>This script does the heavy lifting — installing every tool we need:</p><pre>#!/bin/bash<br><br>echo &quot;🚀 Setting up DeployFlow DevContainer...&quot;<br><br># Update system<br>apt-get update<br>apt-get install -y curl wget git vim tmux zsh build-essential \<br>  ca-certificates gnupg lsb-release<br><br># Install Docker CLI<br>mkdir -p /etc/apt/keyrings<br>curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \<br>  gpg --dearmor -o /etc/apt/keyrings/docker.gpg<br>echo &quot;deb [arch=$(dpkg --print-architecture) \<br>  signed-by=/etc/apt/keyrings/docker.gpg] \<br>  https://download.docker.com/linux/ubuntu \<br>  $(lsb_release -cs) stable&quot; | \<br>  tee /etc/apt/sources.list.d/docker.list &gt; /dev/null<br>apt-get update<br>apt-get install -y docker-ce-cli<br><br># Install Node.js 24<br>curl -fsSL https://deb.nodesource.com/setup_24.x | bash -<br>apt-get install -y nodejs<br><br># Install Go 1.24 (ARM64 - WRONG! This will cause problems...)<br>wget https://go.dev/dl/go1.24.0.linux-arm64.tar.gz<br>rm -rf /usr/local/go &amp;&amp; tar -C /usr/local -xzf go1.24.0.linux-arm64.tar.gz<br>rm go1.24.0.linux-arm64.tar.gz<br>echo &#39;export PATH=$PATH:/usr/local/go/bin&#39; &gt;&gt; ~/.bashrc<br>export PATH=$PATH:/usr/local/go/bin<br><br># Install kubectl (ARM64 - WRONG! This too...)<br>curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s \<br>  https://dl.k8s.io/release/stable.txt)/bin/linux/arm64/kubectl&quot;<br>chmod +x kubectl<br>mv kubectl /usr/local/bin/<br><br># Install Helm<br>curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash<br><br># Install Neovim<br>apt-get install -y neovim<br><br># Install Oh My Zsh<br>sh -c &quot;$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)&quot; &quot;&quot; --unattended<br>chsh -s $(which zsh)<br><br># Install k3d<br>curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash<br><br># Create k3d cluster<br>k3d cluster create deployflow-cluster \<br>  --api-port 6550 \<br>  --servers 1 \<br>  --agents 0 \<br>  --port &quot;3000:30000@server:0&quot; \<br>  --port &quot;8080:30001@server:0&quot; \<br>  --wait<br><br># Set up kubeconfig<br>mkdir -p ~/.kube<br>k3d kubeconfig get deployflow-cluster &gt; ~/.kube/config<br>export KUBECONFIG=~/.kube/config<br><br># Verify cluster<br>kubectl cluster-info<br>kubectl get nodes<br><br># Install Node.js dependencies<br>if [ -d &quot;node-service&quot; ]; then<br>  cd node-service &amp;&amp; npm install &amp;&amp; cd ..<br>fi<br><br># Initialize Go module<br>if [ -d &quot;go-service&quot; ]; then<br>  cd go-service &amp;&amp; go mod tidy &amp;&amp; cd ..<br>fi<br><br>echo &quot;✨ DevContainer setup complete!&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mv9tSVf9rkAtV7Wne4_Euw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kRD694z-2w0Yhg-s3-wHXA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C0P--KDLQdxOJIn1R5KJ0Q.png" /></figure><p>Make it executable:</p><pre>chmod +x .devcontainer/post-create.sh</pre><p>Did you spot the problem? I downloaded ARM64 binaries for Go and kubectl. My Lenovo has an Intel x86_64 CPU. This will fail spectacularly.</p><p>But I didn’t know that yet.</p><p>Building the DevContainer</p><p>With all files created, time to build:</p><pre>cd ~/projects/deployflow<br><br># Build and start DevContainer<br>devcontainer up --workspace-folder .</pre><p>The build process began:</p><pre>@devcontainers/cli 0.80.2. Node.js v22.20.0. linux 6.x.x x86_64.<br>Start: Run: docker buildx version<br>Start: Run: docker inspect --type image ubuntu:22.04<br>[... pulling ubuntu:22.04 ...]<br>Start: Starting container<br>Start: Running postCreateCommand...</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XYysc5pBNZFgnIxsmb5oqg.png" /></figure><p>For the next 10 minutes, I watched:</p><pre>🚀 Setting up DeployFlow DevContainer...<br>📦 Updating system...<br>📦 Installing Docker CLI...<br>📦 Installing Node.js 24...<br>📦 Installing Go 1.24...<br>📦 Installing kubectl...<br>📦 Installing Helm...<br>📦 Installing Neovim...<br>📦 Installing Oh My Zsh...<br>🐳 Installing k3d...<br>🎯 Creating k3d cluster...<br>INFO[0001] Creating node &#39;k3d-deployflow-cluster-server-0&#39;<br>INFO[0007] Cluster &#39;deployflow-cluster&#39; created successfully!<br>✅ Verifying cluster...</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k8Q7qGesWhwxdwn_31mhOw.png" /></figure><p><strong>Final output</strong>:</p><pre>✨ DevContainer setup complete!<br>🚀 Ready to deploy!<br>{&quot;outcome&quot;:&quot;success&quot;}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vNxuXskloyGezeRDtBOw4g.png" /></figure><p><strong>Success! Or so I thought</strong>…</p><p><strong>Entering the Container</strong></p><pre># Execute into the running container<br>devcontainer exec --workspace-folder . bash</pre><p><strong>New prompt</strong>:</p><pre>root@3f9460f9cc90:/workspaces/deployflow#</pre><p>I was inside! Time to verify everything worked.</p><p>The Architecture Nightmare Begins</p><p>Let’s test the tools:</p><p><strong>Test 1</strong>: Node.js</p><pre>root@3f9460f9cc90:/workspaces/deployflow# node --version<br>v24.11.1</pre><p>✅ <strong>Perfect</strong>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lk4MiennldPFjVUpOfUrkg.png" /></figure><p><strong>Test 2</strong>: Go</p><pre>root@3f9460f9cc90:/workspaces/deployflow# go version<br>bash: /usr/local/go/bin/go: cannot execute binary file: Exec format error</pre><p>❌ Wait, what?</p><p><strong>Test 3</strong>: kubectl</p><pre>root@3f9460f9cc90:/workspaces/deployflow# kubectl version --client<br>bash: /usr/local/bin/kubectl: cannot execute binary file: Exec format error</pre><p>❌ Same error!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8_qqp1mjTCw3YQkIACpdiA.png" /></figure><p>That error message — “cannot execute binary file: Exec format error” — would haunt my next 30 minutes.</p><p><strong>Debugging</strong>: The Architecture Mismatch</p><p>When a binary won’t execute with “Exec format error”, it usually means one thing: wrong CPU architecture.</p><p><strong>Check the container’s architecture</strong>:</p><pre>uname -m<br># Output: x86_64<br><br>dpkg --print-architecture<br># Output: amd64</pre><p>The container is x86_64 (amd64).</p><p><strong>Check what I downloaded</strong>:</p><p><strong><em>Looking back at my post-create.sh script</em></strong>:</p><pre>wget https://go.dev/dl/go1.24.0.linux-arm64.tar.gz  # ← ARM64!<br>curl -LO &quot;.../bin/linux/arm64/kubectl&quot;               # ← ARM64!</pre><p><strong>The Problem</strong>:</p><ul><li>Lenovo ThinkPad: Intel x86_64 CPU</li><li>DevContainer: Inherits x86_64 architecture from host</li><li>Downloaded binaries: ARM64 (for Apple Silicon Macs)</li><li>ARM64 binaries cannot execute on x86_64 systems</li></ul><p><strong>Why did I download ARM64</strong>?</p><p>I originally planned this project on my MacBook Air M1 (which IS ARM64). When I pivoted to using the Lenovo, I forgot to update the binary URLs in my script.</p><p>The lesson: Always verify target architecture before downloading binaries.</p><p><strong>The Fix</strong>: Correct Architecture Binaries</p><p><strong>Fixing Go</strong></p><pre># Remove incorrect Go binary<br>rm -rf /usr/local/go<br><br># Download correct x86_64 version<br>cd /tmp<br>wget https://go.dev/dl/go1.24.0.linux-amd64.tar.gz<br>tar -C /usr/local -xzf go1.24.0.linux-amd64.tar.gz<br>export PATH=$PATH:/usr/local/go/bin<br>echo &#39;export PATH=$PATH:/usr/local/go/bin&#39; &gt;&gt; ~/.bashrc<br><br># Test Go<br>go version</pre><p><strong>Output</strong>:</p><pre>go version go1.24.0 linux/amd64</pre><p>✅ <strong>Success</strong>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l7zOjH-nT-D1WlqDOWrHEA.png" /></figure><p><strong>Fixing kubectl</strong></p><pre># Remove incorrect kubectl<br>rm /usr/local/bin/kubectl<br><br># Download correct x86_64 version<br>cd /tmp<br>curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s \<br>  https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl&quot;<br>chmod +x kubectl<br>mv kubectl /usr/local/bin/<br><br># Test kubectl<br>kubectl version --client</pre><p><strong>Output</strong>:</p><pre>Client Version: v1.34.2<br>Kustomize Version: v5.7.1</pre><p>✅ <strong>Success</strong>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELomJW8ZVgNGoZpjkyb2Vw.png" /></figure><p>Verifying K3d Cluster</p><p>With kubectl now working, k3d cluster actually created successfully:</p><pre># Check k3d installation<br>k3d version<br># Output: k3d version v5.8.3<br><br># List clusters<br>k3d cluster list</pre><p><strong>Output</strong>:</p><pre>NAME                 SERVERS   AGENTS   LOADBALANCER<br>deployflow-cluster   1/1       0/0      true</pre><p>The cluster exists! It was created during the post-create script, even though we couldn’t test it at the time due to the broken kubectl.</p><p>What We Learned</p><p><strong>Lesson 1</strong>: Architecture Compatibility is Critical</p><pre>❌ Problem: Downloaded ARM64 binaries on x86_64 system<br>⏱️ Impact: 30 minutes of debugging<br>✅ Solution: Always verify `uname -m` before downloading<br>🛡️ Prevention: Use $(dpkg --print-architecture) in scripts</pre><p><strong><em>The better post-create.sh would be</em></strong>:</p><pre># Detect architecture<br>ARCH=$(dpkg --print-architecture)<br><br># Download correct binary<br>if [ &quot;$ARCH&quot; = &quot;amd64&quot; ]; then<br>    wget https://go.dev/dl/go1.24.0.linux-amd64.tar.gz<br>elif [ &quot;$ARCH&quot; = &quot;arm64&quot; ]; then<br>    wget https://go.dev/dl/go1.24.0.linux-arm64.tar.gz<br>fi</pre><p><strong><em>Lesson 2</em></strong>: DevContainers Are Powerful But Not Magic</p><p><strong><em>DevContainers give you</em></strong>:</p><ul><li>✅ Isolated environments</li><li>✅ Reproducible setups</li><li>✅ Easy sharing (devcontainer.json)</li></ul><p><strong><em>DevContainers don’t protect you from</em></strong>:</p><ul><li>❌ Downloading wrong binaries</li><li>❌ Architecture mismatches</li><li>❌ Network issues</li><li>❌ Basic shell script errors</li></ul><p>You still need to understand what you’re building.</p><p><strong>Lesson 3</strong>: Manual File Creation Has Real Value</p><p>Because I created every file manually, I:</p><ul><li>✅ Understood the post-create.sh script intimately</li><li>✅ Could quickly identify and fix the architecture issue</li><li>✅ Knew exactly what each tool was supposed to do</li><li>✅ Built genuine troubleshooting skills</li></ul><p>If I had just cloned a template: I’d have been completely lost when things broke.</p><p><strong>Lesson 4</strong>: Error Messages Are Your Friends</p><p>“cannot execute binary file: Exec format error”</p><p>This error message was actually very helpful — it told me exactly what was wrong. Many beginners see errors as scary. Experienced engineers see them as diagnostic tools.</p><p><strong><em>The debugging process</em></strong>:</p><p>1. Error message → “cannot execute”<br>2. Hypothesis → “wrong permissions or wrong architecture”<br>3. Check → `uname -m` shows x86_64<br>4. Compare → script shows arm64 downloads<br>5. Root cause → architecture mismatch<br>6. Fix → download correct architecture<br>7. Verify → commands work</p><p><strong>The Current State</strong></p><p>After fixing the architecture issues, I had:</p><p>✅ <strong>Working Tools</strong>:</p><ul><li>Node.js v24.11.1</li><li>Go v1.24.0 (linux/amd64)</li><li>kubectl v1.34.2</li><li>Helm v3.x</li><li>k3d v5.8.3</li><li>Neovim, Zsh, Docker CLI</li></ul><p>✅ <strong>Running Infrastructure</strong>:</p><ul><li>DevContainer successfully built</li><li>k3d cluster created and running</li><li>All project files in place</li></ul><p>✅ <strong>Skills Gained</strong>:</p><ul><li>Binary architecture debugging</li><li>DevContainer troubleshooting</li><li>Systematic problem-solving</li><li>Manual configuration experience</li></ul><p><strong>Time invested</strong>: ~2 hours (1 hour file creation, 30 min debugging, 30 min setup)</p><p><strong>What’s Next</strong>: The Real Challenge</p><p>With tools working and a cluster running, you’d think we’re ready to deploy.</p><p>Not quite.</p><p><strong>In Part 3</strong>, we’ll face the most challenging networking puzzle yet:</p><ul><li>kubectl can see the cluster… but can’t connect to it</li><li>5 different troubleshooting approaches</li><li>Docker network isolation issues</li><li>k3d LoadBalancer failures</li><li>Finally achieving `kubectl get nodes` success</li></ul><p><strong>The twist</strong>: Even when the cluster is running and kubectl is installed correctly, getting them to actually communicate would require understanding Docker networking at a deep level.</p><p><strong>Quick Reference</strong>: Architecture Debugging</p><p>For your future self (and other engineers):</p><pre># Check system architecture<br>uname -m                    # Shows: x86_64, aarch64, armv7l, etc.<br>dpkg --print-architecture   # Shows: amd64, arm64, armhf, etc.<br><br># Common architectures<br>x86_64  = amd64   (Intel/AMD)<br>aarch64 = arm64   (Apple Silicon, AWS Graviton)<br>armv7l  = armhf   (Raspberry Pi, older ARM)<br><br># Test if binary is correct architecture<br>file /usr/local/go/bin/go<br># Good: &quot;ELF 64-bit LSB executable, x86-64&quot;<br># Bad:  &quot;ELF 64-bit LSB executable, ARM aarch64&quot;<br><br># Prevention in scripts<br>ARCH=$(dpkg --print-architecture)<br>case &quot;$ARCH&quot; in<br>    amd64)  URL_SUFFIX=&quot;linux-amd64&quot; ;;<br>    arm64)  URL_SUFFIX=&quot;linux-arm64&quot; ;;<br>    *)      echo &quot;Unsupported arch: $ARCH&quot;; exit 1 ;;<br>esac<br>wget https://example.com/binary-${URL_SUFFIX}.tar.gz</pre><p>The Real DevOps Story</p><p><strong>Tutorials say</strong>:</p><p>&gt; “Run this command and everything works!”</p><p><strong>Reality is</strong>:</p><p>&gt; “Run this command, get an error, spend 30 minutes debugging, learn about CPU architectures, fix it, document it, and become a better engineer.”</p><p>This is the difference between following tutorials and building real projects. The failures teach more than the successes.</p><p><strong>Connect &amp; Continue</strong></p><p><em>Part 3 drops Friday</em>! The networking deep dive where we discover that just because something is “running” doesn’t mean you can connect to it.</p><ul><li>🔗<strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>📧 <strong><em>Email</em></strong>: benjaminLburton@outlook.com</li></ul><p><strong><em>Discussion Question</em></strong>: What’s your most memorable “wrong architecture” debugging session? Drop a comment below!</p><p>#Kubernetes #DevOps #DevContainers #Docker #Debugging #ARM64 #x86_64 #TroubleshootingStories #AWS #CommunityBuilder</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5f653ea4dac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeployFlow Part 1: Building Production Kubernetes on a Budget: The Setup Journey]]></title>
            <link>https://mrburton.medium.com/part-1-building-production-kubernetes-on-a-budget-the-setup-journey-f8697becc5db?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8697becc5db</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Mon, 08 Dec 2025 00:26:40 GMT</pubDate>
            <atom:updated>2025-12-16T03:04:26.187Z</atom:updated>
            <content:encoded><![CDATA[<p>How I built a production-grade infrastructure on a Lenovo laptop while delivering packages in NYC winter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PPC3fFTf1adg7dos5nJ7HQ.png" /></figure><p><strong>Quick Links to Series</strong>:</p><p>📚 <strong>Part 1</strong>: <strong><em>The Setup Journey</em></strong> ← You are here<br>📚 <strong>Part 2</strong>: <a href="https://mrburton.medium.com/deployflow-part-2-devcontainers-from-scratch-manual-configuration-and-architecture-nightmares-b5f653ea4dac"><strong><em>Manual Config &amp; Architecture Nightmares</em></strong></a> (Available Now)<br>📚 <strong>Part 3</strong>: <strong><em>Kubernetes Networking Deep Dive</em></strong> (Coming 12/12/25)<br>📚 <strong>Part 4</strong>: <strong><em>CI/CD &amp; Production Lessons</em></strong> (Coming 12/15/25)</p><p><strong>The Challenge</strong></p><p>It’s December in <strong><em>New York</em></strong>. I’m working the midnight shift delivering packages for <strong><em>Amazon</em></strong>, freezing my ass off (I’m from California), and thinking about one thing: how do I prove I can do DevOps work without having done DevOps work?</p><p><strong>The answer</strong>? Build something real. Not a tutorial. Not a “follow along with me” project. Something that would break, force me to troubleshoot, and demonstrate actual problem-solving skills.</p><p><strong>The Goal</strong>: Create a production-ready CI/CD pipeline with containerized microservices on Kubernetes. From scratch. Using command-line tools, and Documenting every failure.</p><p><strong>The Reality</strong>: This turned into a 5–6 hour troubleshooting session that taught me more about DevOps than months of tutorials ever could.</p><p><strong><em>This is Part 1 of that journey</em></strong>.</p><p><strong>The Hardware Reality Check</strong></p><p>Let me be honest about something most tutorials skip: hardware matters.</p><p>I started planning this project on my MacBook Air M1. Great machine. 8GB RAM. Perfect for everyday work. Completely insufficient for what I was attempting.</p><p>Here’s the math that changed my approach:</p><pre>Kubernetes Requirements (Minimum):<br>├─ K3s per node: 512MB RAM<br>├─ Multi-node cluster: 2-3GB<br>├─ DevContainer overhead: 1-2GB  <br>├─ Development tools: 1-2GB<br>└─ Total needed: 5-8GB minimum<br><br>My MacBook: 8GB total<br>Reality: Not enough headroom</pre><p>The Decision: Use my Lenovo ThinkPad T490 (32GB RAM, Intel x86_64, running Arch Linux) as the primary development machine, and SSH in from my Mac for workflow management and screenshots.</p><p><strong>Why This Matters for DevOps</strong></p><p>This isn’t just a hardware constraint story. It’s a fundamental DevOps principle: resource planning is critical.</p><p>In production, you can’t just throw hardware at problems. You need to:</p><p>1. Understand actual resource requirements<br>2. Plan for overhead and headroom<br>3. Make informed decisions about architecture<br>4. Know when to pivot your approach</p><p>Real DevOps work means working within constraints, not pretending they don’t exist.</p><p><strong>Network Architecture: Making It Work</strong></p><p>Now I had a new problem: How do I work comfortably on my Mac while running everything on the Lenovo?</p><p><strong><em>My Setup</em></strong>:</p><ul><li>iPhone 13 Mini (cellular hotspot)</li><li>MacBook Air (WiFi connection)</li><li>Lenovo ThinkPad (USB tethering)</li></ul><p><strong><em>Network Topology</em></strong>:</p><pre>iPhone Hotspot (172.20.10.0/28)<br>    ├─ MacBook Air (WiFi) → 172.20.10.X<br>    └─ Lenovo (USB Tethering) → 172.20.10.6</pre><p><strong>Why USB tethering for the Lenovo</strong>?</p><ul><li>More stable than WiFi for long development sessions</li><li>Lower latency for SSH operations</li><li>Conserves iPhone battery vs dual WiFi connections</li></ul><p>Finding the Lenovo’s IP Address</p><p><strong><em>First challenge</em></strong>: What’s the Lenovo’s IP address?</p><p><strong>On Arch Linux, I ran</strong>:</p><pre># List all network interfaces<br>ip link</pre><p><strong>My Output showed</strong>:</p><pre>1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; ...<br>2: enp0s20f0u1: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; ...  ← USB tether<br>3: wlan0: &lt;BROADCAST,MULTICAST&gt; ...</pre><p>That enp0s20f0u1 is the USB tethering interface. The naming follows systemd’s predictable network interface naming scheme — USB ethernet devices get unique identifiers.</p><pre># Get IP address from USB interface<br>ip addr show enp0s20f0u1</pre><p><strong>My Output</strong>:</p><pre>inet 172.20.10.6/28 brd 172.20.10.15 scope global dynamic</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qQILW2SAHjpHS9aKahfwXA.jpeg" /></figure><p><strong><em>Perfect</em></strong>. My Lenovo is at 172.20.10.6</p><p><strong>SSH Access: The Missing Piece</strong></p><p>With the IP in hand, I tried to SSH from my Mac:</p><pre>ssh archv@172.20.10.6</pre><p><strong><em>Error encountered</em></strong>:</p><pre>ssh: connect to host 172.20.10.6 port 22: No route to host</pre><p>Problem: OpenSSH server wasn’t installed on Arch by default. (Arch is minimal — you install only what you need.)</p><p><strong><em>On the Lenovo</em></strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*peIjcbYMTJEXK6gu44Hrrg.jpeg" /></figure><pre># Install OpenSSH server<br>sudo pacman -S openssh<br><br># Start the SSH service<br>sudo systemctl start sshd<br><br># Verify it&#39;s running<br>sudo systemctl status sshd<br><br># Enable automatic start on boot<br>sudo systemctl enable sshd</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PaRZNWdfjBkJc4yaPJFbOg.jpeg" /></figure><p><strong><em>Output</em></strong>:</p><pre>● sshd.service - OpenSSH Daemon<br>     Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled)<br>     Active: active (running) since Wed 2025-12-03 22:00:00 UTC</pre><p><strong><em>Quick firewall check</em></strong>:</p><pre>sudo systemctl status iptables<br># Output: inactive (dead)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WBMcP2yGxlCXRd9EkUxEKw.jpeg" /></figure><p>No firewall blocking, SSH port 22 is open by default.</p><p><strong><em>Back on my Mac</em></strong>:</p><pre>ssh archv@172.20.10.6<br># Prompted for password<br># Successfully connected!</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cHMqY6S6prcZeCaJjWrIZg.png" /></figure><p><strong>Success! Now I have my workflow</strong>:</p><ul><li>💻 Mac terminal for SSH commands and workflow management</li><li>📸 Mac screenshot tools (Cmd+Shift+4) for documentation</li><li>⚙️ Lenovo executing all actual development work</li><li>📱 iPhone providing stable internet connectivity</li></ul><p><strong>Docker: The Foundation</strong></p><p>Before DevContainers, I needed Docker.</p><p><strong><em>Why Docker First</em></strong>?</p><p>DevContainers require a container runtime. Docker is the standard choice because:</p><ul><li>Wide compatibility</li><li>Extensive documentation</li><li>DevContainer CLI native support</li><li>Industry standard tooling</li></ul><p><strong>Installation on Arch</strong></p><pre># Install Docker package<br>sudo pacman -S docker<br><br># Verify installation<br>docker --version<br># Output: Docker version 27.3.1, build 41ca978<br><br># Start Docker service<br>sudo systemctl start docker<br><br># Enable Docker to start on boot<br>sudo systemctl enable docker<br><br># Check Docker status<br>sudo systemctl status docker</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9aUsd4PoXIoxaW_TGtC6RA.png" /></figure><p><strong><em>Output</em></strong>:</p><pre>● docker.service - Docker Application Container Engine<br>     Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)<br>     Active: active (running) since Wed 2025-12-03 22:15:00 UTC</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vyKpDMVqnS68d27kFaYyVA.png" /></figure><p><strong>User Permissions</strong></p><p><strong>Problem</strong>: Running docker commands required `sudo` every time.</p><p><strong><em>Solution</em></strong>:</p><pre># Add current user to docker group<br>sudo usermod -aG docker $USER<br><br># Apply group changes without logging out<br>newgrp docker<br><br># Test Docker access<br>docker ps</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KAsTh-JN_wjWLd9gVvG3nA.png" /></figure><p><strong><em>Output</em></strong>:</p><pre>CONTAINER ID   IMAGE   COMMAND   CREATED   STATUS   PORTS   NAMES<br>(empty - no containers running yet)</pre><p><strong><em>Verification test</em></strong>:</p><pre>docker run hello-world<br># Successfully pulled and ran test container<br># Confirmed Docker working without sudo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fHiBuod6bLEpWkzeDmKcNA.png" /></figure><p>✅ Docker installed and accessible without sudo.</p><p><strong>DevContainer CLI</strong>: Why Not VS Code?</p><p>Here’s where I made a deliberate choice that would differentiate my portfolio project: I would use DevContainers via CLI, not VS Code.</p><p><strong><em>Technical Reasons</em></strong>:</p><p>1. Deeper understanding — You learn the mechanics, not just the GUI<br>2. More control — Direct access to build process and configuration<br>3. Better for automation — Scripts over clicking<br>4. Works over SSH — No GUI overhead on remote machines<br>5. Real-world scenarios — Production environments are often headless</p><p><strong>Portfolio Value</strong>:</p><p>Most developers use DevContainers through VS Code. By using the CLI:</p><ul><li>✅ Demonstrates command-line proficiency</li><li>✅ Shows understanding beyond GUI tools</li><li>✅ Proves ability to work in headless environments</li><li>✅ Differentiates from tutorial-followers</li></ul><p>Installing Node.js and npm</p><p>DevContainer CLI requires Node.js:</p><pre># Install Node.js and npm<br>sudo pacman -S nodejs npm<br><br># Verify installation<br>node --version<br># Output: v22.20.0<br><br>npm --version  <br># Output: 10.9.2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*keh5ol-K0uLNmd4KgP9PwQ.png" /></figure><p>Installing DevContainer CLI</p><p><strong><em>First attempt</em></strong>:</p><pre>sudo npm install -g @devcontainers/cli<br># Tries to install to: /usr/lib/node_modules/@devcontainers/cli<br># ❌ Requires sudo (system directory)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w31Cv_-vB1Jby_fJ70E4aQ.png" /></figure><p><strong><em>Error</em></strong>:</p><pre>npm error code EACCES<br>npm error syscall mkdir<br>npm error path /usr/lib/node_modules/@devcontainers<br>npm error errno -13<br>npm error Error: EACCES: permission denied</pre><p><strong>The Problem</strong>: Global npm packages require write access to system directories.</p><p><strong>Bad Solution</strong>: `sudo npm install -g <a href="http://twitter.com/devcontainers/cli">@devcontainers/cli</a>` <br>_(Works but not best practice — mixing sudo with npm creates ownership issues)_</p><p><strong><em>Proper Solution</em></strong>: Configure npm for user-space globals:</p><pre># Create directory for global packages<br>mkdir -p ~/.npm-global<br><br># # Changes global install location to: ~/.npm-global/lib/node_modules/<br>npm config set prefix &#39;~/.npm-global&#39;<br><br># Add to PATH<br>echo &#39;export PATH=~/.npm-global/bin:$PATH&#39; &gt;&gt; ~/.bashrc<br>source ~/.bashrc<br><br># Now install without sudo (into user-space directory)<br>npm install -g @devcontainers/cli<br># Now installs to: ~/.npm-global/lib/node_modules/@devcontainers/cli <br># No sudo needed (your home directory)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wKejZNYvbPCaovAx_zMz-w.png" /></figure><p><strong>Why this approach is better</strong>:</p><ul><li>No permission issues</li><li>No sudo required for global installs</li><li>Follows Linux best practices</li><li>Demonstrates proper system administration</li></ul><p><strong><em>Verify installation</em></strong>:</p><pre>devcontainer --version<br># Output: @devcontainers/cli 0.80.2</pre><p>Understanding DevContainer Commands</p><pre>devcontainer --help</pre><p><strong>Key commands discovered</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y45jdlDE_aX_yaKzS8-qrQ.png" /></figure><p>Important understanding:</p><ul><li>up — Creates/starts container from config</li><li>exec — Runs commands inside running container</li><li>No down command — Use docker stop instead</li><li>— workspace-folder - flag specifies project directory</li></ul><p><strong>The Workflow Established</strong></p><p>By the end of this setup phase, I had:</p><p>✅ <strong><em>Hardware</em></strong>:</p><ul><li>Lenovo ThinkPad (32GB RAM) — Primary development machine</li><li>MacBook Air (8GB RAM) — SSH client &amp; documentation</li><li>iPhone hotspot — Network connectivity</li></ul><p>✅ <strong><em>Network</em></strong>:</p><ul><li>SSH access configured (172.20.10.6)</li><li>Stable USB tethering connection</li><li>Comfortable remote workflow</li></ul><p>✅ <strong><em>Foundation Software</em></strong>:</p><ul><li>Docker 27.3.1 installed and accessible</li><li>DevContainer CLI 0.80.2 ready to use</li><li>Node.js 22.20.0 for tooling</li></ul><p>✅ <strong><em>Development Philosophy</em></strong>:</p><ul><li>CLI-first approach</li><li>Manual configuration over templates</li><li>Deep understanding over convenience</li><li>Document every decision and failure</li></ul><p><strong>What I Learned</strong></p><p><strong><em>1. Resource Planning Matters</em></strong></p><p>Don’t start a project without understanding resource requirements. 8GB RAM might work, but 32GB gives you room to actually develop without constant swapping and slowdowns.</p><p><strong><em>2. Network Topology is Foundation</em></strong></p><p>SSH access isn’t just convenient — it’s a fundamental DevOps skill. Understanding network interfaces, IP addressing, and service configuration is critical.</p><p><strong><em>3. Proper Tool Installation</em></strong></p><p>Taking time to configure tools properly (like npm global packages) saves debugging time later and demonstrates professional practices.</p><p><strong><em>4. CLI Proficiency Sets You Apart</em></strong></p><p>In a world of GUI-driven development, command-line expertise differentiates you. Production environments don’t have GUIs.</p><p><strong>What’s Next</strong></p><p>With a foundation in place, Part 2 will dive into the <strong>real</strong> work:</p><ul><li>Creating 14 project files manually via command line</li><li>Building our first DevContainer</li><li>Discovering a critical architecture mismatch</li><li>Learning why “cannot execute binary file” means you downloaded the wrong CPU architecture</li></ul><p><strong><em>Spoiler</em></strong>: Everything will seem perfect until I try to run `go version`. Then chaos.</p><p><strong>Quick Command Reference</strong></p><p>Here are the key commands from Part 1 for future reference:</p><pre># Network Discovery<br>ip link                              # List interfaces<br>ip addr show INTERFACE               # Get IP address<br><br># SSH Setup<br>sudo pacman -S openssh               # Install SSH server<br>sudo systemctl start sshd            # Start SSH<br>sudo systemctl enable sshd           # Enable on boot<br><br># Docker Setup  <br>sudo pacman -S docker                # Install Docker<br>sudo systemctl start docker          # Start Docker<br>sudo usermod -aG docker $USER        # Add user to docker group<br>newgrp docker                        # Apply group changes<br><br># DevContainer CLI<br>mkdir -p ~/.npm-global               # Create global npm directory<br>npm config set prefix &#39;~/.npm-global&#39; # Configure npm<br>npm install -g @devcontainers/cli    # Install DevContainer CLI</pre><p><strong>Connect &amp; Follow</strong></p><p>Part 2 drops next Wednesday! Don’t miss the architecture debugging.</p><ul><li>🔗 <strong><em>GitHub Repository</em></strong>: [DeployFlow]<br><a href="https://github.com/BenjaminBurton/deployflow">https://github.com/BenjaminBurton/deployflow</a></li><li>💼 <strong><em>LinkedIn</em></strong>: <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></li><li>☁️ <strong><em>AWS Community Builder</em></strong>: <br><a href="https://builder.aws.com/community/@lavell">https://builder.aws.com/community/@lavell</a></li><li>🌐 <strong><em>Website</em></strong>: [Coming Soon]</li></ul><p>#Kubernetes #DevOps #DevContainers #Docker #Containers #SSH #ArchLinux #SelfTaught #AWS #CommunityBuilder</p><p><strong>About the Author</strong>:</p><p>La’vell Burton a DevOps engineer and AWS Community Builder (Containers category) currently working as a delivery driver while building a portfolio to break into tech. This series documents a real production-grade project built in a single intensive day of failures, troubleshooting, and all.</p><p>From package delivery in NYC winter to production Kubernetes. This is the journey. Stay tuned for Part 2 on 12/10/25.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8697becc5db" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nexus Repository Manager Demystified: From Installation to Artifact Deployment]]></title>
            <link>https://mrburton.medium.com/nexus-repository-manager-demystified-from-installation-to-artifact-deployment-9d88c56dcf9c?source=rss-c8a15df53e11------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d88c56dcf9c</guid>
            <dc:creator><![CDATA[Mr. Burton]]></dc:creator>
            <pubDate>Wed, 15 Oct 2025 01:43:50 GMT</pubDate>
            <atom:updated>2025-10-15T01:43:50.204Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jVg21am2rqCY-eoGWbBEzQ.jpeg" /></figure><p><strong>Introduction</strong>: Managing dependencies and artifacts is critical. Developers, DevOps engineers or being part of a CI/CD pipeline team, ensuring that your libraries, binaries, and packages are properly stored, versioned, and accessible can make or break your workflow. Although I am planning to test drive Google Artifact Registry right now I am learning using Sonatype Nexus Artifact Repository, and to be honest this is a game changer for me. This is giving me the ability to understand how to communicate with the developers and the troubles they go through, interesting.</p><p>In this article, we’ll explore how Nexus works, why it’s essential for modern development pipelines, and walk you through the steps to upload, manage, and serve your artifacts — whether from your local machine, GitLab, or a CI/CD pipeline. By the end, you’ll understand how to leverage Nexus to streamline your builds and make artifact management a seamless part of your development process.</p><p>NOTE: Nexus requires at least 2–4GB RAM to run comfortably. Personally I used 8GB on an AWS Ubuntu Instance</p><p>For starters we need to SSH into our server so we can get started</p><pre>ssh -i your-key.pem ubuntu@your-ec2-public-ip</pre><p>Next Install Java ( Nexus requires Java 11+ )</p><pre># First update the software<br>sudo apt update<br><br># Install Java<br>sudo apt install openjdk-17-jdk -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RGI-ARm53qdLdc73MAX62A.png" /></figure><p>Check Java installation</p><pre>java -version</pre><p>Download Nexus</p><pre>wget https://download.sonatype.com/nexus/3/nexus-3.85.0-03-linux-x86_64.tar.gz</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TwigYI0f3gasku00XEvt5A.png" /></figure><p>Extract Nexus</p><pre>tar -xvzf nexus-3.85.0–03-linux-x86_64.tar.gz<br>sudo mv nexus-3.85.0–03 /opt/nexus<br>sudo mv sonatype-work /opt/sonatype-work</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iptOdu8C7vnrUxKmrSYeFA.png" /></figure><p>Create a Nexus user ( for security )</p><pre>sudo adduser - disabled-password - gecos &quot;&quot; nexus<br>sudo chown -R nexus:nexus /opt/nexus /opt/sonatype-work</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bm9juYaJksC273aBbGzWBQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Td2lluijl3M0qRqJJuepUg.png" /></figure><p>Configure Nexus to run as this user</p><pre>sudo vim /opt/nexus/bin/nexus.rc</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_nu6JmzVh04r6yDdVx-BwQ.png" /></figure><p>Change or add this line if it’s not already there (some distros have the line)</p><pre>run_as_user=&quot;nexus&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*32tn-1YKu3ayNne5zKtMAg.png" /></figure><p>Switch into the Nexus user</p><pre>sudo su - nexus</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QdnWkGmxVDU2svgOiEfbsw.png" /></figure><p>Start Nexus</p><pre>/opt/nexus/bin/nexus start</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-vGvygOdCbgHXN8NmWaY4Q.png" /></figure><p>Check status</p><pre>/opt/nexus/bin/nexus status</pre><p>The first start might take a few minutes.</p><p>Change back into admin</p><pre>exit</pre><p>Enable Nexus to start on the boot</p><pre>sudo vim /etc/systemd/system/nexus.service</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nbknDnbfLdDgAQCkB4IcGg.png" /></figure><p>Add</p><pre>[Unit]<br>Description=nexus service<br>After=network.target<br>[Service]<br>Type=forking<br>LimitNOFILE=65536<br>User=nexus<br>Group=nexus<br>ExecStart=/opt/nexus/bin/nexus start<br>ExecStop=/opt/nexus/bin/nexus stop<br>[Install]<br>WantedBy=multi-user.target</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BF7jEhxYoCc5L0FjDmDv7A.png" /></figure><p>Enable and start</p><pre>sudo systemctl daemon-reload<br>sudo systemctl enable nexus<br>sudo systemctl start nexus<br>sudo systemctl status nexus</pre><p>Open the Nexus port in EC2 security group<br>- Inbound TCP traffic on 8081</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3dUGvUuvob8csXsbXPq57Q.png" /></figure><p>Access Nexus<br>- Open a browser and go to</p><pre>http://{ec2-public-ip}:8081</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4mM1EOz5vWyCYad04rqwA.png" /></figure><p>After logging in we need to get the default password so we can login as Admin and create a user</p><p>Default admin password location. Running this command will give you the password for admin to login and start using Nexus.</p><pre>cat /opt/sonatype-work/nexus3/admin.password</pre><p>Login</p><p>Username: admin</p><p>Password: xxxxxxxxxx</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqN_xHS1D_Xqqe5nNZg9Gg.png" /></figure><p>After logging in go through the setup task wizard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2PH-sk00krIl4s3xoeNf-Q.png" /></figure><p>Create a new password</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LIiRRwdpU4u_zeP94DhRsg.png" /></figure><p>Finish the setup wizard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4-ol5dqZZl3QHiuilmA96g.png" /></figure><p>Agree to the terms</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lFkc1FsPM2dT0IxLm_zOCw.png" /></figure><p>Configure Anonymous Access I set mine to disable</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Gi72d9RtwZ-jkj75EHaJw.png" /></figure><p>Setup complete</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PTVOiL00QHXNG1t48-9fxg.png" /></figure><p>Once we have logged in we want to create a Role and attach the necessary permissions From the left sidebar go to Settings in settings click the dropdown for security and go to roles Click create role in the top right</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KTwryGzD9Ee7wh_JTG7x0g.png" /></figure><p>The type is Nexus role</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A7QdrdmdsVt1EGd0Ts_2JA.png" /></figure><p>Role ID — artifact-uploader<br>Role Name — Artifact Uploader<br>Role Description — Can upload artifacts to hosted repositories</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Hu51v7mVgY2RVWfP7UL7w.png" /></figure><p>then click Modify Applied Privileges and give it</p><pre>nx-repository-view-maven2-maven-snapshots-*</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-4v1jb3q73tty8PPG77uKQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XEai69dBGTYJp7ZYhte4hg.png" /></figure><p>Then click confirm and save the role</p><p>Next create the user Back under Security go to Users<br>Click Create local user</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rOabsexKPtg_BC-xsGrGdA.png" /></figure><p>Fill in all the following boxes<br>User ID — uploader<br>First Name — Artifact <br>Last Name — Uploader<br>Email — contactemail<a href="/service/mailto:defidevops@outlook.com">@gmail.com</a><br>Status — Active<br>Password — xxxxxxxx</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S3cbuyAE0HWMvPAQ9X8eqA.png" /></figure><p>Roles → Move the role we created → Artifact Uploader into the granted box<br>Granting the user the role permissions<br>Then Click create local user</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ApbnPl6JVtx6m0B94DWV3w.png" /></figure><p>Now before moving on we need to test the User is active and be able to log in.</p><p>Test the new user<br>Log out of Admin, and<br>Login with your User — Username &amp; Password</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MoTtNlmkLqIXEeVyTOUCEw.png" /></figure><p>Now lets configure the Gradle project and see if we can get things working the way its supposed to be working. In the build.gradle file</p><pre>publishing {<br> publications {<br> maven(MavenPublication) {<br> artifact(&quot;build/libs/my-app-$version&quot; + &quot;.jar&quot;) {<br> extension &#39;jar&#39;<br>     }<br>   }<br> }<br> <br> repositories {<br> maven {<br> name &#39;nexus&#39;<br> # URL of Repository in Repository manager<br> url &quot;http://[nexus ip]:[nexus port]/repository/maven-snapshots/&quot;<br> credentials {<br> username xxx<br> password xxx<br>       }<br>     }<br>   }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AcifF7XPGBy56WxoPaQH5g.png" /></figure><p>Also we need to configure authentication in the `gradle.properties` file</p><pre>repoUser = uploader<br>repoPassword = yourpassword</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WKULTrGHOr7WPIFfUZ-QAA.png" /></figure><p>I ran into a Problem here. This didn’t work for me</p><pre>credentials {<br>username project.repoUser<br>password project.repoPassword<br>}</pre><p>This Worked for me</p><pre>credentials {<br>username System.getenv(&quot;NEXUS_USER&quot;)<br>password System.getenv(&quot;NEXUS_PASS&quot;)<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zub2njVtXPLOqMzHPg7POw.png" /></figure><p>In the Terminal i ran the export for my credentials since its not permanent</p><pre>export NEXUS_USER=uploader<br>export NEXUS_PASS=uploader-password</pre><p>Once I fixed this i was able to run</p><pre>gradle build</pre><p>and the build was a success</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ObzTkDnRizujSzm8GtCm5w.png" /></figure><p>At this point I just needed to run gradle publish to send it to Nexus</p><pre>gradle publish</pre><p>Also I forgot to change to my app name, small fix but was easy to find.</p><pre># Changed this line<br>artifact(&quot;build/libs/my-app-$version&quot;+&quot;.jar&quot;){<br><br><br># Replaced my-app with mygradle-app like I named it<br>artifact(&quot;build/libs/mygradle-app-$version&quot;+&quot;.jar&quot;){</pre><p>Now since we are logged in with admin we need to go to browse and that will show us our users components. Mission complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wPOANjEhIOzWBD3stfR7iQ.png" /></figure><p>To make sure we have this process down we will also do it for our maven project. To do this we will have to configure the pom.xml file. The same way we did in gradle project the same will be applied to maven which is the Artifact &amp; the address of the repository , so we will set the URL of Nexus repository of maven-snapshots. The second part will be to configure the credentials so maven can actually upload the Jar file to Nexus using a Nexus user that we created.<br>Now lets configure the plugin</p><pre># These two blocks is the first part we need to configure<br> &lt;plugin&gt;<br>   &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>     &lt;artifactId&gt;maven-deploy-plugin&lt;/artifactId&gt;<br>   &lt;version&gt;3.1.1&lt;/version&gt;<br> &lt;/plugin&gt;<br>&lt;plugin&gt;<br>   &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>   &lt;artifactId&gt;maven-deploy-plugin&lt;/artifactId&gt;<br> &lt;/plugin&gt;<br><br># This is the second part we need to configure<br> &lt;distributionManagement&gt;<br>   &lt;snapshotRepository&gt;<br>     &lt;id&gt;nexus-snapshots&lt;/id&gt;<br>     &lt;url&gt;http://3.236.112.65:8081/repository/maven-snapshots/&lt;/url&gt;<br>   &lt;/snapshotRepository&gt;<br> &lt;/distributionManagement&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NG3gzIKk05jtlB_7zNESEQ.png" /></figure><p>And to configure credentials in maven it is done in a hidden folder named .m2 in the home directory of your computer. Inside is the repository folder that holds all the locally downloaded dependencies of the maven project, and inside this folder we will create a file named settings.xml where maven’s global credentials can be defined</p><pre>ls -a | grep .m2<br>.m2<br>ls .m2<br>repository  settings.xml  wrapper</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZeYzWLn5wR1jxgGvmrkdIg.png" /></figure><p>This will be available for all maven projects moving forward</p><pre>cd .m2</pre><p>Create the file</p><pre>vim settings.xml</pre><p>Inside the file the contents for configuration</p><pre>&lt;settings&gt;<br>   &lt;servers&gt;<br>     &lt;server&gt;<br>       &lt;id&gt;nexus-snapshots&lt;/id&gt;<br>       &lt;username&gt;uploader&lt;/username&gt;<br>       &lt;password&gt;587fe4ab-xxxx-xxxx-xxxx-xxxxxxxxxxxx&lt;/password&gt;<br>     &lt;/server&gt;<br>   &lt;/servers&gt;<br>&lt;/settings&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rxn1lry6SaUtH2HFmJpNvw.png" /></figure><p>After setting the configuration change directory into your maven-app</p><pre># As an example my path is below<br>cd /Users/lavell/java-maven-app<br>ls<br>pom.xml src # files &amp; folders (contents of the project)</pre><p>Everything is configured now, and all we need to do is execute the command to build the jar file.</p><pre>mvn package</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8XwR_uRFOewv5gFDZReiGg.png" /></figure><p>Now we can view the Jar file that was created</p><pre>ls target</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*htUL94p2JdfOPKVF4KEJeQ.png" /></figure><p>Build is successful and now we can send it to Nexus</p><pre>mvn deploy</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*octU6IYYHplprHVSHnf80A.png" /></figure><p>and now back in Nexus we see both Jar files have been successfully uploaded</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WWjGwJLvHJWBgkfs3odCnw.png" /></figure><p><strong>Conclusion</strong>: This was a critical milestone for me I was able to establish a secure, scalable, and automated artifact management workflow. Configuring and integrating Sonatype Nexus Repository Manager with both Gradle and Maven, so this was more than just a technical achievement. This was another step in the direction of the goal I have set for myself of getting a job in Tech.</p><p>Through this setup, I built an end-to-end system that allows development teams to:</p><ul><li><strong>Standardize software delivery</strong> by storing all build artifacts in a centralized repository.</li><li><strong>Enhanced security and control</strong>, ensuring only authorized users can publish or retrieve artifacts.</li><li><strong>Improved efficiency and traceability</strong>, enabling version control of every deployed component.</li><li><strong>Reduced dependency on public repositories</strong>, strengthening compliance and internal governance.</li><li><strong>Support for CI/CD automation</strong>, as artifacts can now be programmatically pushed or pulled during build and deployment pipelines.</li></ul><p>In business terms, this implementation transforms what used to be a manual, and error-prone process into something reliable, repeatable. When they say they want you to hit the ground running, this shows I can contribute to faster release cycles, and improved developer productivity which ultimately makes management happy.</p><p>If you are looking for me or want to follow my journey I can be reached on LinkedIn @ <a href="https://www.linkedin.com/in/benjaminlburton/">https://www.linkedin.com/in/benjaminlburton/</a></p><p>I appreciate any feedback, and thank you for taking the time to read my article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d88c56dcf9c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>