<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rester L. McGlown Jr. on Medium]]></title>
        <description><![CDATA[Stories by Rester L. McGlown Jr. on Medium]]></description>
        <link>https://medium.com/@rester.mcglown?source=rss-a905c2ab903f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*-2EPwB3SakTuGqLg</url>
            <title>Stories by Rester L. McGlown Jr. on Medium</title>
            <link>https://medium.com/@rester.mcglown?source=rss-a905c2ab903f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:09:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rester.mcglown/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Docker Swarm Orchestration -Building a Self-Healing Web Tier for a Live Sports-News Platform]]></title>
            <link>https://medium.com/@rester.mcglown/docker-swarm-orchestration-building-a-self-healing-web-tier-for-a-live-sports-news-platform-696c1a5500ab?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/696c1a5500ab</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-swarm]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Wed, 16 Sep 2026 02:02:26 GMT</pubDate>
            <atom:updated>2026-09-16T02:02:26.178Z</atom:updated>
            <content:encoded><![CDATA[<p>Live sports content is one of the least forgiving workloads on the internet. Traffic doesn’t ramp up gradually, it spikes the moment a game starts, and if your web tier can’t absorb that spike or recover instantly from a crash, you lose readers in the exact moment they showed up. For this project, I built and stress-tested a self-healing, auto-scaling web tier for a fictional platform called Pulseline Media, using a single-node Docker Swarm cluster on AWS EC2.</p><p>This write-up isn’t just a list of commands that worked. Two real, non-trivial problems came up during the build, a placement-constraint bug and a networking issue that looked like an application bug but wasn’t, and I’ve documented the actual diagnostic process for both, not just the fix.</p><h3>Project Architecture and Objectives</h3><figure><img alt="Infographic diagram showing an AWS EC2 host running a Docker Swarm manager node with three web replicas and a Redis container on an overlay network, a client column with curl validation and the security group, and a source column showing GitHub and Docker Hub feeding code and images into the host, with a Key Outcomes footer listing recovery time, zero failed requests, replica count, and issues resolved." src="https://cdn-images-1.medium.com/max/1024/1*iJiuAvh8Lj2WEXIvcPr2nw.png" /><figcaption>Architecture overview of the Pulseline Media web tier, a single-node Docker Swarm cluster on AWS EC2 running three load-balanced Flask replicas backed by Redis, with code and images sourced from GitHub and Docker Hub.</figcaption></figure><p>A single EC2 instance runs as both Swarm manager and (its only) worker. Three replicas of a Flask web application sit behind Swarm’s routing mesh on port 80, talking to a single Redis instance over an overlay network purely by service name. A Docker secret and a Docker config are mounted into the web containers. Code is delivered to the instance via GitHub, and the application image is versioned and pulled from Docker Hub.</p><p>Going into this build, I set out to:</p><ul><li>Stand up a working single-node Docker Swarm cluster on a cloud VM</li><li>Deploy and scale a service using Swarm’s declarative desired-state model</li><li>Deploy a real multi-service application (Flask + Redis) as a stack, communicating over an overlay network</li><li>Prove self-healing directly, by killing a container on purpose and measuring the recovery</li><li>Apply placement constraints using custom node labels</li><li>Securely inject a secret and a non-secret config file into a running service</li><li>Perform a zero-downtime rolling update between two application versions, then a deliberate rollback</li><li>Practice real-world troubleshooting: diagnosing failures methodically rather than guessing</li></ul><h3>The Problem I Wanted to Solve</h3><p>Anyone can run docker run and get a container up. The harder, more realistic question is: what happens when that container dies at 2 AM during a live event? Does someone get paged, or does the system heal itself? This project exists to answer that question concretely, not theoretically, by actually killing things on purpose and measuring what happens.</p><h3>Why Build This in Stages (Basic, Intermediate, Advanced)</h3><p>I deliberately built this up in layers rather than deploying the full stack on day one. The Basic tier proves Swarm’s core mechanism, desired-state reconciliation, using the simplest possible workload (sleep infinity containers), so that when I introduced real application complexity in the Intermediate tier, I already trusted the underlying orchestration layer. The Advanced tier then layers in the production-adjacent concerns, secrets, configs, placement constraints, versioned rollouts, that a lab-only setup doesn&#39;t strictly need but a real deployment can&#39;t skip.</p><h3>Tools and Services Used</h3><ul><li>AWS EC2 (Ubuntu Server 24.04 LTS)</li><li>Docker Engine and Docker Swarm</li><li>Docker Stack (Compose file, version 3.8)</li><li>Flask (Python 3.12)</li><li>Redis 7 (Alpine)</li><li>Docker Secrets and Docker Configs</li><li>Docker overlay networking</li><li>Git and GitHub</li><li>Docker Hub</li></ul><h3>Stage 1: Standing Up the Swarm</h3><p>The first step was launching the EC2 instance itself, Ubuntu Server 24.04 LTS on a t3.micro, with a security group locking SSH down to my own IP while leaving HTTP open for the web tier.</p><figure><img alt="AWS EC2 console showing the pulseline-swarm-node instance in a running state with 3/3 status checks passed." src="https://cdn-images-1.medium.com/max/1024/1*rwnKWsB7U8JoUoguC50mrg.png" /><figcaption>The pulseline-swarm-node EC2 instance running, passing all status checks, on a t3.micro instance.</figcaption></figure><p>With the instance running, I installed Docker from Ubuntu’s own package repository and added my user to the docker group so I wouldn&#39;t need sudo in front of every command:</p><pre>sudo apt update &amp;&amp; sudo apt install -y docker.io<br>sudo usermod -aG docker $USER</pre><figure><img alt="Terminal output showing successful Docker installation and an empty docker ps table with headers only." src="https://cdn-images-1.medium.com/max/569/1*m9noeqk_qJX0GfwKlby1bQ.png" /><figcaption>Docker installed and working without sudo, confirmed with docker --version and a clean docker ps.</figcaption></figure><p>With Docker confirmed working, initializing the Swarm is a single command:</p><pre>docker swarm init<br>docker node ls</pre><figure><img alt="Terminal output of docker swarm init, docker node ls showing Leader status, and docker service ps showing 3 running tasks." src="https://cdn-images-1.medium.com/max/1024/1*rk9-a_DBiAWjNr4ANFsFXQ.png" /><figcaption>Swarm initialized, the node confirmed as Leader, and a basic 3-replica service deployed successfully.</figcaption></figure><p>docker node ls confirmed my one node as Leader, meaning it&#39;s authoritative for scheduling on this cluster. To prove the mechanism before touching any real application code, I deployed the simplest possible service:</p><pre>docker service create --name basic-web --replicas 3 ubuntu:24.04 sleep infinity<br>docker service ps basic-web</pre><p>All three replicas came up running immediately, each with its own task ID.</p><h3>Stage 2: Scaling Up and Down</h3><pre>docker service scale basic-web=5<br>docker service ps basic-web</pre><pre>docker service scale basic-web=2<br>docker service ps basic-web</pre><figure><img alt="Terminal output of docker service scale commands and docker service ps tables showing 5 then 2 running replicas." src="https://cdn-images-1.medium.com/max/969/1*vcs3G-u1l2hLodJCqVtBvw.png" /><figcaption>Scaling the basic service up to 5 replicas and back down to 2, demonstrating Swarm’s desired-state reconciliation.</figcaption></figure><p>Scaling to 5 replicas converged cleanly. Scaling back down to 2, I expected to see the removed replicas listed with a Shutdown status, task history for debugging. Instead I got a clean two-row table with nothing else. It turns out that&#39;s actually correct: an explicit scale-down removes those replica slots entirely, since nothing failed unexpectedly, you told Swarm to reduce, and it did. Task history only shows up when a running task dies unexpectedly while the desired count stays the same, which is exactly what I tested later in Stage 6.</p><h3>Stage 3: Getting Real Code Onto the Instance</h3><p>For this project I used Git instead of copying files by hand. I pushed the application code to a GitHub repository and cloned it directly onto the EC2 instance:</p><pre>git clone https://github.com/rester12/Docker-Swarm-Orchestration.git<br>cd Docker-Swarm-Orchestration</pre><figure><img alt="Terminal output of git clone completing successfully and ls confirming the app files are present." src="https://cdn-images-1.medium.com/max/787/1*aUU_bwdmHmJZlJpvvqpyNQ.png" /><figcaption>Cloning the project repository directly onto the EC2 instance from GitHub.</figcaption></figure><p>Then built and pushed the application image, tagged with an explicit version rather than latest:</p><pre>cd app<br>docker build -t rester12/pulseline-web:1.0.0 .<br>docker push rester12/pulseline-web:1.0.0</pre><figure><img alt="Terminal output of a successful docker build and docker push showing all layers pushed with a final digest." src="https://cdn-images-1.medium.com/max/794/1*NNAeoGDQIKYT1u8iFuZOjw.png" /><figcaption>Building the Flask web application image and pushing version 1.0.0 to Docker Hub.</figcaption></figure><p>That version-tagging decision felt like a small detail at the time. It turned out to matter a lot later, once I got to the rolling update.</p><h3>Stage 4: Overlay Networking and Deploying the Stack</h3><pre>docker network create -d overlay pulseline-net<br>docker network ls</pre><figure><img alt="Terminal output showing pulseline-net listed with driver overlay and scope swarm among Docker’s default networks." src="https://cdn-images-1.medium.com/max/843/1*IXwoEh5DBBAorls5YXjS8g.png" /><figcaption>Creating the overlay network and confirming its driver and scope with docker network ls.</figcaption></figure><p>Overlay networks are Swarm’s cross-host layer, using VXLAN tunneling so containers can find each other by name even across different physical machines. On a single node the difference from a plain bridge network isn’t visible, but it’s the driver Swarm actually needs for docker stack deploy and for the DNS-based service discovery my app depends on.</p><pre>docker stack deploy -c stack.yml pulseline<br>docker stack services pulseline</pre><p>Deploying the stack created a <em>second</em> overlay network, pulseline_pulseline-net, distinct from the one I&#39;d just made manually. Unless you mark a network external: true in the stack file, Compose creates and namespaces its own. My manually-created network turned out to be a nice way to inspect the overlay driver in isolation, but not actually the network the stack ended up using.</p><h3>Problem #1: Both Services Stuck at Zero Replicas</h3><p>Both services deployed, but neither would schedule. docker stack services pulseline sat at 0/1 and 0/3 indefinitely. Running docker service ps pulseline_web --no-trunc gave me the real error:</p><pre>&quot;no suitable node (scheduling constraints not satisfied on 1 node)&quot;</pre><p>My stack.yml had placement constraints tied to node labels (tier == web for the web service, tier == cache for Redis), but I hadn&#39;t applied those labels to my node yet. So I applied them:</p><pre>docker node update --label-add tier=web &lt;node-id&gt;<br>docker node update --label-add tier=cache &lt;node-id&gt;</pre><p>This made things <em>worse</em>, not better. Redis came up fine, but the web service was now permanently stuck, its already-scheduled tasks got actively rejected with &quot;assigned node no longer meets constraints.&quot; The reason took a minute to click: a Docker node label is one key holding one value. Setting tier=cache second silently overwrote tier=web. I was trying to give one node two different values under the same key, which just isn&#39;t how labels work.</p><figure><img alt="Terminal output of docker stack deploy and docker stack services showing pulseline_web at 3/3 and pulseline_redis at 1/1 replicas." src="https://cdn-images-1.medium.com/max/1024/1*JTA7q0q3l1TMAiPcuGj3Rg.png" /><figcaption>The stack deployed successfully after fixing the node label placement-constraint issue, both services fully converged.</figcaption></figure><p>The real fix was two independent label keys instead of one shared key:</p><pre>docker node update --label-add web=true $(docker node ls -q)<br>docker node update --label-add cache=true $(docker node ls -q)</pre><p>with the constraints in stack.yml updated to node.labels.web == true and node.labels.cache == true. Once I updated the file and redeployed, both services converged within seconds. This is a mistake that only bites you on a single node standing in for multiple roles, on a real multi-node cluster each node would carry one label and the conflict would never come up, but it&#39;s exactly the kind of thing you only really learn by hitting it.</p><h3>Stage 5: Proving Web-to-Redis Communication</h3><pre>curl http://localhost/</pre><figure><img alt="Terminal output of five curl responses showing an incrementing hits value and three different container hostnames." src="https://cdn-images-1.medium.com/max/695/1*G7dqtCXheKji--NMmPIg0A.png" /><figcaption>Repeated requests to the web tier showing the Redis-backed hit counter climbing and the hostname rotating across replicas.</figcaption></figure><h3>Problem #2: A Hanging curl Request</h3><p>This one was more subtle. curl http://localhost/ just hung, no error, no response, nothing, even though docker stack services showed both services fully up. I worked through it methodically instead of guessing:</p><ul><li>docker ps confirmed the containers were genuinely running</li><li>docker service logs for both services showed completely clean startup, no errors on either side</li><li>docker exec ... getent hosts redis confirmed DNS resolution worked, redis resolved to a real IP</li><li>A raw TCP check straight from a web container (cat &lt; /dev/null &gt; /dev/tcp/redis/6379) confirmed the network path to Redis was fully open</li></ul><p>Every layer I checked came back clean, which was actually the useful signal, it meant the problem wasn’t in the app-to-Redis path at all. Going back to curl -v&#39;s output, I noticed it was connecting over IPv6 (::1) for &quot;localhost.&quot; Docker Swarm&#39;s routing mesh only load-balances IPv4 traffic, so that connection was quietly accepted at the socket level and then went nowhere. Forcing IPv4 fixed it instantly:</p><pre>curl -4 http://localhost/</pre><p>The hit counter climbed cleanly and the hostname value rotated across all three replicas, real proof the routing mesh and service discovery were both working the whole time. I’d just been asking the wrong protocol.</p><h3>Stage 6: Killing a Container on Purpose</h3><pre>docker ps<br>docker kill &lt;container_id&gt;<br>docker service ps pulseline_web</pre><figure><img alt="Terminal output of docker service ps showing a failed task with exit code 137 and a new running task started seconds later." src="https://cdn-images-1.medium.com/max/1024/1*DmIIElyF0cGk6hUtlDaUvw.png" /><figcaption>A manually killed web container replaced automatically by Swarm in about 5 seconds, confirmed by the exit code and new task timestamp.</figcaption></figure><p>This is the moment the whole project was really building toward. The killed task’s exit code, 137, confirmed it was genuinely killed (signal 9, SIGKILL) rather than crashing on its own. From that kill to a brand new replacement task fully running in the same slot took about 5 seconds. No alert, no human, nothing. Swarm just noticed its desired state (3 running) didn&#39;t match reality (2 running) anymore and fixed it.</p><h3>Stage 7: Secrets and Configs</h3><pre>echo &quot;some-fake-but-realistic-api-key-value&quot; | docker secret create pulseline_api_key -<br>docker config create pulseline_app_config pulseline_app_config.txt</pre><figure><img alt="Terminal output of docker secret ls and docker config ls listing pulseline_api_key and pulseline_app_config." src="https://cdn-images-1.medium.com/max/724/1*ozNk4_RXBsWiY5rYa0r4pQ.png" /><figcaption>The Docker secret and config objects created and confirmed present in the Swarm.</figcaption></figure><p>A Docker secret and a Docker config aren’t the same thing, and the distinction matters. A secret is for actual sensitive values, it’s encrypted at rest and only ever mounted in-memory inside the container at /run/secrets/&lt;name&gt;, never as an environment variable that could leak through docker inspect or a crash log. A config is for non-sensitive settings you still want as a file rather than baked into the image.</p><pre>docker stack deploy -c stack.yml pulseline<br>docker service inspect pulseline_web --pretty</pre><figure><img alt="Terminal output of docker service inspect — pretty showing the mounted config target and secret name under the web service." src="https://cdn-images-1.medium.com/max/953/1*iSrU-Gkr4LhQgibWbFmiUg.png" /><figcaption>The redeployed stack with the secret and config successfully mounted into the running web service.</figcaption></figure><p>The inspect output confirmed both were mounted, the config at /app/app_config.txt, the secret by name only, no value ever visible in the output.</p><h3>Stage 8: A Zero-Downtime Rolling Update</h3><p>I bumped the app to version 2.0.0 (a visibly different message and a new version field), built and pushed it, then triggered the update:</p><pre>docker build -t rester12/pulseline-web:2.0.0 .<br>docker push rester12/pulseline-web:2.0.0</pre><figure><img alt="Terminal output of a successful docker build and docker push for the pulseline-web 2.0.0 image tag." src="https://cdn-images-1.medium.com/max/833/1*TWJJMJRXIklE-AU489jvkA.png" /><figcaption>Building and pushing version 2.0.0 of the web application image ahead of the rolling update.</figcaption></figure><pre>docker service update --image rester12/pulseline-web:2.0.0 pulseline_web<br>docker service ps pulseline_web<br>curl -4 http://localhost/</pre><figure><img alt="Terminal output of docker service update, docker service ps showing all replicas on 2.0.0, and curl responses confirming the new version with an increasing hit count." src="https://cdn-images-1.medium.com/max/1024/1*NP8zVCs8dbCMvx3Z5rUA8A.png" /><figcaption>The zero-downtime rolling update to version 2.0.0, with the hit counter climbing continuously throughout.</figcaption></figure><p>My stack.yml specified parallelism: 1, a 10s delay, and order: start-first, so Swarm replaced exactly one replica at a time, starting each new container and confirming it healthy before stopping the old one. I ran curl repeatedly through the whole rollout, and the hit counter never stopped climbing. That&#39;s the actual definition of zero-downtime, not a claim in a deploy log, but requests that kept succeeding the entire time.</p><h3>Stage 9: Rolling It Back</h3><pre>docker service update --rollback pulseline_web<br>docker service ps pulseline_web<br>curl -4 http://localhost/</pre><figure><img alt="Terminal output of docker service update — rollback, docker service ps showing all replicas back on 1.0.0, and curl responses confirming the original message with a continuing hit count." src="https://cdn-images-1.medium.com/max/1024/1*jPDtZAjWwUjQRF5DY3VTxw.png" /><figcaption>The deliberate rollback to version 1.0.0, confirmed by the reverted response and an uninterrupted hit counter.</figcaption></figure><p>Swarm keeps the previous service spec around, so rollback isn’t a manual re-deploy, it’s a single command that reverses the same staggered replacement process. All three replicas returned to 1.0.0, confirmed by the response reverting to the original message, and again, the hit counter climbed without interruption the entire time.</p><h3>Validation</h3><p>Across every disruptive test in this project, the shared Redis-backed hit counter served as a simple, continuous liveness check. If it stopped climbing, something was actually broken. It never stopped, through scaling, the container kill, the rolling update, or the rollback.</p><h3>Problems I Encountered and How I Solved Them</h3><p>The two real incidents are documented in full above (the node-label overwrite, and the IPv6 routing-mesh hang), but the pattern behind solving both was the same: don’t guess, isolate. For the label issue, --no-trunc on docker service ps gave the exact error instead of a vague &quot;pending&quot; state. For the networking issue, checking DNS resolution and raw TCP connectivity separately from the application layer narrowed the problem down before I ever looked at curl&#39;s verbose output and spotted the IPv6 detail.</p><h3>Security and Safety Decisions</h3><ul><li>SSH access restricted to my own IP in the security group; HTTP left open since the web tier needs to be public</li><li>Real secret values never committed to source control, .gitignore excludes the actual secrets file while keeping a placeholder example</li><li>Secrets injected via Docker’s in-memory secret mount rather than environment variables</li><li>The Swarm join token (visible in docker swarm init output) treated as sensitive and never reused or shared outside this lab</li></ul><h3>Resource Cleanup and Cost Awareness</h3><pre>docker stack rm pulseline<br>docker swarm leave --force</pre><p>Followed by terminating the EC2 instance, deleting its security group, and double-checking for any leftover EBS volumes or Elastic IPs, since those are the usual sources of surprise charges after an instance is gone.</p><h3>What I Would Add Before Production Use</h3><ul><li>A genuine multi-node Swarm cluster, to validate cross-host networking and manager quorum, not just single-node behavior standing in for multiple roles</li><li>A production WSGI server (Gunicorn) instead of Flask’s built-in development server</li><li>Container-level health checks, so an unhealthy-but-still-running container gets replaced too, not just a crashed one</li><li>Centralized logging and metrics instead of reading docker service logs by hand</li><li>Rolling updates and rollbacks triggered by a CI/CD pipeline instead of run manually</li></ul><h3>What I Learned</h3><p><strong>Desired-state reconciliation is the whole story.</strong> Scaling, self-healing, and rolling updates all turned out to be the same underlying mechanism, Swarm comparing what it wants to what actually exists and correcting the gap, just triggered differently each time.</p><p><strong>A node label is one key, one value.</strong> Not a set of tags. Trying to make a single node stand in for two roles taught me that the hard way.</p><p><strong>A hang and a failure point you in different directions.</strong> Zero bytes received after a successful-looking connection meant checking the network layer first, not the application code, and the eventual cause was one layer below where I was looking.</p><p><strong>Measure zero-downtime, don’t assume it.</strong> Running continuous requests during the rolling update and rollback gave me actual evidence, not just a convergence message from Swarm saying it was done.</p><p><strong>Small platform details deserve documentation.</strong> Windows has no chmod; icacls does the same job for SSH key permissions. That kind of practical friction rarely makes it into Linux-centric tutorials, but it&#39;s real and worth writing down.</p><h3>Project Repository</h3><p><a href="https://github.com/rester12/Docker-Swarm-Orchestration">GitHub - rester12/Docker-Swarm-Orchestration: Self-healing, auto-scaling web tier for a live sports-news platform, built with a single-node Docker Swarm cluster on AWS EC2 - Flask + Redis service stack with overlay networking, rolling updates and rollback, secrets/configs, and placement constraints.</a></p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=696c1a5500ab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Containerizing Java, Python, and JavaScript Apps - Multi-Stage Docker Builds, Non-Root Users, and…]]></title>
            <link>https://medium.com/@rester.mcglown/containerizing-java-python-and-javascript-apps-multi-stage-docker-builds-non-root-users-and-79d29e102b13?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/79d29e102b13</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Sun, 13 Sep 2026 22:32:03 GMT</pubDate>
            <atom:updated>2026-09-14T00:44:35.219Z</atom:updated>
            <content:encoded><![CDATA[<h3>Containerizing Java, Python, and JavaScript Apps - Multi-Stage Docker Builds, Non-Root Users, and Docker Hub Tagging</h3><p>A hands-on project containerizing three separate internal tools, one Python, one Java, one Node, each taken from a bare single-stage Dockerfile all the way to a hardened, versioned, non-root image published on Docker Hub, with a real semantic-version-plus-commit-hash tagging convention simulating how a CI pipeline would actually ship these.</p><h3>Project Architecture and Objectives</h3><p>This project containerizes three small internal tools that previously only ran on someone’s laptop, one Python service, one Java service, and one Node/JavaScript service, and gets each of them to the point where a teammate could pull a known-good, versioned image and run it anywhere Docker runs. Unlike a multi-service stack, these three tracks are independent end to end: three separate Dockerfiles, three separate builds, three separate pushes, but all converging on the same conventions.</p><figure><img alt="Diagram showing Python, Java, and Node build pipelines each producing a multi-stage image with a non-root user, all pushed to Docker Hub under the rester12 account." src="https://cdn-images-1.medium.com/max/1024/1*4qv1uVcbPg1y4zj0iCYbKw.png" /><figcaption>Three independent language tracks, Python, Java, and Node, each following the same path from a multi-stage Dockerfile through a non-root runtime user to a versioned, dual-tagged push to Docker Hub.</figcaption></figure><p>The objectives were to:</p><ul><li>Write a Dockerfile for a simple Python service, build and run it, confirm it responds on its mapped port, and push the image to Docker Hub with a real version tag</li><li>Repeat for a Java service and a Node/JavaScript service, choosing an appropriate base image for each, pushing both with meaningful tags</li><li>Add a .dockerignore to all three projects and compare image sizes before and after</li><li>Convert all three Dockerfiles to multi-stage builds and document the resulting size reduction</li><li>Establish a single tagging convention, semantic version plus short commit hash, and simulate a CI pipeline’s build-once, tag-twice, push-twice flow</li><li>Add a non-root USER to all three final Dockerfiles and confirm all three containers actually run without root privileges</li></ul><h3>The Problem I Wanted to Solve</h3><p>The scenario behind this one was Solstice Devs, a small consultancy with three internal tools, Python, Java, and Node, each living entirely on one developer’s laptop. That’s a familiar trap: a tool works fine until the one person who can run it is unavailable, or a new teammate needs to pick it up and has no idea what’s actually required to make it run. The goal wasn’t just “make three Dockerfiles”, it was to make each of these three tools reproducible and handoff-ready, in a way that held up to the same standard regardless of which language it happened to be written in.</p><h3>Why Three Languages Instead of One</h3><p>Writing a single, polished Dockerfile is a narrower skill than writing three good ones across ecosystems that behave nothing alike. Python’s dependency model, Java’s build-then-run split via Maven, and Node’s TypeScript compile step each impose a different shape on what “install dependencies” and “produce a runtime artifact” actually mean. Doing all three side by side, under the same tagging convention and the same hardening requirements, is what actually tests whether I understand the underlying Docker concepts, multi-stage builds, build context, non-root users, rather than just knowing one language’s idioms well enough to fake it.</p><h3>Tools and Technologies Used</h3><ul><li>Docker Desktop, Docker Engine, and CLI</li><li>Python 3.12 and FastAPI</li><li>Java 17, Spring Boot, and Maven</li><li>Node.js 20 and NestJS</li><li>Multi-stage Docker builds</li><li>Docker Hub as the image registry</li><li>Git, for generating short commit hashes used in image tags</li><li>Windows PowerShell</li><li>Git and GitHub</li></ul><h3>Phase 1 - Python: FastAPI, Build, and First Push</h3><p>The Python tool was rebuilt as a small FastAPI service, chosen over Flask specifically because FastAPI’s automatic, interactive Swagger documentation at /docs gives a stronger visual result for the same amount of code, worth favoring when a project is also meant to be a portfolio piece. It exposes a root endpoint and a /health endpoint. Its first Dockerfile was deliberately simple, a single stage:</p><pre>FROM python:3.12-slim<br>WORKDIR /app<br>COPY requirements.txt .<br>RUN pip install --no-cache-dir -r requirements.txt<br>COPY . .<br>EXPOSE 8000<br>CMD [&quot;uvicorn&quot;, &quot;main:app&quot;, &quot;--host&quot;, &quot;0.0.0.0&quot;, &quot;--port&quot;, &quot;8000&quot;]</pre><pre>docker build -t solstice-python-service:1.0.0 .</pre><figure><img alt="Terminal output of a successful Docker build for solstice-python-service:1.0.0." src="https://cdn-images-1.medium.com/max/1024/1*akm2oAvpV5dLbn2OJJil0A.png" /><figcaption>The Python service’s first Dockerfile build, a single stage installing FastAPI and its dependencies and tagging the result 1.0.0.</figcaption></figure><pre>docker run -d -p 8000:8000 --name solstice-python-service solstice-python-service:1.0.0</pre><figure><img alt="Terminal output of docker ps showing the solstice-python-service container running on port 8000." src="https://cdn-images-1.medium.com/max/1024/1*DHoao63q-CTQJcCyyZYPMA.png" /><figcaption>Docker ps confirming the Python container running and its port 8000 mapping in place.</figcaption></figure><p>With the container up, I checked all three surfaces a real user or teammate would actually touch: the root endpoint, the health endpoint, and the interactive docs.</p><figure><img alt="Browser window showing a JSON response from the Python service’s root endpoint." src="https://cdn-images-1.medium.com/max/485/1*K7aLR16mVsrHlYnIESPeag.png" /><figcaption>The Python service’s root endpoint responding at localhost:8000, confirming FastAPI is running inside the container.</figcaption></figure><figure><img alt="Browser window showing a JSON health check response from the Python service." src="https://cdn-images-1.medium.com/max/313/1*V_h2pakYmQ-TTkCnI6bvSA.png" /><figcaption>The /health endpoint responding with a healthy status and version number.</figcaption></figure><figure><img alt="FastAPI Swagger UI showing the Solstice Devs Python Service with GET endpoints for the root path and /health." src="https://cdn-images-1.medium.com/max/1024/1*uV_nCQygmd-EutELsMRE5w.png" /><figcaption>FastAPI’s automatically generated interactive Swagger documentation at /docs, listing both available endpoints.</figcaption></figure><p>With everything responding correctly, the image was tagged and pushed to a public Docker Hub repository, deliberately using a real version tag instead of latest, since latest tells a teammate nothing about what they&#39;re actually running.</p><pre>docker tag solstice-python-service:1.0.0 rester12/solstice-python-service:1.0.0<br>docker push rester12/solstice-python-service:1.0.0</pre><figure><img alt="Terminal output of docker push completing for rester12/solstice-python-service:1.0.0." src="https://cdn-images-1.medium.com/max/902/1*Ulmt4ScZpGDX7xedqkyCCg.png" /><figcaption>Pushing the tagged 1.0.0 image to a public Docker Hub repository.</figcaption></figure><figure><img alt="Docker Hub repository page for rester12/solstice-python-service showing the 1.0.0 tag." src="https://cdn-images-1.medium.com/max/1024/1*_yxuDVbBO6Agi_j2XTe1Rg.png" /><figcaption>The Docker Hub repository page confirming the 1.0.0 image is live and pullable.</figcaption></figure><h3>Phase 2 - Java: Spring Boot, Build, and First Push</h3><p>The Java tool was scaffolded with Spring Initializr, Maven, Java 17, Spring Web, with a HelloController exposing the same root and /health shape as the Python service, for consistency across all three. The first Dockerfile ran the entire Maven build inside the maven image itself:</p><pre>FROM maven:3.9-eclipse-temurin-17<br>WORKDIR /app<br>COPY pom.xml .<br>COPY src ./src<br>RUN mvn clean package -DskipTests<br>EXPOSE 8080<br>CMD [&quot;sh&quot;, &quot;-c&quot;, &quot;java -jar target/*.jar&quot;]</pre><h3>The Problem I Encountered: An Invisible Character Broke the Build</h3><p>The very first build of this Dockerfile failed with a message that gave almost no useful clue at a glance:</p><pre>illegal character: &#39;﻿&#39;</pre><p>pointing at line 1 of HelloController.java, a file that looked completely normal in the editor. The actual cause was upstream of the Java code entirely: I&#39;d written the file using Set-Content -Encoding utf8 in Windows PowerShell 5.1, and that flag writes a UTF-8 byte-order mark, an invisible three-byte sequence, at the very start of the file. Java&#39;s compiler treats that BOM as an illegal character and refuses to compile, which only ever surfaces once something actually tries to compile the file, in this case, deep inside docker build&#39;s Maven step, far from where the real mistake was made. The fix was to rewrite the file with -Encoding ascii instead, safe since the source is plain ASCII, and I adopted that as a standing rule for every .java, .py, and .ts file I wrote by hand for the rest of the project.</p><p>Once rebuilt successfully, the container was run and checked the same way as Python:</p><pre>docker build -t solstice-java-service:1.0.0 .</pre><figure><img alt="Terminal output of a successful Docker build for solstice-java-service:1.0.0." src="https://cdn-images-1.medium.com/max/1024/1*kG9fsyEE_ExYfY30JT6k9g.png" /><figcaption>The Java service’s Dockerfile building successfully after fixing the UTF-8 BOM issue that had blocked the Maven compile step.</figcaption></figure><pre>docker run -d -p 8080:8080 --name solstice-java-service solstice-java-service:1.0.0</pre><figure><img alt="Terminal output of docker ps listing the running solstice-java-service and solstice-python-service containers." src="https://cdn-images-1.medium.com/max/1024/1*ZSdMII8x_27lYNyObbhNpw.png" /><figcaption>Docker ps showing both the Java and Python containers running side by side.</figcaption></figure><figure><img alt="Browser window showing a JSON response from the Java service’s root endpoint." src="https://cdn-images-1.medium.com/max/470/1*G89EybUX_CZLk-w0cDvX2g.png" /><figcaption>The Java service’s root endpoint responding at localhost:8080, confirming Spring Boot is running inside the container.</figcaption></figure><p>Rather than pushing Java to Docker Hub right away, I moved on to scaffolding Node next, and came back to actually push Java a little later in the project, once I realized the push itself hadn’t happened yet, a good reminder that “I built and ran it” and “I published it” are two different claims, and it’s worth checking which one is actually true before writing it down as done.</p><pre>docker tag solstice-java-service:1.0.0 rester12/solstice-java-service:1.0.0<br>docker push rester12/solstice-java-service:1.0.0</pre><figure><img alt="Terminal output of docker push completing for rester12/solstice-java-service:1.0.0." src="https://cdn-images-1.medium.com/max/871/1*hLPAOfCt5-Rha1SnjYBmYA.png" /><figcaption>Pushing the Java 1.0.0 image to Docker Hub, confirmed after initially assuming this step had already happened.</figcaption></figure><h3>Phase 3 - .dockerignore: A Fix That Doesn’t Always Fix Anything</h3><p>A .dockerignore was added to all three projects, but whether it changes anything at all turns out to depend entirely on how the Dockerfile actually copies files in. Java&#39;s Dockerfile only ever runs COPY pom.xml . and COPY src ./src, explicit, narrow copies, so its .dockerignore had almost nothing to exclude in the first place:</p><pre>target/<br>.git/<br>.idea/<br>*.iml<br>.mvn/<br>mvnw<br>mvnw.cmd</pre><figure><img alt="Docker images output showing solstice-java-service size before adding a .dockerignore file." src="https://cdn-images-1.medium.com/max/1024/1*qyZES4cHrTjSae9lNw_xJg.png" /><figcaption>Java’s image size before adding a .dockerignore.</figcaption></figure><figure><img alt="Docker images output showing solstice-java-service size unchanged after adding a .dockerignore file." src="https://cdn-images-1.medium.com/max/1024/1*DjsixgiQm6g1LcW-ci342Q.png" /><figcaption>Java’s image size after adding a .dockerignore, essentially unchanged since the Dockerfile only copies pom.xml and src explicitly.</figcaption></figure><p>Sure enough, the before-and-after images came back essentially identical, which isn’t a failed test, it’s the correct result given how that Dockerfile is written. Python’s Dockerfile uses a blanket COPY . ., so I tested it against a clean project directory first:</p><pre>python-app/.dockerignore<br>__pycache__/<br>*.pyc<br>.venv/<br>venv/<br>.git/<br>.gitignore<br>.vscode/<br>.idea/<br>*.md</pre><figure><img alt="Docker images output showing solstice-python-service size before adding a .dockerignore file." src="https://cdn-images-1.medium.com/max/1024/1*f2JjgDtRXRx2POVBsgxiFQ.png" /><figcaption>Python’s image size before adding a .dockerignore, in an already-clean project directory.</figcaption></figure><figure><img alt="Docker images output showing solstice-python-service size unchanged after adding a .dockerignore file in a clean directory." src="https://cdn-images-1.medium.com/max/1024/1*v1gJQLMlOhRs7gK4iHQ9kA.png" /><figcaption>Python’s image size after adding a .dockerignore, unchanged since there was nothing extraneous in the build context yet.</figcaption></figure><p>Again, no real difference, because the directory was already clean, there was nothing extraneous for the .dockerignore to exclude. To actually demonstrate the mechanism doing something, I intentionally created a Python virtual environment folder inside the project directory first, the kind of thing that realistically ends up in a build context by accident:</p><figure><img alt="Docker images output showing a larger solstice-python-service size with a virtual environment folder present in the build context." src="https://cdn-images-1.medium.com/max/1024/1*qZI2b0TEMHaZV3m_RHna2A.png" /><figcaption>A more realistic test: with a virtual environment folder intentionally added to the project directory, the image size grows, showing what actually ends up in a build context by accident.</figcaption></figure><p>With the .venv folder present and not yet excluded, the image size actually grew. Adding .venv/ to the .dockerignore and rebuilding brought it right back down to the clean-directory size:</p><figure><img alt="Docker images output showing solstice-python-service size reduced back to its clean-directory size after excluding the virtual environment folder." src="https://cdn-images-1.medium.com/max/1024/1*m7eJWjKUbCAVYWZ0hCVInA.png" /><figcaption>With .venv added to the .dockerignore, the size drops right back down, real proof the exclusion is working rather than just sitting unused in the file.</figcaption></figure><p>That before/with-venv/after sequence is the real proof, a .dockerignore earns its keep specifically in the scenario it exists for: an accidentally bloated build context, not a directory that was already tidy.</p><h3>Phase 4 - Node: NestJS, a Real Troubleshooting Moment, and First Push</h3><p>The Node tool was scaffolded with NestJS, ESM modules with Vitest rather than CommonJS with Jest, chosen over a bare Express app for the same reason FastAPI was chosen over Flask: a more structured, more standout result for a comparable amount of effort. Its first Dockerfile installed dependencies, compiled the TypeScript output, and ran the compiled JavaScript directly:</p><pre>FROM node:20<br>WORKDIR /app<br>COPY package*.json ./<br>RUN npm install<br>COPY . .<br>RUN npm run build<br>EXPOSE 3000<br>CMD [&quot;node&quot;, &quot;dist/main.js&quot;]</pre><pre>docker build -t solstice-node-service:1.0.0 .</pre><figure><img alt="Terminal output of a successful Docker build for solstice-node-service:1.0.0." src="https://cdn-images-1.medium.com/max/1024/1*puGBLPrV2I34ezMWUvj4UA.png" /><figcaption>The Node service’s Dockerfile building successfully, compiling the NestJS TypeScript source and tagging the result 1.0.0.</figcaption></figure><p>The first attempt to run this container produced nothing at all, docker ps -a showed no container, not even one in an exited state, and docker logs reported that no such container existed. I hadn&#39;t changed a single thing, so I ran the identical docker run command again, and it worked, the container came up cleanly and NestJS&#39;s own startup logs confirmed both routes mapped correctly. I never pinned down a definitive root cause, most likely a transient Docker Desktop hiccup, but I&#39;m including it here rather than pretending the first attempt didn&#39;t happen, because a real troubleshooting log is more useful, to me later and to anyone reading this, than a version of events where everything worked the first time.</p><pre>docker tag solstice-node-service:1.0.0 rester12/solstice-node-service:1.0.0<br>docker push rester12/solstice-node-service:1.0.0</pre><figure><img alt="Terminal output of docker push completing for rester12/solstice-node-service:1.0.0." src="https://cdn-images-1.medium.com/max/1024/1*X6Kj4i5WT99gUpUioOBwIg.png" /><figcaption>Pushing the Node 1.0.0 image to Docker Hub for the first time.</figcaption></figure><figure><img alt="Browser window showing a JSON response from the Node service&#39;s root endpoint." src="https://cdn-images-1.medium.com/max/470/1*gNCsXqXsX5A-hj023zXHdg.png" /><figcaption>The Node service&#39;s root endpoint responding at localhost:3000 after a retry resolved an initial container-start failure.</figcaption></figure><p>With all three services built, run, and pushed at least once, docker ps showed all three running side by side, confirmation that three completely different runtimes, Python, Java, and Node, could all be operated the same way once containerized.</p><figure><img alt="Terminal output of docker ps listing the running Python, Java, and Node service containers together." src="https://cdn-images-1.medium.com/max/867/1*Eka0GCxH1MwuCmAFkLZooA.png" /><figcaption>Docker ps showing all three services, Python, Java, and Node, running simultaneously.</figcaption></figure><p>Node’s own .dockerignore comparison came later than the others, since its COPY . . pattern meant node_modules and the compiled dist folder were realistic candidates for bloating the build context the same way Python&#39;s .venv had:</p><pre>node-app/.dockerignore<br>node_modules/<br>dist/<br>.git/<br>.vscode/<br>.idea/<br>*.md<br>npm-debug.log</pre><figure><img alt="Docker images output showing a larger solstice-node-service size before adding a .dockerignore file." src="https://cdn-images-1.medium.com/max/1024/1*cYfdZLQ2Lmtb74wRm3hrCQ.png" /><figcaption>Node’s image size before adding a .dockerignore, with node_modules and the compiled dist folder both present in the build context.</figcaption></figure><figure><img alt="Docker images output showing a reduced solstice-node-service size after adding a .dockerignore file." src="https://cdn-images-1.medium.com/max/1024/1*IhAG1m0u0oCHnPlftEHQRg.png" /><figcaption>Node’s image size after adding a .dockerignore, showing a real reduction once node_modules and dist were excluded from the build context.</figcaption></figure><p>With node_modules and dist excluded, the size dropped in a way that was actually visible, the same underlying lesson as Python&#39;s venv demo: a .dockerignore matters exactly as much as what&#39;s sitting in the directory when you build. Node&#39;s rebuilt image reflected the same fix once the smaller version was pushed, matching the size drop already shown in the docker images comparison above.</p><figure><img alt="Docker Desktop&#39;s local image list showing solstice-node-service:1.0.0." src="https://cdn-images-1.medium.com/max/1024/1*HBPmzDnrQevtsmHpoJ0_VA.png" /><figcaption>Docker Desktop&#39;s local image list for solstice-node-service:1.0.0.</figcaption></figure><h3>Phase 5 - Multi-Stage Builds: Same Application, Much Less Baggage</h3><p>All three Dockerfiles were converted to multi-stage builds: a build stage that installs dependencies and compiles the application, and a slim runtime stage that copies over only the finished artifact, discarding every compiler, package manager, and build cache the build stage needed but the running application never will.</p><p><strong>Python:</strong></p><pre>FROM python:3.12-slim AS builder<br>WORKDIR /app<br>COPY requirements.txt .<br>RUN pip install --no-cache-dir --user -r requirements.txt</pre><pre>FROM python:3.12-slim<br>WORKDIR /app<br>COPY --from=builder /root/.local /root/.local<br>COPY . .<br>ENV PATH=/root/.local/bin:$PATH<br>EXPOSE 8000<br>CMD [&quot;uvicorn&quot;, &quot;main:app&quot;, &quot;--host&quot;, &quot;0.0.0.0&quot;, &quot;--port&quot;, &quot;8000&quot;]</pre><pre>docker build -t solstice-python-service:2.0.0 .</pre><figure><img alt="Terminal output of a successful multi-stage Docker build for solstice-python-service:2.0.0." src="https://cdn-images-1.medium.com/max/1024/1*WkXTaxDFjnFJHguc1BCN2Q.png" /><figcaption>Python’s Dockerfile rebuilt as a multi-stage build and tagged 2.0.0.</figcaption></figure><figure><img alt="Docker images output comparing solstice-python-service 1.0.0 and 2.0.0 image sizes." src="https://cdn-images-1.medium.com/max/822/1*Lkyvv9eryCpPtX-snR0OTA.png" /><figcaption>Python’s 1.0.0 single-stage image compared against its 2.0.0 multi-stage image, showing a real size reduction.</figcaption></figure><p><strong>Java:</strong></p><pre>FROM maven:3.9-eclipse-temurin-17 AS builder<br>WORKDIR /app<br>COPY pom.xml .<br>COPY src ./src<br>RUN mvn clean package -DskipTests</pre><pre>FROM eclipse-temurin:17-jre<br>WORKDIR /app<br>COPY --from=builder /app/target/*.jar app.jar<br>EXPOSE 8080<br>CMD [&quot;java&quot;, &quot;-jar&quot;, &quot;app.jar&quot;]</pre><pre>docker build -t solstice-java-service:2.0.0 .</pre><figure><img alt="Terminal output of a successful multi-stage Docker build for solstice-java-service:2.0.0." src="https://cdn-images-1.medium.com/max/1024/1*yS919Lc7leWfqh4qadkUbw.png" /><figcaption>Java’s Dockerfile rebuilt as a multi-stage build, swapping a full Maven-plus-JDK toolchain for a bare JRE runtime, tagged 2.0.0.</figcaption></figure><p>This produced by far the most dramatic reduction of the three, swapping a full Maven-plus-JDK toolchain for a bare JRE runtime isn’t a trim, it’s a different category of image.</p><figure><img alt="Docker images output comparing solstice-java-service 1.0.0 and 2.0.0 image sizes showing a large reduction." src="https://cdn-images-1.medium.com/max/1024/1*1x-fgHnAz4XOndqXNqjM1A.png" /><figcaption>Java’s 1.0.0 single-stage image compared against its 2.0.0 multi-stage image, the most dramatic size reduction of the three languages.</figcaption></figure><p><strong>Node:</strong></p><pre>FROM node:20 AS builder<br>WORKDIR /app<br>COPY package*.json ./<br>RUN npm install<br>COPY . .<br>RUN npm run build</pre><pre>FROM node:20-slim<br>WORKDIR /app<br>COPY package*.json ./<br>RUN npm install --omit=dev<br>COPY --from=builder /app/dist ./dist<br>EXPOSE 3000<br>CMD [&quot;node&quot;, &quot;dist/main.js&quot;]</pre><pre>docker build -t solstice-node-service:2.0.0 .</pre><figure><img alt="Terminal output of a successful multi-stage Docker build for solstice-node-service:2.0.0." src="https://cdn-images-1.medium.com/max/1024/1*JX96iQSMG46PbOsJ91teKA.png" /><figcaption>Node’s Dockerfile rebuilt as a multi-stage build and tagged 2.0.0.</figcaption></figure><figure><img alt="Docker images output comparing solstice-node-service 1.0.0 and 2.0.0 image sizes." src="https://cdn-images-1.medium.com/max/1024/1*xoO2g7ig3OnOPNFdSXrJIw.png" /><figcaption>Node’s 1.0.0 single-stage image compared against its 2.0.0 multi-stage image, showing a real size reduction.</figcaption></figure><h3>Single-Stage vs. Multi-Stage Builds</h3><p>The application code never changed across any of these conversions, what changed is what the image is carrying alongside it. A single-stage build keeps every tool used to build the application in the final image, whether or not the running application ever touches it again. A multi-stage build throws the builder stage away entirely once it’s done its job, keeping only the finished artifact in the image that actually ships. Same behavior at runtime, meaningfully less surface area, both in download size and in what an attacker would have available to work with if they ever got inside.</p><h3>Phase 6 - A Real Tagging Convention and a Simulated CI Push</h3><p>With all three services on solid multi-stage Dockerfiles, I established one tagging convention across all three: every image gets a semantic version tag and a short Git commit hash tag, exactly what a CI pipeline would attach to a build on every merge to main. To make the commit hash real rather than a placeholder, I initialized an actual Git repository at the project root:</p><pre>git init<br>git add .<br>git commit -m &quot;Initial commit: Dockerfiles for Python, Java, and Node services&quot;<br>git rev-parse --short HEAD</pre><p>That returned 77bd404. Each image was then built once and tagged twice, simulating the build-once, tag-twice, push-twice pattern a CI pipeline runs on every release:</p><pre>docker tag solstice-python-service:2.0.0 rester12/solstice-python-service:2.0.0<br>docker tag solstice-python-service:2.0.0 rester12/solstice-python-service:77bd404<br>docker push rester12/solstice-python-service:2.0.0<br>docker push rester12/solstice-python-service:77bd404</pre><p>The same pattern was repeated for Java and Node, and each repository’s tags page was checked directly in the browser afterward, confirming both tags actually appeared rather than just trusting the CLI’s success output.</p><figure><img alt="Docker Hub tag dropdown for rester12/solstice-python-service showing 1.0.0, 2.0.0, and 77bd404 tags." src="https://cdn-images-1.medium.com/max/1024/1*pgDDLlirTrjv2A__TgEGeg.png" /><figcaption>The Python Docker Hub repository showing both the 2.0.0 semantic-version tag and the 77bd404 short commit-hash tag, simulating a CI pipeline’s dual-tag push.</figcaption></figure><figure><img alt="Docker Hub tag dropdown for rester12/solstice-java-service showing 1.0.0, 2.0.0, and 77bd404 tags." src="https://cdn-images-1.medium.com/max/1024/1*JyuZJTOfc9mhIYV8y49OkA.png" /><figcaption>The Java Docker Hub repository showing both the 2.0.0 and 77bd404 tags after the dual-tag push.</figcaption></figure><figure><img alt="Docker Hub tag dropdown for rester12/solstice-node-service showing 1.0.0, 2.0.0, and 77bd404 tags." src="https://cdn-images-1.medium.com/max/1024/1*zYwIOZPGMtG4wUyOdpo0jw.png" /><figcaption>The Node Docker Hub repository showing both the 2.0.0 and 77bd404 tags after the dual-tag push.</figcaption></figure><h3>Phase 7 - Non-Root Users: The Last Requirement, and a Second Invisible Bug</h3><p>The final requirement was to run all three containers as a non-root user, meaningfully reducing what an attacker gains if they ever achieved code execution inside one of them. Python and Java both needed a user created explicitly, since neither base image ships with one:</p><pre># python-app/Dockerfile (runtime stage, excerpt)<br>RUN useradd --create-home --shell /bin/bash appuser<br>COPY --from=builder /root/.local /home/appuser/.local<br>COPY . .<br>RUN chown -R appuser:appuser /app<br>ENV PATH=/home/appuser/.local/bin:$PATH<br>USER appuser</pre><pre># java-app/Dockerfile (runtime stage, excerpt)<br>RUN useradd --create-home --shell /bin/bash appuser<br>COPY --from=builder /app/target/*.jar app.jar<br>RUN chown appuser:appuser app.jar<br>USER appuser</pre><p>Node’s official images are the exception here: node:20-slim already ships a built-in node user, so its runtime stage only needed to hand that existing user ownership of the app directory:</p><pre># node-app/Dockerfile (runtime stage, excerpt)<br>RUN chown -R node:node /app<br>USER node</pre><h3>The Second Problem I Encountered: A Silent PowerShell Truncation</h3><p>Writing the Python Dockerfile’s ENV PATH=/home/appuser/.local/bin:$PATH line through a double-quoted PowerShell here-string (@&quot;...&quot;@) caused PowerShell itself to interpolate $PATH as an empty PowerShell variable before the file was ever written to disk. The line silently became ENV PATH=/home/appuser/.local/bin:, nothing after the colon, wiping out /usr/bin, /bin, and every other standard system path from the resulting image. There was no error at build time at all; the first sign of trouble was that even whoami and id couldn&#39;t be found once the container was actually running. The fix was switching to a single-quoted here-string (@&#39;...&#39;@), which writes its contents completely literally, with no PowerShell interpolation at all. It&#39;s the same underlying category of mistake as the earlier Java BOM issue: a Windows text-writing default doing something invisible, surfacing only much later and in a completely different place than where the actual mistake was made.</p><p>Each image was rebuilt and tagged 3.0.0, run, and checked directly from inside the running container:</p><pre>docker exec solstice-python-service whoami<br>docker exec solstice-python-service id</pre><figure><img alt="Terminal output of whoami and id commands run inside the Python container, both confirming the non-root appuser." src="https://cdn-images-1.medium.com/max/891/1*5_kQte3yLeI8PeT0rK2swg.png" /><figcaption>Confirming the Python container runs as a non-root appuser rather than root, after fixing a PowerShell here-string bug that had silently broken the container’s PATH.</figcaption></figure><figure><img alt="Terminal output of whoami and id commands run inside the Java container, both confirming the non-root appuser." src="https://cdn-images-1.medium.com/max/881/1*3ufM_ixk1hlymYPjIn90Bw.png" /><figcaption>Confirming the Java container runs as a non-root appuser rather than root.</figcaption></figure><figure><img alt="Terminal output of whoami and id commands run inside the Node container, both confirming the non-root node user." src="https://cdn-images-1.medium.com/max/826/1*yqAcdqhuQ5tb3LKrgexNjQ.png" /><figcaption>Confirming the Node container runs as the built-in non-root node user that ships with the official Node images.</figcaption></figure><p>All three came back with their intended non-root user, appuser for Python and Java, node for Node, rather than root, closing out the final requirement of the project.</p><h3>Security and Safety Decisions</h3><ul><li><strong>Non-root by default, everywhere.</strong> Every final image runs as a dedicated, unprivileged user rather than root, whether that meant creating one from scratch (Python, Java) or reusing the one an image already ships with (Node).</li><li><strong>Version tags over </strong><strong>latest, always.</strong> Every image was pushed with a real semantic version and a commit-hash tag; latest was never relied on, so anyone pulling any of these three images knows exactly what they&#39;re getting.</li><li><strong>Multi-stage builds as a security boundary, not just a size optimization.</strong> A build stage’s compilers and package managers are real, potentially exploitable software; a runtime image that never ships them removes an entire category of attack surface, independent of the size savings.</li><li><strong>Two invisible-character bugs, two loud lessons.</strong> The UTF-8 BOM and the PowerShell here-string interpolation both failed silently at write-time and loudly much later, a reminder that Windows text-writing defaults deserve deliberate attention rather than blind trust.</li></ul><h3>Resource Cleanup</h3><pre>docker rm -f solstice-python-service solstice-java-service solstice-node-service<br>docker rmi solstice-python-service:1.0.0 solstice-python-service:2.0.0 solstice-python-service:3.0.0<br>docker rmi solstice-java-service:1.0.0 solstice-java-service:2.0.0 solstice-java-service:3.0.0<br>docker rmi solstice-node-service:1.0.0 solstice-node-service:2.0.0 solstice-node-service:3.0.0</pre><p>The published images themselves stay on Docker Hub as the actual deliverable; the local containers and superseded local tags were the only things worth cleaning up once the project wrapped.</p><h3>What I Would Add Before Production Use</h3><ul><li>Wire an actual CI pipeline (GitHub Actions, for example) to run this build-tag-push flow automatically on every merge, instead of by hand.</li><li>Add a HEALTHCHECK instruction to each Dockerfile so an orchestrator could detect an unhealthy service directly rather than only whether the container process is alive.</li><li>Pin base image digests instead of tags, for fully reproducible builds regardless of what a tag might later point to.</li><li>Add automated vulnerability scanning (Docker Scout or Trivy, for example) to each image as part of the build, before it’s ever pushed.</li></ul><h3>What I Learned</h3><ul><li><strong>A </strong><strong>.dockerignore&#39;s value depends entirely on the Dockerfile it&#39;s paired with.</strong> Java&#39;s explicit, narrow COPY instructions meant its .dockerignore had nothing meaningful to exclude; Python&#39;s and Node&#39;s blanket COPY . . meant theirs mattered the moment anything extraneous, a .venv, a node_modules, showed up in the build context.</li><li><strong>Multi-stage builds don’t pay off equally across languages.</strong> Java’s toolchain-heavy build stage made its reduction dramatic; Python’s and Node’s builder stages were already fairly lean, so their improvements were real but smaller.</li><li><strong>Windows text-writing defaults can fail silently in ways that surface somewhere else entirely.</strong> A UTF-8 BOM broke a Java compile step deep inside a Docker build; a PowerShell here-string’s variable interpolation broke a container’s PATH with zero output at build time. Both were invisible until something downstream tried to use the file.</li><li><strong>Simulating a CI tagging flow by hand makes it obvious why pipelines automate it.</strong> Doing build-once, tag-twice, push-twice correctly and consistently across three separate projects, by hand, every time, is exactly the kind of repetitive, error-prone task a pipeline exists to remove.</li><li><strong>Not every base image treats production hardening the same way.</strong> Node’s official images ship a non-root user out of the box; Python’s and Java’s don’t, a small but real difference in how much each ecosystem’s tooling anticipates this step being necessary.</li></ul><h3>Project Repository</h3><p>The full project, including all three Dockerfiles and this write-up’s source commands, is available on GitHub:</p><p><a href="https://github.com/rester12/Containerizing-Java-Python-JavaScript-Apps">GitHub - rester12/Containerizing-Java-Python-JavaScript-Apps: Containerizing Java, Python, and JavaScript Apps - Multi-Stage Docker Builds, Non-Root Users, and Docker Hub Tagging</a></p><h3>Closing Thoughts</h3><p>None of the interesting parts of this project were the happy path. The Dockerfiles that just worked on the first try aren’t what I remember; the BOM that broke a build with an invisible character, and the here-string that silently gutted a container’s PATH, are. Both were caught by actually running the thing and checking it, not by assuming a Dockerfile did what it looked like it should do. That’s really the throughline across all three languages here: containerizing three completely different tools isn’t hard because Docker is complicated, it’s hard because each ecosystem hides its own small, specific ways of breaking, and finding them takes actually building and running the thing, three separate times.</p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79d29e102b13" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Compose-WordPress + MySQL: Network Segmentation, Secrets, and Healthcheck-Gated Startup]]></title>
            <link>https://medium.com/@rester.mcglown/docker-compose-wordpress-mysql-network-segmentation-secrets-and-healthcheck-gated-startup-0c72836bd4ae?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/0c72836bd4ae</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[wordpress]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 19:24:10 GMT</pubDate>
            <atom:updated>2026-09-11T19:27:39.320Z</atom:updated>
            <content:encoded><![CDATA[<p>A hands-on Docker Compose project standing up a WordPress blogging stack, then deliberately hardening it in stages: named volumes for persistence, phpMyAdmin for direct database visibility, network segmentation to isolate the database layer, Docker secrets instead of plaintext credentials, and a healthcheck that stops WordPress from racing MySQL on startup.</p><h3>Project Architecture and Objectives</h3><p>This project stands up a WordPress site backed by MySQL, with phpMyAdmin alongside it for database inspection, entirely through Docker Compose rather than individual docker run commands. Compose let the whole stack, services, volumes, networks, and secrets, live in one declarative file instead of being wired together by hand.</p><figure><img alt="Diagram of a Docker Compose architecture showing WordPress on a frontend and backend network, MySQL and phpMyAdmin on a backend-only network, with named volumes and secrets attached to each service." src="https://cdn-images-1.medium.com/max/1024/1*PudGcwLMABFB3b78z1k09w.png" /><figcaption>The final stack’s architecture, showing WordPress bridging both the frontend and backend networks while MySQL and phpMyAdmin stay backend-only, with named volumes and Docker secrets backing the stateful and credential-handling pieces.</figcaption></figure><p>The objectives were to:</p><ul><li>Write a docker-compose.yml that brings up WordPress and MySQL together with a single command</li><li>Persist MySQL’s data and WordPress’s uploads in named volumes, and prove that data survives docker compose down without -v</li><li>Add phpMyAdmin on a backend-only network to inspect WordPress’s database directly</li><li>Segment traffic into explicit frontend and backend networks, keeping MySQL and phpMyAdmin off the public-facing network entirely</li><li>Replace plaintext environment-variable credentials with Docker Compose secrets, and confirm the containers actually read credentials from mounted secret files</li><li>Add a MySQL healthcheck and use depends_on: condition: service_healthy so WordPress waits for a genuinely ready database instead of just a started container</li></ul><h3>The Problem I Wanted to Solve</h3><p>The scenario behind this project was The Daily Byte, a small blog network, wanting a reproducible way to stand up its WordPress hosting stack without manually wiring together containers, networks, and volumes every time. That framing shaped the whole build: a “reproducible stack” only means something if a stranger, or future me, can clone the repo, drop in their own credentials, and run one command. Every choice after the first working version, the volumes, the network split, the secrets, the healthcheck, was really in service of that one requirement.</p><h3>Why I Kept Adding Constraints Instead of Stopping at “It Works”</h3><p>The stack technically worked after the first version: two services, plaintext credentials, one flat network. It would have been easy to call that done. I didn’t, because a working demo and a defensible deployment aren’t the same thing, and the gap between them is exactly where the interesting engineering decisions live. Each later phase, network segmentation, secrets, the healthcheck, exists because the simpler version had a real, specific weakness, not because more configuration is inherently better.</p><h3>Tools and Technologies Used</h3><ul><li>Docker Desktop, Docker Engine, and Docker Compose</li><li>WordPress (latest) as the CMS</li><li>MySQL 8.0 as the database backend</li><li>phpMyAdmin for direct database inspection</li><li>Docker named volumes for persistent storage</li><li>Docker Compose secrets for credential management</li><li>Docker Compose healthchecks and depends_on conditions</li><li>Docker bridge networks for traffic segmentation</li><li>Windows PowerShell</li><li>Git and GitHub</li></ul><h3>Phase 1 — WordPress and MySQL with Docker Compose</h3><p>The stack started as a minimal two-service docker-compose.yml, using the official wordpress and mysql:8.0 images with database credentials passed as plain environment variables. Bringing the whole thing up was a single command:</p><pre>docker compose up -d</pre><figure><img alt="Terminal output of docker compose ps showing a running mysql and wordpress container." src="https://cdn-images-1.medium.com/max/1024/1*R1zDHR4XZAZQbYPo494jVw.png" /><figcaption>Docker compose ps confirming the initial two-service stack, WordPress and MySQL, both running after a single docker compose up -d.</figcaption></figure><p>Visiting localhost:8080 loaded WordPress&#39;s own setup wizard, starting with a language selection.</p><figure><img alt="WordPress installation wizard language selection screen with English (United States) highlighted." src="https://cdn-images-1.medium.com/max/1024/1*Rzxu8TIv8DUnYvKWRKVgAw.png" /><figcaption>WordPress’s setup wizard, reached at localhost:8080, prompting for a default language before installation continues.</figcaption></figure><p>After choosing a site title and creating an admin account, the wizard confirmed WordPress had installed successfully and connected to MySQL on the first try, no manual database setup required.</p><figure><img alt="WordPress installation success screen showing the admin username and a Log In link." src="https://cdn-images-1.medium.com/max/1024/1*RzJY9mAmhGDOKSYBvHxbuw.png" /><figcaption>The setup wizard’s success screen, confirming WordPress installed and connected to MySQL, with the new admin username shown.</figcaption></figure><p>Logging in dropped me straight into the WordPress dashboard, confirming the whole chain, Compose, MySQL, and WordPress, was working end to end.</p><figure><img alt="WordPress admin dashboard showing site health, at-a-glance, and activity widgets after initial setup." src="https://cdn-images-1.medium.com/max/1024/1*tColi4iQZ_2fyDNUkm_fPg.png" /><figcaption>The WordPress admin dashboard after logging in for the first time, confirming the full stack, WordPress talking to MySQL, is working end to end.</figcaption></figure><h3>Phase 2 — Persistent Volumes and Proving It Immediately</h3><p>Named volumes were added for both stateful paths: MySQL’s data directory and WordPress’s uploads directory. Rather than trusting the documentation that named volumes survive a container being removed, I tested it directly. I created a test post, then tore the stack down and brought it back up without touching the volumes:</p><pre>docker compose down<br>docker compose up -d</pre><figure><img alt="Terminal output of docker compose ps showing mysql and wordpress containers recreated seconds earlier." src="https://cdn-images-1.medium.com/max/1024/1*q9Ysv6FzhF8wbrti5l2Ozg.png" /><figcaption>Docker compose ps showing both containers freshly recreated after a docker compose down and up, run to test whether named-volume data would survive.</figcaption></figure><p>The test post was still there on the front end, and WordPress returned straight to a logged-in dashboard rather than the setup wizard, meaning none of the state I cared about had actually lived inside the container that just got deleted.</p><figure><img alt="WordPress single post view titled “The Daily Byte — Test post” describing a persistence test." src="https://cdn-images-1.medium.com/max/1024/1*Szd2Zr3H0a7JY-cjB_zEqw.png" /><figcaption>The WordPress blog’s front end showing a test post, “The Daily Byte — Test post”, still present after the containers were recreated, confirming the named volumes held the data.</figcaption></figure><p>Opening the same post in the block editor confirmed it further: it showed as already published, with revision history intact, not freshly created.</p><figure><img alt="WordPress block editor showing the test post’s status as Published with two revisions." src="https://cdn-images-1.medium.com/max/1024/1*z9W1PAdKorbEGEAsUlPBPw.png" /><figcaption>The same test post open in WordPress’s block editor, showing it as published with existing revisions, confirming it wasn’t recreated fresh after the container rebuild.</figcaption></figure><h3>Phase 3 — Adding phpMyAdmin for Direct Database Visibility</h3><p>With WordPress’s own persistence confirmed, I added a phpMyAdmin service pointed at the same MySQL instance, giving direct visibility into the tables WordPress was actually writing to, without needing to shell into the MySQL container and run raw SQL by hand.</p><pre>docker compose up -d</pre><figure><img alt="Terminal output showing phpMyAdmin’s image being pulled and started, followed by docker compose ps listing mysql, phpmyadmin, and wordpress containers." src="https://cdn-images-1.medium.com/max/1024/1*vyxpzb-G6R95Q7PizudaTQ.png" /><figcaption>Docker compose up -d pulling and starting the new phpMyAdmin service alongside the existing stack, with docker compose ps confirming all three containers running.</figcaption></figure><p>phpMyAdmin came up cleanly at localhost:8081.</p><figure><img alt="phpMyAdmin welcome and login screen prompting for a username and password." src="https://cdn-images-1.medium.com/max/1024/1*lEBknDU1ZcUsuMja5rQv5A.png" /><figcaption>phpMyAdmin’s login screen at localhost:8081, reachable from the browser despite sitting on a backend-only network internally.</figcaption></figure><p>Logging in with the WordPress database user surfaced the full wordpress database and its complete set of wp_-prefixed tables, exactly what WordPress&#39;s own admin screens never show directly.</p><figure><img alt="phpMyAdmin sidebar showing the wordpress database expanded with its full list of wp_ prefixed tables." src="https://cdn-images-1.medium.com/max/1024/1*r-V80mB4KaxVRNNGbK0x-w.png" /><figcaption>phpMyAdmin, logged in and connected to the mysql service, listing WordPress’s full set of database tables, proving direct database visibility without needing shell access into the MySQL container.</figcaption></figure><h3>Phase 4 — Network Segmentation, Secrets, and a Healthcheck</h3><p>The final phase introduced three changes together, because they reinforce the same underlying idea: don’t trust more than you have to, by default.</p><p><strong>Network segmentation.</strong> Two networks were defined, frontend and backend. MySQL and phpMyAdmin were placed on backend only; WordPress was placed on both, since it&#39;s the only service that genuinely needs to be reachable from a browser while also needing database access. After the split, everything still worked exactly as before, which is the point, segmentation shouldn&#39;t change legitimate traffic, only block illegitimate paths.</p><pre>docker compose up -d</pre><figure><img alt="Terminal output of docker compose ps showing mysql, phpmyadmin, and wordpress containers running after network segmentation was added." src="https://cdn-images-1.medium.com/max/1024/1*Y0pjE-u_LNW-PjOs4JnalA.png" /><figcaption>Docker compose ps after introducing the explicit frontend and backend networks, showing all three services still running correctly under the new network topology.</figcaption></figure><p>phpMyAdmin’s own server status page confirmed it was still reaching MySQL correctly over TCP/IP within the backend network.</p><figure><img alt="phpMyAdmin server status panel showing database server, web server, and phpMyAdmin version details." src="https://cdn-images-1.medium.com/max/1024/1*4Vyd3HCHSCJ551mUFFz5qw.png" /><figcaption>phpMyAdmin’s server information panel confirming it reached MySQL over the backend network via TCP/IP, verifying the network segmentation didn’t break legitimate backend-to-backend traffic.</figcaption></figure><p><strong>Docker Compose secrets.</strong> Plaintext credentials in environment variables were replaced with Compose secrets backed by local files, using the _FILE-suffixed environment variables both the MySQL and WordPress images support (MYSQL_PASSWORD_FILE, WORDPRESS_DB_PASSWORD_FILE, and so on), which tell each image to read the credential from a file mounted at /run/secrets/ instead of trusting the environment variable&#39;s literal value.</p><figure><img alt="Terminal output of docker compose ps showing all three containers running shortly after being started." src="https://cdn-images-1.medium.com/max/1024/1*wy-d7u6DJfXMCGE02jkG_w.png" /><figcaption>Docker compose ps immediately before verifying Docker secrets, showing the stack running normally under the updated configuration.</figcaption></figure><p>I didn’t just take that mechanism on faith either. I exec’d into the running MySQL container to confirm both that the environment variable actually pointed at the secret file, and that the file contained the real credential:</p><pre>docker exec docker-project-4-compose-wordpress-mysql-1 env | Select-String &quot;MYSQL_PASSWORD&quot;<br>docker exec docker-project-4-compose-wordpress-mysql-1 cat /run/secrets/db_password</pre><figure><img alt="Terminal output showing MYSQL_PASSWORD_FILE pointing to /run/secrets/db_password, followed by the file’s contents." src="https://cdn-images-1.medium.com/max/1024/1*vD7KlwuOqpcEauJH32IVAQ.png" /><figcaption>Confirming inside the MySQL container that its password is being read from a mounted Docker secret file rather than a plaintext environment variable, and that the file itself contains the expected credential.</figcaption></figure><p><strong>Healthcheck and conditional startup.</strong> A healthcheck was added to the MySQL service using mysqladmin ping, reading the root password from the mounted secret file rather than a hardcoded value. WordPress&#39;s depends_on was changed from the default list form to the map form with an explicit condition:</p><pre>depends_on:<br>  mysql:<br>    condition: service_healthy</pre><p>Bringing the stack up afterward showed exactly the ordering I was after: MySQL reached a Healthy status, and only then did WordPress finish starting.</p><pre>docker compose up -d</pre><figure><img alt="Terminal output of docker compose up and docker compose ps showing MySQL marked healthy and WordPress started afterward." src="https://cdn-images-1.medium.com/max/1024/1*yaVrQs09sZEFFNIIFG9kmA.png" /><figcaption>Docker compose up -d showing MySQL reach a Healthy status before WordPress finishes starting, confirming depends_on: condition: service_healthy is gating startup order as intended.</figcaption></figure><h3>Proving Persistence End-to-End, One Last Time</h3><p>With the fully hardened stack running, network segmentation, secrets, and healthcheck all in place, I repeated the persistence test from Phase 2, this time against the complete configuration. The stack was brought down and back up without removing volumes:</p><pre>docker compose down<br>docker compose up -d</pre><p>The test post and the default WordPress post were both still present on the front end afterward.</p><figure><img alt="WordPress blog homepage showing two posts still present after the containers were recreated." src="https://cdn-images-1.medium.com/max/1024/1*HiydKipDjvOdKo2KCZqDJw.png" /><figcaption>The WordPress blog after a full docker compose down and up, showing the test post and default post both still present, confirming data persisted across the final, fully-configured version of the stack.</figcaption></figure><p>Logging into phpMyAdmin confirmed the same thing from the database side: the underlying tables were unchanged, holding the same data as before the containers were recreated.</p><figure><img alt="phpMyAdmin showing the wordpress database tables unchanged after container recreation." src="https://cdn-images-1.medium.com/max/1024/1*iMr6cdJhNzb8XHxMpJkqVQ.png" /><figcaption>phpMyAdmin confirming the WordPress database tables and data were unchanged after the same container recreation, completing the persistence proof from the database side.</figcaption></figure><h3>Engineering Decisions</h3><ul><li><strong>Docker Compose over individual </strong><strong>docker run commands.</strong> Compose was used to declare the entire stack, services, volumes, networks, and secrets, in a single reproducible file, rather than wiring each container together by hand.</li><li><strong>Separate volumes for MySQL data and WordPress uploads.</strong> Rather than a single shared volume, each service’s persistent state was isolated in its own named volume, keeping their lifecycles independently manageable.</li><li><strong>Two networks instead of one flat network.</strong> frontend and backend were split explicitly so that MySQL and phpMyAdmin, which never need to be reachable from outside the Docker host except through their published ports for local testing, aren&#39;t sitting on the same network as anything public-facing.</li><li><strong>Docker secrets over environment variables for credentials.</strong> The _FILE-suffixed environment variables supported by both the MySQL and WordPress images were used specifically so credentials could be sourced from Compose secrets instead of being visible in plaintext via docker inspect or the compose file itself.</li><li><strong>depends_on with a healthcheck condition instead of a plain list.</strong> A plain depends_on: - mysql only waits for the container to start, not for MySQL to actually accept connections; the healthcheck-gated condition was used specifically to close that gap.</li></ul><h3>Security and Operational Considerations</h3><ul><li><strong>Secrets files are excluded from version control.</strong> The secrets/ directory containing the actual credential files is listed in .gitignore; only the docker-compose.yml referencing their paths is committed, so credentials never end up in the repository history.</li><li><strong>Network segmentation limits blast radius.</strong> Keeping MySQL and phpMyAdmin off the frontend network means that even if the WordPress container were compromised, direct network-level access to the database from outside the backend network is still restricted by Docker&#39;s own network isolation.</li><li><strong>Healthchecks prevent a class of startup-time failures.</strong> Without the service_healthy condition, WordPress could attempt its first database connection before MySQL has finished initializing, particularly on a slower host or a first-ever startup when MySQL has to initialize its data directory from scratch. The healthcheck removes that race condition entirely.</li><li><strong>phpMyAdmin’s exposure is a development convenience, not a production posture.</strong> Its port is published directly here for local testing; in a real deployment it would sit behind authentication and a reverse proxy, and ideally wouldn’t be exposed publicly at all.</li></ul><h3>What I Would Improve for Production</h3><ul><li>Move the secrets/ files out of the project directory entirely and integrate with a dedicated secrets manager instead of local files.</li><li>Put a reverse proxy with TLS in front of WordPress rather than exposing raw HTTP directly.</li><li>Restrict or remove phpMyAdmin’s exposed port in favor of an SSH tunnel or VPN-only access.</li><li>Add a healthcheck to the WordPress service itself, so its own readiness, not just MySQL’s, could be monitored and acted on.</li><li>Pin explicit image versions instead of latest for every service, to avoid an unannounced version change silently altering behavior on a future rebuild.</li></ul><h3>Lessons Learned</h3><ul><li>Docker Compose’s YAML structure is strict about indentation in a way that’s easy to get wrong when copy-pasting or hand-editing: a service’s properties (image, environment, volumes, and so on) must be indented one level deeper than the service name itself, or Compose fails with an unhelpful &quot;did not find expected key&quot; error rather than pointing at the exact line.</li><li>depends_on has two distinct forms, a plain list and a map with condition:, and only the map form supports waiting on a healthcheck; the list form only ever waits for a container to start, not for the service inside it to be ready.</li><li>Named volumes prove their value even outside the intended test: WordPress recognizing pre-existing volume data and skipping its setup wizard on a later rebuild wasn’t the planned first step of testing persistence, but it was the same underlying property already demonstrating itself in practice.</li><li>Reading credentials from files at /run/secrets/, rather than passing them as plaintext environment variables, is directly supported by both the WordPress and MySQL images via their _FILE-suffixed variables, requiring no extra tooling like a dedicated secrets manager for a project at this scale.</li><li>Verifying a security mechanism from inside the running container, rather than trusting that the configuration “should” work, is what actually turns a claim into a proof; exec’ing into MySQL to read the secret file directly was a small step that closed a real gap in confidence.</li></ul><h3>Project Repository</h3><p>The full project, including the docker-compose.yml and this write-up&#39;s source commands, is available on GitHub:</p><p><a href="https://github.com/rester12/Docker-Compose-WordPress-MySQL">GitHub - rester12/Docker-Compose-WordPress-MySQL: Multi-container WordPress and MySQL stack orchestrated with Docker Compose, using named volumes, network segmentation, and secrets to prove persistence and secure credential handling.</a></p><h3>Closing Thoughts</h3><p>It would have been faster to stop at the first working version of this stack. What made it worth continuing was that each later constraint, splitting the network, replacing plaintext credentials, gating startup on an actual healthcheck, came from asking what could quietly go wrong with the simpler version, then building something that couldn’t fail that way. None of it was theoretical either: every claim in this write-up, the volumes persisting, the secrets being read from files, the healthcheck actually blocking startup, was something I watched happen, not something I assumed from documentation.</p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0c72836bd4ae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Containerized DevOps Tooling with Jenkins, Portainer, and SonarQube, and Proving…]]></title>
            <link>https://medium.com/@rester.mcglown/building-a-containerized-devops-tooling-with-jenkins-portainer-and-sonarqube-and-proving-eaa6137c67ff?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/eaa6137c67ff</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[cicd]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 19:57:21 GMT</pubDate>
            <atom:updated>2026-09-10T19:57:21.558Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Containerized DevOps Tooling with Jenkins, Portainer, and SonarQube, and Proving Container Storage Persistence</h3><p>A hands-on Docker project focused on running real DevOps tooling entirely in containers, backing every stateful service with named volumes, and proving, through deliberate deletion and recreation, that the data actually survives.</p><h3>Project Architecture and Objectives</h3><p>This project stands up three separate DevOps tools, Jenkins for CI, Portainer for container management, and SonarQube for static code analysis, entirely through the Docker CLI, with no Docker Compose involved. Each tool runs in its own container, and each stateful one is backed by its own named Docker volume, so its data lives independently of the container’s lifecycle rather than inside it.</p><figure><img alt="Diagram of a Docker architecture showing Jenkins, Portainer, and SonarQube containers, each connected to their own named volumes, with Portainer also connected to the Docker socket." src="https://cdn-images-1.medium.com/max/1024/1*xiSA95AMmv8slZjyI0qgBQ.png" /><figcaption>The three-tool architecture, showing Jenkins, Portainer, and SonarQube each backed by their own named volumes, with Portainer additionally granted access to the Docker socket to manage the host’s containers.</figcaption></figure><p>The objectives were to:</p><ul><li>Deploy Jenkins in a container with its home directory persisted in a named volume, and complete its setup wizard</li><li>Deploy Portainer with direct access to the Docker Engine, so it can manage containers through a browser instead of the CLI</li><li>Deploy SonarQube with three separate named volumes, one each for its data, logs, and extensions</li><li>Prove, rather than assume, that both Jenkins and SonarQube retain their configuration, jobs, and user-created data after a full container deletion and rebuild</li><li>Use Portainer’s own UI to inspect running containers as an alternative to docker ps and docker inspect</li></ul><h3>The Problem I Wanted to Solve</h3><p>The scenario behind this project was ForgeWorks Software wanting to trial Jenkins for continuous integration before committing to a dedicated server. That framing mattered to how I approached it: a trial only means something if you can tear it down and rebuild it without losing the work already done. Anyone can point at Docker’s documentation and say a named volume persists data. I wanted to actually watch it happen, by deleting a running container on purpose and confirming what came back.</p><h3>Why I Deliberately Broke Things to Prove Persistence</h3><p>It would have been faster to just stand up all three tools, take screenshots, and call it done. I chose instead to build in genuine destruction tests: create real data in SonarQube and Jenkins, delete both containers entirely, and rebuild them from the exact same commands to see if that data came back untouched. That decision is what turned this from a setup exercise into an actual demonstration of understanding, and it’s also where nearly every interesting problem in this project showed up.</p><h3>Tools and Technologies Used</h3><ul><li>Docker Desktop, Docker Engine, and CLI</li><li>Jenkins (LTS) for CI orchestration</li><li>Portainer Community Edition for container management</li><li>SonarQube (latest tag) for static code analysis</li><li>Docker named volumes for persistent storage</li><li>A bind mount of the Docker socket (/var/run/docker.sock)</li><li>Windows PowerShell</li><li>Git and GitHub</li></ul><h3>Phase 1 — Running Jenkins with a Persistent Named Volume</h3><p>The first step was creating a named volume before Jenkins ever ran, so its configuration directory would exist independently of any single container instance:</p><pre>docker volume create jenkins_home</pre><p>Jenkins was then launched detached, publishing both its web UI port (8080) and a second, easy-to-overlook port, 50000, which Jenkins uses for JNLP, the protocol build agents use to connect back to the controller. No agents connect to it in this project, but publishing it up front avoids having to recreate the container later just to add it:</p><pre>docker run -d -p 8080:8080 -p 50000:50000 --name jenkins -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts</pre><figure><img alt="Terminal output of docker ps showing a running Jenkins container with ports 8080 and 50000 published." src="https://cdn-images-1.medium.com/max/1024/1*BSxYD50X9A3bH7e7MY8Qyg.png" /><figcaption>Docker ps confirming the Jenkins container running with both the web UI port (8080) and JNLP agent port (50000) published.</figcaption></figure><p>Jenkins doesn’t email or display its initial admin password anywhere convenient. It writes that password to a file inside the container’s own filesystem, which meant reaching directly into the running container to read it:</p><pre>docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword</pre><figure><img alt="Jenkins web UI Unlock Jenkins screen prompting for the initial administrator password." src="https://cdn-images-1.medium.com/max/1024/1*KV_7mdheB-p37GbyTvFP2Q.png" /><figcaption>Jenkins’s Unlock screen, waiting for the initial admin password retrieved from inside the container.</figcaption></figure><p>From there, the setup wizard offered a choice between installing a curated set of suggested plugins or hand-picking them. I went with the suggested set, both because it matched the project’s own instructions and because it’s the realistic choice for a team like ForgeWorks that wants to see Jenkins working quickly rather than spend a day curating plugins.</p><figure><img alt="Jenkins setup wizard showing a progress screen for installing suggested plugins." src="https://cdn-images-1.medium.com/max/1024/1*RxKC3GHftF8GQvxEsql-gw.png" /><figcaption>The suggested plugin set installing during Jenkins’s setup wizard.</figcaption></figure><p>After installing plugins and creating an admin account, the setup wizard completed and the main Jenkins dashboard loaded, empty and ready for a first job.</p><figure><img alt="Jenkins dashboard homepage showing the sidebar and an empty jobs list after initial setup." src="https://cdn-images-1.medium.com/max/1024/1*4Ud4LKQfY_ibyPN0pFE41w.png" /><figcaption>The main Jenkins dashboard after completing setup and creating an admin account.</figcaption></figure><h3>Troubleshooting: Docker Desktop Wasn’t Running</h3><p>Before any of this, my very first docker volume create command failed outright:</p><pre>failed to connect to the docker API at npipe:////./pipe/dockerDesktopLinuxEngine; check if the path is correct and if the daemon is running: open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified.</pre><p>That error has nothing to do with Docker commands themselves, it’s the Docker CLI failing to reach the Docker Engine because Docker Desktop hadn’t finished starting. The fix was simply starting Docker Desktop and waiting for it to report itself running before retrying. A small thing, but a reminder that the very first thing worth checking when any docker command fails outright is whether the engine is actually up.</p><h3>Phase 2 — Portainer and the Docker Socket</h3><p>Portainer’s entire purpose is managing Docker itself through a browser, which means it needs a fundamentally different kind of access than Jenkins or SonarQube do. Alongside a named volume for its own configuration, Portainer’s container was launched with a bind mount of the Docker socket, the same interface every docker command on the host actually uses to talk to the Docker Engine:</p><pre>docker volume create portainer_data<br>docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest</pre><p>That distinction is worth sitting with rather than skimming past. Mounting the Docker socket into a container is not a scoped, limited permission, it is functionally equivalent to giving that container root access to the host machine. Any container with socket access can launch a brand-new container that bind-mounts the entire host filesystem into itself, then read or write anything on the machine from inside that new container. This is a well-known point in container security discussions, sometimes summarized simply as “the Docker socket is root.” Mounting it into Portainer means trusting Portainer’s image completely, the same way you’d trust anything you handed root privileges to.</p><figure><img alt="Terminal output of docker ps showing a running Portainer container with ports 8000 and 9443 published." src="https://cdn-images-1.medium.com/max/1024/1*SPEa35jEJ-bCrSI99eACmQ.png" /><figcaption>Docker ps showing the Portainer container running with its ports published, following a clean restart after the setup window timed out.</figcaption></figure><h3>Troubleshooting: Portainer’s Setup Window Timed Out, and the Fix Lived in the Volume, Not the Container</h3><p>Portainer only allows a short window after startup to claim its initial admin account, specifically to prevent a race condition where an unauthorized party claims the account first. On my first attempt, that window closed before I reached the setup screen, and the UI responded with:</p><pre>New Portainer installation<br>Your Portainer instance timed out for security purposes. To re-enable your Portainer instance, you will need to restart Portainer.</pre><p>Restarting the container with docker restart portainer did not fix it. Trying again, the setup screen returned a flat 403. That was the real lesson here: Portainer&#39;s lockout state gets written into its own persistent volume, not into the container. Since a named volume is specifically designed to survive a container restart, the same mechanism that would normally protect my data was now protecting a broken state. Restarting the container brought back the exact same locked volume every time. The actual fix was removing both the container and the volume entirely and starting clean:</p><pre>docker rm -f portainer<br>docker volume rm portainer_data<br>docker volume create portainer_data<br>docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest</pre><p>This time, I went straight to the browser the moment the container was up, claimed the admin account immediately, and it worked. Portainer confirmed it was already connected to and monitoring the local Docker environment, a direct result of that socket mount.</p><figure><img alt="Portainer web UI home screen showing one connected local Docker environment with container and resource counts." src="https://cdn-images-1.medium.com/max/1024/1*6Mg7NqhmSJK53qMbpzJcYg.png" /><figcaption>The Portainer home dashboard, confirming it is connected to and monitoring the local Docker environment through the mounted Docker socket.</figcaption></figure><h3>Phase 3 — SonarQube with Three Separate Named Volumes</h3><p>Jenkins keeps everything under one home directory, so one volume was enough for it. SonarQube’s image is built differently: it writes to three distinct internal paths for three distinct purposes, its analysis data, its logs, and its installed extensions. Rather than lumping all of that into a single volume, I created one for each:</p><pre>docker volume create sonarqube_data<br>docker volume create sonarqube_logs<br>docker volume create sonarqube_extensions</pre><pre>docker run -d -p 9000:9000 --name sonarqube -v sonarqube_data:/opt/sonarqube/data -v sonarqube_logs:/opt/sonarqube/logs -v sonarqube_extensions:/opt/sonarqube/extensions sonarqube:latest</pre><figure><img alt="Terminal output of docker ps showing a running SonarQube container with port 9000 published." src="https://cdn-images-1.medium.com/max/1024/1*mI2fyk98IVlTjSG66DTiaw.png" /><figcaption>Docker ps confirming the SonarQube container running and healthy on port 9000.</figcaption></figure><h3>Troubleshooting: An Image Tag That Didn’t Exist</h3><p>Before landing on latest, I first tried the tag sts, assuming it referred to SonarQube&#39;s Short-Term Support release track. Docker rejected it outright:</p><pre>Unable to find image &#39;sonarqube:sts&#39; locally<br>docker: Error response from daemon: failed to resolve reference &quot;docker.io/library/sonarqube:sts&quot;: docker.io/library/sonarqube:sts: not found</pre><p>Rather than guessing again, I went straight to the actual tags page on Docker Hub and confirmed which tags genuinely exist for this image. sts isn&#39;t one of them. latest is, and while it&#39;s convenient, it&#39;s also a moving target, rerunning the exact same command months from now could silently pull a different SonarQube version. A production deployment would pin an explicit version number instead, precisely to avoid that kind of drift.</p><p>After logging in with SonarQube’s default admin/admin credentials and completing the forced password change (a good practice on its own, since default credentials left in place are a real, common security gap), I made two deliberate changes to set up a genuine persistence test: I set a custom Server Base URL under General Settings, and created a new user named forgeworks-dev.</p><figure><img alt="SonarQube web UI dashboard after initial login." src="https://cdn-images-1.medium.com/max/1024/1*6d5lfjwPn_xsarRUyfkbgQ.png" /><figcaption>The SonarQube dashboard after logging in and completing the forced password change.</figcaption></figure><figure><img alt="SonarQube Administration Users page showing a newly created user named forgeworks-dev." src="https://cdn-images-1.medium.com/max/1024/1*hWzsTnZjPFZ4lYQ5yLHGyA.png" /><figcaption>The forgeworks-dev user created in SonarQube as a persistence test baseline.</figcaption></figure><figure><img alt="SonarQube General Settings page showing a custom Server Base URL value being set before container recreation." src="https://cdn-images-1.medium.com/max/1024/1*q8NFoM6QCuGvhTG7l1iEHg.png" /><figcaption>The custom Server Base URL set under General Settings, establishing the second baseline for the persistence test.</figcaption></figure><h3>Phase 4 — Proving SonarQube’s Persistence</h3><p>This is the part of the project that actually mattered. I stopped and removed the SonarQube container entirely, then recreated it using the exact same command and the same three volumes:</p><pre>docker stop sonarqube<br>docker rm sonarqube<br>docker run -d -p 9000:9000 --name sonarqube -v sonarqube_data:/opt/sonarqube/data -v sonarqube_logs:/opt/sonarqube/logs -v sonarqube_extensions:/opt/sonarqube/extensions sonarqube:latest</pre><figure><img alt="Terminal output of docker ps showing a new SonarQube container ID after deletion and recreation." src="https://cdn-images-1.medium.com/max/1024/1*_6NwJU-NcoEuRUA1tz0kZQ.png" /><figcaption>Docker ps showing a newly recreated SonarQube container with a different container ID, confirming it is a genuinely new instance.</figcaption></figure><p>Logging back in, both the custom Server Base URL and the forgeworks-dev user were still there, exactly as before. From the volumes&#39; perspective, nothing had actually changed, the container that got deleted was never where that data lived in the first place.</p><figure><img alt="SonarQube Users page showing the forgeworks-dev user still present after container recreation." src="https://cdn-images-1.medium.com/max/1024/1*ATYAJl3pNrUI98Edr3xq7w.png" /><figcaption>The forgeworks-dev user still present in SonarQube’s Users list after the container was fully deleted and rebuilt.</figcaption></figure><figure><img alt="SonarQube General Settings page showing a custom Server Base URL value preserved after container recreation." src="https://cdn-images-1.medium.com/max/1024/1*GyU3abBsmVckjx0LJrCksA.png" /><figcaption>The custom Server Base URL still set under General Settings after the container rebuild, confirming configuration persistence.</figcaption></figure><h3>Phase 5 — Proving Jenkins’s Persistence</h3><p>I ran the identical test against Jenkins, with one addition: instead of just a settings change, I wanted to prove that actual build history survives too, not just static configuration. I created a simple pipeline job, forgeworks-ci-pipeline, and ran it once to generate a build record.</p><figure><img alt="Jenkins pipeline job page showing a successful build for forgeworks-ci-pipeline." src="https://cdn-images-1.medium.com/max/1024/1*UuIu9PXuKj-j6ZEOzFc1PQ.png" /><figcaption>The forgeworks-ci-pipeline job created and run successfully in Jenkins, generating build history.</figcaption></figure><p>Then I stopped, removed, and recreated the Jenkins container with the identical original command:</p><pre>docker stop jenkins<br>docker rm jenkins<br>docker run -d -p 8080:8080 -p 50000:50000 --name jenkins -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts</pre><p>On returning to localhost:8080, the unlock screen and plugin installation wizard did not reappear, only a standard login prompt, exactly what should happen since that one-time setup state lives in the persisted volume. Logging in with my original admin credentials dropped me straight into the dashboard with forgeworks-ci-pipeline and its build history sitting exactly where I left it.</p><figure><img alt="Jenkins dashboard showing the forgeworks-ci-pipeline job and build history still present after container recreation." src="https://cdn-images-1.medium.com/max/1024/1*6-R-lxyPOXjqFrhTfvOSuw.png" /><figcaption>The Jenkins dashboard after the container was deleted and recreated, showing forgeworks-ci-pipeline and its build history still intact.</figcaption></figure><h3>Phase 6 — Inspecting Everything Through Portainer Instead of the CLI</h3><p>For the final phase, I switched from docker ps and docker inspect to Portainer&#39;s own web UI to inspect what was actually running. The container list showed Jenkins, SonarQube, and Portainer itself, all healthy, alongside their published ports and creation times, plus a handful of older, unrelated exited containers from previous projects that I hadn&#39;t noticed cluttering the environment until Portainer made them visible at a glance.</p><figure><img alt="Portainer container list view showing three running containers with their images, ports, and status." src="https://cdn-images-1.medium.com/max/1024/1*25VANwwuSGo2XR4Gk8i2DA.png" /><figcaption>The Portainer container list showing Jenkins, SonarQube, and Portainer itself all running, alongside published ports and creation times.</figcaption></figure><p>Clicking into the Jenkins container’s detail view surfaced its environment variables directly, including JENKINS_HOME pointing exactly where expected, along with its entrypoint and full port configuration, all without touching a terminal.</p><figure><img alt="Portainer container detail page for the Jenkins container showing environment variables and port configuration." src="https://cdn-images-1.medium.com/max/1024/1*MI-DhAT1NfqgLLAEUKFDyg.png" /><figcaption>The Jenkins container’s detail view in Portainer, showing its environment variables including JENKINS_HOME, entrypoint, and port configuration.</figcaption></figure><h3>Final Validation</h3><p>Once both persistence tests were complete, I tore the whole environment down deliberately: stopped and removed all three containers, then removed the five named volumes tied to them, and confirmed with docker ps -a and docker volume ls that nothing project-related remained.</p><pre>docker stop jenkins sonarqube portainer<br>docker rm jenkins sonarqube portainer<br>docker volume rm jenkins_home portainer_data sonarqube_data sonarqube_logs sonarqube_extensions</pre><figure><img alt="Terminal output of docker ps -a and docker volume ls showing the project’s containers and volumes no longer present." src="https://cdn-images-1.medium.com/max/1024/1*akmuE5rwrDPlW1qv1CUVDw.png" /><figcaption>Final cleanup confirming all three project containers and their five named volumes were fully removed.</figcaption></figure><h3>Engineering Decisions</h3><ul><li><strong>Named volumes over bind mounts for application state.</strong> Docker manages a named volume’s lifecycle independently of any host directory structure, which better reflects how these tools are actually run in production environments.</li><li><strong>Splitting SonarQube’s storage into three volumes instead of one.</strong> Data, logs, and extensions were separated so each could, in principle, be backed up, rotated, or migrated independently of the others.</li><li><strong>Publishing Jenkins’s JNLP port up front.</strong> Port 50000 wasn’t used directly in this project, but publishing it from the start avoids having to recreate the container later if a build agent needs to connect.</li><li><strong>Verifying the SonarQube tag against the actual Docker Hub page after a failed guess</strong>, rather than guessing a second time, and choosing latest with the explicit understanding that a production deployment would pin a specific version instead.</li></ul><h3>Security and Operational Considerations</h3><ul><li>Mounting the Docker socket into Portainer is functionally equivalent to granting it root access to the host, not a scoped container-management permission, and should only ever be done with a trusted image.</li><li>Both SonarQube’s default admin/admin login and Jenkins&#39;s randomly generated initial password were replaced with unique credentials immediately, rather than left in place.</li><li>Portainer’s short admin-claim window is a deliberate security control against account-hijacking races, not a bug, letting it expire is a safe failure mode.</li><li>In a real production environment, Portainer’s socket access would sit behind a socket proxy that exposes only a narrow subset of the Docker API, rather than the full socket itself.</li></ul><h3>What I Would Improve for Production</h3><ul><li>Move all three services into Docker Compose (or a full orchestrator) for declarative, reproducible management instead of individual docker run commands.</li><li>Put a reverse proxy with TLS in front of each web UI instead of exposing raw HTTP/HTTPS ports directly.</li><li>Replace Portainer’s direct Docker socket mount with a socket proxy that limits its actual API access.</li><li>Pin explicit version numbers for every image instead of relying on latest or an assumed tag.</li><li>Wire real Jenkins build agents into the already-published JNLP port for distributed builds.</li></ul><h3>Lessons Learned</h3><ul><li>A named volume decouples data from a container’s lifecycle by storing it outside the container’s writable layer entirely, deleting a container never touches the volume unless the volume is explicitly removed too.</li><li>That same persistence can work against you as easily as for you. Portainer’s own lockout state was written into its volume, meaning a container restart alone couldn’t undo it, only removing the volume itself gave a clean slate.</li><li>Not every documented-sounding tag or flag actually exists. Assuming sts was a valid SonarQube tag without checking the real tags page produced an avoidable error; checking the source directly resolved it in seconds.</li><li>Multi-service storage isn’t one-size-fits-all. Jenkins needed a single volume for everything, while SonarQube’s image writes to three distinct internal paths, requiring three separate volumes to fully persist its state.</li><li>The very first thing worth checking when a docker command fails outright is whether Docker Desktop itself is actually running.</li></ul><h3>Project Repository</h3><p>The full project, including the Dockerfile, application code, and this write-up’s source commands, is available on GitHub:</p><p><a href="https://github.com/rester12/Docker-Project-Containerizing-DevOps-Tooling">GitHub - rester12/Docker-Project-Containerizing-DevOps-Tooling</a></p><h3>Closing Thoughts</h3><p>Volume persistence is one of those concepts that’s easy to nod along to and easy to get wrong in practice. This project forced the point past theory into something I actually watched happen: containers got deleted, brand-new ones took their place, and jobs, users, plugins, and settings all came back untouched, because none of that state ever actually lived inside the container to begin with.</p><p>The most valuable outcome here wasn’t standing up three separate tools, it was building the habit of proving a system’s behavior with a real test instead of trusting its documentation.</p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eaa6137c67ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Multi-Container Docker Application with User-Defined Networks, Service Discovery…]]></title>
            <link>https://medium.com/@rester.mcglown/building-a-multi-container-docker-application-with-user-defined-networks-service-discovery-5c372c9104d9?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c372c9104d9</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Wed, 09 Sep 2026 16:27:01 GMT</pubDate>
            <atom:updated>2026-09-09T16:27:01.773Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Multi-Container Docker Application with User-Defined Networks, Service Discovery, Resource Limits, and a Custom Flask App</h3><p>A hands-on Docker project focused on container networking, service discovery, resource governance, troubleshooting, and building a custom application, all without Docker Compose</p><p>In this project, I built and validated a small multi-service application, redis, MySQL, two web-based admin GUIs, and eventually a custom Flask app, using nothing but raw docker commands. No Docker Compose, no YAML, no depends_on. The goal wasn&#39;t to avoid Compose forever; it was to genuinely understand what Compose actually does for you before I ever let it do it automatically.</p><p>Rather than stopping once four containers were technically “running,” I progressively worked through the exact failure modes that Compose is designed to paper over: containers that can’t find each other by name, environment variables that only take effect at container creation, memory and CPU limits that need to be proven under real load rather than trusted from a config value, and a custom application that has to handle a dependency going down without crashing outright. The focus throughout was not just making things work, but proving, with real commands and real output, why they worked.</p><h3>Project Architecture and Objectives</h3><figure><img alt="Diagram of a Docker architecture with a Browser connecting to three published services, which in turn connect to Redis or MySQL inside a shared user-defined network called app-net." src="https://cdn-images-1.medium.com/max/1024/1*t-6DJPdUzNJrxq6Dpc5r_A.png" /><figcaption>The five-container architecture, showing the Browser reaching flask-counter, redis-commander, and phpMyAdmin through published ports, while each service resolves its backing store by container name over app-net.</figcaption></figure><p>The final architecture consists of five containers sharing a single user-defined bridge network (app-net):</p><ul><li><strong>redis</strong> — an in-memory cache and, later, the backing store for a custom application’s state</li><li><strong>mysql</strong> — a relational database</li><li><strong>redis-commander</strong> — a web GUI for browsing Redis</li><li><strong>phpmyadmin</strong> — a web GUI for administering MySQL</li><li><strong>flask-counter</strong> — a custom Python/Flask application I wrote and containerized myself, using Redis to persist a hit counter</li></ul><p>The project objectives were to:</p><ul><li>Run multiple containers independently on Docker’s default bridge network and observe its real limitations firsthand, rather than taking them on faith</li><li>Create a user-defined bridge network and prove container-to-container DNS resolution actually works</li><li>Apply CPU and memory limits to running containers and validate them under genuine load, not just by reading the configuration back</li><li>Author a Dockerfile for a real application and connect it to an existing service over a shared network</li><li>Prove that application state can survive a container restart when it’s stored externally</li><li>Deliberately break a dependency and prove the application recovers correctly</li></ul><h3>The Problem I Wanted to Solve</h3><p>The scenario I worked from: a small team is prototyping an app that needs a web tier, a cache, and a database. Before anyone commits to Docker Compose, they want proof that the underlying pieces, container networking, service discovery, resource governance, are actually understood, not just copied from a Compose file that happens to work.</p><p>GUI tools (redis-commander for Redis, phpMyAdmin for MySQL) stood in for a hand-written application in the earlier phases, which kept the focus entirely on Docker networking rather than application code. The final phase replaced that with an actual custom application, so the project ended with something closer to a real deployment than a pure networking exercise.</p><h3>Why I Used Raw Docker Commands Instead of Compose</h3><p>It would have taken about five minutes to write a docker-compose.yml file that did all of this correctly on the first try. That was exactly the problem. Compose&#39;s depends_on, its default network with built-in DNS, and its environment variable handling all exist to solve specific, real problems, and if I&#39;d started with Compose, I would never have actually seen those problems happen.</p><p>Building this by hand meant I hit the default bridge network’s missing DNS resolution directly, as a real error message, not as a line in documentation. I hit the “environment variables are read once at startup” limitation by trying to fix it the wrong way first and watching it fail. That’s the kind of understanding that holds up in an interview question, versus knowing which line to paste into a YAML file.</p><h3>Tools and Technologies Used</h3><ul><li>Docker Desktop (Windows, PowerShell)</li><li>Docker Engine and Docker CLI</li><li>Redis</li><li>MySQL 8.0</li><li>redis-commander</li><li>phpMyAdmin</li><li>Python 3.11 and Flask</li><li>Docker user-defined bridge networks</li><li>Docker resource limits (--memory, --memory-swap, --cpus)</li><li>Git and GitHub</li></ul><h3>Phase 1 — Running Four Containers on the Default Bridge Network</h3><p>I started by running four containers independently, with no custom networking at all, which put all of them on Docker’s default bridge network:</p><pre>docker run -d --name redis redis:latest<br>docker run -d --name mysql -e MYSQL_ROOT_PASSWORD=rootpass mysql:8.0<br>docker run -d --name redis-commander -p 8081:8081 rediscommander/redis-commander:latest<br>docker run -d --name phpmyadmin -p 8080:80 phpmyadmin/phpmyadmin</pre><p>I confirmed all four were running concurrently and checked their live resource usage:</p><pre>docker ps<br>docker stats --no-stream</pre><figure><img alt="Terminal output of docker ps listing four running containers: redis, mysql, redis-commander, and phpmyadmin." src="https://cdn-images-1.medium.com/max/1024/1*P67vThhdzEEn6j1bX32Y7g.png" /><figcaption>PowerShell showing docker ps with redis, mysql, redis-commander, and phpmyadmin all in an “Up” state on Docker’s default bridge network.</figcaption></figure><figure><img alt="Terminal output of docker stats showing CPU, memory, and network usage for four containers." src="https://cdn-images-1.medium.com/max/915/1*k_de6QzjrRaxkyjvJa6JNA.png" /><figcaption>Live resource usage for all four containers captured with docker stats — no-stream.</figcaption></figure><h3>Phase 2 — Diagnosing Why the Default Bridge Network Wasn’t Enough</h3><p>With all four containers technically running, I loaded both GUIs in a browser to see if they could actually reach their databases. Neither could.</p><p><strong>redis-commander</strong> defaulted to 127.0.0.1:6379 and failed immediately:</p><pre>setUpConnection Redis error Error: connect ECONNREFUSED 127.0.0.1:6379</pre><figure><img alt="Terminal log output showing a Redis ECONNREFUSED error at address 127.0.0.1 port 6379." src="https://cdn-images-1.medium.com/max/860/1*lcFJvly9E8U-ais6ZxNMXA.png" /><figcaption>Docker logs for redis-commander showing a connection refused error when it tried to reach Redis at the loopback address 127.0.0.1.</figcaption></figure><figure><img alt="Redis Commander web UI displaying a disconnected Redis instance with no data." src="https://cdn-images-1.medium.com/max/529/1*S5eo1itqxVb5rE3f1s8WLQ.png" /><figcaption>The redis-commander web interface showing a failed connection state before container networking was fixed.</figcaption></figure><p>127.0.0.1 is a loopback address scoped to each container&#39;s own network namespace. It doesn&#39;t matter that the real Redis container was running fine on the same Docker host; a container&#39;s loopback address can never reach a sibling container. This isn&#39;t a bug or a misconfiguration; it&#39;s containers doing exactly what network namespace isolation is designed to do.</p><p><strong>phpMyAdmin</strong> failed in a genuinely different way. It defaulted to a hostname of db and failed with:</p><pre>mysqli::real_connect(): php_network_getaddresses: getaddrinfo for db failed: Name or service not known</pre><figure><img alt="phpMyAdmin login page displaying a “Cannot log in to the MySQL server” error message." src="https://cdn-images-1.medium.com/max/1024/1*iWbDj0GGxChaIgjlvlcvcg.png" /><figcaption>phpMyAdmin’s login screen showing a database connection error caused by an unresolvable hostname.</figcaption></figure><figure><img alt="Terminal log output showing a getaddrinfo failure for hostname “db” with the message “Name or service not known.”" src="https://cdn-images-1.medium.com/max/1024/1*RIgSSjXX8FDm0M4erwN99w.png" /><figcaption>Docker logs for phpmyadmin confirming the same DNS resolution failure from the server side.</figcaption></figure><p>db is a naming convention phpMyAdmin&#39;s default configuration assumes, aimed at a typical Docker Compose stack where the database service is commonly named db. Since I&#39;d named my container mysql, that convention didn&#39;t help me. More importantly, the failure mode itself is different from redis-commander&#39;s: this is a DNS lookup failing outright, &quot;name not known&quot;, rather than a connection being refused at a real address. That distinction turned out to be the most important concept in the whole first phase: <strong>Docker&#39;s default bridge network has no embedded DNS server at all.</strong> There&#39;s no name resolution step to even attempt, which is why a hostname like db or a container&#39;s real name fails immediately, while a loopback address like 127.0.0.1 at least gets far enough to be refused.</p><h3>Phase 3 — Troubleshooting a Browser Caching Illusion</h3><p>Before I even got to the real networking error, I hit a smaller, sillier one. The first time I loaded redis-commander in my browser, it showed a generic “It works!” placeholder page instead of the actual application UI, which briefly looked like a much stranger networking problem than it actually was. Loading the same URL in an incognito window immediately showed the correct redis-commander interface, confirming it was a browser caching artifact from something unrelated that had previously used that port, not a Docker or networking issue at all. It was a good early reminder to isolate variables before assuming the more interesting failure is the real one.</p><h3>Phase 4 — Creating a User-Defined Network for Service Discovery</h3><p>To fix the actual networking problem, I created a user-defined bridge network and attached all four containers to it:</p><pre>docker network create app-net</pre><pre>docker network connect app-net redis<br>docker network connect app-net mysql</pre><pre>docker stop redis-commander phpmyadmin<br>docker rm redis-commander phpmyadmin</pre><pre>docker run -d --name redis-commander --network app-net -p 8081:8081 -e REDIS_HOST=redis rediscommander/redis-commander:latest<br>docker run -d --name phpmyadmin --network app-net -p 8080:80 -e PMA_HOST=mysql phpmyadmin/phpmyadmin</pre><figure><img alt="Terminal output of docker ps showing four containers running on a Docker network named app-net." src="https://cdn-images-1.medium.com/max/1024/1*rWwyQb3nfBdjW4MAF9RuCA.png" /><figcaption>All four containers running again, now attached to the custom app-net bridge network.</figcaption></figure><figure><img alt="JSON output of docker network inspect listing four containers with IP addresses on the 172.19.0.0/16 subnet." src="https://cdn-images-1.medium.com/max/731/1*0MOAMWlRTSqPx7TdDw7oRA.png" /><figcaption>Docker network inspect confirming all four containers are attached to app-net, each with a real IP address on the shared subnet.</figcaption></figure><p>I deliberately handled the four containers two different ways here, and the difference mattered. redis and mysql needed no new configuration, so I attached them to the new network live, with docker network connect, without touching them otherwise. redis-commander and phpmyadmin, on the other hand, needed new environment variables (REDIS_HOST, PMA_HOST), and Docker doesn&#39;t support injecting new environment variables into an already-running container; they&#39;re read once, at startup, and baked into the process. So those two had to be stopped and recreated from scratch, this time with both the network and the correct host variables set together.</p><p><strong>Docker’s default bridge network</strong> provides basic container-to-container IP connectivity, but no automatic service discovery. Containers on it simply cannot resolve each other by name. A <strong>user-defined bridge network</strong>, by contrast, runs an embedded DNS server, so any container attached to it can resolve another attached container purely by its container name. That’s the entire mechanism that lets REDIS_HOST=redis and PMA_HOST=mysql work as real, functioning hostnames instead of placeholder values.</p><h3>Phase 5 — Verifying Container-to-Container DNS Resolution</h3><p>With both GUIs recreated on app-net, I reloaded them in the browser to confirm the fix actually worked, rather than just assuming it did because the containers started without errors.</p><figure><img alt="Redis Commander web UI showing a successful connection labeled redis:6379 with a working command prompt." src="https://cdn-images-1.medium.com/max/391/1*PF6BCuffGj6M5WQ9XvxChA.png" /><figcaption>Redis-commander successfully connected to Redis by container name, shown as “local (redis:6379:0)” with an active CLI prompt.</figcaption></figure><figure><img alt="phpMyAdmin dashboard showing a successful login and server connection details for host mysql." src="https://cdn-images-1.medium.com/max/1024/1*EpTv1NGUdCLzSS5UVXakNg.png" /><figcaption>phpMyAdmin successfully logged in, reporting a connection to the database server “mysql” over TCP/IP.</figcaption></figure><p>redis-commander’s connection label changed from localhost:6379:0 to redis:6379:0, and phpMyAdmin logged in successfully, explicitly reporting it connected to a server named mysql over TCP/IP. Both confirmed that container-name DNS resolution, available only through the user-defined network, was genuinely working, not just theoretically configured.</p><h3>Phase 6 — Applying and Validating Resource Limits Under Load</h3><p>Next, I applied CPU and memory limits to the running redis and mysql containers:</p><pre>docker update --memory=&quot;256m&quot; --memory-swap=&quot;256m&quot; --cpus=&quot;0.5&quot; redis<br>docker update --memory=&quot;512m&quot; --memory-swap=&quot;512m&quot; --cpus=&quot;0.5&quot; mysql</pre><figure><img alt="Terminal output showing docker update memory and CPU limit commands and their resulting inspected values." src="https://cdn-images-1.medium.com/max/991/1*Oi-UUFmc3yRHbU7wmBeeKQ.png" /><figcaption>Terminal output showing the corrected docker update commands succeeding, followed by docker inspect confirming the real memory and CPU limit values.</figcaption></figure><p>Reading a config value back isn’t proof it actually does anything, so I generated genuine CPU-bound load against both containers and watched docker stats live while it ran:</p><pre>docker exec -it redis redis-benchmark -q -n 200000 -c 50<br>docker exec -it mysql mysql -u root -prootpass -e &quot;SELECT BENCHMARK(500000000, MD5(&#39;stress-test&#39;));&quot;</pre><figure><img alt="Terminal output of docker stats showing Redis CPU usage capped near 50 percent under load." src="https://cdn-images-1.medium.com/max/1015/1*Dx6TjqhS7zKSDtKjsW23qA.png" /><figcaption>Docker stats showing Redis’s CPU usage plateaued around 50 percent during a redis-benchmark load test, matching its configured limit.</figcaption></figure><figure><img alt="Terminal output of docker stats showing MySQL CPU usage capped near 50 percent under load." src="https://cdn-images-1.medium.com/max/961/1*mHlpLK9_TMpBEOONOOWSWg.png" /><figcaption>Docker stats showing MySQL’s CPU usage similarly capped near 50 percent while running a CPU-intensive BENCHMARK() query.</figcaption></figure><p>Both containers plateaued right at the configured --cpus=&quot;0.5&quot; ceiling, even though both load-generation tools were actively trying to push past it. That&#39;s the difference between a resource limit that&#39;s advisory and one that&#39;s actually enforced by the Linux kernel&#39;s cgroups underneath Docker.</p><h3>Phase 7 — Troubleshooting the Memory-Swap Conflict and a Missing Benchmark Tool</h3><p>The first attempt at applying the memory limit failed outright:</p><pre>Error response from daemon: Cannot update container ...: Memory limit should be smaller than already set memoryswap limit, update the memoryswap at the same time</pre><p>Docker’s --memory-swap setting (total memory plus swap combined) is coupled to --memory. Since no swap limit had ever been explicitly set, it defaulted to unlimited, which conflicted with the new, smaller memory ceiling I was trying to introduce. The fix was to set both flags together in the same command, with --memory-swap set equal to --memory, which effectively disables any additional swap beyond the RAM cap entirely.</p><p>The second snag came when I tried to generate load against MySQL using mysqlslap, the standard benchmarking client:</p><pre>OCI runtime exec failed: exec: &quot;mysqlslap&quot;: executable file not found in $PATH</pre><p>The official mysql:8.0 image is minimal and doesn&#39;t ship it, and it doesn&#39;t even include which to check what else was missing. Rather than installing extra tooling into a running container just for a benchmark, I used MySQL&#39;s own built-in BENCHMARK() SQL function instead, which generates real CPU-bound load entirely through the standard mysql client, already confirmed present.</p><h3>Phase 8 — Building a Custom Flask and Redis Application</h3><p>For the final phase, I replaced the GUI-based approach with an actual application: a small Flask app that increments a hit counter in Redis on every page load.</p><p><strong>app.py</strong></p><pre>import time</pre><pre>import redis<br>from flask import Flask</pre><pre>app = Flask(__name__)<br>cache = redis.Redis(host=&#39;redis&#39;, port=6379)</pre><pre>def get_hit_count():<br>    retries = 5<br>    while True:<br>        try:<br>            return cache.incr(&#39;hits&#39;)<br>        except redis.exceptions.ConnectionError as exc:<br>            if retries == 0:<br>                raise exc<br>            retries -= 1<br>            time.sleep(0.5)</pre><pre>@app.route(&#39;/&#39;)<br>def hello():<br>    count = get_hit_count()<br>    return &#39;Hello World! I have been seen {} times.\n&#39;.format(count)</pre><figure><img alt="Terminal or editor view of app.py showing Flask and Redis Python code." src="https://cdn-images-1.medium.com/max/550/1*Xdeoez9MWiy3UrxCNZ-v8w.png" /><figcaption>The complete source code for app.py, the Flask application that increments a hit counter in Redis on every page load.</figcaption></figure><p><strong>Dockerfile</strong></p><pre># syntax=docker/dockerfile:1<br>FROM python:3.11-slim<br>WORKDIR /code<br>ENV FLASK_APP=app.py<br>ENV FLASK_RUN_HOST=0.0.0.0<br>COPY requirements.txt requirements.txt<br>RUN pip install --no-cache-dir -r requirements.txt<br>COPY . .<br>EXPOSE 5000<br>CMD [&quot;flask&quot;, &quot;run&quot;]</pre><figure><img alt="Terminal or editor view of a Dockerfile defining a Python 3.11 Flask application image." src="https://cdn-images-1.medium.com/max/485/1*JztwHgSPt4x-hzQ59v0CpA.png" /><figcaption>The Dockerfile used to build the Flask application’s container image.</figcaption></figure><p>The application deliberately stores its state, the hit count, in Redis, an external service, rather than in a Python variable inside the Flask process itself. That’s not incidental; it’s the entire point. An application container that holds its own state in memory is fragile: restart it, replace it, or scale it to two copies, and that state is gone or inconsistent. Pushing it into Redis keeps the Flask container stateless and disposable, which is the pattern real cloud-native applications are built on. The retry loop in get_hit_count() also matters: it gives the application five chances, half a second apart, to reach Redis before giving up, which matters because there&#39;s no guarantee Redis is fully ready the instant this container starts.</p><p>The Dockerfile copies requirements.txt and installs dependencies before copying the rest of the application code. That ordering is deliberate: Docker caches each instruction as a layer, so as long as requirements.txt doesn&#39;t change, a later rebuild that only touches app.py can skip re-running pip install entirely and reuse the cached layer.</p><p>I built and ran it on the same network as everything else:</p><pre>docker build -t flask-counter ./app<br>docker run -d --name flask-counter --network app-net -p 5000:5000 flask-counter</pre><figure><img alt="Terminal output of a successful docker build process for the flask-counter image." src="https://cdn-images-1.medium.com/max/976/1*YxJhZUxYrI4PMG9fIOiaFQ.png" /><figcaption>The full docker build output showing every build step completing successfully.</figcaption></figure><figure><img alt="Terminal output of docker images listing flask-counter and other project images with their sizes." src="https://cdn-images-1.medium.com/max/1024/1*eSwotp6hROD9TmpEDu28xw.png" /><figcaption>Docker images confirming the new flask-counter image alongside the project’s other images.</figcaption></figure><figure><img alt="Terminal output of docker ps showing five running containers including flask-counter." src="https://cdn-images-1.medium.com/max/1024/1*lQMJEL7HDe1LrMYbO17chw.png" /><figcaption>All five containers, including the newly built Flask application, running together on app-net.</figcaption></figure><figure><img alt="Web browser showing the Flask app’s homepage with a hit counter reading one." src="https://cdn-images-1.medium.com/max/330/1*fg1QndvOqOnEBDImhUGuqg.png" /><figcaption>The Flask application running in the browser, displaying “Hello World! I have been seen 1 times.”</figcaption></figure><figure><img alt="Web browser showing the Flask app’s hit counter after multiple refreshes with a higher count." src="https://cdn-images-1.medium.com/max/332/1*w5kOueigdqo_V5AdVVn4UQ.png" /><figcaption>The hit counter climbing after several page refreshes, confirming Redis is correctly persisting each increment.</figcaption></figure><h3>Phase 9 — Troubleshooting a Silent File-Extension Issue</h3><p>While saving the Dockerfile in Notepad, I hit a classic Windows gotcha: without explicitly choosing “All Files” in the save dialog, Notepad silently appended a .txt extension, saving Dockerfile.txt instead of Dockerfile. I caught it by trying to type the file back out and getting a &quot;path not found&quot; error rather than the file&#39;s actual contents, which was the first clue something had saved incorrectly. The fix was simply renaming the file, but it&#39;s a good reminder to verify a file actually saved the way you intended, rather than trusting that a save dialog closing without an error means everything&#39;s fine.</p><h3>Phase 10 — Verifying State Persistence and Dependency-Failure Recovery</h3><p>With the application confirmed working, I ran two deliberate tests to prove the design decisions actually held up, not just in theory.</p><p><strong>State persistence</strong>: I restarted only the Flask container.</p><pre>docker restart flask-counter</pre><figure><img alt="Web browser showing the Flask app’s hit counter continuing to climb after a container restart." src="https://cdn-images-1.medium.com/max/345/1*XaF3LyMmZ53ec-3ggPUQqw.png" /><figcaption>After restarting only the Flask container, the counter continued from its previous value instead of resetting to zero.</figcaption></figure><p>The counter picked up exactly where it left off. If the count had been stored as a plain variable inside the Flask process, this restart would have reset it to zero. Because it lived in Redis instead, the Flask container’s own lifecycle had no effect on it.</p><p><strong>Dependency failure handling</strong>: I stopped Redis while the application was still running.</p><pre>docker stop redis</pre><figure><img alt="Web browser showing a Flask server error page after the Redis dependency was stopped." src="https://cdn-images-1.medium.com/max/1000/1*Wm8X0oNqcU6_YSbqC4I7mA.png" /><figcaption>The Flask application returning a server error after Redis was deliberately stopped.</figcaption></figure><figure><img alt="Terminal log output showing Flask retry attempts and a Redis ConnectionError." src="https://cdn-images-1.medium.com/max/879/1*4JtjyOLr64DNf-WP9Gqa1A.png" /><figcaption>Docker logs for flask-counter showing the application’s retry attempts and eventual connection error while Redis was down.</figcaption></figure><p>Refreshing the page triggered the retry loop in app.py, five attempts, half a second apart, followed by a clean failure once they were exhausted. I then restarted Redis and refreshed again.</p><pre>docker start redis</pre><figure><img alt="Web browser showing the Flask app’s hit counter continuing correctly after Redis was restarted." src="https://cdn-images-1.medium.com/max/343/1*8mi18uSxmTla2bklsQJ1PQ.png" /><figcaption>After restarting Redis, the Flask application recovered automatically and the counter continued from where it left off.</figcaption></figure><p>The application recovered automatically, and the counter continued climbing from its last value, since Redis’s own data was never lost, only briefly unreachable. This is the exact failure mode, no guaranteed startup or recovery order between dependent containers, that Docker Compose’s depends_on and healthchecks exist to solve. Having actually watched it fail and recover by hand gives me a much better answer to &quot;why do you need depends_on&quot; than knowing the syntax alone ever would.</p><h3>Final Validation</h3><p>I finished by tearing everything down cleanly, confirming both the containers and the custom network were fully removed:</p><pre>docker stop redis mysql redis-commander phpmyadmin flask-counter<br>docker rm redis mysql redis-commander phpmyadmin flask-counter<br>docker network rm app-net<br>docker ps -a<br>docker network ls</pre><figure><img alt="Terminal output of docker ps -a and docker network ls showing no remaining project containers or networks." src="https://cdn-images-1.medium.com/max/1024/1*8XRHCUdOFrEyqcpMRB5iag.png" /><figcaption>Final cleanup confirming all five project containers and the app-net network were fully removed.</figcaption></figure><h3>Engineering Decisions</h3><p><strong>User-defined network over default bridge.</strong> I created app-net specifically to gain embedded DNS-based service discovery, which the default bridge network simply doesn&#39;t provide.</p><p><strong>docker network connect for containers that didn&#39;t need to change.</strong> redis and mysql needed no new configuration, so I attached them to the new network live rather than recreating them, to demonstrate that a running container can gain additional network access without any downtime.</p><p><strong>Recreation for containers that needed new environment variables.</strong> redis-commander and phpmyadmin were stopped and recreated rather than modified in place, since Docker doesn&#39;t support injecting new environment variables into an already-running container.</p><p><strong>Paired memory and swap limits.</strong> I always set --memory and --memory-swap together, with swap effectively disabled beyond the memory ceiling, to keep each container&#39;s total memory behavior simple and predictable rather than relying on Docker&#39;s implicit swap defaults.</p><p><strong>External state for the Flask application.</strong> I deliberately built the Flask app to hold no state of its own, pushing the hit counter into Redis instead. That keeps the application container stateless and disposable, which is the pattern that actually scales.</p><h3>Security and Operational Considerations</h3><ul><li>Use official container images from trusted publishers.</li><li>Pin specific image versions or digests for anything beyond local learning, rather than relying indefinitely on latest.</li><li>Never commit real secrets, passwords, or tokens to a repository; the MYSQL_ROOT_PASSWORD used here is a local development-only placeholder.</li><li>Apply explicit CPU and memory limits to any container that could otherwise consume unbounded host resources.</li><li>Confirm a service is actually reachable and functioning, not just that its container shows a “running” status.</li><li>Treat application containers as stateless and disposable; persist real data only in dedicated data services.</li><li>Design application code to retry and fail gracefully when a dependency is temporarily unavailable, rather than crashing outright.</li></ul><h3>What I Would Improve for Production</h3><ul><li>Introduce Docker Compose to declaratively manage all five services and their startup order, now that I understand exactly what problems it’s actually solving.</li><li>Add depends_on and healthchecks so the dependency-failure scenario I tested manually here is handled automatically.</li><li>Replace the plaintext MYSQL_ROOT_PASSWORD with Docker secrets or an environment file excluded from version control.</li><li>Add a production-grade WSGI server (Gunicorn) in place of Flask’s development server.</li><li>Add persistent named volumes for MySQL and Redis so data survives container removal, not just a restart.</li><li>Add a CI pipeline (GitHub Actions) to validate the Docker build on every push.</li></ul><h3>Lessons Learned</h3><ul><li>Docker’s default bridge network provides IP connectivity but no automatic service discovery.</li><li>A user-defined bridge network’s embedded DNS is what actually makes container-name-based configuration work.</li><li>Environment variables are read once, at container startup; changing them means recreating the container, not just reconfiguring the network around it.</li><li>--cpus and --memory limits are real, kernel-enforced ceilings, and I verified that under genuine load rather than taking the configuration on faith.</li><li>Minimal official images often omit convenience tooling; a built-in alternative or a Docker-native command can usually substitute without modifying the image at all.</li><li>Applications that depend on other containers need to handle “not ready yet” and “temporarily unavailable” gracefully, through retry logic, not by assuming the dependency is always there.</li><li>This exact dependency-ordering problem is what Docker Compose’s depends_on and healthchecks exist to solve, and now I&#39;ve actually felt the problem they solve.</li></ul><h3>Project Repository</h3><p>The full project, including the Dockerfile, application code, and this write-up’s source commands, is available on GitHub:</p><p><a href="https://github.com/rester12/Docker-Project-Multi-Container-without-Compose">GitHub - rester12/Docker-Project-Multi-Container-without-Compose: Multi-container Docker networking project, redis, mysql, and a custom Flask+Redis app, built without Compose to prove and demonstrate container networking fundamentals.</a></p><h3>Closing Thoughts</h3><p>What started as four containers running independently on a network I hadn’t thought much about became a much deeper exercise in Docker networking, resource governance, environment variable lifecycle, and application design. The most valuable part wasn’t getting five containers running side by side; it was being able to explain, with real error messages and real commands, exactly why the naive version didn’t work and exactly what fixed it.</p><p>This project gave me a much stronger, hands-on foundation for the networking and orchestration concepts that Docker Compose and, eventually, Kubernetes build on top of, and it reinforced the validate-everything mindset I want to keep carrying into my cloud engineering and DevOps work.</p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c372c9104d9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Containerized Static Website with Docker, Nginx, Bind Mounts, Apache, Gzip, and Custom…]]></title>
            <link>https://medium.com/@rester.mcglown/building-a-containerized-static-website-with-docker-nginx-bind-mounts-apache-gzip-and-custom-f971a6627e39?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/f971a6627e39</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[nginx]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Mon, 07 Sep 2026 04:14:57 GMT</pubDate>
            <atom:updated>2026-09-07T04:37:03.462Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Containerized Static Website with Docker, Nginx, Bind Mounts, Apache, Gzip, and Custom Error Handling</h3><p>A hands-on Docker project focused on container deployment, web-server inspection, bind mounts, troubleshooting, HTTP validation, and production-minded Nginx configuration</p><p>In this project, I built and validated a containerized static website for a fictional real estate company called BrightPath Realty. The goal was to deploy a lightweight marketing site without provisioning or maintaining a traditional virtual machine, while also practicing the operational skills needed to inspect, troubleshoot, validate, and improve a containerized web service.</p><p>Rather than stopping after launching an Nginx container, I progressively expanded the project to include HTTP validation with curl, container inspection, log monitoring, a custom BrightPath website, bind-mounted content, an Apache comparison, externalized Nginx configuration, gzip compression, and custom 404 handling. The focus throughout the project was not only on making the service work, but on understanding why it worked and proving each major change through direct validation.</p><h3>Project Architecture and Objectives</h3><p>The final design used Docker Desktop on Windows to run two independent web-server containers. Nginx served the BrightPath Realty site through host port 8080, while Apache ran separately on host port 8081 for comparison. The BrightPath site content and custom Nginx configuration were both stored on the Windows host and mounted into the Nginx container as read-only bind mounts.</p><figure><img alt="Docker architecture diagram showing a Windows host running Docker Desktop with separate Nginx and Apache containers, read-only bind mounts for the BrightPath site and Nginx configuration, host ports 8080 and 8081 mapped to container port 80, and browser and curl validation for both services." src="https://cdn-images-1.medium.com/max/1024/1*YnfvjNLv7U3-QlHC_rJcrg.png" /><figcaption>Architecture of the BrightPath Realty Docker environment, showing the Windows host, read-only bind mounts, Nginx and Apache containers, host-port mappings, and browser and curl validation.</figcaption></figure><p>The project objectives were to:</p><ul><li>Deploy an official Nginx image.</li><li>Publish the Nginx web service to the Windows host.</li><li>Inspect the running container and Nginx configuration.</li><li>Replace the default site with custom BrightPath content.</li><li>Redesign the deployment with bind mounts.</li><li>Compare Nginx and Apache.</li><li>Externalize the Nginx configuration.</li><li>Enable gzip compression.</li><li>Configure and validate a custom 404 page.</li></ul><h3>The Problem I Wanted to Solve</h3><p>BrightPath Realty needed a lightweight marketing site deployed quickly without provisioning, patching, or maintaining a traditional virtual machine. I wanted to use this project to demonstrate how containers can reduce that operational overhead while still giving me direct control over the web server, site content, configuration, networking, and validation process.</p><p>The project started with a simple goal: run Nginx in a container and make it reachable from the Windows host. From there, I expanded the build so I could inspect the container, understand where Nginx stored its configuration and content, replace the default site, compare Nginx with Apache, and then redesign the deployment using bind mounts and a custom Nginx configuration.</p><p>The main question I wanted to answer was:</p><blockquote>How can I take a basic containerized web server and turn it into a deployment I can inspect, explain, validate, troubleshoot, and improve?</blockquote><h3>Why I Used Containers Instead of a VM</h3><p>A traditional virtual machine could also host a static website, but it would introduce more operating-system management than this project required. With Docker, I could start from an official Nginx image, create a disposable web-server container, publish a host port, and validate the service within minutes.</p><p>The container model also made the project easier to rebuild and compare. I could remove and recreate Nginx without rebuilding an entire server, run Apache beside it on a different host port, and externalize both the site files and the Nginx configuration so they remained outside the lifecycle of the container.</p><p>The container approach gave me several advantages for this project:</p><ul><li>Fast deployment from an official image.</li><li>No need to provision and maintain a dedicated VM.</li><li>Easy container replacement and recreation.</li><li>Independent port mappings for Nginx and Apache.</li><li>Direct access to logs and container metadata.</li><li>Support for bind-mounted content and configuration.</li><li>A clean way to separate host-managed files from the running workload.</li></ul><figure><img alt="PowerShell showing the running brightpath-nginx Docker container with host port 8080 mapped to container port 80." src="https://cdn-images-1.medium.com/max/1024/1*Yr4vYs8uNOsP94H7hruuiQ.png" /><figcaption>The first Nginx container running in detached mode with Windows host port 8080 mapped to container port 80.</figcaption></figure><h3>Tools and Technologies Used</h3><p>The project used the following technologies and tools:</p><ul><li>Docker Desktop</li><li>Docker Engine and Docker CLI</li><li>Windows PowerShell</li><li>Nginx</li><li>Apache HTTP Server</li><li>Linux containers</li><li>HTML5 and CSS3</li><li>curl</li><li>Docker bind mounts</li><li>Custom Nginx configuration</li><li>Git and GitHub</li><li>Markdown documentation</li></ul><h3>Phase 1 — Building and Validating the First Nginx Container</h3><p>I started by verifying that Docker Desktop and the Docker Engine were operational before deploying anything. I used docker version to confirm that the Docker client could communicate with the server-side Docker Engine.</p><figure><img alt="PowerShell showing Docker client and server version information from the docker version command." src="https://cdn-images-1.medium.com/max/507/1*_Wc01gVAyl49712shTSnCA.png" /><figcaption>Docker client and server information confirming that the local Docker CLI could communicate successfully with the Docker Engine.</figcaption></figure><p>After confirming the Docker environment, I pulled the official Nginx image:</p><pre>docker pull nginx:latest</pre><p>This downloaded the image layers required to create an Nginx container locally.</p><figure><img alt="PowerShell showing the docker pull nginx latest command completing successfully." src="https://cdn-images-1.medium.com/max/816/1*dJq1XwUxsPfL922uEeaj3Q.png" /><figcaption>Pulling the official Nginx image from Docker Hub before creating the web-server container.</figcaption></figure><p>With the image available locally, I created the first BrightPath Nginx container:</p><pre>docker run -d --name brightpath-nginx -p 8080:80 nginx:latest</pre><p>The -d option ran the container in detached mode. The --name option assigned the container a readable name, and -p 8080:80 published Windows host port 8080 and forwarded traffic to port 80 inside the container.</p><figure><img alt="PowerShell showing the docker run command creating the brightpath-nginx container with port mapping from host 8080 to container 80." src="https://cdn-images-1.medium.com/max/862/1*vZ2NtQuSL7J11KPyRlstTA.png" /><figcaption>Creating the first BrightPath Nginx container in detached mode with host port 8080 published to container port 80.</figcaption></figure><p>I also inspected the published port directly from Docker metadata instead of relying only on the docker ps summary:</p><pre>docker inspect --format=&#39;{{json .NetworkSettings.Ports}}&#39; brightpath-nginx</pre><p>This confirmed that container port 80 was exposed through Windows host port 8080.</p><figure><img alt="PowerShell showing Docker inspect output with container port 80 mapped to host port 8080." src="https://cdn-images-1.medium.com/max/967/1*r4BJPM1RvnJ7fracCXTJaA.png" /><figcaption>Docker Inspect showing the configured port mapping for the running Nginx container.</figcaption></figure><h4>Verifying the Default Nginx Site</h4><p>I opened http://localhost:8080 in the browser and received the default Nginx welcome page. This confirmed that the request path from the Windows host through Docker’s published port to Nginx inside the container was working correctly.</p><figure><img alt="Browser displaying the default Nginx welcome page at localhost port 8080." src="https://cdn-images-1.medium.com/max/1024/1*gx07xyu0ixp1NK0RvmQB9w.png" /><figcaption>The default Nginx welcome page served successfully through localhost on host port 8080.</figcaption></figure><p>I then validated the service at the HTTP level with curl:</p><pre>curl.exe -I http://localhost:8080</pre><p>The response returned HTTP/1.1 200 OK and identified Nginx as the server, giving me a second validation path independent of the browser.</p><figure><img alt="PowerShell showing curl response headers with HTTP 200 OK and Server nginx." src="https://cdn-images-1.medium.com/max/637/1*qO2C79SJljzTjtgxohhH0A.png" /><figcaption>HTTP header validation confirming a successful 200 response from the Nginx container.</figcaption></figure><h3>Phase 2 — Inspecting the Container and Understanding Nginx</h3><p>After confirming that Nginx was reachable from the Windows host, I wanted to understand what was actually running inside the container. Rather than treating the container as a black box, I used docker exec to open an interactive shell and inspect the Linux environment directly.</p><pre>docker exec -it brightpath-nginx /bin/bash</pre><figure><img alt="PowerShell showing an interactive Bash shell opened inside the running brightpath-nginx container." src="https://cdn-images-1.medium.com/max/710/1*tX74SXR9hJosvqskcrOYhQ.png" /><figcaption>Opening an interactive Bash shell inside the running BrightPath Nginx container with docker exec.</figcaption></figure><h4>Inspecting the Linux Environment</h4><p>Once inside the container, I verified the underlying Linux environment with:</p><pre>cat /etc/os-release</pre><p>This reinforced an important Docker concept: although Docker Desktop was running on Windows, the Nginx workload itself was executing inside a Linux container environment with its own filesystem and userspace.</p><figure><img alt="Terminal showing Linux operating system information from the /etc/os-release file inside the Nginx container." src="https://cdn-images-1.medium.com/max/361/1*NHVWLsL3mnMQdR76uy8b1A.png" /><figcaption>Inspecting the Linux operating system information from inside the running Nginx container.</figcaption></figure><h4>Connecting Nginx Configuration to Docker Networking</h4><p>I then inspected Nginx’s default server configuration to understand how the web server inside the container related to the Docker port mapping I had already configured.</p><pre>cat /etc/nginx/conf.d/default.conf</pre><p>Two directives were especially important:</p><ul><li>listen 80; showed that Nginx was listening on port 80 inside the container.</li><li>root /usr/share/nginx/html; identified the directory Nginx used to serve its static website content.</li></ul><p>This connected the Docker and Nginx layers directly: Windows host port 8080 forwarded requests to container port 80, and Nginx was configured to listen on that same internal port.</p><figure><img alt="Terminal showing Nginx configuration with listen port 80 and document root /usr/share/nginx/html." src="https://cdn-images-1.medium.com/max/450/1*vjHIWzBur-sOBG_EPfud5A.png" /><figcaption>Nginx default server configuration showing the internal listen port and the static-content document root.</figcaption></figure><h4>Tracing the Default Web Page to Its Source File</h4><p>After identifying the document root, I inspected the files Nginx was actually serving:</p><pre>ls -la /usr/share/nginx/html<br>cat /usr/share/nginx/html/index.html</pre><p>The index.html file contained the same default Nginx content I had already seen in the browser. This gave me a complete trace from the browser request back to the exact file being served inside the container.</p><figure><img alt="Terminal showing the default Nginx index.html file stored in /usr/share/nginx/html inside the container." src="https://cdn-images-1.medium.com/max/556/1*MNG0uXdKWHt5yl3B0lRRCw.png" /><figcaption>The default Nginx index.html file inside the container, matching the welcome page previously displayed in the browser.</figcaption></figure><p>At this point, I could explain the full request path instead of simply saying that “Nginx worked.”</p><pre>Browser<br>  ↓<br>localhost:8080<br>  ↓<br>Docker port publishing<br>  ↓<br>container port 80<br>  ↓<br>Nginx<br>  ↓<br>/usr/share/nginx/html/index.html</pre><h3>Phase 3 — Troubleshooting the Minimal Nginx Image</h3><p>While inspecting the running container, I tried to use a familiar Linux process-inspection command:</p><pre>ps aux</pre><p>Instead of returning the running processes, the container responded with:</p><pre>bash: ps: command not found</pre><p>The reason was that the official Nginx image was intentionally minimal and did not include the traditional ps utility.</p><figure><img alt="Terminal showing the ps aux command failing inside the Nginx container with bash reporting ps command not found." src="https://cdn-images-1.medium.com/max/231/1*QLZ3sI3rPqixeQuvx4fqhg.png" /><figcaption>The ps aux command was unavailable inside the minimal Nginx container image.</figcaption></figure><h4>Using Docker-Native Process Inspection</h4><p>Rather than installing additional packages into the running container solely for troubleshooting, I exited the container and used Docker’s own process-inspection command from the Windows host:</p><pre>docker top brightpath-nginx</pre><p>This displayed the Nginx master and worker processes without modifying the container. That was the better troubleshooting choice because it preserved the minimal runtime environment while still giving me the process information I needed.</p><figure><img alt="PowerShell showing docker top output for the brightpath-nginx container with Nginx master and worker processes visible." src="https://cdn-images-1.medium.com/max/1024/1*xi3uENFBEyGPLL93fHb9sg.png" /><figcaption>Using docker top from the Windows host to inspect the Nginx master and worker processes without modifying the container.</figcaption></figure><h4>Validating the Nginx Service After Troubleshooting</h4><p>I then used docker exec for targeted application-level checks instead of opening another interactive shell.</p><pre>docker exec brightpath-nginx nginx -v<br>docker exec brightpath-nginx nginx -t</pre><p>The first command confirmed the installed Nginx version. The second validated the active Nginx configuration and returned a successful syntax test.</p><figure><img alt="PowerShell showing the Nginx version and a successful nginx configuration syntax test inside the running container." src="https://cdn-images-1.medium.com/max/671/1*YDz-EQBcsNH2PesE3en-6w.png" /><figcaption>Verifying the installed Nginx version and confirming that the active configuration passed nginx -t.</figcaption></figure><h4>Troubleshooting Takeaway</h4><p>The key lesson was that a missing administrative command does not automatically indicate a failed container. Minimal images often omit utilities that are not required by the application, so the better response is to first determine whether Docker or application-native tooling can provide the needed visibility.</p><p>In this case, the troubleshooting flow became:</p><ol><li>Attempt process inspection inside the container.</li><li>Identify that ps is not installed.</li><li>Avoid modifying the running image unnecessarily.</li><li>Use docker top from the host.</li><li>Validate Nginx directly with nginx -t.</li></ol><h3>Phase 4 — Replacing the Default Site with BrightPath Realty</h3><p>After understanding the default Nginx configuration and document root, I created the custom BrightPath Realty website on the Windows host. The initial site consisted of an index.html file for the page structure and a styles.css file for presentation.</p><figure><img alt="PowerShell showing the BrightPath Realty site directory containing index.html and styles.css on the Windows host." src="https://cdn-images-1.medium.com/max/819/1*RFj8rFcFd6i_0RW4Q8oGKQ.png" /><figcaption>The BrightPath Realty source files created on the Windows host before deployment into the Nginx container.</figcaption></figure><h4>First Deployment Method: Copying Files into the Running Container</h4><p>For the first custom deployment, I used docker cp to copy the website files directly into the running Nginx container.</p><pre>docker cp .\site\index.html brightpath-nginx:/usr/share/nginx/html/index.html<br>docker cp .\site\styles.css brightpath-nginx:/usr/share/nginx/html/styles.css</pre><figure><img alt="PowerShell showing docker cp commands copying BrightPath index.html and styles.css into the Nginx document root." src="https://cdn-images-1.medium.com/max/993/1*EgSs3Lu1paonXM1CQiZMhw.png" /><figcaption>Copying the custom BrightPath Realty HTML and CSS files into the running Nginx container.</figcaption></figure><h4>Verifying the Files Inside the Container</h4><p>I then verified the destination inside the container rather than assuming the copy had succeeded.</p><pre>docker exec brightpath-nginx ls -la /usr/share/nginx/html</pre><p>The Nginx document root now contained the BrightPath files instead of only the default content.</p><figure><img alt="PowerShell showing index.html and styles.css inside the Nginx container document root." src="https://cdn-images-1.medium.com/max/951/1*RZtEAawIDm4CR-Z15Zwltg.png" /><figcaption>The BrightPath Realty files verified inside the Nginx document root at /usr/share/nginx/html.</figcaption></figure><h4>Verifying the Custom Site in the Browser</h4><p>After refreshing http://localhost:8080, the default Nginx welcome page was replaced by the custom BrightPath Realty site.</p><figure><img alt="Browser displaying the custom BrightPath Realty website at localhost port 8080." src="https://cdn-images-1.medium.com/max/1024/1*ETP5CKhN0rdMDNdI8wdc4w.png" /><figcaption>The custom BrightPath Realty website successfully replacing the default Nginx landing page.</figcaption></figure><h4>Verifying the Custom Content with curl</h4><p>I also verified that the HTTP response contained the new site content rather than relying only on what the browser displayed.</p><pre>curl.exe http://localhost:8080 | Select-String &quot;BrightPath Realty&quot;</pre><figure><img alt="PowerShell showing curl output matching the text BrightPath Realty from the Nginx web response." src="https://cdn-images-1.medium.com/max/903/1*fZDALbtzY6KsDDRYgtRapA.png" /><figcaption>Using curl to confirm that the Nginx HTTP response contained the BrightPath Realty content.</figcaption></figure><h4>Why This Was Not the Final Deployment Design</h4><p>The custom site worked, but this approach had an architectural limitation. The files copied with docker cp lived inside that specific container’s writable filesystem. If I removed the container and created a new one, those copied changes would not automatically follow the replacement container.</p><p>That limitation led to the next design improvement: moving the website content out of the container lifecycle and serving it through a Docker bind mount.</p><h3>Phase 5 — Redesigning the Deployment with Bind Mounts</h3><p>Before changing the deployment model, I first inspected the existing container to confirm that no mounts were attached.</p><pre>docker inspect --format=&#39;{{json .Mounts}}&#39; brightpath-nginx</pre><p>The output showed an empty mount list, confirming that the custom site files were still stored only inside the container filesystem.</p><figure><img alt="PowerShell showing Docker inspect output with an empty mounts list for the brightpath-nginx container." src="https://cdn-images-1.medium.com/max/846/1*7QUMvxtZKW6QDs5f9_mcrQ.png" /><figcaption>Docker Inspect confirming that the original Nginx container had no bind mounts attached.</figcaption></figure><h4>Removing the Container Without Losing the Host Files</h4><p>I then stopped and removed the original Nginx container. The important point was that the BrightPath source files had already been created on the Windows host, so removing the container did not delete the project content.</p><figure><img alt="PowerShell showing the Nginx container removed while the BrightPath site files remain available in the host project directory." src="https://cdn-images-1.medium.com/max/1024/1*LMra3AMRYSLPNaKZVJ4Tww.png" /><figcaption>The original Nginx container removed while the BrightPath website files remained intact on the Windows host.</figcaption></figure><h4>Recreating Nginx with a Read-Only Bind Mount</h4><p>I recreated the Nginx container and mounted the host-side site directory directly into the Nginx document root.</p><pre>docker run -d `<br>  --name brightpath-nginx `<br>  -p 8080:80 `<br>  --mount type=bind,source=&quot;${PWD}\site&quot;,target=/usr/share/nginx/html,readonly `<br>  nginx:latest</pre><p>The readonly option was intentional. Nginx only needed to read the static content, so the container did not need write access to the host project files.</p><figure><img alt="PowerShell showing the docker run command creating the brightpath-nginx container with a read-only bind mount for the site directory." src="https://cdn-images-1.medium.com/max/649/1*igNdxaFfIRXXIJDs-SUmHg.png" /><figcaption>Recreating the Nginx container with the BrightPath site directory mounted into the document root as a read-only bind mount.</figcaption></figure><h4>Verifying the Bind Mount</h4><p>After recreation, I inspected the container metadata again to verify that Docker had attached the mount exactly as intended.</p><figure><img alt="PowerShell showing Docker inspect output for the BrightPath bind mount from the Windows host to the Nginx document root." src="https://cdn-images-1.medium.com/max/1024/1*-Vf_TmYkR5XwTk4gjqVOfA.png" /><figcaption>Docker Inspect showing the BrightPath site directory mounted from the Windows host into /usr/share/nginx/html inside the container.</figcaption></figure><h4>Comparing Host and Container Content</h4><p>I also compared the host directory with the Nginx document root to confirm that both locations were presenting the same files.</p><figure><img alt="PowerShell showing matching BrightPath site files on the Windows host and inside the Nginx document root." src="https://cdn-images-1.medium.com/max/825/1*1Y0Qd5xS8uDv33N5U3hbKg.png" /><figcaption>Comparing the BrightPath files on the Windows host with the files visible inside the Nginx container.</figcaption></figure><h4>Proving the Bind Mount with a Live Content Change</h4><p>The strongest proof came from modifying the host-side website without copying files into the container and without restarting or recreating Nginx.</p><p>I changed the homepage headline to:</p><blockquote>Your trusted path to a brighter home.</blockquote><p>Because the host directory was mounted directly into the container, Nginx immediately served the updated file.</p><figure><img alt="Browser showing the updated BrightPath Realty homepage after the site file was changed on the Windows host through the bind mount." src="https://cdn-images-1.medium.com/max/1024/1*TvCjWAvvp_SFsBIVwkYubQ.png" /><figcaption>The BrightPath homepage reflecting a host-side content change immediately through the bind mount without a container rebuild or restart.</figcaption></figure><h4>Verifying the Live Update with curl</h4><p>I verified the updated content again at the HTTP level:</p><pre>curl.exe http://localhost:8080 | Select-String &quot;trusted path&quot;</pre><figure><img alt="PowerShell showing curl output matching the updated BrightPath headline served from the bind-mounted site directory." src="https://cdn-images-1.medium.com/max/863/1*PZqIr5zrI9XkwDsd2UT9rg.png" /><figcaption>curl confirming that Nginx immediately served the updated BrightPath headline from the bind-mounted host files.</figcaption></figure><h4>Bind Mount vs. Named Volume</h4><p>For this project, a bind mount was a better fit than a named volume because the website source files were part of the project and needed to remain directly accessible on the Windows host.</p><ul><li>A <strong>bind mount</strong> maps a specific host path directly into the container.</li><li>A <strong>named volume</strong> is managed by Docker and stored in Docker-controlled storage.</li><li>Bind mounts are useful when source files must be edited directly from the host.</li><li>Named volumes are often better for persistent application data that does not need to be managed as normal project files.</li></ul><p>This redesign separated the BrightPath content from the container lifecycle. The container became replaceable, while the website source remained persistent and directly manageable from the host.</p><h3>Phase 6 — Running Nginx and Apache Side by Side</h3><p>After completing the bind-mount redesign, I added Apache as a second web server so I could compare two common HTTP server implementations in the same Docker environment.</p><p>Nginx continued to serve the BrightPath Realty site through host port 8080, while Apache ran independently on host port 8081. Both containers listened on port 80 internally, but Docker exposed them through different host ports.</p><pre>docker pull httpd:latest<br>docker run -d --name brightpath-apache -p 8081:80 httpd:latest</pre><figure><img alt="PowerShell showing the brightpath-nginx and brightpath-apache containers running at the same time with host ports 8080 and 8081." src="https://cdn-images-1.medium.com/max/1024/1*ErxdMiPz__I96ftTtdYD_A.png" /><figcaption>Nginx and Apache running simultaneously in separate Docker containers with different host-port mappings.</figcaption></figure><h4>Verifying Apache in the Browser</h4><p>I opened http://localhost:8081 in the browser and received Apache’s default page, confirming that the second container was reachable independently of Nginx.</p><figure><img alt="Browser displaying the Apache default page at localhost port 8081." src="https://cdn-images-1.medium.com/max/306/1*2I4EgvdsxDYJNrw0ABevDQ.png" /><figcaption>Apache’s default page served successfully through localhost on host port 8081.</figcaption></figure><h4>Comparing HTTP Response Headers</h4><p>I then compared the HTTP response headers from both services with curl.</p><pre>curl.exe -I http://localhost:8080<br>curl.exe -I http://localhost:8081</pre><p>Both services returned successful responses, but the Server header identified the implementation handling each request. This provided a simple way to verify that the two host ports were reaching different containers.</p><figure><img alt="PowerShell showing curl response headers from Nginx on port 8080 and Apache on port 8081." src="https://cdn-images-1.medium.com/max/640/1*ejAid8b1_ZIlXN2ccSWOiA.png" /><figcaption>Comparing HTTP response headers from Nginx and Apache to verify that each host port reached a different web server.</figcaption></figure><h4>Comparing Configuration and Directory Structure</h4><p>I also compared the filesystem layout used by each server. The difference was useful because the container images were designed around different configuration paths and document roots.</p><ul><li>Nginx configuration directory: /etc/nginx</li><li>Nginx document root: /usr/share/nginx/html</li><li>Apache configuration file: /usr/local/apache2/conf/httpd.conf</li><li>Apache document root: /usr/local/apache2/htdocs</li></ul><figure><img alt="PowerShell showing the different Nginx and Apache configuration directories and document-root paths inside their containers." src="https://cdn-images-1.medium.com/max/872/1*fahIizVbApfDrmLAd1q8jg.png" /><figcaption>Comparing the configuration locations and document-root structures used by the Nginx and Apache container images.</figcaption></figure><h4>What the Comparison Demonstrated</h4><p>Running both servers side by side reinforced that Docker provides a consistent container-management workflow even when the applications inside the containers are structured differently.</p><p>The comparison also showed that port publishing is independent of the application itself. Both web servers listened on internal port 80, while Docker mapped them to separate host ports so they could run simultaneously without conflict.</p><h3>Phase 7 — Externalizing the Nginx Configuration</h3><p>The next improvement was to move the Nginx configuration out of the container and manage it as part of the project on the Windows host. This made the configuration visible, version-controllable, and replaceable without manually editing the container filesystem.</p><p>I created a custom nginx.conf file that preserved the core Nginx runtime settings while adding two features for the BrightPath deployment: gzip compression and custom 404 handling.</p><figure><img alt="PowerShell or editor showing the custom nginx.conf file containing gzip settings, server configuration, document root, and custom 404 directives." src="https://cdn-images-1.medium.com/max/357/1*ZMFTjaJbSrPyxwHN-Ovqmg.png" /><figcaption>The custom Nginx configuration created on the Windows host with gzip compression and custom 404 handling enabled.</figcaption></figure><h4>Key Configuration Changes</h4><ul><li>gzip on; enabled response compression.</li><li>gzip_vary on; added the appropriate response variation behavior for clients that support compression.</li><li>gzip_min_length 256; prevented compression from being applied to very small responses.</li><li>gzip_types defined which content types Nginx should compress.</li><li>error_page 404 /404.html; mapped missing resources to the custom BrightPath error page.</li><li>try_files $uri $uri/ =404; ensured nonexistent resources returned a true HTTP 404 response.</li></ul><h4>Recreating Nginx with Content and Configuration Mounts</h4><p>I then recreated the Nginx container with two read-only bind mounts: one for the BrightPath website content and one for the custom Nginx configuration.</p><pre>docker run -d `<br>  --name brightpath-nginx `<br>  -p 8080:80 `<br>  --mount type=bind,source=&quot;${PWD}\site&quot;,target=/usr/share/nginx/html,readonly `<br>  --mount type=bind,source=&quot;${PWD}\nginx\nginx.conf&quot;,target=/etc/nginx/nginx.conf,readonly `<br>  nginx:latest</pre><p>The website content and Nginx configuration were now both controlled from the host, while the container remained disposable and replaceable.</p><figure><img alt="PowerShell showing the docker run command creating the brightpath-nginx container with read-only mounts for the site directory and nginx.conf." src="https://cdn-images-1.medium.com/max/749/1*RXUmBhe1M8WzNmEdt3FYEg.png" /><figcaption>Recreating the BrightPath Nginx container with separate read-only bind mounts for site content and custom configuration.</figcaption></figure><h4>Verifying Both Bind Mounts</h4><p>I inspected the container metadata again to verify that both host resources were mounted into the intended container paths.</p><figure><img alt="PowerShell showing Docker inspect output with two read-only bind mounts for the site directory and nginx.conf." src="https://cdn-images-1.medium.com/max/1024/1*-0coFkJBkvnvZTic5UC-yw.png" /><figcaption>Docker Inspect confirming both the BrightPath site mount and the custom Nginx configuration mount.</figcaption></figure><h4>Validating the Custom Configuration</h4><p>Before relying on the new configuration, I validated it directly inside the running container.</p><pre>docker exec brightpath-nginx nginx -t</pre><p>The syntax test completed successfully, confirming that Nginx could parse and load the custom configuration.</p><figure><img alt="PowerShell showing nginx configuration syntax validation completing successfully inside the brightpath-nginx container." src="https://cdn-images-1.medium.com/max/676/1*YmKbIGdHcNkcKvFT9W8Vww.png" /><figcaption>Successful nginx -t validation of the custom bind-mounted Nginx configuration.</figcaption></figure><h4>Why Externalizing the Configuration Mattered</h4><p>Externalizing nginx.conf made the deployment easier to understand and reproduce. Instead of changing configuration inside a running container, the desired server behavior was defined in a host-side file that could be reviewed, versioned, and mounted into any replacement container.</p><p>This also reinforced an important container design principle: application content and configuration should be managed separately from the disposable runtime whenever possible.</p><h3>Phase 8 — Custom 404 Handling and Gzip Compression</h3><p>With the custom Nginx configuration mounted and validated, I tested the two behavior changes that were most important to the final deployment: branded 404 handling and gzip compression.</p><h4>Testing the Custom 404 Page</h4><p>I created a custom 404.html page for BrightPath Realty so that missing resources would return a branded error experience instead of the default Nginx error page.</p><p>The Nginx configuration mapped HTTP 404 errors to that file while preserving the real 404 status code.</p><pre>error_page 404 /404.html;<br><br>location = /404.html {<br>    internal;<br>}<br><br>location / {<br>    try_files $uri $uri/ =404;<br>}</pre><figure><img alt="Browser displaying the custom BrightPath Realty 404 page for a missing property URL." src="https://cdn-images-1.medium.com/max/861/1*-OMumFhf1EjMZbuPw1fj9g.png" /><figcaption>The branded BrightPath Realty 404 page displayed for a nonexistent resource.</figcaption></figure><h4>Confirming the Real HTTP 404 Status</h4><p>A custom error page is only correct if the server still returns the appropriate HTTP status code. I verified that behavior with curl.</p><pre>curl.exe -I http://localhost:8080/property-does-not-exist</pre><p>The response returned HTTP/1.1 404 Not Found, confirming that the branded page did not mask the underlying error status.</p><figure><img alt="PowerShell showing curl response headers with HTTP 404 Not Found for a nonexistent BrightPath URL." src="https://cdn-images-1.medium.com/max/833/1*w9O6Wr3GwERWz72mQhVvpA.png" /><figcaption>curl confirming that the branded error page still returned a real HTTP 404 status.</figcaption></figure><h4>Verifying the 404 in Nginx Logs</h4><p>I also checked the Nginx logs to confirm that the failed request was recorded by the web server.</p><figure><img alt="PowerShell showing Nginx logs containing a request for a missing BrightPath page with a 404 status." src="https://cdn-images-1.medium.com/max/1024/1*FnD4jA-NYhFPWYRZoH217g.png" /><figcaption>Nginx logs showing the request for a nonexistent resource and the resulting 404 response.</figcaption></figure><h4>Testing Gzip Compression</h4><p>I then tested gzip compression by requesting the BrightPath stylesheet with and without the Accept-Encoding: gzip request header.</p><pre>curl.exe -I http://localhost:8080/styles.css<br>curl.exe -I -H &quot;Accept-Encoding: gzip&quot; http://localhost:8080/styles.css</pre><p>When the client advertised gzip support, Nginx responded with Content-Encoding: gzip. Without that request header, the response was sent without gzip encoding.</p><figure><img alt="PowerShell showing two curl header responses for styles.css, with gzip compression enabled only when the Accept-Encoding gzip header is sent." src="https://cdn-images-1.medium.com/max/944/1*LbZeRirUq0rrb55IFBxdrw.png" /><figcaption>Comparing stylesheet responses with and without gzip support, showing Nginx adding Content-Encoding: gzip only when requested by the client.</figcaption></figure><h4>Why These Tests Mattered</h4><p>These final checks validated more than appearance. The 404 tests proved that Nginx handled missing resources correctly at both the user-interface and HTTP-protocol levels, while the gzip test demonstrated that the custom configuration changed real response behavior.</p><p>At this point, the deployment had progressed from a default Nginx container to a custom, externally configured, bind-mounted web service that I could inspect, validate, troubleshoot, and reproduce.</p><h3>Final Validation</h3><p>Before considering the project complete, I ran a final end-to-end validation to confirm that the important parts of the environment were still working together correctly.</p><p>The final checks confirmed that:</p><ul><li>Nginx was running and serving the BrightPath Realty site on host port 8080.</li><li>Apache was running independently on host port 8081.</li><li>The custom Nginx configuration passed nginx -t.</li><li>The BrightPath homepage returned HTTP 200.</li><li>Gzip compression was negotiated successfully for supported content.</li><li>Missing resources returned a real HTTP 404.</li><li>The custom 404 page was served correctly.</li></ul><figure><img alt="Terminal output showing docker ps with brightpath-nginx and brightpath-apache both running, nginx -t confirming successful configuration syntax, curl returning HTTP 200 OK for the BrightPath homepage, gzip compression applied to styles.css when requested, and a proper HTTP 404 response for a nonexistent resource." src="https://cdn-images-1.medium.com/max/1024/1*4lX3R_G9WhGX-S2CIijBvQ.png" /><figcaption>Final end-to-end validation confirming Nginx and Apache running side by side, a successful nginx -t configuration check, a 200 OK response from the BrightPath homepage, gzip compression applied to styles.css, and a proper HTTP 404 for a nonexistent resource.</figcaption></figure><figure><img alt="PowerShell showing the completed BrightPath Docker project directory with nginx configuration, site files, README, and project folders." src="https://cdn-images-1.medium.com/max/867/1*aH4UBF_SUw-2aGzAIdRORg.png" /><figcaption>The completed BrightPath Docker project structure containing the site files, custom Nginx configuration, documentation, and supporting project assets.</figcaption></figure><h3>Engineering Decisions</h3><p>Several decisions in this project were intentional and reflected how I wanted the deployment to behave operationally.</p><ul><li>I used the official Nginx image instead of building an unnecessary custom image for a static-site workload.</li><li>I used Docker port publishing to expose each web server independently.</li><li>I kept the BrightPath site files on the Windows host instead of storing them permanently inside the container filesystem.</li><li>I used read-only bind mounts because Nginx only needed read access to the website and configuration.</li><li>I externalized nginx.conf so the server behavior could be reviewed, versioned, and reproduced.</li><li>I avoided installing extra troubleshooting tools into the running container when Docker-native commands provided the needed visibility.</li><li>I validated changes with both browser testing and command-line HTTP checks.</li></ul><h3>Security and Operational Considerations</h3><p>Although this was a local development project, I still applied several practices that improve safety and maintainability.</p><ul><li>Site content was mounted read-only.</li><li>The Nginx configuration was mounted read-only.</li><li>Unnecessary package installation inside the running container was avoided.</li><li>The runtime container remained replaceable instead of becoming a manually customized server.</li><li>HTTP behavior was validated directly instead of relying only on visual browser testing.</li><li>Configuration syntax was checked with nginx -t before being trusted.</li></ul><h3>What I Would Improve for Production</h3><p>For a production deployment, I would extend this design with additional controls and automation.</p><ul><li>Build a custom Docker image so the website and validated configuration can be versioned as a single deployable artifact.</li><li>Pin image versions instead of relying on latest.</li><li>Add HTTPS with a managed certificate.</li><li>Run the workload behind a reverse proxy or load balancer when appropriate.</li><li>Add container health checks.</li><li>Add automated image scanning and vulnerability management.</li><li>Use CI/CD to build, validate, and deploy changes automatically.</li><li>Centralize application and access logs.</li><li>Add monitoring and alerting.</li><li>Run the container with the minimum privileges required by the workload.</li></ul><h3>Lessons Learned</h3><p>This project reinforced several Docker concepts that became much clearer through hands-on implementation rather than documentation alone.</p><ul><li>A running container is only one layer of the deployment; networking, configuration, storage, and HTTP behavior also need to be understood.</li><li>Container filesystems are disposable, so important project content should not depend on manual changes inside a specific container.</li><li>Bind mounts are useful when host-managed files need to be visible directly inside a container.</li><li>Minimal images may omit familiar administrative utilities, which makes Docker-native troubleshooting commands especially useful.</li><li>Running two web servers side by side helped clarify the difference between Docker’s container-management model and the application-specific structure inside each image.</li><li>A successful browser page is not enough validation by itself; response headers, status codes, logs, and configuration tests provide stronger evidence.</li><li>Externalized configuration makes containerized services easier to review, replace, and reproduce.</li></ul><h3>Project Repository</h3><p>The complete project files, Nginx configuration, BrightPath Realty source code, and implementation documentation are available in the GitHub repository:</p><p><a href="https://github.com/rester12/Docker-Project-Single-Container-Static-Site">GitHub - rester12/Docker-Project-Single-Container-Static-Site: Hands-on Docker portfolio project deploying a static BrightPath Realty site with Nginx, bind mounts, Apache comparison, gzip compression, custom 404 handling, and HTTP validation.</a></p><h3>Closing Thoughts</h3><p>What started as a simple Nginx container became a deeper exercise in container networking, filesystem design, configuration management, troubleshooting, HTTP validation, and web-server comparison. The most valuable part of the project was not just getting a static site online, but understanding how each layer worked and being able to prove that behavior with repeatable commands and validation.</p><p>This project gave me a much stronger foundation for working with containerized applications and reinforced the operational mindset I want to continue building as I develop my cloud engineering and DevOps skills.</p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, containers, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f971a6627e39" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a CI/CD Pipeline: GitLab to AWS CodePipeline to S3]]></title>
            <link>https://medium.com/@rester.mcglown/building-a-ci-cd-pipeline-gitlab-to-aws-codepipeline-to-s3-59bf1803e729?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/59bf1803e729</guid>
            <category><![CDATA[cicd]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 20:37:52 GMT</pubDate>
            <atom:updated>2026-09-06T21:02:28.052Z</atom:updated>
            <content:encoded><![CDATA[<p>How I connected GitLab to AWS CodePipeline, debugged a YAML build failure, and used branch strategy to prove a working CI/CD workflow deploying a personalized resume live to S3</p><p>Continuous deployment sounds simple in theory: push code, watch it go live. In practice, getting there means wiring together a source repository, a build process, and a deployment target, then proving each piece only does what it is supposed to do.</p><p>For this project, I built a complete CI/CD pipeline connecting a GitLab repository to AWS CodePipeline, with AWS CodeBuild validating and stamping the build, and Amazon S3 serving the final static site. Rather than deploying once and calling it done, I used a deliberate branching strategy to prove the pipeline’s trigger behavior was actually working as designed, not just working by accident.</p><p>The complete project repository is linked at the end of this article.</p><h3>Project Architecture and Objectives</h3><figure><img alt="Architecture diagram showing GitLab connected through AWS CodeConnections to AWS CodePipeline, with Source, Build, and Deploy stages, deploying to an Amazon S3 static website bucket." src="https://cdn-images-1.medium.com/max/1024/1*L9KKntzdX2Lc6g9JHZO62g.png" /><figcaption>Architecture of the GitLab-to-S3 CI/CD pipeline used in this project.</figcaption></figure><h3>The problem I wanted to solve</h3><p>The basic requirement sounded straightforward:</p><blockquote>Connect a GitLab repository to AWS CodePipeline, automatically build and validate the site content, and deploy it to a live S3 static website.</blockquote><p>At first, the workflow seemed simple: push code and watch it deploy. But building it correctly required understanding several distinct pieces: source integration through a connection to GitLab, a build stage that validates and transforms the content, a deploy stage that publishes to S3, and a branching strategy that proves the trigger behavior is actually working as intended rather than by coincidence.</p><p>I therefore treated the project as a complete CI/CD workflow rather than a single deploy action.</p><h3>Why the pipeline uses separate stages</h3><p>CodePipeline does not move code directly from a repository to a live website. The workflow is separated into distinct stages: Source, Build, and Deploy.</p><p>The Source stage pulls the latest code from GitLab whenever a push event fires. The Build stage runs CodeBuild against a buildspec.yml file, which validates the HTML and stamps a build timestamp into the page. The Deploy stage takes the build output and publishes it to the S3 bucket configured for static website hosting.</p><p>Understanding these separate stages mattered because a successful push to GitLab does not by itself prove the site is live. Each stage has to succeed independently, and the trigger configuration determines which branch actually causes the pipeline to run at all.</p><h3>Tools and services used</h3><ul><li>GitLab</li><li>GitLab CodeConnections</li><li>AWS CodePipeline (V2)</li><li>AWS CodeBuild</li><li>Amazon S3 (static website hosting)</li><li>VS Code</li><li>Git</li></ul><p>GitLab hosted the source repository, while CodeConnections established the secure link between GitLab and AWS CodePipeline. CodePipeline orchestrated the Source, Build, and Deploy stages, and CodeBuild ran the buildspec-defined validation and timestamp step. Amazon S3 served the final static site through its website hosting feature. VS Code and Git handled all local development, branching, and version control.</p><h3>Building the pipeline in phases</h3><p>I built the project in four phases so I could verify each part of the workflow before moving to the next.</p><pre>Foundation -&gt; Branch-trigger proof -&gt; Personalization -&gt; Documentation</pre><h4>Phase 1: Foundation</h4><p>Phase 1 established the pipeline itself. I created a new GitLab account, group, and project, then cloned the repository locally into VS Code using HTTPS.</p><p>In the AWS Console, I built a CodePipeline named Git-project-2, configured for GitLab as the source provider through a CodeConnections link, with change detection handled by a webhook trigger rather than polling. The trigger was filtered to fire only on push events to the main branch.</p><ul><li>Source stage: GitLab repository, triggered on push to main</li><li>Build stage: AWS CodeBuild, using a buildspec.yml to validate HTML and stamp a build timestamp</li><li>Deploy stage: Amazon S3, extracting the build artifact into the bucket root</li></ul><figure><img alt="Diagram showing the repository structure with buildspec.yml controlling the Build stage, and index.html, resume.css, and image files deploying directly to the root of the S3 bucket." src="https://cdn-images-1.medium.com/max/1024/1*5rXf-NC8w2yQmTPGZQMwFw.png" /><figcaption>buildspec.yml controls the build process; the site files deploy directly to the S3 bucket root.</figcaption></figure><p>After creating the pipeline, I confirmed the S3 static website endpoint returned a 404 NoSuchKey error for index.html. This was the expected result: it proved the full Source-Build-Deploy chain ran successfully end to end, even though no HTML had been pushed yet.</p><h4>Phase 2: Proving the branch-trigger behavior</h4><p>The second phase was designed to prove the trigger configuration actually worked, rather than just assuming it did because the pipeline existed.</p><p>I created a test branch, added a sample resume template to the repository, renamed the main HTML file to index.html so S3 would serve it as the default document, and pushed the branch to GitLab.</p><p>Because the webhook trigger was filtered to the main branch only, pushing to test did not start the pipeline. I checked the S3 site again and confirmed it still returned the same 404 error, proving that branch isolation was working as intended.</p><p>I then merged test into main. That push fired the webhook, the pipeline ran through Source, Build, and Deploy automatically, and refreshing the S3 site now rendered the resume template’s placeholder content. Nothing about the files had changed between the two checks, only which branch they were on, which was the entire point of the exercise.</p><figure><img alt="Diagram comparing two branch outcomes: pushing to the test branch does not trigger the pipeline and the S3 site stays at a 404 error, while merging into main triggers the webhook, runs the pipeline, and the S3 site goes live." src="https://cdn-images-1.medium.com/max/1024/1*ojLUp4cCp4WJoG0bOPO79A.png" /><figcaption>Pushing to test leaves the site unchanged. Merging into main triggers the pipeline and deploys live.</figcaption></figure><h4>Phase 3: Personalizing the resume</h4><p>With the pipeline and branch-trigger behavior both verified, the final build step was making the resume actually mine.</p><p>I created a second branch, test2, off of main, and edited index.html directly, replacing the placeholder name, title, and contact information with my own. I committed the change, pushed test2 to GitLab, and merged it into main.</p><p>The merge triggered the pipeline again. Once Build and Deploy both completed, refreshing the S3 site showed my personalized resume live, served through the same pipeline that had deployed the placeholder version moments earlier.</p><h3>Problems I encountered and how I solved them</h3><p>The most valuable part of this project was debugging a build failure that had nothing to do with logic and everything to do with YAML syntax.</p><figure><img alt="Structural diagram of the CodeBuild stage showing the install phase installing html5validator, the build phase validating HTML and stamping a build timestamp, and the resulting artifacts passed to the Deploy stage." src="https://cdn-images-1.medium.com/max/1024/1*4xhU8Xfg0tJkXn4Ge1ti8w.png" /><figcaption>The Build stage installs html5validator, validates the HTML, and stamps a build timestamp before passing artifacts to Deploy.</figcaption></figure><h4>A colon inside a shell command broke the buildspec.yml parser</h4><p>The buildspec.yml included a command that stamped a build timestamp into the page using sed. The command’s replacement text contained the literal phrase “Last built: $BUILD_TIME,” and that colon followed by a space was the problem.</p><p>YAML treats an unquoted colon followed by a space as the start of a key-value pair. Instead of reading the entire sed command as one plain string, the parser split it apart mid-line and interpreted the rest as a nested structure. CodeBuild’s error made this visible directly in the logs: Expected Commands[4] to be of string type, found subkeys instead.</p><p>The fix was to wrap the entire command in single quotes, forcing YAML to treat it as one literal string regardless of any colons inside it. Since the command already used double quotes internally, single-quoting the outer line avoided any escaping conflicts.</p><pre>version: 0.2<br><br>phases:<br>  install:<br>    runtime-versions:<br>      python: 3.12<br>    commands:<br>      - pip install html5validator<br>  build:<br>    commands:<br>      - echo &quot;Validating HTML...&quot;<br>      - html5validator --root . --also-check-css=false || echo &quot;Validation found issues, continuing anyway&quot;<br>      - echo &quot;Stamping build time...&quot;<br>      - BUILD_TIME=$(date -u &quot;+%Y-%m-%d %H:%M:%S UTC&quot;)<br>      - &#39;sed -i &quot;s/&lt;!--BUILD_TIMESTAMP--&gt;/Last built: $BUILD_TIME/&quot; index.html || echo &quot;No index.html yet, skipping stamp&quot;&#39;<br>artifacts:<br>  files:<br>    - &#39;**/*&#39;</pre><p>Once corrected, CodeBuild ran successfully, and the pipeline completed all three stages end to end.</p><h3>Security and safety decisions</h3><p>Security considerations were built into the pipeline configuration itself rather than added afterward.</p><ul><li>Used AWS CodeConnections for GitLab authentication instead of storing a personal access token directly in AWS</li><li>Let AWS auto-generate dedicated service roles for CodePipeline and CodeBuild, scoped to this pipeline rather than reusing a broad existing role</li><li>Kept the S3 bucket policy scoped to allow public read access only, with no write or delete permissions exposed publicly</li><li>Kept no credentials, tokens, or secrets committed to the GitLab repository</li></ul><h3>Resource cleanup and cost awareness</h3><p>Unlike projects involving EC2 instances, this pipeline did not require manual teardown to avoid ongoing charges. CodePipeline, CodeBuild, and S3 are all event-driven or on-demand services: CodePipeline only runs when triggered by a push, CodeBuild only runs during an actual build, and S3 storage for a handful of small static files is effectively negligible.</p><p>Repository hygiene still mattered. After merging the resume template, the repository briefly contained a set of macOS metadata files created during the zip extraction. I removed them with a simple Git command to keep the repository focused only on the files the project actually needed.</p><h3>What I would add before production use</h3><p>This project is a foundational CI/CD lab rather than a production deployment workflow. A real production setup would benefit from:</p><ul><li>Branch protection rules on main, requiring merge requests instead of direct pushes</li><li>Automated testing beyond HTML validation, such as link checking or accessibility scans</li><li>A CDN in front of S3, such as CloudFront, for caching and HTTPS support</li><li>Environment separation, such as distinct staging and production pipelines</li><li>Monitoring and alerting on pipeline failures rather than manual console checks</li><li>Infrastructure as code, such as CloudFormation or Terraform, instead of manual console configuration</li></ul><h3>What I learned</h3><p>This project pushed me past just running commands and into understanding why a CI/CD pipeline behaves the way it does.</p><h4>A successful push does not mean a successful deploy</h4><p>Git confirming a push does not mean the site is live. Source, Build, and Deploy each have to succeed independently, and only checking the live endpoint proves the entire chain actually worked.</p><h4>Trigger configuration is what makes branching strategy meaningful</h4><p>Creating a branch does nothing on its own. It was the webhook filter on main that gave the test and main branches different real-world consequences, turning an abstract Git concept into something I could see happen live in the browser.</p><h4>YAML errors are often about structure, not content</h4><p>The buildspec.yml failure had nothing to do with the shell command’s logic. A single colon in the wrong place changed how the entire file was parsed. Reading the actual error message, rather than assuming the problem, led directly to the fix.</p><h3>Project Repository</h3><p>The complete pipeline configuration, buildspec.yml, resume files, and commit history are available here:</p><p><a href="https://gitlab.com/git-project-22550732/git-project-22">Git Project 2 / Git Project 22 · GitLab</a></p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59bf1803e729" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Git: Building a Complete Git and GitHub Workflow with WSL 2]]></title>
            <link>https://medium.com/@rester.mcglown/git-building-a-complete-git-and-github-workflow-with-wsl-2-23e14614ccc2?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/23e14614ccc2</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[wsl]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 18:45:26 GMT</pubDate>
            <atom:updated>2026-09-06T21:01:39.478Z</atom:updated>
            <content:encoded><![CDATA[<h4>How I configured a Linux development environment, learned the Git staging and commit workflow, secured GitHub authentication with SSH, and published my first repository</h4><p>Version control is a foundational skill in modern software development, DevOps, and cloud engineering.</p><p>For this project, I built a complete Git and GitHub workflow using Windows Subsystem for Linux (WSL 2), Ubuntu, Git, and SSH authentication. I configured Git locally, created and tracked a file, staged and committed changes, configured secure SSH access to GitHub, and pushed the completed repository to the remote main branch.</p><p>The finished project demonstrates the complete Git workflow from the working directory to the staging area, local repository, and GitHub remote repository.</p><p>The complete project repository is linked at the end of this article.</p><h3>Project Architecture and Objectives</h3><figure><img alt="Git and GitHub workflow architecture showing WSL 2 Ubuntu, the local Git working directory, staging area, local repository, SSH authentication, and the remote GitHub repository." src="https://cdn-images-1.medium.com/max/1024/1*t1rBlbSCACV4POjVKP3wKA.png" /><figcaption>Architecture of the Git and GitHub workflow used in this project.</figcaption></figure><h3>The problem I wanted to solve</h3><p>The basic requirement sounded straightforward:</p><blockquote>Create a local Git workflow, securely connect it to GitHub, and push the completed project to the remote repository.</blockquote><p>At first, the workflow seemed simple: create a file, commit it, and push it to GitHub. But completing the process correctly required understanding several separate Git concepts, including the working directory, staging area, local repository, remote repository, branch tracking, and SSH authentication.</p><p>I therefore treated the project as a complete version-control workflow rather than a single push command.</p><h3>Why the Git workflow uses separate stages</h3><p>Git does not move a changed file directly from the working directory to GitHub. The workflow is separated into distinct stages: the working tree, staging area, local repository, and remote repository.</p><p>The working tree contains the files I am actively editing. The staging area defines the exact content that will be included in the next commit. A commit records that staged snapshot in the local Git repository, and git push publishes the local commit to the remote GitHub repository.</p><p>Understanding these separate states was important because a clean working tree does not necessarily prove that the latest commit has been pushed to GitHub. The local and remote repository states must be validated independently.</p><h3>Tools and services used</h3><p>I completed the project using:</p><ul><li>Windows Subsystem for Linux 2 (WSL 2)</li><li>Ubuntu 24.04 LTS</li><li>Bash and standard Linux commands</li><li>Git 2.43.0</li><li>GitHub</li><li>OpenSSH with an Ed25519 key pair</li><li>Markdown and Mermaid diagrams</li></ul><p>WSL 2 provided the Linux environment while allowing me to continue working from a Windows workstation. Ubuntu supplied the package management, Bash shell, Linux filesystem behavior, permissions, and command-line tools used throughout the project.</p><p>Git handled local version control, while GitHub hosted the remote repository. OpenSSH provided secure key-based authentication between the local Linux environment and GitHub.</p><p>Markdown and Mermaid were used to document the project and represent the architecture and Git data flow inside the repository.</p><h3>Preparing the local Windows environment</h3><p>I used Windows Subsystem for Linux 2 (WSL 2) with Ubuntu 24.04 LTS so I could work in a Linux environment while keeping Windows as the host operating system.</p><pre>wsl --update<br>wsl --status<br>wsl --list --verbose</pre><p>Inside Ubuntu, I verified the active Linux user, current working directory, and administrative access:</p><pre>whoami<br>pwd<br>sudo -v</pre><p>I then updated the Ubuntu packages, installed Git, and confirmed the installed Git version:</p><pre>sudo apt update<br>sudo apt upgrade -y<br>sudo apt install -y git<br>git --version</pre><p>I stored the repository under the Linux home directory instead of the Windows-mounted /mnt/c filesystem. This kept the project on the Linux-native filesystem and avoided unnecessary cross-filesystem overhead.</p><h3>Building the Git workflow in stages</h3><p>I built the project in separate stages so I could verify each part of the Git workflow before moving to the next:</p><pre>Configure Git -&gt; Clone repository -&gt; Create file -&gt; Stage changes -&gt; Commit locally -&gt; Configure SSH -&gt; Push to GitHub</pre><h4>Configuring Git</h4><p>I first configured the Git identity, default branch name, and Linux-oriented line-ending behavior:</p><pre>git config --global user.name &quot;rester&quot;<br>git config --global user.email &quot;rmcglown@hotmail.com&quot;<br>git config --global init.defaultBranch main<br>git config --global core.autocrlf input<br>git config --global --list</pre><p>The author name and email are stored in Git commit metadata. They identify who created a commit, but they do not authenticate the connection to GitHub.</p><h4>Cloning the GitHub repository</h4><p>After configuring Git, I cloned the GitHub repository into my Linux home directory and moved into the project folder:</p><pre>cd ~<br>git clone https://github.com/rester12/Git-and-GitHub-Workflow-with-WSL-2-SSH-Authentication.git<br>cd Git-and-GitHub-Workflow-with-WSL-2-SSH-Authentication<br>pwd<br>git status</pre><p>The pwd command confirmed that I was working inside the Linux-native project directory, while git status verified that Git recognized the repository and showed the current branch and working-tree state.</p><h4>Creating and staging the project file</h4><p>I created a simple text file so I could observe how Git detects, stages, and tracks a new file:</p><pre>echo &quot;This is my first Git project.&quot; &gt; file.txt<br>cat file.txt<br>git status</pre><p>At this point, Git detected file.txt as an untracked file. The file existed in the working tree, but it had not yet been added to the staging area.</p><pre>git add file.txt<br>git status</pre><p>After running git add, the file moved into the staging area and was ready to be included in the next commit. This demonstrated the difference between creating a file in the working tree and explicitly selecting that change for a Git snapshot.</p><h4>Creating the first commit</h4><p>With file.txt staged, I created the first local commit:</p><pre>git commit -m &quot;Add initial project file&quot;<br>git status<br>git log --oneline</pre><p>The commit saved the staged snapshot in the local Git repository. Running git status confirmed that the working tree was clean, while git log — oneline displayed the new commit in the local history.</p><h4>Configuring SSH authentication</h4><p>I used SSH authentication so Git could securely connect to GitHub without requiring a password for each push.</p><pre>ssh-keygen -t ed25519 -C &quot;rmcglown@hotmail.com&quot;</pre><p>This created an Ed25519 public/private key pair. I kept the private key on the local system and added only the public key to my GitHub account.</p><pre>eval &quot;$(ssh-agent -s)&quot;<br>ssh-add ~/.ssh/id_ed25519<br>cat ~/.ssh/id_ed25519.pub</pre><p>After adding the public key to GitHub, I tested the SSH connection before changing the repository remote.</p><pre>ssh -T git@github.com</pre><h4>Changing the remote from HTTPS to SSH</h4><p>The repository was originally cloned with an HTTPS remote, so I changed the origin URL to SSH before pushing the project.</p><pre>git remote -v<br>git remote set-url origin git@github.com:rester12/Git-and-GitHub-Workflow-with-WSL-2-SSH-Authentication.git<br>git remote -v</pre><p>The second git remote -v check confirmed that both fetch and push operations now used the SSH URL.</p><h4>Pushing the main branch to GitHub</h4><p>With SSH authentication working and the remote configured correctly, I pushed the local main branch to GitHub:</p><pre>git push -u origin main</pre><p>The -u option established origin/main as the upstream tracking branch. After that, future pushes could use the shorter git push command.</p><h4>Validating the local and remote repository</h4><p>I finished the workflow by validating the branch state, working tree, remote configuration, and commit history:</p><pre>git status<br>git branch -vv<br>git remote -v<br>git log --oneline --decorate --graph --all</pre><p>These checks confirmed that the working tree was clean, the main branch was tracking origin/main, the remote used SSH, and the expected commit history existed locally.</p><h3>Testing and validating the Git workflow</h3><p>I did not treat a successful push message as enough evidence that the workflow was complete. I validated the local repository, the remote repository, and the relationship between them independently.</p><pre>git status<br>git branch -vv<br>git remote -v<br>git log --oneline --decorate --stat<br>git rev-parse HEAD<br>git ls-remote origin refs/heads/main</pre><p>The validation confirmed:</p><ul><li>The local branch was main.</li><li>main tracked origin/main.</li><li>SSH authentication identified the correct GitHub account.</li><li>file.txt contained the expected project content.</li><li>The local and remote main references matched after publication.</li><li>The working tree was clean after the push.</li></ul><p>Comparing the local and remote commit identifiers provided stronger verification than relying only on the output from git push. It proved that both repositories referenced the same published commit.</p><h3>Problems I encountered and how I solved them</h3><p>The troubleshooting was one of the most valuable parts of the project.</p><h4>Bash displayed a &gt; continuation prompt</h4><p>My first ssh-keygen command contained an unmatched double quotation mark, which caused Bash to display the &gt; continuation prompt instead of executing the command.</p><p>I used Ctrl+C to cancel the incomplete command and entered it again with matching quotation marks.</p><h4>Git reported that the upstream was gone</h4><p>Because the cloned GitHub repository was initially empty, origin/main did not exist yet. Git therefore reported that the upstream branch was gone.</p><p>The first git push -u origin main created the remote main branch and established origin/main as the upstream tracking branch.</p><h4>GitHub reported successful authentication but no shell access</h4><p>When I tested the SSH connection, GitHub confirmed successful authentication but also reported that it does not provide shell access.</p><p>This was expected behavior. GitHub accepts SSH connections for Git operations, but it does not provide a general interactive server shell.</p><h4>Commit identity did not authenticate the push</h4><p>The Git user.name and user.email settings identify the author stored in commit metadata, but they do not authenticate access to GitHub.</p><p>SSH authentication solved a different problem by proving that the local connection was authorized to access the GitHub account and repository.</p><h3>Security and safety decisions</h3><p>Security was an important part of the workflow, especially when configuring authentication between the local Linux environment and GitHub.</p><ul><li>Used an Ed25519 SSH key pair for GitHub authentication while keeping the private key on the local system.</li><li>Kept the private SSH key out of the Git repository and added only the public key to GitHub.</li><li>Verified the SSH connection before changing the repository remote and pushing the project.</li><li>Kept passwords, private SSH keys, access tokens, and other sensitive credentials out of Git commits.</li></ul><h3>Resource cleanup and cost awareness</h3><p>This project did not create temporary cloud resources, but cleanup and repository hygiene were still part of the workflow.</p><p>I kept the repository focused on the files required for the project and avoided committing credentials, private SSH keys, temporary files, or unrelated local content.</p><p>The SSH private key remained outside the repository, while only the public key was added to GitHub for authentication. Keeping sensitive and temporary data out of version control reduces security risk and keeps the project history clean.</p><h3>What I would add before production use</h3><p>This project is a foundational Git and GitHub lab rather than a production development workflow. A larger team environment would benefit from:</p><ul><li>Branch protection rules for the main branch</li><li>Pull requests and peer review before merging changes</li><li>A documented branching strategy</li><li>Automated testing through CI/CD</li><li>Repository-level security scanning</li><li>A .gitignore file tailored to the application and development environment</li><li>Signed commits where stronger identity verification is required</li><li>Clear contributor and repository ownership documentation</li></ul><p>These controls would extend the same Git fundamentals used in this project into a more structured collaborative software-delivery workflow.</p><h3>What I learned</h3><p>This project helped me move beyond simply memorizing Git commands and understand how the complete local-to-remote workflow fits together.</p><h4>Git has multiple states</h4><p>A file can exist in the working tree, be selected in the staging area, be recorded in the local repository, and still not exist in the remote GitHub repository. Understanding those states made the workflow much easier to reason about.</p><h4>A clean working tree does not prove the remote is current</h4><p>git status can report a clean working tree even when the latest local commit has not been pushed. I learned to validate the local and remote references separately instead of relying on one command.</p><h4>Commit identity and authentication are different</h4><p>Git user.name and user.email identify the author recorded in a commit, while SSH authentication controls access to GitHub. These two concepts solve different problems even though both are part of the same workflow.</p><h4>Validation should be explicit</h4><p>Comparing branch tracking, remote configuration, commit history, and local and remote commit identifiers gave me much stronger evidence that the project was published correctly.</p><h3>Project Repository</h3><p>The complete Git project, workflow documentation, architecture diagram, troubleshooting notes, and validation steps are available here:</p><p><a href="https://github.com/rester12/Git-and-GitHub-Workflow-with-WSL-2-SSH-Authentication">GitHub - rester12/Git-and-GitHub-Workflow-with-WSL-2-SSH-Authentication</a></p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23e14614ccc2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python: Automating Amazon Aurora Cost Control with AWS Lambda and Boto3]]></title>
            <link>https://medium.com/@rester.mcglown/python-automating-amazon-aurora-cost-control-with-aws-lambda-and-boto3-b683187b3c60?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b683187b3c60</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[amazon-aurora]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 20:36:54 GMT</pubDate>
            <atom:updated>2026-09-06T21:03:12.078Z</atom:updated>
            <content:encoded><![CDATA[<h4>How I built, tested, deployed, and safely validated a dry-run-first Lambda function that stops tagged development clusters</h4><p>Cloud resources do not stop generating costs simply because nobody is using them.</p><p>That is especially important in development environments, where a database might be needed during working hours but sit idle at night or over a weekend. Amazon Aurora supports temporarily stopping a DB cluster, which makes it useful for development and test workloads that do not require continuous availability. However, automating that action introduces a serious risk: a poorly designed script could stop the wrong database.</p><p>For this project, I built a Python application that discovers Amazon Aurora clusters, reads their tags, checks their status, and stops only clusters that meet a strict set of safety conditions. I then deployed the same logic to AWS Lambda, tested it against a temporary Aurora MySQL cluster, reviewed the results in CloudWatch Logs, and removed the temporary resources after validation.</p><p>The finished project combines Python, Boto3, AWS CLI, Amazon RDS and Aurora, Lambda, IAM, CloudWatch Logs, pytest, Git, and GitHub Actions.</p><p>The complete project repository is linked at the end of this article.</p><h3>Project Architecture and Objectives</h3><figure><img alt="Architecture diagram showing PowerShell and the AWS CLI interacting with a Python AWS Lambda function that discovers Aurora clusters, reads their tags, applies tag, status, and dry-run safety checks, stops an approved Aurora MySQL cluster, and sends execution logs to CloudWatch." src="https://cdn-images-1.medium.com/max/1024/1*8JMvFX_fxl8TxSHLyxDQKA.png" /><figcaption>Architecture of the Python Aurora cost automation workflow.</figcaption></figure><h3>The problem I wanted to solve</h3><p>The basic requirement sounded straightforward:</p><blockquote>Find Aurora clusters tagged environment=dev and stop them when they are available.</blockquote><p>The unsafe implementation would be equally straightforward:</p><pre>for cluster in all_clusters:<br>    stop_cluster(cluster)</pre><p>That code has no protection against production resources, incorrectly tagged resources, already-stopped clusters, or incomplete AWS responses. Infrastructure automation should fail safely. If the program cannot prove that a resource is an approved target, it should skip the resource.</p><p>I therefore converted the requirement into a guarded workflow:</p><pre>Discover every DB cluster<br>    -&gt; retrieve its ARN<br>    -&gt; retrieve its tags<br>    -&gt; require environment=dev<br>    -&gt; require status=available<br>    -&gt; require DRY_RUN=false<br>    -&gt; request the cluster stop<br>    -&gt; record the result</pre><p>The three most important controls are:</p><ol><li>An exact tag match: environment=dev</li><li>An exact status match: available</li><li>Dry-run mode enabled by default</li></ol><p>All three must align before a real stop request can be sent.</p><h3>Why Aurora is stopped at the cluster level</h3><p>Aurora is organized around a DB cluster that can contain a writer instance and one or more reader instances. The correct lifecycle operation is therefore StopDBCluster, not StopDBInstance.</p><p>AWS stops the Aurora instances as part of the cluster operation while retaining the cluster’s storage and metadata. According to the <a href="https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-cluster-stop-start.html">Amazon Aurora documentation</a>, stopped clusters do not incur DB instance-hour charges, but storage and backup-related charges can continue. AWS also automatically starts a stopped Aurora cluster after seven days so it does not fall behind on required maintenance.</p><p>That seven-day behavior is why real cost-automation workflows are normally scheduled instead of treated as one-time actions. A production design can pair this Lambda function with an Amazon EventBridge scheduled rule or EventBridge Scheduler so the function runs nightly, reevaluates the approved development clusters, and stops a qualifying cluster again after an automatic restart. The same tag, status, dry-run, logging, and IAM controls still apply on every scheduled invocation.</p><h3>Tools and services used</h3><p>I developed the project on Windows with:</p><ul><li>Windows PowerShell</li><li>Visual Studio Code</li><li>Python and a virtual environment</li><li>Boto3 and Botocore</li><li>pytest</li><li>AWS CLI v2 and AWS CloudShell</li><li>Amazon Aurora MySQL-Compatible Edition</li><li>AWS Lambda</li><li>AWS Identity and Access Management (IAM)</li><li>Amazon CloudWatch Logs</li><li>Git, GitHub, and GitHub Actions</li></ul><p>The AWS CLI and Boto3 communicate with the same AWS service APIs, but they serve different roles in the project.</p><p>I used the AWS CLI for explicit administrative tasks such as verifying my identity, inspecting resources, creating the temporary lab infrastructure, deploying Lambda, invoking the function, and checking resource status.</p><p>I used Boto3 — the AWS SDK for Python — inside the application so Python could read AWS responses, evaluate safety rules, and decide whether an action was permitted.</p><p>For example, the CLI equivalent of the discovery request is:</p><pre>aws rds describe-db-clusters</pre><p>The application performs the same type of request through Boto3:</p><pre>rds_client.describe_db_clusters()</pre><p>The important difference is that the Python application can continue from that response into controlled decision-making.</p><h3>Preparing the local Windows environment</h3><p>I created an isolated virtual environment so the project’s dependencies would not interfere with other Python installations:</p><pre>py -m venv .venv<br>.\.venv\Scripts\Activate.ps1<br>python -m pip install --upgrade pip<br>python -m pip install -r .\requirements-dev.txt</pre><p>The development requirements are:</p><pre>boto3[crt]<br>pytest</pre><p>The [crt] extra became important because I authenticated locally with the browser-based AWS CLI login flow. Boto3 recognized the login credential provider but initially raised a missing-dependency error. Installing boto3[crt] added the AWS Common Runtime dependency required to use those temporary credentials.</p><p>Before making AWS calls, I confirmed the active identity and Region:</p><pre>aws sts get-caller-identity<br>aws configure get region</pre><p>This is a habit worth keeping. Before any cloud automation creates, changes, or deletes resources, I want to know exactly which account, identity, and Region it will affect.</p><p>Credentials were never placed in the Python source code or committed to GitHub. Locally, Boto3 used the authenticated AWS credential provider. In Lambda, temporary credentials came from the function’s execution role.</p><h3>Building the application in safe stages</h3><p>I did not begin with permission to stop a live database. I built the project in increasingly powerful stages:</p><pre>Local logic -&gt; Read-only discovery -&gt; Tag evaluation -&gt; Dry run -&gt; Controlled stop -&gt; Lambda deployment</pre><p>This approach made each new capability observable and testable before the next one was added.</p><h4>1. Exact tag matching</h4><p>AWS returns tags as a list of dictionaries. The tag function loops through that data and returns True only when both the key and value match:</p><pre>def has_target_tag(tags, target_key, target_value):<br>    &quot;&quot;&quot;Return True when the requested key-value tag is present.&quot;&quot;&quot;<br>    for tag in tags:<br>        key = tag.get(&quot;Key&quot;)<br>        value = tag.get(&quot;Value&quot;)<br><br>        if key == target_key and value == target_value:<br>            return True<br><br>    return False</pre><p>Using .get() is deliberate. If a tag is incomplete or malformed, the function returns False instead of raising a KeyError. Untagged resources also return False.</p><p>This gives the automation a safe default:</p><blockquote>Missing or uncertain metadata never qualifies a resource for a destructive action.</blockquote><h4>2. Pagination-aware discovery</h4><p>The application uses the RDS paginator rather than assuming all clusters will fit into one API response:</p><pre>def list_db_clusters(rds_client):<br>    &quot;&quot;&quot;Return all DB clusters visible to the current AWS identity.&quot;&quot;&quot;<br>    clusters = []<br>    paginator = rds_client.get_paginator(&quot;describe_db_clusters&quot;)<br><br>    for page in paginator.paginate():<br>        page_clusters = page.get(&quot;DBClusters&quot;, [])<br>        clusters.extend(page_clusters)<br><br>    return clusters</pre><p>Pagination might not matter in a small lab account containing one cluster but supporting it from the beginning prevents the discovery logic from silently ignoring resources as the account grows.</p><h4>3. Retrieving tags through the cluster ARN</h4><p>The cluster’s Amazon Resource Name is required to retrieve its tags:</p><pre>def get_cluster_tags(rds_client, cluster_arn):<br>    &quot;&quot;&quot;Return the tags attached to one DB cluster.&quot;&quot;&quot;<br>    response = rds_client.list_tags_for_resource(<br>        ResourceName=cluster_arn<br>    )<br><br>    return response.get(&quot;TagList&quot;, [])</pre><p>The main workflow checks that an ARN exists before calling this function. A cluster response without an ARN is logged and skipped.</p><h4>4. Separating evaluation from action</h4><p>The decision logic is kept separate from the AWS stop call:</p><pre>def evaluate_cluster(cluster, tags):<br>    identifier = cluster.get(&quot;DBClusterIdentifier&quot;, &quot;unknown&quot;)<br>    engine = cluster.get(&quot;Engine&quot;, &quot;unknown&quot;)<br>    status = cluster.get(&quot;Status&quot;, &quot;unknown&quot;)<br><br>    tag_matches = has_target_tag(<br>        tags,<br>        target_key=&quot;environment&quot;,<br>        target_value=&quot;dev&quot;,<br>    )<br><br>    is_available = status == &quot;available&quot;<br>    would_stop = tag_matches and is_available<br><br>    return {<br>        &quot;identifier&quot;: identifier,<br>        &quot;engine&quot;: engine,<br>        &quot;status&quot;: status,<br>        &quot;tag_matches&quot;: tag_matches,<br>        &quot;is_available&quot;: is_available,<br>        &quot;would_stop&quot;: would_stop,<br>    }</pre><p>The core safety expression is intentionally small:</p><pre>would_stop = tag_matches and is_available</pre><p>Small decision functions are easier to understand and test than AWS calls mixed into a large handler.</p><h4>5. Dry-run mode by default</h4><p>The program reads a DRY_RUN environment variable. If the setting is absent, the function remains in safe mode:</p><pre>def is_dry_run_enabled():<br>    &quot;&quot;&quot;Return True unless DRY_RUN is explicitly disabled.&quot;&quot;&quot;<br>    setting = os.getenv(&quot;DRY_RUN&quot;, &quot;true&quot;).strip().lower()<br><br>    false_values = {&quot;false&quot;, &quot;0&quot;, &quot;no&quot;}<br>    return setting not in false_values</pre><p>This design requires an explicit decision to enable live behavior. A missing variable, capitalization difference, or unexpected value cannot accidentally activate the stop operation.</p><p>The action function then handles three possible outcomes:</p><ul><li>skipped: the cluster failed one or more safety checks</li><li>dry-run: the cluster qualified, but no stop request was sent</li><li>stop-requested: the cluster qualified and live mode was explicitly enabled</li></ul><pre>def apply_cluster_action(rds_client, evaluation, dry_run):<br>    if not evaluation[&quot;would_stop&quot;]:<br>        return {<br>            &quot;action&quot;: &quot;skipped&quot;,<br>            &quot;reason&quot;: &quot;Cluster did not pass all safety checks.&quot;,<br>        }<br><br>    if dry_run:<br>        return {<br>            &quot;action&quot;: &quot;dry-run&quot;,<br>            &quot;reason&quot;: &quot;Cluster qualifies, but no stop request was sent.&quot;,<br>        }<br><br>    response = rds_client.stop_db_cluster(<br>        DBClusterIdentifier=evaluation[&quot;identifier&quot;]<br>    )<br><br>    return {<br>        &quot;action&quot;: &quot;stop-requested&quot;,<br>        &quot;status&quot;: response.get(&quot;DBCluster&quot;, {}).get(<br>            &quot;Status&quot;,<br>            &quot;unknown&quot;,<br>        ),<br>    }</pre><p>The real AWS mutation is isolated behind the two independent guards.</p><h3>Testing without stopping real databases</h3><p>Infrastructure automation needs tests that prove both what the code does and what it refuses to do.</p><p>I wrote 15 pytest tests covering:</p><ul><li>the exact target tag</li><li>the wrong tag value</li><li>a missing tag key</li><li>an empty tag list</li><li>incomplete tag dictionaries</li><li>an available development cluster</li><li>a stopped development cluster</li><li>an available production cluster</li><li>an untagged cluster</li><li>dry-run mode enabled by default</li><li>explicit disabling of dry-run mode</li><li>no stop call during a dry run</li><li>a stop call for a qualified cluster in live mode</li><li>no stop call for an unqualified cluster</li><li>Lambda handler execution in safe mode</li></ul><p>For the action tests, I used a fake RDS client:</p><pre>class FakeRDSClient:<br>    &quot;&quot;&quot;Record stop requests without calling AWS.&quot;&quot;&quot;<br><br>    def __init__(self):<br>        self.stop_requests = []<br><br>    def stop_db_cluster(self, DBClusterIdentifier):<br>        self.stop_requests.append(DBClusterIdentifier)<br><br>        return {<br>            &quot;DBCluster&quot;: {<br>                &quot;DBClusterIdentifier&quot;: DBClusterIdentifier,<br>                &quot;Status&quot;: &quot;stopping&quot;,<br>            }<br>        }</pre><p>This test proves that a qualified cluster reaches the stop method:</p><pre>def test_live_mode_calls_stop_api_for_qualified_cluster():<br>    fake_client = FakeRDSClient()<br><br>    evaluation = {<br>        &quot;identifier&quot;: &quot;aurora-dev-cluster&quot;,<br>        &quot;would_stop&quot;: True,<br>    }<br><br>    result = apply_cluster_action(<br>        fake_client,<br>        evaluation,<br>        dry_run=False,<br>    )<br><br>    assert result[&quot;action&quot;] == &quot;stop-requested&quot;<br>    assert fake_client.stop_requests == [<br>        &quot;aurora-dev-cluster&quot;<br>    ]</pre><p>The companion tests prove that dry-run and unqualified paths never add anything to stop_requests.</p><p>The final local result was:</p><pre>15 passed</pre><p>The same suite also runs automatically through GitHub Actions whenever changes are pushed to the repository.</p><h3>Creating the temporary Aurora test environment</h3><p>After the read-only code and safety tests passed, I created a temporary Aurora MySQL development cluster in us-east-1.</p><p>The cluster was configured with:</p><ul><li>one writer instance</li><li>no public accessibility</li><li>a custom DB subnet group spanning multiple Availability Zones</li><li>deletion protection disabled for the disposable lab</li><li>the exact cluster tag environment=dev</li></ul><p>The cluster tag — not the instance name — was the authorization signal used by the application.</p><p>Before every controlled test, I verified the target directly:</p><pre>aws rds describe-db-clusters `<br>  --db-cluster-identifier aurora-dev-cluster `<br>  --query &quot;DBClusters[0].Status&quot; `<br>  --output text</pre><p>I also retrieved the cluster ARN and inspected its tags with list-tags-for-resource.</p><p>The first live interaction remained a dry run. The application discovered the real cluster and reported:</p><pre>Has environment=dev: True<br>Status is available: True<br>Passed all safety checks: True<br>Action: dry-run</pre><p>The cluster remained available, proving that evaluation did not imply mutation.</p><p>Only after that verification did I temporarily set:</p><pre>$env:DRY_RUN = &quot;false&quot;</pre><p>The next execution requested the stop, and the cluster transitioned through stopping to stopped. I immediately removed the environment variable so subsequent local executions returned to safe mode.</p><p>I then ran the program again. Because the cluster status was no longer available, the program returned Action: skipped and did not make a second stop request.</p><p>That repeated run was important: it demonstrated safe, idempotent behavior for the state being managed.</p><h3>Deploying the workflow to AWS Lambda</h3><p>The same application supports both local execution and Lambda execution.</p><p>The Lambda entry point is intentionally small:</p><pre>def lambda_handler(event, context):<br>    &quot;&quot;&quot;AWS Lambda entry point.&quot;&quot;&quot;<br>    dry_run = is_dry_run_enabled()<br>    rds_client = create_rds_client()<br><br>    run_workflow(<br>        rds_client,<br>        dry_run,<br>    )<br><br>    return {<br>        &quot;statusCode&quot;: 200,<br>        &quot;dryRun&quot;: dry_run,<br>        &quot;message&quot;: &quot;Aurora workflow completed.&quot;,<br>    }</pre><p>The function was deployed with:</p><ul><li>Handler: lambda_function.lambda_handler</li><li>Memory: 128 MB</li><li>Timeout: 30 seconds</li><li>Environment variable: DRY_RUN=true</li></ul><p>AWS documents the Python handler as the method Lambda invokes to process an event. The handler continues until it returns, exits, or reaches its timeout. See <a href="https://docs.aws.amazon.com/lambda/latest/dg/python-handler.html">Defining Lambda function handlers in Python</a>.</p><p>I first invoked the deployed function while the target cluster was stopped. Lambda discovered it but skipped it because the status guard failed.</p><p>For the final end-to-end test, I:</p><ol><li>Started the Aurora cluster.</li><li>Waited for both the cluster and writer to become available.</li><li>Invoked Lambda with DRY_RUN=true.</li><li>Confirmed the cluster remained available.</li><li>Changed the Lambda environment variable to DRY_RUN=false.</li><li>Waited for the configuration update to complete.</li><li>Invoked Lambda once in live mode.</li><li>Immediately restored DRY_RUN=true.</li><li>Confirmed the Aurora cluster reached stopped.</li><li>Invoked Lambda again and confirmed the stopped cluster was skipped.</li></ol><p>This validated the complete path from Lambda’s IAM identity through Boto3, RDS discovery, tag retrieval, status evaluation, the guarded stop request, and logging.</p><h3>IAM: trust and least-privilege permissions</h3><p>The Lambda role required two different policy concepts.</p><p>The trust policy allows the Lambda service to assume the role:</p><pre>{<br>  &quot;Effect&quot;: &quot;Allow&quot;,<br>  &quot;Principal&quot;: {<br>    &quot;Service&quot;: &quot;lambda.amazonaws.com&quot;<br>  },<br>  &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>}</pre><p>The permissions policy grants only the service actions used by the function:</p><pre>rds:DescribeDBClusters<br>rds:ListTagsForResource<br>rds:StopDBCluster<br>logs:CreateLogGroup<br>logs:CreateLogStream<br>logs:PutLogEvents</pre><p>AWS recommends granting workloads temporary credentials through IAM roles and applying least-privilege permissions. See <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html">Security best practices in IAM</a>.</p><p>For this controlled lab, some RDS permissions use Resource: “*”. Before production use, I would narrow the stop permission to approved cluster ARNs wherever the service supports the required resource-level authorization. I would also add an explicit cluster allowlist in the application so tags are not the only approval boundary.</p><h3>Observability with CloudWatch Logs</h3><p>Each execution reports:</p><ul><li>whether dry-run mode is enabled</li><li>the number of clusters discovered</li><li>the cluster identifier and engine</li><li>the status before action</li><li>the retrieved tags</li><li>whether each safety check passed</li><li>the selected action</li><li>the AWS response status when a stop is requested</li></ul><p>AWS Lambda sends function output to CloudWatch Logs when the execution role has the required logging permissions. The default log group follows the /aws/lambda/&lt;function-name&gt; naming pattern. See <a href="https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html">Sending Lambda function logs to CloudWatch Logs</a>.</p><p>These logs gave me evidence for each important state:</p><pre>Action: dry-run<br>Action: stop-requested<br>Action: skipped</pre><p>For a production implementation, I would replace the presentation-oriented print() output with structured JSON logging and add CloudWatch metrics and alarms for stop requests, skipped resources, and API failures.</p><h3>Problems I encountered and how I solved them</h3><p>The troubleshooting was one of the most valuable parts of the project.</p><h4>Boto3 could not use the browser-login credentials</h4><p>The AWS CLI was authenticated successfully, but Boto3 raised:</p><pre>MissingDependencyException:<br>Using the login credential provider requires an additional dependency.</pre><p>Installing the CRT extra resolved it:</p><pre>python -m pip install --upgrade &quot;boto3[crt]&quot;</pre><h4>The default DB subnet group did not exist</h4><p>The first cluster-creation attempt failed with DBSubnetGroupNotFoundFault because the account did not contain an RDS subnet group named default.</p><p>I listed the subnets in the default VPC, confirmed they spanned multiple Availability Zones, created aurora-dev-subnet-group, and retried the cluster command with that subnet-group name.</p><h4>A stopped cluster blocked writer deletion</h4><p>During cleanup, AWS returned InvalidDBClusterStateFault when I attempted to delete the writer while the Aurora cluster was stopped.</p><p>I started the cluster, waited until both the cluster and writer were available, deleted the writer instance, waited for its deletion, and then deleted the empty cluster.</p><p>The lesson was that safe infrastructure cleanup must respect resource dependencies and valid service states.</p><h4>Windows PowerShell rejected utf8NoBOM</h4><p>Windows PowerShell 5.1 does not support that encoding name. Because the Lambda test event contained only {}, ASCII was safe and avoided a byte-order mark:</p><pre>&#39;{}&#39; | Set-Content `<br>  -Path &quot;.\test-event.json&quot; `<br>  -Encoding Ascii</pre><h4>Tests could not import a new function</h4><p>After the test suite was expanded, pytest could not import apply_cluster_action. The updated source had not been saved correctly. I replaced and saved the complete file, verified the function import, cleared Python caches, ran syntax checks, and reran pytest.</p><p>This reinforced a simple debugging sequence: verify the file path, verify the source actually contains the symbol, test the import directly, and only then rerun the full suite.</p><h3>Security and safety decisions</h3><p>Several design choices were intentionally conservative:</p><ul><li>AWS credentials were never hard-coded or committed.</li><li>The database was not publicly accessible.</li><li>The exact tag key and value were required.</li><li>The exact available state was required.</li><li>Dry-run mode was the default locally and in Lambda.</li><li>Live mode was enabled for one monitored invocation and immediately disabled.</li><li>API errors for one cluster were handled without guessing about that resource.</li><li>The stop call was isolated and tested with a fake client.</li><li>Temporary AWS resources were removed after validation.</li></ul><p>The strongest principle behind the project was:</p><blockquote>If the automation is uncertain, skip the resource.</blockquote><h3>Resource cleanup and cost awareness</h3><p>After end-to-end testing, I deleted:</p><ul><li>the Aurora writer instance</li><li>the Aurora cluster</li><li>unwanted snapshots and retained backups, if present</li><li>the Lambda function</li><li>the CloudWatch log group</li><li>the Lambda inline policy and execution role</li><li>the custom RDS subnet group</li><li>temporary local deployment ZIP and invocation-reponse files</li></ul><p>I kept the source code, tests, policy documents, dependency file, and Git history.</p><p>Cleanup matters because a stopped Aurora cluster can still generate storage and backup costs, and AWS automatically restarts a temporarily stopped cluster after seven days. A cloud lab is not complete until its resources have been audited and removed or intentionally retained.</p><h3>What I would add before production use</h3><p>This project is a tested portfolio lab, not a production-ready database control system. A production version should include:</p><ul><li>an explicit allowlist of approved cluster identifiers or ARNs</li><li>tighter IAM resource restrictions</li><li>structured JSON logging</li><li>custom CloudWatch metrics and alarms</li><li>EventBridge scheduling that accounts for Aurora’s automatic restart after seven days</li><li>separate start and recovery automation</li><li>formal approval before live-mode changes</li><li>tag configuration outside the source code</li><li>engine and topology eligibility checks</li><li>integration tests in an isolated AWS account</li><li>infrastructure as code for repeatable deployment</li><li>version-pinned packaged dependencies</li><li>documented ownership, rollback, and incident procedures</li></ul><p>I would also reconsider whether an environment variable alone should enable a production stop action. A stronger design could require both a deployment-time configuration and a separate approval signal.</p><h3>What I learned</h3><p>This project moved beyond writing a short Boto3 script. It required me to think about how cloud automation should behave when data is missing, state changes between requests, credentials are temporary, service dependencies affect cleanup, and the same code must run locally and inside Lambda.</p><p>My main lessons were:</p><ol><li>AWS CLI and Boto3 expose similar service operations, but Boto3 makes those operations programmable.</li><li>Aurora lifecycle actions operate at the cluster level.</li><li>Tags are useful targeting metadata, but destructive automation needs additional safeguards.</li><li>Dry-run behavior should be the default, not an optional afterthought.</li><li>Separating evaluation from action makes code easier to test and review.</li><li>Tests must prove that unsafe paths do not call AWS.</li><li>IAM trust policies and permissions policies solve different problems.</li><li>CloudWatch logs are part of the operational design, not just a debugging convenience.</li><li>Repeated execution is an important part of end-to-end validation.</li><li>Resource cleanup is part of the engineering work.</li></ol><p>The result is a small but complete cloud automation system: designed defensively, tested locally, deployed to Lambda, validated against a real Aurora cluster, observed through CloudWatch, published to GitHub, and cleaned up afterward.</p><h3>Project Repository</h3><p>The full Python source, tests, IAM policy, Lambda trust policy, GitHub Actions workflow, and setup documentation are available here:</p><p><a href="https://github.com/rester12/Python-Aurora-Cost-Automation">GitHub - rester12/Python-Aurora-Cost-Automation: Python - Safety-first AWS Lambda automation that stops available Aurora clusters tagged environment=dev.</a></p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b683187b3c60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Linux Server Health Monitoring and Troubleshooting Toolkit]]></title>
            <link>https://medium.com/@rester.mcglown/building-a-linux-server-health-monitoring-and-troubleshooting-toolkit-726d1bb445ff?source=rss-a905c2ab903f------2</link>
            <guid isPermaLink="false">https://medium.com/p/726d1bb445ff</guid>
            <category><![CDATA[cloud-engineering]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Rester L. McGlown Jr.]]></dc:creator>
            <pubDate>Thu, 30 Jul 2026 01:05:09 GMT</pubDate>
            <atom:updated>2026-09-06T21:03:43.240Z</atom:updated>
            <content:encoded><![CDATA[<h4>How I built an automated monitoring and alerting toolkit on AWS, then deliberately triggered four failures to prove it could detect and report real problems.</h4><h3>Why I Built This</h3><p>My first two AWS and Linux projects focused on building and securing infrastructure. This project focused on the other half of operations: detecting when something goes wrong, sending a useful alert, and having a clear process for responding.</p><p>I built the environment on an Ubuntu 24.04 LTS EC2 instance to expand my Linux experience beyond Amazon Linux. A Bash monitoring script checks disk usage, available memory, CPU load, Nginx, and SSH. A systemd timer runs the checks every five minutes, Amazon SNS sends email alerts, logrotate manages the logs, and Nginx serves an HTML status dashboard.</p><p>The most important part was testing the system. I deliberately stopped a service, created controlled memory pressure, increased disk usage, and introduced an invalid Nginx configuration to confirm what the toolkit could detect — and to identify where its monitoring still had limitations.</p><h3>Architecture at a Glance</h3><p>The solution runs on an Ubuntu 24.04 LTS EC2 instance inside an AWS VPC. A systemd timer starts the Bash health monitor every five minutes. The script checks disk usage, available memory, normalized CPU load, Nginx, and SSH; writes timestamped logs; and updates an HTML status dashboard served by Nginx.</p><p>An IAM instance role allows the monitor to publish only to one Amazon SNS topic. Alerts are sent when a check changes from OK to WARNING and again when it recovers, preventing repeated notifications while the same issue remains active.</p><figure><img alt="AWS architecture diagram showing an Ubuntu EC2 instance running Nginx and a Bash health monitor triggered by systemd every five minutes. The monitor checks disk, memory, CPU, Nginx, and SSH, writes logs, updates an HTML dashboard, and sends failure and recovery alerts through Amazon SNS using a least-privilege IAM role." src="https://cdn-images-1.medium.com/max/1024/1*KVHUaNHKXnqqBTtefCXlzg.png" /><figcaption>Project architecture: automated Linux health monitoring, state-transition alerting, logging, and dashboard reporting on AWS.</figcaption></figure><h3>A More Restrictive Security Foundation</h3><p>I restricted SSH access to my current public IP instead of allowing port 22 from the entire internet. This was more secure, but it also exposed a real operational tradeoff: when my CloudShell source IP changed, I had to update the security-group rule before reconnecting.</p><p>That experience reinforced why production teams often use AWS Systems Manager Session Manager. It removes the need for an inbound SSH rule and avoids maintaining a changing administrative IP address.</p><p>For alerting, I attached a purpose-built IAM role to the EC2 instance. The policy allowed only the sns:Publish action to one SNS topic, so no long-lived AWS credentials had to be stored on the server.</p><h3>Designing the Health Monitor</h3><p>The Bash script performs five operational checks: disk usage, available memory, normalized CPU load, Nginx health, and SSH service status. Every result passes through one logging function so the timestamp and message format remain consistent.</p><p>For memory, I used the available value reported by Linux rather than raw used memory. Linux uses otherwise idle memory for cache, so raw usage can create false warnings. Available memory provides a better estimate of what the kernel can give to a new process.</p><p>CPU required similar care. Load average is a count of queued work, not a percentage, so I compared the one-minute load with the number of processors reported by nproc. I also checked both the Nginx service state and its listening port because an active process does not always prove that the expected network service is available.</p><h3>State-Transition Alerting</h3><p>A monitor that only writes to a log is easy to ignore. I used Amazon SNS so the server could send email notifications, but I did not want the same unresolved problem to generate an alert every five minutes.</p><p>The script stores the previous state for each check. It sends an alert when a check changes from OK to WARNING, stays quiet while the warning continues, and sends a separate recovery message when the check returns to OK.</p><pre>if [ &quot;$status&quot; = &quot;WARNING&quot; ] &amp;&amp; [ &quot;$previous_state&quot; != &quot;WARNING&quot; ]; then<br>    aws sns publish \<br>        --topic-arn &quot;$SNS_TOPIC_ARN&quot; \<br>        --subject &quot;Linux Monitor Alert: $check_name&quot; \<br>        --message &quot;$message&quot;<br>elif [ &quot;$status&quot; = &quot;OK&quot; ] &amp;&amp; [ &quot;$previous_state&quot; = &quot;WARNING&quot; ]; then<br>    aws sns publish \<br>        --topic-arn &quot;$SNS_TOPIC_ARN&quot; \<br>        --subject &quot;Linux Monitor Recovery: $check_name&quot; \<br>        --message &quot;$message&quot;<br>fi</pre><p>I validated both sides of the lifecycle by stopping Nginx, receiving one failure alert, restoring the service, and receiving a separate recovery notification.</p><h3>Automation, Log Rotation, and the Dashboard</h3><p>A systemd timer runs the monitor every five minutes without manual intervention. The service runs as the unprivileged ubuntu user, continuing the least-privilege approach used throughout the project.</p><p>Log rotation produced one of the most useful troubleshooting lessons. logrotate initially rejected the configuration because /var/log is group-writable on Ubuntu. After adding a su directive, the file rename still failed because logrotate’s privilege change did not preserve supplementary groups. Using su ubuntu syslog gave the rotation process the correct operating group and resolved the issue.</p><p>Each monitoring run also regenerates an HTML status page served by Nginx. The dashboard provides a quick visual view of the latest disk, memory, CPU, Nginx, and SSH results without requiring someone to read the raw log.</p><h3>Breaking It on Purpose</h3><p>The toolkit was only useful if it could detect real failures, so I tested four controlled scenarios in the isolated lab.</p><p><strong>Service down:</strong> I stopped Nginx and allowed the systemd timer to detect the failure without manually running the script. The monitor sent one failure alert and a separate recovery alert after the service was restored.</p><p><strong>High memory:</strong> I used a time-limited stress-ng workload. My first test ended before the next scheduled check, which demonstrated how polling intervals can miss short-lived events. A longer retest was captured successfully and raised both memory and CPU warnings.</p><p><strong>High disk usage:</strong> I created a temporary file large enough to cross the warning threshold without completely filling the filesystem. A search limited to /var/log found nothing because the file was under /home/ubuntu. A broader filesystem search located it and revealed a gap in the original runbook.</p><p><strong>Invalid Nginx configuration:</strong> Nginx rejected the broken configuration but safely kept the old process running. The monitor still reported the service and port as healthy, exposing an important blind spot: process health does not prove that the configuration on disk is valid.</p><h3>What This Project Taught Me</h3><p>A health check becomes operationally useful when it sends a meaningful alert to someone who can respond. Logging creates evidence, but alerting creates action.</p><p>State-transition alerting is essential for reducing alert fatigue. When notifications are limited to failures and recoveries, each message communicates a real change instead of repeating the same unresolved condition.</p><p>A running process and a valid service configuration are not the same thing. Nginx’s safe reload behavior exposed this clearly and showed why deeper validation such as nginx -t and HTTP content checks — should be included in a future version.</p><p>The incident runbook also improved through testing. The disk scenario proved that a targeted search should include a broader fallback, reinforcing that operational documentation should evolve when real testing exposes a gap.</p><h3>Skills Demonstrated</h3><p>This project demonstrates hands-on experience with AWS CLI, Amazon EC2, VPC security groups, IAM instance roles, Amazon SNS, Ubuntu server administration, Bash scripting, systemd services and timers, logrotate, Nginx, Linux monitoring, incident runbooks, root-cause analysis, and controlled failure testing.</p><h3>Project Repository</h3><p>The complete monitoring script, systemd files, logrotate configuration, example settings, installation script, and incident runbook are available in the GitHub repository:</p><p><a href="https://github.com/rester12/AWS-Linux-Server-Monitoring-Toolkit">GitHub - rester12/AWS-Linux-Server-Monitoring-Toolkit: AWS EC2 Linux monitoring project featuring Bash health checks, systemd automation, SNS alerts, log rotation, an Nginx status dashboard, and controlled failure testing.</a></p><h3>About the Author</h3><p>Rester L. McGlown Jr. is an AWS Certified Solutions Architect and Cloud Practitioner with more than 15 years of enterprise IT experience supporting mission-critical systems. He is building hands-on Cloud engineering and DevOps experience through projects focused on AWS, Linux, automation, security, monitoring, troubleshooting, and operational documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=726d1bb445ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>