<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Onyeani Njoku Victor on Medium]]></title>
        <description><![CDATA[Stories by Onyeani Njoku Victor on Medium]]></description>
        <link>https://medium.com/@njokuvic?source=rss-1bd5c72e315c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*p5WEcs5PsnbuniEmBxq_5Q.jpeg</url>
            <title>Stories by Onyeani Njoku Victor on Medium</title>
            <link>https://medium.com/@njokuvic?source=rss-1bd5c72e315c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:31:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@njokuvic/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Your Development Environment Doesn’t Need Kubernetes — How We Cut Infrastructure Costs by 96%…]]></title>
            <link>https://njokuvic.medium.com/your-development-environment-doesnt-need-kubernetes-how-we-cut-infrastructure-costs-by-96-1748a7b1e776?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1748a7b1e776</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[digitalocean]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devsecops]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Tue, 02 Jun 2026 06:26:51 GMT</pubDate>
            <atom:updated>2026-06-02T06:39:21.454Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A real conversation, a simple question, and a decision that saved the company hundreds of dollars a month.</em></p><h3>It Started With a Question</h3><p>It wasn’t a grand infrastructure review. It wasn’t a cost-cutting mandate from leadership. It started with a casual conversation with the Senior backend engineer.</p><p>We were talking about the platform — a multi-tenant Loan Management SaaS built on a microservices architecture — when I asked a question that seemed almost too obvious:</p><p><strong>“How many people are actually using this development environment right now?”</strong></p><p>The answer stopped me cold.</p><p>Developers. QA engineers. That was it.</p><p>A handful of internal team members were the sole consumers of an environment running on DigitalOcean Kubernetes Service (DOKS) — the same managed Kubernetes cluster powering production. We were paying for high availability, automatic node scaling, managed control planes, and enterprise-grade orchestration so that a small internal team could test their pull requests.</p><p>We were spending <strong>$300 every single month</strong> for that privilege.</p><h3>The Honest Audit</h3><p>Once the question was asked, we couldn’t unsee it. We did a quick deep dive into what the development environment was actually being asked to do:</p><ul><li><strong>8 microservices</strong> — backend APIs and workers handling the core business logic of the platform</li><li><strong>3 Single Page Applications</strong> — frontend interfaces for different user roles across the product</li><li><strong>A handful of developers</strong> running feature branches and integration tests</li><li><strong>QA engineers</strong> validating functionality before production deploys</li></ul><p>That’s it. No real user traffic. No unpredictable load spikes. No need for self-healing node pools or multi-zone availability. The workload was completely predictable, relatively light, and entirely internal.</p><p>We were using a sledgehammer to crack a nut — and paying for the sledgehammer every month.</p><h3>The Decision: Docker Swarm on a VPS</h3><p>The solution wasn’t complicated. In fact, that was the whole point.</p><p>We provisioned a <strong>Contabo VPS — 4 vCPU, 8GB RAM</strong> — at just under $10 per month. Then we stood up <strong>Docker Swarm</strong> and migrated all 11 services across.</p><p>Docker Swarm was the right call for several reasons:</p><p><strong>1. It’s Kubernetes without the complexity tax</strong> Swarm gives you service orchestration, rolling updates, overlay networking, and secrets management out of the box. For a development environment running predictable internal workloads, that’s everything you need and nothing you don’t.</p><p><strong>2. The mental model is simpler</strong> Developers can reason about a Swarm stack without a Kubernetes certification. A docker stack deploy command is immediately understandable. There&#39;s no ingress controller to debug, no CRDs to manage, no complex autoscaling configurations to tune. The barrier to spin things up or tear them down is dramatically lower.</p><p><strong>3. It maps naturally to Docker Compose</strong> Most developers already write Docker Compose files for local development. Swarm uses the same format with a thin deploy key on top. The learning curve is nearly flat - if your team already knows Compose, they already know 80% of Swarm.</p><h3>What the Architecture Looked Like</h3><p>On the new VPS we ran a single-node Docker Swarm managing all 8 backend microservices and a couple of frontend SPAs. Each service ran in its own Docker Swarm service with proper overlay networking, environment variables injected via an external secrets management platform, and nginx handling routing at the edge.</p><p>The services communicated with each other over the Swarm overlay network by service name — exactly as they did in Kubernetes — so no application code needed to change. From the services’ perspective, nothing had moved.</p><h3>The Overkill Confession</h3><p>Here’s something I’ll be honest about: the 4 vCPU / 8GB RAM spec was slightly overkill.</p><p>After the microservices were properly separated — APIs decoupled from their workers and schedulers, each running as a lean single-process container — the actual memory consumption was remarkably low. The platform hummed along comfortably with significant headroom to spare.</p><p>If I were sizing this today, I’d say <strong>4 vCPU / 6GB RAM is the sweet spot</strong> for a setup like this. You get enough headroom for all services to run simultaneously, a comfortable buffer for builds and deployments, and you’re not over-provisioning unnecessarily.</p><p>That said, going slightly over is always better than going under for a development environment. The last thing you want is developers hitting memory pressure in the middle of a feature sprint.</p><h3>The Numbers</h3><p>Let’s be direct about what this looked like financially:</p><p>Before: Kubernetes — ~$300/month<br>After: VPS — ~$10/month<br>Saving: ~$290/month | ~$3,480/year | 96% reduction</p><p>For a growing fintech SaaS company, that’s not a rounding error. That’s nearly $3,500 per year redirected away from infrastructure nobody needed toward things that actually move the business forward.</p><h3>What We Didn’t Lose</h3><p>This is the part that matters most for engineers evaluating a similar move.</p><p><strong>Developer experience stayed the same.</strong> Developers push code, the CI/CD pipeline builds the image, Docker Swarm pulls it and does a rolling update. From the developer’s perspective, nothing changed except the environment felt snappier because there was no Kubernetes overhead.</p><p><strong>Feature parity was complete.</strong> Every service that ran in DOKS ran in Swarm. Same environment variables, same secrets, same networking topology. Services could still discover and talk to each other by name over the overlay network exactly as they did in Kubernetes.</p><p><strong>QA confidence remained high.</strong> The QA team could test against an environment that faithfully represented production service behaviour. The only difference was the orchestration layer underneath — which QA never interacts with anyway..</p><p><strong>The path to production stayed clean.</strong> Developers validate in the Swarm environment, changes get promoted to production on Kubernetes. The environments are intentionally different — which is actually a healthy forcing function. Engineers can’t accidentally assume that what works in dev will skip proper production checks.</p><h3>The Bigger Lesson</h3><p>The real insight from this exercise wasn’t about Docker versus Kubernetes. It was about asking a question before assuming.</p><p><strong>“Who is actually using this, and what do they actually need?”</strong></p><p>Kubernetes is a phenomenal piece of engineering. It handles complexity that would otherwise require teams of people to manage manually. But complexity has a cost — in dollars, in cognitive overhead, in operational burden. When the workload doesn’t justify that complexity, you’re paying the cost without receiving the benefit.</p><p>A development environment used by a small internal team does not need:</p><ul><li>Automatic node scaling</li><li>Multi-zone high availability</li><li>Managed control plane SLAs</li><li>Complex ingress controllers</li><li>Horizontal Pod Autoscalers</li></ul><p>It needs to run your services reliably so developers can ship features. Docker Swarm on a modest VPS does that extraordinarily well, at a fraction of the cost.</p><h3>When Should You Keep Kubernetes for Dev?</h3><p>To be fair, this approach isn’t universal. There are genuine cases where Kubernetes in development makes sense:</p><ul><li>Your team is actively building Kubernetes operators or controllers and needs to test against the real API</li><li>Your production setup uses CRDs or custom resources that Swarm can’t replicate</li><li>You have a very large distributed team where the overhead of managing a Swarm setup exceeds the cost of managed Kubernetes</li><li>Your services have complex Kubernetes-native dependencies that can’t be abstracted away</li></ul><p>If none of those apply to you — and for most early to mid-stage companies they won’t — the simpler path is almost always the right one.</p><h3>What You Can Do Today</h3><p>If this story resonates, here’s a practical starting point:</p><ol><li><strong>Ask the question.</strong> Who is using your development environment right now? What are they actually doing with it?</li><li><strong>Audit the cost.</strong> Pull up your cloud bill. Is what you’re spending proportional to what that environment delivers?</li><li><strong>Evaluate Swarm.</strong> If you’re already writing Docker Compose files for local development, you’re 80% of the way to a Swarm setup. The deploy key and docker stack deploy are the remaining 20%.</li><li><strong>Right-size the VPS.</strong> For a setup similar to the aforementioned — multiple microservices, a few frontends, internal team only — 4 vCPU / 6GB RAM is a solid target. Contabo, Hetzner, and DigitalOcean Droplets are all good options in this range.</li><li><strong>Keep production on Kubernetes.</strong> The savings come from recognising that not every environment has production-level requirements. Production absolutely should have the reliability, scalability, and observability that Kubernetes provides. Dev doesn’t need to match it.</li></ol><h3>Final Thought</h3><p>The best infrastructure decision isn’t always the most sophisticated one. Sometimes the best decision is the one that asks whether you need the sophistication at all.</p><p>We moved 8 microservices and 3 frontend applications off a $300/month Kubernetes cluster onto a $10/month VPS running Docker Swarm. The developers noticed nothing except that things felt a bit snappier. The QA team noticed nothing at all. The cloud bill noticed everything..</p><p><strong>Right-size your environments. Question your assumptions. The savings might surprise you.</strong></p><p><em>If this helped you think differently about your own infrastructure, consider sharing it with your team. And if you’ve done something similar — or decided Kubernetes was worth keeping — I’d love to hear about it in the comments.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1748a7b1e776" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automate Server Monitoring with Prometheus, Grafana, and Node Exporter Using a Bash Script]]></title>
            <link>https://towardsaws.com/automate-server-monitoring-with-prometheus-grafana-and-node-exporter-using-a-bash-script-4ce24d76dde4?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ce24d76dde4</guid>
            <category><![CDATA[prometheus-monitoring]]></category>
            <category><![CDATA[bash-scipt]]></category>
            <category><![CDATA[node-exporter]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[grafana]]></category>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Sun, 23 Nov 2025 15:09:56 GMT</pubDate>
            <atom:updated>2025-11-24T03:48:22.578Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u_61vZbj1c3jxn8EZBi5MQ.jpeg" /><figcaption>Pictorial Illustration</figcaption></figure><p><em>Monitoring a small fleet of servers powering our workload often comes with a paradoxical challenge, we need powerful, industry-standard tools like Prometheus and Grafana, but we want to avoid the overhead of managing these servers with a full-scale Configuration Management tool like Ansible or Chef .</em></p><p><em>My goal was clear: zero unnecessary complexity. I needed a clean, fast, and repeatable method to bootstrap a complete monitoring stack on any new EC2 instance in minutes.</em></p><p><em>This article presents a robust, single-file Bash script that orchestrates the entire process: it handles the native installation of Node Exporter via systemd, spins up the Dockerized Prometheus and Grafana services, and — most critically — bridges the networking gap. The result is a truly resilient, ready-to-go monitoring solution that is lightweight.</em></p><pre>#!/bin/bash<br><br># Exit on any error and treat unset variables as error<br>set -eu<br><br>echo &quot;Setting up monitoring stack with Grafana, Prometheus, and Node Exporter&quot;<br><br># -----------------------------<br># Install Node Exporter directly on the host<br># -----------------------------<br>echo &quot;Installing Node Exporter on the host system...&quot;<br>NODE_EXPORTER_VERSION=&quot;1.5.0&quot;<br><br>cd /tmp<br>wget -q &quot;https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz&quot;<br>tar xzf &quot;node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz&quot;<br>sudo cp &quot;node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter&quot; /usr/local/bin/<br>sudo useradd --no-create-home --shell /bin/false node_exporter || true<br><br># -----------------------------<br># Create systemd service for Node Exporter<br># -----------------------------<br>cat &gt; node_exporter.service &lt;&lt; EOF<br>[Unit]<br>Description=Node Exporter<br>After=network.target<br><br>[Service]<br>User=node_exporter<br>Group=node_exporter<br>Type=simple<br>ExecStart=/usr/local/bin/node_exporter<br><br>[Install]<br>WantedBy=multi-user.target<br>EOF<br><br>sudo mv node_exporter.service /etc/systemd/system/<br>sudo systemctl daemon-reload<br>sudo systemctl enable node_exporter<br>sudo systemctl start node_exporter<br><br># Clean up downloaded files<br>rm -rf &quot;node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64&quot; &quot;node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz&quot;<br><br># -----------------------------<br># Create necessary directories<br># -----------------------------<br>mkdir -p ./prometheus<br>mkdir -p ./grafana/provisioning/datasources<br>mkdir -p ./grafana/provisioning/dashboards<br><br># -----------------------------<br># Create Prometheus configuration of prometheus and node exporter<br># -----------------------------<br>cat &gt; ./prometheus/prometheus.yml &lt;&lt; EOF<br>global:<br>  scrape_interval: 15s<br>  evaluation_interval: 15s<br><br>scrape_configs:<br>  - job_name: &#39;prometheus&#39;<br>    static_configs:<br>      - targets: [&#39;localhost:9090&#39;]<br><br>  - job_name: &#39;host-node-exporter&#39;<br>    static_configs:<br>    - targets: [&#39;host.docker.internal:9100&#39;] <br>EOF<br><br># -----------------------------<br># Create Grafana datasource configuration <br># -----------------------------<br>cat &gt; ./grafana/provisioning/datasources/datasource.yml &lt;&lt; EOF<br>apiVersion: 1<br><br>datasources:<br>  - name: Prometheus<br>    type: prometheus<br>    access: proxy<br>    url: http://prometheus:9090<br>    isDefault: true<br>    editable: false<br>EOF<br><br># -----------------------------<br># Create Grafana dashboard configuration<br># -----------------------------<br>cat &gt; ./grafana/provisioning/dashboards/dashboard.yml &lt;&lt; EOF<br>apiVersion: 1<br><br>providers:<br>  - name: &#39;Default&#39;<br>    orgId: 1<br>    folder: &#39;&#39;<br>    type: file<br>    disableDeletion: false<br>    updateIntervalSeconds: 10<br>    allowUiUpdates: true<br>    options:<br>      path: /var/lib/grafana/dashboards<br>      foldersFromFilesStructure: true<br>EOF<br><br># -----------------------------<br># Create Docker Compose file (without Node Exporter)<br># -----------------------------<br>cat &gt; docker-compose.yml &lt;&lt; EOF<br>version: &#39;3&#39;<br><br>services:<br>  prometheus:<br>    image: prom/prometheus:latest<br>    container_name: prometheus<br>    restart: unless-stopped<br>    volumes:<br>      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml<br>    command:<br>      - &#39;--config.file=/etc/prometheus/prometheus.yml&#39;<br>    ports:<br>      - &quot;9090:9090&quot;<br>    networks:<br>      - monitor<br>    extra_hosts:<br>      - &quot;host.docker.internal:host-gateway&quot;<br><br>  grafana:<br>    image: grafana/grafana:latest<br>    container_name: grafana<br>    restart: unless-stopped<br>    volumes:<br>      - ./grafana/provisioning:/etc/grafana/provisioning<br>    ports:<br>      - &quot;3000:3000&quot;<br>    networks:<br>      - monitor<br>    environment:<br>      - GF_SECURITY_ADMIN_USER=admin<br>      - GF_SECURITY_ADMIN_PASSWORD=admin<br>      - GF_USERS_ALLOW_SIGN_UP=false<br><br>networks:<br>  monitor:<br>    driver: bridge<br>EOF<br><br>echo &quot;Starting Docker containers...&quot;<br>docker-compose up -d<br><br>echo &quot;Monitoring stack deployed successfully!&quot;<br>echo &quot;Grafana: http://host-ip:3000 (login with admin/admin)&quot;<br>echo &quot;Prometheus: http://host-ip:9090&quot;<br>echo &quot;Node Exporter is running on http://host-ip:9100 (managed by systemd service)&quot;</pre><p>With this setup, you now have a <strong>fully automated monitoring stack</strong> running on your servers. Prometheus is collecting metrics, Node Exporter is exposing host-level stats, and Grafana gives you a beautiful, interactive dashboard to visualize everything — all deployed with a single Bash script and a bit of Docker.</p><p>You can now:</p><ul><li>Add more servers by just running the same script on each host.</li><li>Extend your Prometheus configuration to scrape additional services or containers.</li><li>Build custom dashboards in Grafana to monitor exactly what matters most to you.</li></ul><p>This approach keeps things <strong>lightweight, repeatable, and easy to manage</strong> — no Ansible, no complex orchestration, just a simple script that gets the job done. Whether you’re monitoring a handful of servers or scaling up slowly, this stack gives you <strong>visibility and control</strong> over your infrastructure in minutes.</p><h3><strong>Recommendation for a Well-Architected and Distributed Monitoring System</strong>: Centralized Prometheus with Host-Based Node Exporters</h3><p>For a robust and scalable monitoring architecture, it is ideal to run <strong>Node Exporter on every host</strong> you want to monitor, while <strong>running Prometheus and Grafana on a separate, dedicated monitoring server</strong>. To get the most out of this setup, it’s important to ensure that the prometheus.yml file includes the IP addresses of all hosts where <strong>Node Exporter</strong> is running. Additionally, if you are using servers with dynamically assigned IPs that may change when the server restarts, you should consider using <strong>Prometheus service discovery (SD). </strong>This allows Prometheus to automatically track such servers using labels and tags, ensuring your monitoring remains accurate and up-to-date. If you only have static IPs, you don’t need SD. The explained approach has several advantages:</p><ol><li><strong>Avoids Single Points of Failure:</strong> By keeping Prometheus separate from the application hosts, a crash or failure on one of the monitored servers won’t affect your monitoring stack.</li><li><strong>Simplified Host Configuration:</strong> Only Node Exporter runs on the individual servers. You don’t need to deploy the full monitoring stack (Prometheus, Grafana, dashboards, etc.) on each host, reducing resource usage and complexity.</li><li><strong>Centralized Metrics Collection:</strong> Prometheus on a dedicated server can scrape metrics from all Node Exporters across your infrastructure. This allows you to <strong>track all hosts and applications from a single location</strong>.</li><li><strong>Scalability and Maintainability:</strong> Adding new servers is as simple as deploying Node Exporter on the host. Prometheus automatically collects metrics from the new hosts if they are added to its scrape configuration.</li></ol><h3>Conclusion:</h3><p>Setting up a monitoring stack with <strong>Prometheus, Grafana, and Node Exporter</strong> using a simple Bash script reduces complexity and ensures consistency, as the same script can be run on all host machines. By running <strong>Node Exporter </strong>on each host and centralizing <strong>Prometheus and Grafana</strong> on a dedicated monitoring server, you create a <strong>resilient, scalable, and maintainable monitoring architecture</strong>. This approach avoids single points of failure, minimizes resource usage on production servers, and gives you a <strong>clear, centralized view of all your metrics</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ce24d76dde4" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/automate-server-monitoring-with-prometheus-grafana-and-node-exporter-using-a-bash-script-4ce24d76dde4">Automate Server Monitoring with Prometheus, Grafana, and Node Exporter Using a Bash Script</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating MongoDB Backups in Kubernetes: Mastering Jobs and CronJobs for Data Resilience]]></title>
            <link>https://njokuvic.medium.com/automating-mongodb-backups-in-kubernetes-mastering-jobs-and-cronjobs-for-data-resilience-f08c844dcc30?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f08c844dcc30</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Wed, 12 Nov 2025 04:42:15 GMT</pubDate>
            <atom:updated>2025-11-12T04:56:38.015Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*imgfee-Ybl-K0aaoXT4Nlg.jpeg" /><figcaption>Automated MongoDB Backup Architecture Flow</figcaption></figure><p>As a DevOps engineer, you learn quickly that managing application deployment is only half the battle; the other half is ensuring <strong>data resilience</strong>. When moving our critical MongoDB service into a Kubernetes cluster, my priority shifted to automating reliable, scheduled backups.</p><p>Initially, I looked at the simple Kubernetes Job object. the tool for carrying out one-time tasks. However, daily operations require recurrence, leading me to the essential batch processor: the CronJob.</p><p>This article walks through the exact Kubernetes manifests and the architectural shift required to move from a manual backup trigger to a fully automated, daily data protection routine, using a<strong> PersistentVolumeClaim (PVC) </strong>for guaranteed storage.</p><h3><strong>Running MongoDB on Kubernetes with StatefulSets and Persistent Storage:</strong></h3><p>First, we need to run a MongoDB instance inside our Kubernetes cluster. We can accomplish this by applying the following YAML configuration, which spins up MongoDB using a StatefulSet along with a supporting Service.</p><p>mongodb-deployment.yaml</p><pre>apiVersion: apps/v1<br>kind: StatefulSet<br>metadata:<br>  name: mongodb<br>spec:<br>  serviceName: &quot;mongodb&quot;<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: mongodb<br>  template:<br>    metadata:<br>      labels:<br>        app: mongodb<br>    spec:<br>      containers:<br>      - name: mongodb<br>        image: mongo:latest<br>        ports:<br>        - containerPort: 27017<br>        volumeMounts:<br>        - name: mongodb-data<br>          mountPath: /data/db<br>  volumeClaimTemplates:<br>  - metadata:<br>      name: mongodb-data<br>    spec:<br>      accessModes: [ &quot;ReadWriteOnce&quot; ]<br>      resources:<br>        requests:<br>          storage: 1Gi<br><br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: mongodb<br>spec:<br>  ports:<br>  - port: 27017<br>    targetPort: 27017<br>  selector:<br>    app: mongodb</pre><p>Applying the above deployment:</p><pre>kubectl apply -f mongodb-deployment.yaml</pre><h3>The Persistence Problem: Guarding Against Ephemeral Storage</h3><p>The core philosophy of Kubernetes dictates that Pods and containers are <strong>ephemeral</strong> and disposable. Any data saved inside a Pod’s temporary filesystem is lost the moment the Pod is terminated, restarted, or evicted.</p><p>To protect our MongoDB backups, we need three things:</p><ol><li><strong>A Backup Command:</strong> mongodump to extract the data.</li><li><strong>A Durable Destination:</strong> A storage volume that exists outside the Pod.</li><li><strong>A Scheduled Trigger:</strong> The CronJob to run the task every midnight.</li></ol><p>We start by solving the durable destination problem with a <strong>PersistentVolumeClaim (PVC)</strong>.</p><h3>Phase1:Defining the Persistent Volume Claim (PVC) and Job</h3><p>The<strong> PersistentVolumeClaim (PVC)</strong> provides durable storage — similar to an AWS EBS volume or Azure Disk — but in this setup, we are using a standard Kubernetes storage class. This guarantees that backup files persist safely, even if the backup Pod fails or restarts.</p><p>Before scheduling automated daily backups, we first need to verify that the core backup command works correctly. A <strong>Kubernetes Job</strong> is ideal for this because it guarantees that the container task runs to completion.</p><p>I deployed the Job once to confirm that the mongodump command successfully connected to the MongoDB service (mongodb-service) and wrote data to the mounted PVC.</p><p>mongodb-backup-job.yaml</p><pre>apiVersion: v1<br>kind: PersistentVolumeClaim<br>metadata:<br>  name: mongo-backup-pvc<br>spec:<br>  # ReadWriteOnce: Can be mounted read-write by a single node.<br>  accessModes:<br>    - ReadWriteOnce<br>  resources:<br>    requests:<br>      # 1Gigabytes of storage being requested for backups. This size should fit your data volume.<br>      storage: 1Gi<br>  storageClassName: standard<br>---<br>apiVersion: batch/v1<br>kind: Job<br>metadata:<br>  name: mongodb-onetime-backup<br>spec:<br>  # If the container exits with an error (non-zero code), it will be restarted up to 4 times.<br>  backoffLimit: 4 <br>  template:<br>    spec:<br>      containers:<br>      - name: mongo-backup<br>        image: mongo:latest<br>        command: [&quot;/bin/bash&quot;, &quot;-c&quot;]<br>        args:<br>        # Executes mongodump, connecting to the &#39;mydatabase&#39; via the &#39;mongodb-service&#39;<br>        - &quot;mongodump --uri=mongodb://mongodb-service:27017/mydatabase --out=/backup-data &amp;&amp; echo One-time Backup complete.&quot;<br>        volumeMounts:<br>        - name: backup-storage<br>          mountPath: /backup-data<br>      restartPolicy: OnFailure # Essential for enabling Job retries<br>      volumes:<br>      - name: backup-storage<br>        persistentVolumeClaim:<br>          claimName: mongo-backup-pvc # Mounts the durable storage volume</pre><p>Applying the Job configuration:</p><pre>kubectl apply -f mongodb-backup-job.yaml</pre><p>Check the status of the Job and the logs to ensure the backup was completed successfully:</p><pre>kubectl get jobs<br>kubectl get pods<br>kubectl logs &lt;name of pod&gt;</pre><h3>Phase Two: The Daily Routine with CronJob</h3><p>The immediate problem with the Job was its nature: it only runs once. Since our data is changing constantly, we needed the task to execute reliably every day at midnight.</p><p>The <strong>CronJob</strong> object acts as the scheduler. It defines a recurring schedule (using standard cron syntax) and, at the specified time, creates and launches a new instance of the underlying <strong>Job</strong> object.</p><h3>The CronJob Definition (Daily Backup)</h3><p>This manifest handles the entire automation, guaranteeing that a new backup Job is kicked off at midnight (0 0 * * *) every day.</p><p>mongodb-backup-cronjob.yaml</p><pre>apiVersion: batch/v1<br>kind: CronJob<br>metadata:<br>  name: mongodb-daily-backup<br>spec:<br>  # Cron Schedule: &quot;0 0 * * *&quot; This schedule makes the job execute daily<br>  schedule: &quot;0 0 * * *&quot; <br>  jobTemplate:<br>    spec:<br>      # Job settings are nested here. If this scheduled Pod fails, it will retry 4 times.<br>      backoffLimit: 4 <br>      template:<br>        spec:<br>          containers:<br>          - name: mongo-backup<br>            image: mongo:latest<br>            command: [&quot;/bin/bash&quot;, &quot;-c&quot;]<br>            args:<br>            - &quot;mongodump --uri=mongodb://mongodb-service:27017/mydatabase --out=/backup-data &amp;&amp; echo Daily Backup complete.&quot;<br>            volumeMounts:<br>            - name: backup-storage<br>              mountPath: /backup-data<br>          restartPolicy: OnFailure<br>          volumes:<br>          - name: backup-storage<br>            persistentVolumeClaim:<br>              claimName: mongo-backup-pvc # Mounts the durable storage volume</pre><p>Applying the CronJob configuration:</p><pre>kubectl apply -f mongodb-backup-cronjob.yaml</pre><p>Monitor the CronJob to ensure it creates Jobs as scheduled at midnight:</p><pre>kubectl get cronjobs<br>kubectl get jobs<br>kubectl get pods</pre><h3>Next Steps for Production Systems:</h3><p>The mongodump process within the CronJob Pod should be extended to include a second action which is <strong>transfer</strong>.</p><p>Instead of stopping at local storage, the CronJob container’s command must be modified to sequence the dump, compression, and then the transfer to an external cloud platform like <strong>AWS S3</strong>, <strong>Azure Blob Storage</strong>, or <strong>Google Cloud Storage (GCS)</strong>. This ensures that, even if the entire Kubernetes infrastructure is destroyed, your data is safe and recoverable from a completely independent, highly durable service.</p><h3><strong>Conclusion:</strong></h3><p>With a <strong>StatefulSet</strong> managing the primary MongoDB data, a dedicated <strong>PersistentVolumeClaim (PVC)</strong> handling the backup storage, and the <strong>CronJob</strong> ensuring daily execution, we have successfully decoupled data protection from application deployment. This established architecture guarantees that our backups are not only <strong>durable</strong> and <strong>repeatable</strong> but also self-recovering from transient failures — a non-negotiable standard for maintaining data integrity and business continuity in a Kubernetes environment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f08c844dcc30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cost Optimization on AWS: Event-Driven Automation for Stopping EC2 Instances Using GitHub Webhooks…]]></title>
            <link>https://njokuvic.medium.com/cost-optimization-on-aws-event-driven-automation-for-stopping-ec2-instances-using-github-webhooks-17a459b294f8?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/17a459b294f8</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Mon, 31 Mar 2025 07:02:35 GMT</pubDate>
            <atom:updated>2025-03-31T11:46:06.149Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cost Optimization on AWS: Event-Driven Automation for Stopping EC2 Instances Using GitHub Webhooks, API Gateway, and Lambda</h3><p>Managing a fleet of EC2 instances can be challenging, especially when cost efficiency is a priority. Take a look at the convenience of an automated system that dynamically shuts down idle servers based on events thereby saving you both time and money. This is where event-driven automation comes into play! by leveraging GitHub webhooks, AWS API Gateway, and Lambda functions, you can build a highly efficient and responsive system to manage EC2 instances. This solution not only reduces manual overhead but ensures optimal cost management by stopping instances when they’re no longer needed. Let’s walk through setting up a robust, event-driven architecture for managing EC2 instances, with a focus on simplicity and scalability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zd67AMSCOA15USysHcYCug.jpeg" /><figcaption>Workflow Pipeline of the Infrastructure</figcaption></figure><p><strong>Scenario: </strong>Our DevOps team relies on a development lab to test new application releases. However, management has raised concerns about the increasing costs of running the lab 24/7. To optimize expenses, we need a solution to automatically stop our two EC2 instances after all engineers have clocked out, ensuring resources are only used when needed.</p><p><strong>Prerequisites: </strong>Here are a vital list of requirements needed to getting started along with helpful links.</p><ol><li>AWS Account: Sign Up for an AWS account to deploy resources.</li></ol><p><a href="https://aws.amazon.com/free/">Free Cloud Computing Services - AWS Free Tier</a></p><p>The AWS account is pivotal to having access to AWS Services like API Gateway, Lambda Function and Cloudwatch for effective monitoring.</p><p>2. Postman: A tool for testing APIs, used to send requests to the API endpoint which further invokes the associated Lambda function.</p><p><a href="http://www.postman.com/downloads/">www.postman.com/downloads/</a></p><p>3. Github Account: A GitHub account is essential for utilizing GitHub Actions to schedule and trigger webhook requests to the API endpoint in an automated manner.</p><p><a href="http://www.github.com/desktop/">www.github.com/desktop/</a> — — — — link to download and setup github on your local machine.</p><p><a href="http://www.github.com/features/actions/">www.github.com/features/actions/</a> — — — — This page provides an overview of GitHub Actions and how to use them for CI/CD workflows.</p><p>4. Knowledge of Python: A basic knowledge of python is very ideal to set things up and running.</p><p><strong>Approach:</strong></p><p>Getting started with this task an AWS account is very vital as almost everything will be carried out with the AWS Cloud provider from the API Gateway to the Lambda function as well as provisioning Cloudwatch.</p><h3>Granular IAM Role Creation</h3><p>The process began with creating a granular IAM role that granted only the necessary permissions required by the Lambda function. The permissions included:</p><ul><li><strong>Stop Instances Permission</strong>: To allow the Lambda function to stop instances in AWS.</li><li><strong>CloudWatch Permissions</strong>: To enable Lambda to send logs to CloudWatch for monitoring purposes.</li></ul><p>This ensured that the principle of least privilege was maintained, providing security and minimizing unnecessary access.</p><h3>Lambda Function Creation</h3><p>After creating the IAM role, I proceeded with creating the Lambda function:</p><ol><li><strong>Runtime Selection</strong>: Python 3.9+ was chosen as the runtime environment for the function.</li><li><strong>Execution Role</strong>: The Lambda function’s default execution role was replaced with the previously created IAM role.</li><li><strong>Python Code Deployment</strong>: A Python script was developed and deployed to the Lambda function. The script’s primary logic was to stop specified EC2 instances. below is an excerpt of the code</li></ol><pre>import boto3<br>import json<br><br>region_name = &quot;us-east-1&quot;<br>ec2 = boto3.client(&quot;ec2&quot;, region_name=region_name)<br><br>def lambda_handler(event, context):<br>    try:<br>        # Parse the incoming request body<br>        body = json.loads(event[&quot;body&quot;])<br>        instance_ids = body[&#39;instance_ids&#39;]<br><br>        # Stop the specified EC2 instances<br>        response = ec2.stop_instances(InstanceIds=instance_ids)<br>        stopping_instances = response[&#39;StoppingInstances&#39;]<br>        stopped_ids = [instance[&#39;InstanceId&#39;] for instance in stopping_instances]<br>        message = f&quot;Instances are being stopped: {stopped_ids}&quot;<br>        print(message)<br><br>        # Determine the trigger source<br>        if &#39;httpMethod&#39; in event:<br>            trigger_source = &quot;API Gateway&quot;  # Handles API Gateway logic<br>        else:<br>            trigger_source = &quot;Webhook&quot;  # Handles Webhook logic<br><br>        # Log the trigger source<br>        print(f&quot;Triggered by: {trigger_source}&quot;)<br><br>        # Return the response<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: json.dumps({<br>                &#39;message&#39;: f&quot;{trigger_source} trigger detected&quot;,<br>                &#39;stopped_instances&#39;: stopped_ids<br>            }),<br>            &#39;headers&#39;: {<br>                &#39;Content-Type&#39;: &#39;application/json&#39;<br>            }<br>        }<br><br>    except Exception as e:<br>        # Log and handle exceptions<br>        message = f&quot;Error stopping instances: {str(e)}&quot;<br>        print(message)<br><br>        return {<br>            &#39;statusCode&#39;: 500,<br>            &#39;body&#39;: json.dumps({&#39;message&#39;: message}),<br>            &#39;headers&#39;: {<br>                &#39;Content-Type&#39;: &#39;application/json&#39;<br>            }<br>        }<br><br></pre><h4>Testing the Lambda Function</h4><ul><li>A manual test was conducted by invoking the Lambda function with sample instance IDs.</li><li>The function successfully stopped the specified instances and logged the activity in CloudWatch, confirming its functionality.API Gateway Setup</li></ul><p>With the Lambda function validated, an API Gateway endpoint was configured to trigger the function:</p><ol><li><strong>HTTP API Type</strong>: An HTTP API endpoint was created to facilitate the connection.</li><li><strong>Endpoint Configuration</strong>:</li></ol><ul><li>The endpoint was linked to the Lambda function.</li><li>The <strong>path</strong> /shutdown was added to the endpoint (https://.execute-api.us-east-1.amazonaws.com/shutdown). Without this path, the endpoint would not successfully trigger the Lambda function.</li></ul><h3>Postman Testing</h3><p>To ensure the API Gateway endpoint was functional, <strong>Postman</strong> was used for testing:</p><ol><li><strong>Request Type</strong>: A POST request was selected since the API Gateway was designed to handle POST requests.</li><li><strong>Header Configuration</strong>:</li></ol><ul><li>Key: Content-Type</li><li>Value: application/json</li></ul><p>3.<strong> Request Body</strong>:</p><ul><li>The body format was set to raw and included the instance IDs to be stopped:</li></ul><p>Postman provided a convenient interface for crafting and testing HTTP requests, making it easy to verify that the API Gateway correctly triggered the Lambda function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-g38LtJhfIwfKGUoCDISWw.png" /></figure><p><strong>Note:</strong> The API endpoint was intentionally not fully disclosed for security reasons. This prevents unauthorized users from stopping instances and mitigates potential malicious activities.</p><h3>Webhook Integration</h3><p>Once the API Gateway endpoint was validated using Postman, it was integrated into a webhook system for event-based triggers. This enabled automated invocations of the Lambda function based on specific events or schedules. Simply put if clients do not make request to the servers at night, it can be put to sleep until they are turned back on to serve clients. Below is a simple workflow that carries out the scheduled stop of the instances at 8pm daily.</p><pre>name: Trigger Lambda via Webhook<br><br>on:  <br>  schedule:<br>    - cron: &#39;00 20 * * *&#39; # Runs every day at 8:00 UTC<br><br>jobs:<br>  trigger-lambda:<br>    runs-on: ubuntu-latest<br>    steps:<br>      - name: Send Webhook to API Gateway<br>        run: |<br>          curl -X POST -H &quot;Content-Type: application/json&quot; \<br>          -d &#39;{&quot;instance_ids&quot;:[&quot;i-0b76fa72ccda9bdae&quot;,&quot;i-0ffa983bd0e4faff5&quot;]}&#39; \<br>          https://.execute-api.us-east-1.amazonaws.com/shutdown<br><br></pre><p><strong>Zusammenfassung:</strong></p><p>This setup ensures seamless communication between the API Gateway and Lambda function. By incorporating Postman for initial testing, potential issues were identified and resolved before integrating the webhook for automated triggers. The system is robust, secure, and adheres to best practices in AWS architecture.</p><p><strong>When the Cloud meets the Clouds:</strong></p><p>We’ve been automating within the cloud, but what if our automation could interact with <strong>the world around us</strong>? A new challenge awaits — where data, weather, and automation collide in a way that makes every decision smarter. You won’t want to miss this! Los geht’s!.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17a459b294f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From AWS Console to Terraform: The Benefits of Using AWS2TF for infrastructure as Code]]></title>
            <link>https://njokuvic.medium.com/from-aws-console-to-terraform-the-benefits-of-using-aws2tf-for-infrastructure-as-code-658b12070258?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/658b12070258</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Thu, 06 Mar 2025 15:44:23 GMT</pubDate>
            <atom:updated>2025-03-07T13:10:46.746Z</atom:updated>
            <content:encoded><![CDATA[<p>Lately, I’ve been fascinated by how seamlessly cloud resources managed exclusively through platforms like AWS, GCP, and Azure can be transformed into Terraform code using AWS2TF. Imagine being able to manage your entire infrastructure declaratively — where deploying or tearing down an entire building is as simple as executing a single Terraform command, rather than manually deleting each individual resource. This approach not only streamlines operations but also boosts productivity and minimizes errors. Let’s dive into the practical use of AWS2TF and explore how it empowers us as Cloud Engineers.</p><p><strong>Prerequisites: </strong>Below is a consolidated list of requirements for getting started with AWS2TF, along with helpful links.</p><ol><li>Linux distribution or WSL for Windows:</li></ol><p>Ubuntu: <a href="https://ubuntu.com/download">https://ubuntu.com/download</a>/</p><p>Fedora: <a href="https://getfedora.org/">https://getfedora.org</a>/</p><p>WSL for Windows: Install WSL: <a href="https://ubuntu.com/desktop/wsl">https://ubuntu.com/desktop/wsl</a></p><p>2. Python: Ensure Python is Installed on your Linux environment and not directly on Windows.</p><p><a href="https://www.python.org/downloads/">Download Python</a></p><p>(On Ubuntu, you can install it using: sudo apt-get install python3)</p><p>3. AWS Account: Sign Up for an AWS account to deploy resources.</p><p><a href="https://aws.amazon.com/free/">Free Cloud Computing Services - AWS Free Tier</a></p><p>4. Terraform: Install and configure Terrraform to manage Infrastructure as code.</p><p><a href="https://www.terraform.io/downloads">https://www.terraform.io/downloads</a> — Download and Install Terraform</p><p><a href="https://learn.hashicorp.com/terraform">https://learn.hashicorp.com/terraform</a> — Learn the basics of Terraform to get started.</p><p>5. Code Editor: Preferably VS Code</p><p><a href="https://code.visualstudio.com/">Visual Studio Code - Build what comes next</a></p><p>6. AWS CLI: This is vital in other to communicate with your resources in AWS using the command line.</p><ul><li><a href="https://aws.amazon.com/cli/">AWS CLI</a></li><li><a href="https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html">Installing or updating to the latest version of the AWS CLI</a></li></ul><p>7. AWS2TF Repository and Documentation: Check out the official AWS2TF Repository to get a whole hang of its usage.</p><p><a href="https://github.com/aws-samples/aws2tf">https://github.com/aws-samples/aws2tf</a></p><h3><strong>Approach:</strong></h3><p>Using AWS2TF is a straightforward process once the necessary packages are installed. The official documentation of AWS2TF serves as the main guy throughout the setup and usage. AWS2TF leverages Python scripts to transform AWS resources into Terraform code seamlessly. Notably, it supports the conversion of over 100 AWS resources, simplifying the migration of existing infrastructure to Terraform-managed configurations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uDMAX-GCxwO39IYKJaaJSg.png" /><figcaption><strong>An Illustration of how the repository looks</strong></figcaption></figure><p>It is worth noting that <strong>AWS2TF</strong> is a more Holistic approach than the <strong>terraform import</strong> command to bring resources into Terraform. While <strong>terraform import</strong> allows you to import a resource into the state file, it does not explicitly generate Terraform code for the underlying resources and configurations associated with the imported resource. This limitation makes AWS2TF a more comprehensive solution for converting existing AWS infrastructure into Terraform-managed code.</p><p>Take for example, if<strong> terraform import</strong> is being used to bring in an EC2 instance using its ID, the command will only import the instance itself and its properties, such as the associated VPC or security group. However, it does not explicitly generate the Terraform code for those underlying resources. This limitation makes managing dependencies and related configurations more challenging. <strong>AWS2TF</strong> overcomes this by generating complete Terraform configurations for both the primary resource and its associated components, providing a more holistic view of your Infrastructure.</p><p><strong>A Step-by-Step Guide to Converting an EC2 Instance, Deployed within a VPC and Protected by a Security Group into Terraform Code</strong></p><p>After making sure python is properly installed within a linux environment, the next step is as delightful and effortless as taking a leisurely stroll to your favorite Eiscreme shop.</p><ol><li>Firstly, Before using AWS2TF, you need to obtain and set up the aws2tf.py file in your environment. Typically, you’ll clone or download the repository from GitHub, then ensure the script is in your working directory and executable with Python. This way, you can run it to transform your deployed AWS resources into Terraform code. The README file in the github repositroy will come in handy here.</li><li>Make sure to deploy an ec2 instance within the default vpc on aws and attach a key pair to it for SSH access as well as a security group enabling ports that sits well with you like port 80 for HTTP access and 22 for SSH access. As seen below is an ec2 instance in the running state whose built would be extracted and imported declaratively into terraform.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SVT37FL8vl5NIkNgzA0IfA.png" /><figcaption>Copy of Demo Instance for Practice</figcaption></figure><p>3. Navigate to the aws2tf path where you would find some files like the aws2tf.py file, the generated file that stores the resources converted based on the region it was created. Grabbing the ec2-instance id and running the command <strong>./aws2tf.py -t aws_instance -i i-0db5bad8a6e1f92f2 </strong>starts the conversion process which should take about 10minutes max. The statement <strong>Terraform files &amp; state in sub-directory: generated/tf-396913711544-us-east-1 </strong>clearly signifies that the conversion was successful and we can proceed to seeing the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DIQ52iSGp5N4DpnFDAr2rA.png" /><figcaption>The process until conversion</figcaption></figure><p>4.Navigate to the <strong>generated/tf-396913711544-us-east-1 </strong>directory—this is where the transformed resources are located. For a clearer view, I opened the directory in VS Code. On the left panel, you can see the created resources (I only expanded two folders). All necessary properties for deploying the instance via Terraform, including the VPC, subnet, and other configurations, were successfully extracted. Notably, the state file is also present for proper tracking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ESQyFrqUkTZVf0ckbzml4A.png" /><figcaption>View of the instance and vpc resource extracted from AWS</figcaption></figure><p><strong>Conclusion (Zusammenfassung):</strong><br>The walk through clearly demonstrates that AWS2TF is an invaluable tool in a Cloud Engineer’s arsenal, simplifying the transformation of deployed AWS resources into manageable Terraform code. Although my experience has been limited to AWS, I will look into examining similar solutions for other cloud providers and will share my insights once I do.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=658b12070258" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging Ansible, to deploy NGINX as Reverse Proxy for CTFd and Multi-Database Architectures…]]></title>
            <link>https://njokuvic.medium.com/leveraging-ansible-to-deploy-nginx-as-reverse-proxy-for-ctfd-and-multi-database-architectures-a1649701fc43?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1649701fc43</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Tue, 04 Mar 2025 06:01:36 GMT</pubDate>
            <atom:updated>2025-03-04T06:22:04.660Z</atom:updated>
            <content:encoded><![CDATA[<h3>Leveraging Ansible, to deploy NGINX as Reverse Proxy for CTFd and Multi-Database Architectures Using Terraform</h3><p>Securing your infrastructure isn’t just a box to check — it’s the heartbeat of your system. To supercharge our security, we enlisted Nginx as a reverse proxy — a dynamic gatekeeper that stands between incoming client requests and our CTFD server. Think of Nginx as the secret agent that masks the server’s true identity, ensuring that every request is funneled through its protective barrier. This setup not only hides the CTFD server’s IP address, adding an extra shield against attacks, but also ensures that the server can only be reached via this trusted intermediary. In essence, Nginx transforms into a robust fortress, guarding our core assets while streamlining and securing every interaction. Nginx not only acts as a security agent but also helps with load balancing to the backend resources.</p><p>Security which is a vital part of any Infrastructure To enforce security while setting up this Infrastructure, Nginx was introduced to act as reverse proxy i.e a middle man between the clients sending requests and the ctfd server that process the requests and give back information to the clients. This configuration effectively masks the IP address of the CTFD server which in turn adds an extra layer of protection. The ctfd server can only be accessed through the nginx server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WjKqwIJ3NbU815fKQajXkw.jpeg" /><figcaption><strong>Architecture Diagram</strong></figcaption></figure><p><strong>Terminologies and Applications:</strong></p><ul><li><strong>Amazon Elastic Cloud Compute(EC2): </strong>AWS EC2 Instance is a very vital part of Amazon Web services as it provides virtual servers to host and manage the different components of this Infrastructure from the nginx server which acts as a reverse proxy to the ctfd server that serves the nginx. It also enhances scalability through auto-scaling groups as well as storage with the EBS as a persistent block store and the Instance store for ephemeral storage purpose.</li><li><strong>Virtual Private Cloud(VPC): </strong>VPC is an isolated section of the AWS cloud where resources are defined and deployed in a private and secured environment. It houses public subnets which contains resources that needs to be accessed from the internet and private subnet holds resources that do not need direct access such as backend services or databases. In relation to this project, only the public subnet was used to deploy all resources whilst nginx added an extra security layer such that clients do not access the ctfd server directly but do so through the nginx server.</li><li><strong>Internet gateway: </strong>The Internet gateway is an highly available VPC component that allows communication between resources within the public subnet and the internet. It allows the nginx server connect to the internet to enable its accessibility by clients.</li><li><strong>Ansible: </strong>Ansible is an automation tool used for configuration management as well as automation of tasks. It allows one to define and manage an Infrastructure using simple human readable YAML file called playbooks. it was used to make necessary installations on the various servers according to what they need to run like the nginx installation and every other tasks.</li><li>Terraform: Terraform is an IAC tool(Infrastructure as code) which is a declarative language as it enables one to define and provision data center Infrastructure using a high-level configuration language that allows for management and automation of the lifecycle of our Infrastructure.</li></ul><p>Requirements:</p><ul><li>Code Editor: Visual Studio Code (preferrably)</li><li><a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">AWS CLI v2</a> configured with your IAM credentials for programmatic access</li><li><a href="https://developer.hashicorp.com/terraform/downloads">Terraform</a> CLI within your local device.</li></ul><p>File Structure:</p><pre>.<br>├── ansible<br>│   ├── ansible.cfg<br>│   ├── group_vars<br>│   │   └── all<br>│   ├── inventory<br>│   ├── playbook.yml<br>│   └── roles<br>│       ├── common<br>│       │   └── tasks<br>│       │       ├── main.yml<br>│       ├── ctfd<br>│       │   ├── handlers<br>│       │   │   ├── main.yml<br>│       │   ├── tasks<br>│       │   │   ├── main.yml<br>│       │   └── templates<br>│       │       ├── ctfd.service.j2<br>│       │       ├── env.j2<br>│       ├── mariadb<br>│       │   ├── handlers<br>│       │   │   ├── main.yml<br>│       │   └── tasks<br>│       │       ├── main.yml<br>│       ├── nginx<br>│       │   ├── handlers<br>│       │   │   ├── main.yml<br>│       │   ├── tasks<br>│       │   │   ├── main.yml<br>│       │   └── templates<br>│       │       ├── nginx.conf.j2<br>│       └── redis<br>│           ├── handlers<br>│           │   ├── main.yml<br>│           └── tasks<br>│               ├── main.yml<br>├── cloud-init.yml<br>├── data.tf<br>├── main.tf<br>├── modules<br>│   ├── compute<br>│   │   ├── instances.tf<br>│   │   ├── outputs.tf<br>│   │   ├── variables.tf<br>│   ├── network<br>│   │   ├── network.tf<br>│   │   ├── outputs.tf<br>│   │   ├── variables.tf<br>│   └── securitygroup<br>│       ├── main.tf<br>│       ├── output.tf<br>│       ├── variables.tf<br>├── providers.tf<br>├── templates<br>│   ├── inventory.tpl<br>│   ├── vars.tpl<br>├── terraform.tfstate<br>├── terraform.tfstate.backup<br>├── variables.tf<br></pre><p><strong>Breaking down the tree Diagram above:</strong></p><p>The <strong>Ansible folder</strong> contains several important components that control the behavior of Ansible when managing systems, running roles, and executing playbooks:</p><ol><li><strong>ansible.cfg</strong>: This file is used to describe settings that control Ansible&#39;s behavior during operations. It defines configurations related to roles, playbooks, inventory management, and other operations within Ansible.</li><li><strong>inventory file</strong>: The inventory file defines the list of hosts i.e ip addresses(or groups of hosts) that Ansible will manage and run tasks on. It helps specify the target systems on which tasks or playbooks will be executed.</li><li><strong>playbook.yml</strong>: A playbook is a YAML file that defines a set of tasks, configurations, or processes to be executed on a group of hosts. It acts as a declarative script where you outline the steps you want to automate, such as installing software packages, configuring services, or managing files across remote systems.</li><li><strong>roles</strong>: Roles are a way to organize playbooks and tasks into reusable, modular components. They help break down complex automation workflows, making it easier to maintain large Ansible projects by separating responsibilities into smaller, reusable units.</li></ol><p>ansible.cfg file</p><pre>[defaults]<br>user = ubuntu<br>host_key_checking = False<br>private_key_file = id_ed25519</pre><p>playbook.yml file</p><pre>---<br>- hosts: redis<br>  become: true<br>  roles:<br>    - redis<br>    <br>- hosts: db<br>  become: true<br>  roles:<br>    - mariadb<br><br>- hosts: ctfd<br>  become: true<br>  roles:<br>    - ctfd<br><br>- hosts: nginx<br>  become: true<br>  roles:<br>    - nginx</pre><p>common roles file</p><pre>---<br># tasks file for roles/common<br><br>- name: Wait for apt lock to be released<br>  become: true<br>  ansible.builtin.command: lsof /var/lib/dpkg/lock-frontend<br>  register: result<br>  until: result.rc != 0<br>  retries: 10<br>  delay: 10<br><br>- name: Update apt cache<br>  ansible.builtin.apt:<br>    update_cache: yes<br>  become: true<br><br>- name: Install common packages<br>  ansible.builtin.apt:<br>    name:<br>      - acl<br>    state: present<br>  become: true</pre><p>The <strong>cloud-init.yaml</strong> file is used to define configuration for <strong>Cloud-Init</strong>, a tool that automates the initialization of cloud instances during their first boot. Cloud-Init is often used in cloud environments like AWS, Azure, and OpenStack to configure instances when they are launched. the file below simply ensures docker is installed, configured and ready for use.</p><pre>#cloud-config<br><br># making it retry 5 times to download the docker gpg key, due to some weird bug where one instance always fails to get it.<br>bootcmd:<br>  - until ping -c1 8.8.8.8; do sleep 1; done  # Ensures the network is ready<br>  - mkdir -p /etc/apt/keyrings<br>  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc<br>  - chmod a+r /etc/apt/keyrings/docker.asc<br>  - echo &quot;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable&quot; &gt; /etc/apt/sources.list.d/docker.list<br><br><br>package_update: true<br>#package_upgrade: true<br><br>packages:<br>  - apt-transport-https<br>  - ca-certificates<br>  - curl<br>  - gnupg-agent<br>  - software-properties-common<br>  - lsb-release<br>  - docker-ce<br>  - docker-ce-cli<br>  - containerd.io<br><br>write_files:<br>- path: /etc/docker/daemon.json<br>  content: |<br>    {<br>      &quot;mtu&quot;: 1442<br>    }<br><br>runcmd:<br>  - systemctl restart docker<br>  - systemctl enable docker.service<br>  - systemctl enable containerd.service<br><br># Import SSH keys from my GitHub<br>users:<br>  - name: ubuntu<br>    shell: /bin/bash<br>    groups: [sudo]<br>    sudo: ALL=(ALL) NOPASSWD:ALL<br>    ssh_import_id:<br>      - gh:ludovici96<br>    lock_passwd: true</pre><p>The <strong>data.tf</strong> file in a Terraform project is primarily used to define data sources. These data sources enable you to retrieve and utilize information about existing infrastructure resources that weren&#39;t created by your Terraform setup. This feature is particularly beneficial for integrating with resources managed outside of your Terraform state or when you need to reference already existing cloud resources. The <strong>AMI </strong>and <strong>template files </strong>for Ansible variables were included in the data.tf file in this project.</p><pre>data &quot;aws_ami&quot; &quot;ubuntu&quot; {<br>  most_recent = true<br><br>  filter {<br>    name   = &quot;name&quot;<br>    values = [&quot;ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*&quot;]<br>  }<br><br>  filter {<br>    name   = &quot;virtualization-type&quot;<br>    values = [&quot;hvm&quot;]<br>  }<br> <br>  owners = [&quot;839399074955&quot;]<br>}<br><br>data &quot;template_file&quot; &quot;ansible_vars&quot; {<br>  depends_on = [ module.private_ec2, module.vpc]<br>  template = file(&quot;${path.module}/templates/vars.tpl&quot;)<br><br>  vars = {<br>    redis_internal_ip = module.private_ec2.infra_private_instance_private_ips[&quot;Redis_Instance&quot;]<br>    db_internal_ip    = module.private_ec2.infra_private_instance_private_ips[&quot;db_Instance&quot;]<br>    ctfd_internal_ip  = module.private_ec2.infra_private_instance_private_ips[&quot;ctfd_Instance&quot;]<br>  }<br>}</pre><p>The <strong>main.tf </strong>file is where the overall Infrastructure is defined. It is also termed the root module for the Infrastructure as It can call out various child modules available.</p><pre>terraform {<br>  required_version = &quot;&gt;= 0.14.0&quot;<br><br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 3.0&quot;<br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>  region = var.aws_region<br>}<br><br><br>module &quot;vpc&quot; {<br>  source = &quot;./modules/vpc&quot;<br>  availability_zone = &quot;us-east-1a&quot;<br>  public_subnet_cidr = &quot;10.x.x.0/24&quot;<br>  private_subnet_cidr = &quot;10.x.x.0/24&quot;<br>}<br><br>module &quot;pub_sg&quot; {<br>  vpc_id = module.vpc.vpc_id<br>  count = 1<br>  sg_name = var.sg_name1<br>  source = &quot;./modules/security-groups&quot;  <br>}<br><br>module &quot;pri_sg&quot;{<br>  vpc_id = module.vpc.vpc_id<br>  source = &quot;./modules/security-groups&quot;<br>  count = 3<br>  sg_name = var.sg_name2<br>}<br><br>module &quot;public_ec2&quot; {<br>  count = 1<br>  source = &quot;./modules/ec2&quot;<br>  security_group = module.pub_sg.infra_public_sg_id<br>  subnet_id = module.vpc.public_subnet_ids<br>  associate_public_ip_address = var.associate_public_ip_address_1<br>}<br><br>module &quot;private_ec2&quot; {<br>  count = 3<br>  source = &quot;./modules/ec2&quot;<br>  security_group = module.pub_sg.infra_private_sg_id<br>  subnet_id = module.vpc.public_subnet_ids  <br>  associate_public_ip_address = var.associate_public_ip_address_2<br>}<br><br><br>module &quot;ansible&quot; {<br>  source = &quot;./modules/ansible&quot;<br>  public_ips = module.public_ec2.infra_public_instance_public_ips<br>  private_ips = [module.private_ec2.infra_private_instance_private_ips[*]<br>  ]<br>  private_key_path = &quot;./aaron.pem&quot;<br>  ansible_user      = &quot;ubuntu&quot;<br>}<br><br><br><br><br><br><br><br></pre><p><strong>Modules Folder</strong>: For Terraform modules, the child module is the module that is called out by the root module. It serves as a building block for several resources like the the compute in this case which defines the ec2 instance servers and necessary resources needed to make it work effectively, the network which defines the vpc configuration as well as the security group which defines how traffic is routed to various resources.</p><p>The provider.tf file contains the provider block configuration which houses the version, and the source of the required provider within the Terraform block, specifying that the “hashicorp/aws” provider, version “~&gt; 5.0,” is necessary. The provider block configures the AWS provider with the specified region dynamically set to the value of the “var.aws_region” variable.</p><pre>terraform {<br>  required_version = &quot;&gt;= 0.14.0&quot;<br><br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.0&quot; <br>    }<br>  }<br>}<br><br># Configure the AWS Provider<br>provider &quot;aws&quot; {<br> region = &quot;us-east-1&quot;<br>}</pre><p>Terraform.tfstate/terraform.tfstate.backup are the state file and statefile backup which is a sources of truth and transparency. It clearly tracks the current state and Infrastructure information when a terraform apply is successfully carried out. The security of the state-files can be improved by configuring its storage in an s3 bucket for AWS(Amazon web services), azure Blob storage for Microsoft Azure and Google Cloud Storage for Google cloud based deployment. storing them in cloud based storage services isn’t just enough as state locking is also required to store a lock when a terraform apply or terraform plan operation is running. This prevents other Terraform processes from making changes to the state simultaneously. Dynamo Db is used for AWS, Blob storage with lease mechanism is used for Azure and Firestone for GCP to achieve state locking.</p><p>variables.tf file contains values that are supposed to be hardcoded into the main.tf files directly. This file provides an excellent way to reduce hardcoding and make the configuration more flexible, reusable, and maintainable. Instead of embedding fixed values directly in your Terraform files, you can define variables and reference them throughout your configuration. This approach allows you to manage inputs centrally.</p><pre><br><br>variable &quot;small_instance_type&quot; {<br>    description = &quot;This defines the instance type for the redis and db instance&quot;<br>    type = string<br>    default = &quot;t2.micro&quot; <br>}<br><br>variable &quot;medium_instance_type&quot; {<br>    description = &quot;This defines the instance type for ctfd and nginx&quot;<br>    type = string<br>    default = &quot;&quot;<br>}<br><br>variable &quot;redis_image_id&quot; {<br>    description = &quot;value&quot;<br>    type = string<br>    default = data.aws_ami.ubuntu.id<br>}<br><br>variable &quot;db_image_id&quot; {<br>    description = &quot;value&quot;<br>    type = string<br>    default = data.aws_ami.ubuntu.id  <br>}<br><br>variable &quot;ctfd_image_id&quot; {<br>    description = &quot;value&quot;<br>    type = string<br>    default = data.aws_ami.ubuntu.id  <br>}<br><br>variable &quot;nginx_image_id&quot; {<br>    description = &quot;value&quot;<br>    type = string<br>    default = data.aws_ami.ubuntu.id <br>}</pre><p>After Successful Deployment of the Infrastructure with a terraform apply -auto-approve command, Below is the welcome page of the ctfd server through the DNS address of the nginx server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nrVdiRv_8UVkS4SUck3e7w.png" /><figcaption>Welcome Page of CTFD Server</figcaption></figure><p><strong>Dankeschön for your continous reading, I hope this helps to improve your Infrastructure security going forward without exposing backend resources.</strong></p><p>keep your eyes on this space as more exciting Cloud solutions would be dropped regularly.</p><p><strong>Also If you’re interested in the full code or need additional resources, feel free to reach out, and I’ll provide the GitHub link.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1649701fc43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Augmenting Modularity: Utilizing the power of Modules in Terraform for Efficient Infrastructure…]]></title>
            <link>https://njokuvic.medium.com/augmenting-modularity-utilizing-the-power-of-modules-in-terraform-for-efficient-infrastructure-3fca6fe41bfd?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/3fca6fe41bfd</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Sun, 16 Jun 2024 06:54:32 GMT</pubDate>
            <atom:updated>2024-06-18T02:13:00.089Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Augmenting Modularity: Utilizing the power of Modules in Terraform for Efficient Infrastructure Manangement.</strong></p><p>Regardless of Terraform’s prominence as a leading Infrastructure as Code (IAC) tool, its full importance hasn’t graced the pages of this blog until now. In this post, we embark on a journey to elevate our infrastructure management to the pinnacle of best practices. By harnessing the power of modules within our local environment, we aim to improve our Terraform workflows with reusability, scalability, and version control. Join me as I delve into the realm of Terraform and unleash a vital part of it.</p><p>This demonstration assumes that you are familiar with the Terraform workflow. If you are new to Terraform, you can get started with this <a href="https://developer.hashicorp.com/terraform/tutorials/aws-get-started">Get Started collection</a> .</p><p>For this demonstration, you will need:</p><ul><li>VS code which is a CLI for writing efficient and understandable code</li><li>Terraform v1.2+ installed locally.</li><li>AWS Credentials <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication">configured for use with Terraform</a>. so as to interact with the AWS APIs to fetch necessary updates and information.</li></ul><p>Before going into details, I would talk about what a module is in Teraform. Basically, Terraform modules are well arranged configuration files that encapsulate a piece of an Infrastructure. It contains one or few <strong><em>.tf </em></strong>files that resides in their various directory, this way the complexity of an infrastructure would be reduced, it could be reused and makes understanding of the Infrastructure easier. There are basically three widely known terraform modules:</p><ul><li><strong>Child module</strong>: This is where the main work starts from as it is the module that will be called by another module outside where it was created but still within the scope of the infrastructure creation.</li><li><strong>Root module:</strong> This module serves as the module which calls other modules like the child module and the Public module we will see soon. to call other modules it would use a source path to do that. <strong>e.g source= “/module/vpc/”</strong> for calling a child module or <strong>source = “terraform-aws-modules/vpc/aws”</strong> to call a public module.</li><li>Public modules are published in the terraform registry and could be incorporated into the root module. This will help in having a much deeper insight into modules<a href="https://developer.hashicorp.com/terraform/language/modules"> https://developer.hashicorp.com/terraform/language/modules</a></li></ul><p>Getting started, A separate folder was dedicated to creating the child module which contained two files describing the necessary resources that are needed for proper setup of a VPC(Virtual Private Cloud) system as well as an S3(Simple storage Service) bucket on AWS Cloud.</p><figure><img alt="module" src="https://cdn-images-1.medium.com/max/271/1*S03bBhZzASaNkDPkOdhm3w.png" /><figcaption>Detailed picture showing the module that was created</figcaption></figure><p>As can be clearly seen in the picture above, the VPC child module files has three parts which are the main.tf file that defines the necessary resources to get a VPC up and running based off my specific requirements. an output.tf file was also part of the files in the VPC child modules which will give out some necessary output when the resource has been completely deployed to the aws cloud platform. The output of the <strong>vpc id </strong>was what was needed. The variables.tf file is also another means of of attaining best practices by defining input variables to reducing hardcoding of resources in the main.tf file. The diagram below will illustrate what the variables.tf file in the vpc child module contains. placeholders were used to replace the ip addresses as they are very sensitive information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DE03gOR6wC7vX89CkVfOqA.png" /><figcaption>Desciption of the variables.tf file</figcaption></figure><p>The main.tf file for the VPC module which serves as a building block for the VPC environment contains all the necessary resources needed to spin up a VPC on AWS which will later be called out by the root module when Crucial. Below is a snippet of what the main.tf file looks like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FRYTiwRrHMBgeMOgl2JlrQ.png" /><figcaption>Pictorial Illustration of the main.tf file in the VPC module</figcaption></figure><p>Now, let’s explore the root module side of things which would call out the child module. As earlier explained, the root module is a stand alone module that encapsulates the child modules which are reusable components that can be called to create specific resources. It can call either a child module or a Public module which is published in the terrafom registry. It has a very important part called source which just details the path to which the child module or public module exists. e.g <strong>source = “./modules/vpc”</strong> which describes where the child module of a certain resource exists or <strong>source = “terraform-aws-modules/vpc/aws”</strong> for a public module and it’s worth taking note that the public modules also have designated versions that would be included. below will illustrate how a child module and public module can be called.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*t4pPZqzAEJ78V9FiU6i2dg.png" /><figcaption>Representation of a Child module being called within a root module</figcaption></figure><p>as explained above, we can clearly see the module”vpc” part clearly calling the child module through the source path to fetch all information it has.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OPePnrdq6JtgQ9PylDEdsQ.png" /><figcaption>Sample of a public module being called within a root module</figcaption></figure><p>The Public module exists in the terraform registry and can be called by the root module when needed but some necessary things must be included like the source which tells the cloud provider and the specific resource to be called out as well as the version.</p><p>Here, we draw the curtain on modules. Keep these ideas in mind as you explore Implementation of modules in terraform. Reflect on the potential efficacy and scalability benefits they can bring to your infrastructure. Lets work together to leverage these insights and harness the full power of Terraform modular approach for your next deployment!. Tschüss. you can also refer to the repository via <a href="https://github.com/Aaronthegreat-star/Terraform-modules.git">github</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3fca6fe41bfd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS-Infrastructure-with-Terraform]]></title>
            <link>https://njokuvic.medium.com/aws-infrastructure-with-terraform-bb36f53c8e45?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb36f53c8e45</guid>
            <category><![CDATA[aws-vpc-terraform]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[aws-ec2]]></category>
            <category><![CDATA[aws-load-balancer]]></category>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Mon, 19 Feb 2024 04:11:24 GMT</pubDate>
            <atom:updated>2024-02-19T04:11:24.785Z</atom:updated>
            <content:encoded><![CDATA[<p>Terraform which is one of the leading Infrastructure as a code(IAC) tool was used to build this infrastructure. This project describes the installation of an apache web server on several ec2 instances residing in the private subent of a <strong>VPC.</strong> a shell script that installs the apache webserver on an ubuntu ami was written as the user-data to install the apache webserver at launch of the instances. A Demo s3 bucket was also attached as a backend which could serve various purpose from static website hoisting, storage of files as well as backup of data from the ec2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6NX8HJHiPBmADP8-Ot6gGw.png" /><figcaption>Pictorial representation of the Infrastructure</figcaption></figure><p><strong>Pre-requesites:</strong> There are a few resources needed to get this project up and running before we begin. A handful of them will be explained in detail below.</p><blockquote><strong><em>EC2 Instanc</em></strong><em>e: This is a virtual server/machine provided by Amazon web Services(AWS) that runs on the cloud providing computing capacity. Users can choose the instance type based on their computing needs such as amount of CPU, memory, storage and networking capacity. EC2 instances can easily be scaled up and down based on demand i.e multiple instances can be launched to handle increased workloads and terminated when they are no longer in use. Amazon machine images(AMI) are also very important component of an ec2 instance as it contains all the necessary software updates and configurations which the instances are launched with.</em></blockquote><blockquote><strong><em>Virtual Private Cloud (VPC): </em></strong><em>A</em><strong><em> VPC</em></strong><em> is a secured or isolated section of the AWS cloud Infrastructure where varieties of AWS resources ranging from the almighty EC2 instance to the Dynamo DB which is used as a store for our data can be placed reason being that our resources cannot be accessed by external entities that could cause harm on them. A VPC has several subnets associated with different Availability Zones(AZ) which are basically just bits of the IP address range. Subnets allows for organization of resources within the VPC and control of traffic flow. a very important part of the VPC is the internet gateway(IGW) as it enables effective communication between the VPC and the internet, it allows resources within the VPC to connect to the internet. best believe we have seen how traffic wardens appropriately control traffic on the road and direct users on where to pass. An AWS service by name </em><strong>Route Tables</strong><em> in the VPC environment also does this effectively as it defines how traffic is being directed within the VPC to external networks. Security groups are also very crucial as they control the type of traffic entering and leaving the VPC environment. if an</em><strong> http</strong><em> traffic is to be allowed from external Users into the VPC environment, port </em><strong>80 </strong><em>will be opened as this enables </em><strong>http </strong><em>access whereas for </em><strong>https</strong><em>, port </em><strong>443 </strong><em>will be opened.</em></blockquote><blockquote><strong><em>Load Balancer: </em></strong><em>This is a device that distributes incoming network traffic across several servers preventing a single server from being over worked. This plays very important role in ensuring scalability, reliability and better performance of web applications. There are three varieties available</em> <em>from AWS. below are the three(3) types.</em></blockquote><blockquote><em>i. Application Load Balancer(ALB): This allows for the routing of http and https traffic alone.</em></blockquote><blockquote><em>ii. Network Load Balancer(NLB): Network load balancer enables the access of TCP,UDP and TLS traffic to out resources.</em></blockquote><blockquote><em>iii. Classic Load Balancer(CLB): This is a deprecated version as it isn’t being recognized any longer.</em></blockquote><blockquote><strong><em>Shell Scripting:</em></strong><em> This will be needed for bootstrapping.</em></blockquote><blockquote><em>For this architecture, the Application Load Balancer was being used as it allows http and https traffic into the resources.</em></blockquote><p>Writing of the terraform files: The terraform files were written across different files directory so as not to complicate things. these directories helped simplified things. The following are the various files that were used in the Infrastructure build:</p><ol><li>main.tf: This describes the terraform provider being used. The provider used to build up this architecture was aws with version greater than 4.0 and us-east-1 as the deployment region for the resources.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aIbEd8AfqyCy3pFayFfDGA.png" /><figcaption>pictorial description of the main.tf file</figcaption></figure><p>2. Loadbalancer.tf: This consists of all resources needed for the load balancer to function properly ranging from the target group which is used to route incoming traffic to different target, aws listener which set up to listen for incoming traffic and forward it to the specified target group as well as the listener rule that route incoming traffic to different target group based on specific criteria. such as the source ip.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yMTJx8leHGlGqyVyRdW7yw.png" /><figcaption>Pictorial Illustration of the Loadbalancer.tf file</figcaption></figure><p>3. security-group.tf: The kind of traffic that is permitted to enter and exit the resources in order for the Infrastructure to operate properly is described in this file. Both port 80 and port 443, which allow https and http traffic, were enabled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S85OWsv107Ggont_M7vyfg.png" /><figcaption>Pictorial description of the security-group.tf file</figcaption></figure><p>4. userdata.sh: This is a shell script that basically installs the apache web page on the various ec2 instances so that when launched by accessing the load balancer DNS, it displays the apache web page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kCTtGJLLv5pI1p8Tni84ng.png" /><figcaption>Illustration of the userdata.sh file</figcaption></figure><p>5. parameters.tf: The parameters.tf file contains the launch template which describes the ami as well as the instance type and other important attributes needed for the ec2 instance to boot appropriately. The auto scaling group properties which gives room for bringing up or removing other instances due to the amount of users visiting or leaving the platform follows the definition of the launch template. It is also imperative to note that the script(userdata.sh) was also introduced here as the userdata which provides the ec2 instances the instructions on what to display when launched. an s3 bucket which could be used for different purposes from storage of objects to static website hoisting was also defined and given a role which was attached to the launch template in order for the s3 bucket to get access to the ec2 instances although for this project, the s3 bucket was just a demo and wasn’t really used for anything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UD1UFCxdv9b0Vts3c_TqJg.png" /><figcaption>Pictorial Description of the parameter.tf file</figcaption></figure><p>6. vpc.tf: This file primarily specifies the placement of the different EC2 instances and the transmission of the network connection, enabling them to access the internet and download the packages required to maintain it operational and to serve clients outside of the virtual private cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*14sZoaZz2YEtuGntFP_3Aw.png" /><figcaption>Diagramatical Illustration of the vpc.tf file</figcaption></figure><p>After setting up the files appropriately, a terraform plan was ran within the IDE environment to set the plan for the project after which a terraform apply was ran to deploy the Infrastructure to aws after checking for any error.</p><p>When the Application load balancer DNS was queried, the page below was the final outcome which was created by the aforementioned files.</p><p><strong>Github link</strong> to the project: <a href="https://github.com/Aaronthegreat-star/AWS-Infrastructure-with-terraform.git">https://github.com/Aaronthegreat-star/AWS-Infrastructure-with-terraform.git</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TRC23-XSNNj2A7X7P-GM9w.png" /><figcaption>The result from the script installed on the various ec2 instances</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb36f53c8e45" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Highly Available Three-tier Wordpress Application on AWS with Terraform]]></title>
            <link>https://njokuvic.medium.com/deployment-of-a-wordpress-application-within-a-vpc-virtual-private-cloud-environemnt-a-three-tier-1d14dc281244?source=rss-1bd5c72e315c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d14dc281244</guid>
            <dc:creator><![CDATA[Onyeani Njoku Victor]]></dc:creator>
            <pubDate>Wed, 01 Nov 2023 05:52:08 GMT</pubDate>
            <atom:updated>2023-11-01T11:23:21.772Z</atom:updated>
            <content:encoded><![CDATA[<p>Terraform, which is one of the outstanding Infrastructure as a code(IAC) tool was used owning to its re-usability, scalability as well as aiding collaboration among members of a team. The aws terraform documentation file v2 was<a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs"> referenced</a> in other to deploy this application seamlessly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h7HmVMmno_s8tb1FZUFDQA.png" /><figcaption><strong>Detailed pictorial representation of the Architecture</strong></figcaption></figure><p>A VPC(virtual private cloud) environment was created and modified to fit the desired conditions to deploy the application so as to make it much secured and prevent access to our virtual servers private information hence no public ip address was assigned to each of the ec2 instances.</p><p>A good number of files which are inter related besides the terraform configuration files were created in other not to make our terraform environment clumsy and difficult to understand. these files are listed below with a little bit of explanation as to how they were created.</p><blockquote><strong>vpc.tf</strong>: <em>This contains all the necessary resources needed for the vpc environemnt to function properly. six subnets were created which are one two public subnets as well as four private subnets that takes care of the resources. these resources are the ec2 instances</em><strong>(aws virtual server</strong><em>) that were auto scaled in two distinct subnets and AZs as well as two Aurora DBs in two different subnets and AZs . The public subnet was created to provide a network segment that is directly accesible from the internet while an Internet gateway was also introduced to ensure our resources within our vpc communicate effectively. a nat gateway was also one of the vital resources available as it enables our ec2 instances in the private subnets initiate outbound connections to the internet while remaining hidden from incoming internet traffic. the pictorial illustration below shows a bit of the terraform file that defines the vpc environment.</em></blockquote><blockquote><em>PS: </em><strong>.tf</strong><em> is the file extension used for terraform configuration files</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QP2nf1HBCEM1jLX4KDQNbA.png" /><figcaption><strong>A part of the terraform code showing how the vpc was defined</strong></figcaption></figure><blockquote><strong>security-group.tf<em>: </em></strong><em>The</em><strong><em> </em></strong><em>security-group.tf file was created to simply define the ports on which traffic can be directed in and out of the various resources. From the architecture diagram, it was clearly seen that a load balancer was introduced to balance the request from our users to the various ec2 instances. port </em><strong>80 and 443</strong><em> were opened because we needed to send</em><strong> http and https</strong><em> traffics from our users to the virtual servers through the load balancer. Port 3306 was also enabled which is the port that allows traffic to the database. the virtual servers weren’t left out in the assigning of security groups as port</em><strong> 80 and 65535 </strong><em>were both opened to allow </em><strong>http</strong><em> traffic in and allow all traffic out of our servers by opening the port number range of </em><strong>0 to 65535.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqyX6C1O0L29GvCucrI-sw.png" /><figcaption><strong>Excerpt from the security-group file</strong></figcaption></figure><blockquote><strong>main.tf<em>: </em></strong><em>This file is the brain behind the deployment just like the engine that drives the wheel of a vehicle, it contains the required provider which is aws as well as the region the application will be deployed. </em><strong>eu-east-1 <em>Ireland </em></strong><em>was the desired region in other to serve our clients within this region with little or no latency effect. below shows an image description of this.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*92oUGlcoyXztT7Eu1gVWLA.png" /><figcaption>Detailed image of the main.tf file</figcaption></figure><blockquote><strong>script.sh:</strong> <em>This file contains a series of shell scripts which were used in the user data section of the ec2 instances. it just basically launches the instances with pre installed information ranging from updating the </em><strong>apt</strong><em> package as an </em><strong>ubuntu ami </strong><em>was used, installing a LAMP(Linux Apache My sql and php) stack to spin it up at launch. the wordpress related files were also downloaded and untarred to get the necessary information needed for wordpress to function properly.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fmSHtxjGgj7NO7Q3rkPC9w.png" /><figcaption>Installation of several packages that would work on launch of the ec2 instances</figcaption></figure><blockquote><strong>output.tf: </strong><em>This file basically shows us some information that could be used to access the wordpress webpage within the terraform environment after terraform init,plan and apply(the three important terraform commands to deploy resources) has been carried out. a couple of these resources are:</em></blockquote><blockquote><strong>RDS endpoint</strong>: <em>it gives us a DNS hostname or an ip address that serves as the entry point for connecting to the database hosted within the vpc environment.</em></blockquote><blockquote><strong>LB endpoint:</strong><em> an LB endpoint simply refers to a load balancer endpoint. since load balancers are a critical part in a cloud infrastructure as they distribute incoming traffic across multiple servers to ensure high availability and reliability it is important to be among what is needed in the output section which will return a DNS hostname or ip address so we can access the wordpress web page in the backend servers.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R0nf9MCh-2e5yW4OUiq9Yg.png" /><figcaption>Required output needed when deployment has been done</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-CuOEUnEHspyezIaEj1J-Q.png" /><figcaption>The outputs upon deployment</figcaption></figure><blockquote><strong>db.tf: </strong><em>The db.tf file stands for database.tf, This file contains the necessary data required to deploy two aurora my sql database in two different AZs and two private subnets. three important resources were created.</em></blockquote><blockquote><strong>aws_rds_cluster_instance </strong><em>which just defines the number of aurora-mysql to be created, the size of the aurora-mysql and a little more information which will be clearly seen in the image below</em></blockquote><blockquote><strong>aws_rds_cluster</strong><em> was also one of the resources in the db.tf file which describes everything the aurora-mysql db needs to operate perfectly well.</em></blockquote><blockquote><strong>aws_db_subnet_group </strong><em>was another resource defined to partition each of the aurora-mysql dbs into two private subnets within the vpc.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x1gX0xf58V0SyBpP-jHSoA.png" /><figcaption>an excerpt of the db.tf file</figcaption></figure><blockquote><strong>attribute.tf: </strong><em>This was called the attribute.tf file as it contains all other attributes needed for the word press application to function perfectly well. I will say it best explains the ec2 instance(aws virtual server) properties which are:</em></blockquote><blockquote><strong>Launch template</strong><em> that describes the properties of the ec2 instances that will be placed in the auto scaling group.</em></blockquote><blockquote><strong>Auto scaling group </strong><em>also defines how the ec2 instances will be scaled depending on the demand by the users.</em></blockquote><blockquote><strong>Load balancer</strong><em> properties were also defined which will balance incoming traffic request to several virtual servers present in the auto scaling group.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7h_iMjtiGqcd5psEUYtb9w.png" /><figcaption>A snippet of the attribute.tf file</figcaption></figure><p>Final Result after deployment</p><p>when all necessary resource has been defined, a terraform init is used to intialize a Terraform configuration in a directory after which a terraform apply is used to apply the changes defined in the terraform configuration of the infrastructure. Lastly a terraform plan is initiated to create an execution plan for the infrastructure and tells us any possible error encountered while trying to deploy the infrastructure to aws cloud. after all these were done, the LB endpoint was used to access the wordpress webpage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G-HZZBY8uyD0LGw1ETm7Tg.png" /><figcaption>The word press application up and running</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d14dc281244" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>