<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ijeawele Nkwocha on Medium]]></title>
        <description><![CDATA[Stories by Ijeawele Nkwocha on Medium]]></description>
        <link>https://medium.com/@dnkwocha14?source=rss-91dad3dbfe5f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZbrMmaZQIaa96IqAISsYIQ@2x.jpeg</url>
            <title>Stories by Ijeawele Nkwocha on Medium</title>
            <link>https://medium.com/@dnkwocha14?source=rss-91dad3dbfe5f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 13:12:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dnkwocha14/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How CI/CD Transformed Odyssey Co. from Deployment Chaos to Release Confidence]]></title>
            <link>https://medium.com/@dnkwocha14/how-ci-cd-transformed-odyssey-co-from-deployment-chaos-to-release-confidence-60ef83622ac2?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/60ef83622ac2</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[continuous-deployment]]></category>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Fri, 12 Sep 2025 11:11:22 GMT</pubDate>
            <atom:updated>2025-09-12T11:11:22.861Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TvFR5wncJGQpYZhP66PGTg.png" /></figure><h3>The Breaking Point</h3><p>Once again, there has been a 2-month postponement to the release of Artifact2.0, a product owned by Odyssey Co., a leader in the financial tech industry that helps people send and receive money via social media platforms. The testing team has found that 5 of the packages used are now redundant, and there are many security vulnerabilities to be exploited in them. There have been many complaints from customers about sending money and getting transaction failures, even after the recipients have received it. Each patch has to be manually tested to see what works, but it takes so much time to see a change.</p><p>The software team has already moved on to working on new features for 3.0, and the junior dev, who hasn’t quite mastered documentation, carried out a lot of work on the app. Now he is finding it difficult to remember what he did. The senior developers are busy fixing errors, and the whole team is tired from constantly fighting fires.</p><h3>The Pattern of Problems</h3><p>This is the 5th time this has happened this year; changes are not pushed when they should be, and errors are only discovered when deployment is about to take place. The tests aren’t automated, and even something as simple as changing a variable name in the codebase requires so much effort post-change. A developer has to go through individual directories in the codebase to run tests, then confirm that the validation is good. To run the validations, the application has to be up and running. To confirm that the application is running, you have to open a web browser and confirm that the APIs are hit. A lot of manual effort goes into just testing. If there are 10 changes made, minute ones like a variable name or function name, quality checks will have to be manually carried out to check each time.</p><p>Now it is the 3rd quarter of the year, and many plans Odyssey Co. has made to be achieved haven’t been met, and resources and time are being expended to fix issues.</p><h3>The Final Straw</h3><p>The final straw came when a critical bug made it to production during peak hours, causing thousands of failed transactions. The department has met and needs a more effective way to ensure that tests are done on every push to the shared repository. They also want to reduce the manual effort involved in deploying new features. They want it to be continuous; they want to make a change by 3 pm, and by 3:30 pm, testing has been carried out, and the change is already live. They want to focus all their time and energy on working on a product, and every repeatable step done after pushing should be automated. The solution: Continuous Integration and Continuous Deployment (CI/CD).</p><h3>Continuous Integration and Continuous Deployment: A Better Approach</h3><p>What is CI/CD, and how is it going to drastically change things for Odyssey Co.? These are two independent processes that almost always work hand-in-hand. Let’s look at each individually:</p><h3>Understanding Continuous Integration</h3><p>In a development team, there is usually a shared repository where each developer pushes their code to. But not any kind of code should be pushed there; despite it not being the main branch, every piece of code, even as little as a variable name change, should be tested with respect to the whole code to see if it functions as it should. Continuous integration ensures that unit tests and integration tests run automatically on every code push. To trigger this, a webhook is usually created. A webhook is an automated message sent from one application to another when a specific event occurs, allowing for real-time data synchronisation and automated workflows between different services.</p><p>Remember, for some tests, like unit tests, to work, your application has to be up and running, so you have to make sure that every dependency and package needed for it to run are installed while running the CI.</p><h3>Types of Tests in CI</h3><p>Here are the types of tests usually done in continuous integration:</p><h4>Core Functional Tests</h4><ul><li><strong>Unit tests</strong>: These are the quickest and most frequent tests, run immediately after a new code commit. They verify individual, isolated components (or “units”) of the code, such as a function or method, to ensure it functions as expected.</li><li><strong>Integration tests</strong>: After units have been confirmed to work individually, these tests ensure that different modules or services interact correctly. This may include checking interactions with databases, APIs, or other microservices.</li><li><strong>End-to-End (E2E) tests</strong>: Also known as functional tests, these simulate a complete user journey through the application, from the user interface (UI) to the back-end systems. E2E tests are slower and require more resources, so they are executed later in the pipeline than unit tests.</li><li><strong>Regression tests</strong>: This involves re-running tests on existing functionality to confirm that new code changes have not introduced new bugs or caused any existing features to fail.</li><li><strong>Smoke tests</strong>: A quick set of tests to verify that the most basic and critical functions of the application work after a new build is deployed to a test environment. A failed smoke test halts the rest of the pipeline, as it signals a critical error.</li></ul><h4>Quality and Security Tests</h4><ul><li><strong>Static Application Security Testing (SAST)</strong>: This analysis is performed on the code while it is not running. It checks for common coding errors and security vulnerabilities, such as SQL injections and cross-site scripting (XSS).</li><li><strong>Software Composition Analysis (SCA)</strong>: This involves scanning and analysing any open-source components and third-party dependencies used in the project. The purpose is to identify licensing issues and known security vulnerabilities in those dependencies.</li><li><strong>Code quality checks</strong>: Tools like linters analyse the code for formatting inconsistencies, bad practices, and other non-functional issues. The pipeline can be configured to fail if a quality standard is not met.</li></ul><h3>Continuous Deployment vs. Continuous Delivery</h3><p>Yes, CD stands for two things; they are similar, but they don’t mean the same thing.</p><p>In <strong>Continuous Deployment</strong>, the application or the artifact is deployed automatically after Continuous Integration has been completed. From code push, after all the necessary tests and checks have been done, it is automatically deployed to cloud platforms like AWS, Azure, or Google Cloud, containerised environments using Kubernetes, or serverless functions.</p><p>Whereas, in <strong>Continuous Delivery</strong>, after all the tests and everything has been carried out, it requires human input, someone to click a button before it is then deployed. This can be done if it’s an application that needs to be downloaded from an app store, especially when it involves customers updating their apps by getting a higher version from the app store. So deployment is only done when there are significant updates or bug fixes.</p><p>In continuous deployment, every change is automatically deployed, whereas in continuous delivery, deployment requires someone to click a button.</p><h3>Key Features of CD</h3><ol><li>It ensures you are properly authenticated into the system you are deploying to, such as AWS or Kubernetes.</li><li>It ensures that the code that is deployed is working as expected once deployed.</li><li>As part of the CD process, tests are run to ensure that the application is running correctly.</li></ol><p>CI/CD drastically shortens the Software Development Life Cycle and increases software delivery, which is exactly what Odyssey Co. needs. But how can they achieve this? Using CI/CD tools.</p><h3>Popular CI/CD Tools</h3><p>There are quite a number of tools widely used today for CI/CD, but the ones mentioned in this article are the most popular.</p><h3>1. Jenkins</h3><p>An open-source, self-hosted tool known for its extensive customisation through plugins.</p><h3>2. GitLab CI/CD</h3><p>Integrated within the GitLab platform, offering a robust, security-focused option for GitLab users.</p><h3>3. GitHub Actions</h3><p>A cloud-based tool built into GitHub, ideal for automating workflows directly from your repositories.</p><h3>4. CircleCI</h3><p>A popular cloud-based CI/CD platform known for its speed and auto-scaling capabilities.</p><h3>5. Azure DevOps</h3><p>A comprehensive DevOps platform from Microsoft that includes CI/CD capabilities.</p><h3>Choosing the Right Tool for Your Needs</h3><p>But how can Odyssey Co. choose which of them to use or, if necessary, combine up to two tools to best suit their needs?</p><h3>When to Use Jenkins</h3><p>For Odyssey Co.’s form of business, having a fintech product with a lot of users and sensitive information, having a self-hosted tool for carrying out CI/CD works perfectly. Although it has a steep learning curve, it will mean hiring a professional with ample experience or training existing staff to use it.</p><p>Jenkins should be chosen over other CI/CD tools in scenarios demanding deep customisation, on-premises control, and extensive integration with legacy systems or specialised tools.</p><p>A primary use case is when an organisation requires highly tailored and complex build, test, and deployment pipelines that are not readily achievable with opinionated, cloud-native CI/CD solutions.</p><h3>When to Use GitHub Actions</h3><p>If GitHub is already used as the shared repository and hosting the project, building a CI/CD pipeline using GitHub Actions will be seamless, especially integration with repository events and features. When source code, issues, pull requests, and other development artifacts reside on GitHub, using GitHub Actions provides a streamlined and efficient CI/CD workflow.</p><p>It eliminates the need for managing separate CI/CD infrastructure, reducing operational overhead and accelerating pipeline setup.</p><h3>When to Use GitLab CI/CD</h3><p>Suppose a team or organisation is already using GitLab for version control and issue tracking, or desires to consolidate its development tools into a single platform. In that case, GitLab CI/CD offers seamless integration.</p><p>This eliminates the need to integrate and manage separate tools for each stage of the DevOps pipeline (planning, coding, testing, deploying, monitoring), simplifying workflows and reducing overhead.</p><p>One thing that makes it stand out is the built-in features for project management, security scanning, and container registry, alongside CI/CD, which provide a cohesive environment.</p><h3>When to Use CircleCI</h3><p>CircleCI offers excellent native support for Docker, including Docker layer caching and the ability to build and test applications within containerised environments. This makes it ideal for projects that heavily utilise Docker for development and deployment.</p><p>CircleCI offers in-depth and intuitive integrations with popular version control systems, such as GitHub and Bitbucket, streamlining workflows for teams already using these platforms.</p><h3>When to Use Azure DevOps</h3><p>Organisations heavily invested in the Microsoft ecosystem, utilising Azure cloud services, Office 365, or other Microsoft technologies, can achieve seamless integration and synergy by adopting Azure DevOps. This minimises integration complexities and leverages existing infrastructure and skillsets.</p><h3>Factors for Choosing a CI/CD Tool</h3><h3>Deployment Model</h3><p>Decide between cloud-hosted (e.g., CircleCI, GitHub Actions) or self-hosted (e.g., Jenkins, GitLab CI) options.</p><h3>Project Size &amp; Complexity</h3><p>Choose tools that can handle your specific build and pipeline requirements.</p><h3>Integration</h3><p>Select a tool that integrates well with your existing version control systems, cloud providers, and other development tools.</p><h3>Scalability</h3><p>Ensure the tool can scale with your team’s needs, whether through auto-scaling cloud services or self-hosted solutions.</p><h3>Conclusion</h3><p>Odyssey Co. has now been released from the shackles of manual testing and deployment, with hours of error-prone human involvement now automated. The transformation brings faster deployment cycles, reduced production issues, improved team morale, and most importantly, a reliable service for their customers. Instead of fighting fires, the development team can focus on innovation and building features that matter.</p><p>For organisations still struggling with manual deployments, delayed releases, and production surprises, the message is clear: CI/CD isn’t just a nice-to-have anymore, it’s essential for staying competitive. The tools are mature, the benefits are proven, and the cost of delay grows with every postponed release.</p><p>Start small, automate incrementally, and watch as your development process transforms from chaos to confidence. Your developers will thank you, your customers will notice the difference, and your business will reap the rewards of faster, more reliable software delivery.</p><p>Now CI/CD isn’t done evolving, and there are many exciting developments to expect, such as deeper AI/ML integration for predictive analytics and autonomous pipelines, increased DevSecOps adoption with security embedded throughout the pipeline, and widespread use of GitOps to manage infrastructure and deployments. The future of software development is automated, tested, and continuous, and it’s arriving faster than ever.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60ef83622ac2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Designed a Cloud Architecture for a Multi-Vendor E-Commerce MVP @ BIVY TECH]]></title>
            <link>https://medium.com/@dnkwocha14/how-i-designed-a-cloud-architecture-for-a-multi-vendor-e-commerce-mvp-bivy-tech-e8679bab7abe?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8679bab7abe</guid>
            <category><![CDATA[cloud-architecture]]></category>
            <category><![CDATA[ecs-fargate]]></category>
            <category><![CDATA[ecommerce]]></category>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Fri, 11 Jul 2025 20:25:03 GMT</pubDate>
            <atom:updated>2025-07-11T20:25:03.285Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In today’s digital marketplace, building a scalable and secure multi-vendor e-commerce platform requires careful consideration of cloud architecture patterns, microservices design, and modern DevOps practices. This blog post outlines my approach to designing a cloud-native architecture for a multi-vendor e-commerce MVP, similar to platforms like Amazon, Jumia, or AliExpress.</p><p><strong>My Approach:</strong> I’ve designed a microservices-based architecture using AWS cloud services, focusing on scalability, security, and cost-effectiveness. The solution emphasizes separation of concerns, fault tolerance, and automated deployment pipelines to ensure rapid development and reliable operations.</p><h3>Architecture Diagram</h3><p>I drew my architecture diagram using draw.io.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g1ozh6m2FES4m2tmTLDoog.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BQHNkTh2jdX3tw4VieHFCQ.png" /></figure><h3>⚙️ Core Components</h3><h3>Frontend Hosting</h3><ul><li><strong>Service:</strong> Amazon S3 + CloudFront CDN</li><li><strong>Framework:</strong> Next.js with Server-Side Rendering (SSR)</li></ul><p><strong>Why:</strong> S3 provides cost-effective static hosting while CloudFront ensures global content delivery with sub-100ms latency. Next.js offers excellent SEO capabilities and performance optimization for e-commerce sites.</p><h3>Backend API</h3><ul><li><strong>Service:</strong> Amazon ECS with Fargate (Serverless containers)</li><li><strong>Load Balancer:</strong> Application Load Balancer (ALB) with SSL termination</li><li><strong>API Gateway:</strong> AWS API Gateway for rate limiting and API management</li></ul><p><strong>Microservices:</strong></p><ul><li><strong>User Service</strong> (Node.js): Authentication, user profiles, vendor management</li><li><strong>Product Service</strong> (Django): Product catalog, inventory, search</li><li><strong>Order Service</strong> (Node.js): Order processing, payment integration</li></ul><p><strong>Why:</strong> ECS Fargate eliminates server management while providing container orchestration. Microservices architecture enables independent scaling and deployment of different business components.</p><h3>Database Strategy</h3><ul><li><strong>Primary Database:</strong> Amazon RDS PostgreSQL (Multi-AZ) for transactional data</li><li><strong>NoSQL Database:</strong> Amazon DocumentDB (MongoDB-compatible) for product catalogs and user sessions</li><li><strong>Cache Layer:</strong> Amazon ElastiCache (Redis) for session management and frequently accessed data</li><li><strong>File Storage:</strong> Amazon S3 for product images and static assets</li></ul><p><strong>Why:</strong> PostgreSQL ensures ACID compliance for financial transactions, while DocumentDB handles flexible product schemas. Redis provides sub-millisecond response times for cached data.</p><h3>CI/CD Pipeline</h3><ul><li><strong>Version Control:</strong> GitHub with feature branch workflow</li><li><strong>CI/CD:</strong> GitHub Actions for automated testing and deployment</li><li><strong>Container Registry:</strong> Amazon ECR for Docker image storage</li><li><strong>Deployment:</strong> Blue-green deployment strategy using ECS</li><li><strong>Infrastructure as Code:</strong> AWS CloudFormation templates</li></ul><p><strong>Why:</strong> GitHub Actions integrates seamlessly with AWS services, providing cost-effective CI/CD. Blue-green deployment ensures zero-downtime updates.</p><h3>Security &amp; Monitoring</h3><ul><li><strong>Security:</strong> AWS WAF, VPC with private subnets, IAM roles and policies</li><li><strong>Monitoring:</strong> Amazon CloudWatch for metrics, AWS X-Ray for distributed tracing</li><li><strong>Logging:</strong> Centralized logging with CloudWatch Logs</li><li><strong>Secrets Management:</strong> AWS Secrets Manager for API keys and database credentials</li></ul><h3>Scalability Features</h3><h3>Horizontal Scaling</h3><ul><li><strong>Auto Scaling Groups:</strong> ECS services automatically scale based on CPU/memory utilization</li><li><strong>Database Scaling:</strong> Read replicas for PostgreSQL, MongoDB sharding support</li><li><strong>CDN Scaling:</strong> CloudFront automatically scales globally</li></ul><h3>Performance Optimization</h3><ul><li><strong>Caching Strategy:</strong> Multi-layer caching (Redis, CloudFront, Application-level)</li><li><strong>Database Optimization:</strong> Connection pooling, query optimization, indexing strategy</li><li><strong>API Optimization:</strong> GraphQL for efficient data fetching, pagination for large datasets</li></ul><h3>Security Best Practices</h3><ul><li><strong>Network Security:</strong> VPC with private subnets, security groups with least privilege</li><li><strong>Data Protection:</strong> Encryption at rest and in transit, secure API authentication</li><li><strong>Compliance:</strong> PCI DSS compliance for payment processing, GDPR compliance for user data</li></ul><h3>Cost Optimization</h3><h3>Resource Management</h3><ul><li><strong>Fargate Spot Instances:</strong> Up to 70% cost savings for non-critical workloads</li><li><strong>S3 Intelligent Tiering:</strong> Automatic cost optimization for storage</li><li><strong>Reserved Instances:</strong> RDS reserved instances for predictable database workloads</li><li><strong>Auto Scaling:</strong> Automatic resource adjustment based on demand</li></ul><h3>Monitoring &amp; Alerting</h3><ul><li><strong>Cost Monitoring:</strong> AWS Cost Explorer with budget alerts</li><li><strong>Performance Monitoring:</strong> CloudWatch dashboards for real-time metrics</li><li><strong>Health Checks:</strong> Application Load Balancer health checks with automated recovery</li></ul><h3>What I Learned</h3><h3>Technical Challenges</h3><p>The most significant challenge was balancing cost optimization with performance requirements. Initially, I considered using Amazon EKS for container orchestration, but realized that ECS Fargate provides better cost efficiency for an MVP while maintaining scalability options.</p><h3>Design Decisions</h3><p><strong>Microservices vs Monolith:</strong> I chose microservices architecture to enable independent team development and service-specific scaling. However, I kept the services relatively coarse-grained to avoid the complexity of managing too many small services in an MVP.</p><p><strong>Database Strategy:</strong> The hybrid approach of using both SQL and NoSQL databases was crucial. PostgreSQL handles transactional integrity for orders and payments, while DocumentDB provides flexibility for product catalogs with varying attributes across different vendors.</p><h3>Security Insights</h3><p>Implementing proper security in a multi-vendor environment requires careful consideration of data isolation. I learned that using separate IAM roles for each vendor’s data access, combined with application-level authorization, provides robust security without compromising performance.</p><h3>DevOps Lessons</h3><p>Setting up automated testing in the CI/CD pipeline is crucial for maintaining code quality.</p><h3>Cloud-Native Thinking</h3><p>The biggest insight was understanding that cloud-native design isn’t just about using cloud services, it’s about designing for failure, embracing automation, and building systems that can self-heal and scale automatically.</p><h3>Future Improvements</h3><p>For scaling beyond MVP, I would consider:</p><ul><li>Implementing event-driven architecture using AWS EventBridge</li><li>Adding disaster recovery across multiple AWS regions</li></ul><p>This architecture provides a solid foundation for a multi-vendor e-commerce platform that can handle significant growth while maintaining security, performance, and cost efficiency. The modular design allows for easy addition of new features and services as the business evolves.</p><p>Want an implementation of this?<br>Contact me: <a href="http://mail.to:dnkwocha14@gmail.com/">dnkwocha14@gmail.com</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CzZnrAVgvPrCVCoPK91hew.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8679bab7abe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy 11 Microservices on AWS EKS with Production-Grade Security]]></title>
            <link>https://medium.com/@dnkwocha14/deploy-11-microservices-on-aws-eks-with-production-grade-security-b6296b9edde2?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6296b9edde2</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[helm]]></category>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Fri, 04 Jul 2025 11:07:42 GMT</pubDate>
            <atom:updated>2025-07-14T11:48:54.047Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A complete walkthrough of deploying an e-commerce application with AWS Secrets Manager, External Secrets Operator, and NGINX Ingress</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0QZQCu9CMxl5e3SpyexBmw.png" /></figure><h3>What You’ll Build</h3><p>An e-commerce application with 11 microservices running on AWS EKS, featuring:</p><ul><li>AWS Secrets Manager for secure configuration</li><li>External Secrets Operator for automated secret sync</li><li>NGINX Ingress with security headers and rate limiting</li><li>Horizontal Pod Autoscaling for high-traffic services</li><li>Production-ready Dockerfiles and Kubernetes manifests</li></ul><p><strong>Time:</strong> 2–3 hours | <strong>Cost:</strong> ~$50/month</p><p><strong>Complete code:</strong> <a href="https://github.com/DivineTheAnalyst/microservices-k8s">GitHub Repository</a></p><h3>The Microservices</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*Nehb7g4Zpkr8tPEbOPEuRw.png" /></figure><h3>Step 1: Build Production Containers</h3><p>I started with Google’s microservices demo but rebuilt all the Dockerfiles for production use.</p><h3>Why Custom Dockerfiles?</h3><p>The original containers were demo-quality. I needed:</p><ul><li>Smaller image sizes</li><li>Security hardening</li><li>Multi-stage builds</li><li>Non-root users</li></ul><h3>Example: Frontend Dockerfile</h3><pre># Build stage<br>FROM golang:1.21-alpine AS builder<br>WORKDIR /app<br>COPY go.mod go.sum ./<br>RUN go mod download<br>COPY . .<br>RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o frontend .</pre><pre># Production stage<br>FROM alpine:3.18<br>RUN apk --no-cache add ca-certificates tzdata<br>WORKDIR /root/<br>COPY --from=builder /app/frontend .<br>EXPOSE 8080<br>USER 1000<br>CMD [&quot;./frontend&quot;]</pre><h3>Build Script for All Services</h3><p>Create build_and_push.sh:</p><pre>#!/bin/bash<br>DOCKERHUB_USERNAME=&quot;your-username&quot;<br>VERSION=&quot;v1.0&quot;</pre><pre>SERVICES=(<br>    &quot;frontend&quot; &quot;cartservice&quot; &quot;productcatalogservice&quot;<br>    &quot;currencyservice&quot; &quot;paymentservice&quot; &quot;shippingservice&quot;<br>    &quot;emailservice&quot; &quot;checkoutservice&quot; &quot;recommendationservice&quot;<br>    &quot;adservice&quot; &quot;shoppingassistantservice&quot;<br>)</pre><pre>echo &quot;Building and pushing images...&quot;</pre><pre>for service in &quot;${SERVICES[@]}&quot;; do<br>    echo &quot;Processing $service...&quot;<br>    <br>    docker build -t $DOCKERHUB_USERNAME/$service:$VERSION ./src/$service<br>    docker push $DOCKERHUB_USERNAME/$service:$VERSION<br>    <br>    echo &quot;✅ $service:$VERSION pushed&quot;<br>done</pre><pre>echo &quot;🎉 All images ready!&quot;</pre><p>Run it: chmod +x build_and_push.sh &amp;&amp; ./build_and_push.sh</p><h3>Step 2: Set Up AWS EKS</h3><h3>Install Prerequisites</h3><pre># Install AWS CLI<br>curl &quot;https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip&quot; -o &quot;awscliv2.zip&quot;<br>unzip awscliv2.zip<br>sudo ./aws/install</pre><pre># Install kubectl<br>curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl&quot;<br>chmod +x ./kubectl<br>sudo mv ./kubectl /usr/local/bin</pre><pre># Install eksctl<br>curl -L &quot;https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_Linux_amd64.tar.gz&quot; -o eksctl.tar.gz<br>tar -xvf eksctl.tar.gz<br>sudo mv eksctl /usr/local/bin</pre><h3>Configure AWS</h3><pre># Configure AWS credentials<br>aws configure<br># Enter your Access Key, Secret Key, Region (us-east-1), Output format (json)</pre><h3>Create EKS Cluster</h3><pre>eksctl create cluster \<br>  --name ije-cluster \<br>  --region us-east-1 \<br>  --nodegroup-name ije-nodes \<br>  --node-type t3.medium \<br>  --nodes 2 \<br>  --nodes-min 1 \<br>  --nodes-max 4 \<br>  --managed</pre><pre># Configure kubectl<br>aws eks --region us-east-1 update-kubeconfig --name ije-cluster</pre><pre># Test connection<br>kubectl get nodes</pre><h3>Step 3: Set Up Secret Management</h3><h3>Why AWS Secrets Manager?</h3><p>The microservices need to communicate with each other. Instead of hardcoding service addresses in YAML files, I use AWS Secrets Manager for:</p><ul><li>Centralized secret management</li><li>Automatic rotation capability</li><li>Access logging and auditing</li><li>Encryption at rest and in transit</li></ul><h3>Create Secrets in AWS</h3><pre>aws secretsmanager create-secret \<br>  --name microservices/services \<br>  --secret-string &#39;{<br>    &quot;PRODUCT_CATALOG_SERVICE_ADDR&quot;:&quot;productcatalogservice:3550&quot;,<br>    &quot;CURRENCY_SERVICE_ADDR&quot;:&quot;currencyservice:7000&quot;,<br>    &quot;CART_SERVICE_ADDR&quot;:&quot;cartservice:7070&quot;,<br>    &quot;RECOMMENDATION_SERVICE_ADDR&quot;:&quot;recommendationservice:5002&quot;,<br>    &quot;SHIPPING_SERVICE_ADDR&quot;:&quot;shippingservice:8081&quot;,<br>    &quot;CHECKOUT_SERVICE_ADDR&quot;:&quot;checkoutservice:5050&quot;,<br>    &quot;AD_SERVICE_ADDR&quot;:&quot;adservice:9555&quot;,<br>    &quot;PAYMENT_SERVICE_ADDR&quot;:&quot;paymentservice:3002&quot;,<br>    &quot;EMAIL_SERVICE_ADDR&quot;:&quot;emailservice:5001&quot;,<br>    &quot;SHOPPING_ASSISTANT_SERVICE_ADDR&quot;:&quot;shoppingassistantservice:8005&quot;,<br>    &quot;REDIS_HOST&quot;:&quot;redis:6379&quot;<br>  }&#39; \<br>  --region us-east-1</pre><h3>Install External Secrets Operator</h3><pre>helm repo add external-secrets https://charts.external-secrets.io<br>helm install external-secrets external-secrets/external-secrets \<br>  -n external-secrets-system \<br>  --create-namespace</pre><pre># Verify installation<br>kubectl get pods -n external-secrets-system</pre><h3>Configure Secret Store</h3><p>Create aws-secret-store.yaml:</p><pre>apiVersion: external-secrets.io/v1beta1<br>kind: SecretStore<br>metadata:<br>  name: aws-secrets-store<br>  namespace: online-boutique<br>spec:<br>  provider:<br>    aws:<br>      service: SecretsManager<br>      region: us-east-1<br>      auth:<br>        secretRef:<br>          accessKeyID:<br>            name: aws-credentials<br>            key: access-key-id<br>          secretAccessKey:<br>            name: aws-credentials<br>            key: secret-access-key</pre><h3>Create External Secret</h3><p>Create external-secret.yaml:</p><pre>apiVersion: external-secrets.io/v1beta1<br>kind: ExternalSecret<br>metadata:<br>  name: frontend-config<br>  namespace: online-boutique<br>spec:<br>  refreshInterval: 30s<br>  secretStoreRef:<br>    kind: SecretStore<br>    name: aws-secrets-store<br>  target:<br>    name: frontend-config-secret<br>    creationPolicy: Owner<br>  data:<br>  - secretKey: PRODUCT_CATALOG_SERVICE_ADDR<br>    remoteRef:<br>      key: microservices/services<br>      property: PRODUCT_CATALOG_SERVICE_ADDR<br>  - secretKey: CURRENCY_SERVICE_ADDR<br>    remoteRef:<br>      key: microservices/services<br>      property: CURRENCY_SERVICE_ADDR<br>  # Add all other service addresses</pre><p>Apply the configurations:</p><pre>kubectl create namespace online-boutique<br>kubectl apply -f aws-secret-store.yaml<br>kubectl apply -f external-secret.yaml</pre><h3>Step 4: Deploy the Microservices</h3><h3>Frontend Deployment</h3><p>Create frontend-deployment.yaml:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: frontend-deployment<br>  namespace: online-boutique<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: frontend<br>  template:<br>    metadata:<br>      labels:<br>        app: frontend<br>    spec:<br>      containers:<br>      - name: frontend<br>        image: your-username/frontend:v1.0<br>        ports:<br>        - containerPort: 8080<br>        <br>        # Health checks<br>        livenessProbe:<br>          httpGet:<br>            path: /<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>        <br>        readinessProbe:<br>          httpGet:<br>            path: /<br>            port: 8080<br>          initialDelaySeconds: 10<br>          periodSeconds: 5<br>        <br>        # Resource limits<br>        resources:<br>          requests:<br>            memory: &quot;64Mi&quot;<br>            cpu: &quot;100m&quot;<br>          limits:<br>            memory: &quot;256Mi&quot;<br>            cpu: &quot;500m&quot;<br>        <br>        # Get environment variables from secret<br>        envFrom:<br>        - secretRef:<br>            name: frontend-config-secret<br>        <br>        # Security<br>        securityContext:<br>          runAsNonRoot: true<br>          runAsUser: 1000</pre><pre>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: frontend<br>  namespace: online-boutique<br>spec:<br>  selector:<br>    app: frontend<br>  ports:<br>  - port: 8080<br>    targetPort: 8080<br>  type: ClusterIP</pre><h3>Add Horizontal Pod Autoscaling</h3><p>Create frontend-hpa.yaml:</p><pre>apiVersion: autoscaling/v2<br>kind: HorizontalPodAutoscaler<br>metadata:<br>  name: frontend-hpa<br>  namespace: online-boutique<br>spec:<br>  scaleTargetRef:<br>    apiVersion: apps/v1<br>    kind: Deployment<br>    name: frontend-deployment<br>  minReplicas: 2<br>  maxReplicas: 10<br>  metrics:<br>  - type: Resource<br>    resource:<br>      name: cpu<br>      target:<br>        type: Utilization<br>        averageUtilization: 70</pre><h3>Deploy All Services</h3><p>Repeat the deployment pattern for all 11 services. Each service needs:</p><ul><li>Deployment with health checks and resource limits</li><li>Service for internal communication</li><li>HPA for high-traffic services (frontend, cart, product catalog)</li></ul><pre># Apply all deployments<br>kubectl apply -f frontend-deployment.yaml<br>kubectl apply -f frontend-hpa.yaml<br># Repeat for other services</pre><pre># Check deployment status<br>kubectl get pods -n online-boutique<br>kubectl get hpa -n online-boutique</pre><h3>Step 5: Set Up NGINX Ingress</h3><h3>Install NGINX Ingress Controller</h3><pre>kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/aws/deploy.yaml</pre><pre># Wait for deployment<br>kubectl wait --namespace ingress-nginx \<br>  --for=condition=ready pod \<br>  --selector=app.kubernetes.io/component=controller \<br>  --timeout=300s</pre><h3>Configure Ingress with Security</h3><p>Create ingress.yaml:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: app-ingress<br>  namespace: online-boutique<br>  annotations:<br>    # Security headers<br>    nginx.ingress.kubernetes.io/configuration-snippet: |<br>      add_header X-Frame-Options &quot;DENY&quot; always;<br>      add_header X-Content-Type-Options &quot;nosniff&quot; always;<br>      add_header X-XSS-Protection &quot;1; mode=block&quot; always;<br>    <br>    # Rate limiting<br>    nginx.ingress.kubernetes.io/rate-limit: &quot;100&quot;<br>    nginx.ingress.kubernetes.io/rate-limit-rps: &quot;10&quot;</pre><pre>spec:<br>  ingressClassName: nginx<br>  rules:<br>  - http:<br>      paths:<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: frontend<br>            port:<br>              number: 8080<br>      <br>      - path: /api/checkout<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: checkoutservice<br>            port:<br>              number: 5050<br>      <br>      - path: /api/cart<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: cartservice<br>            port:<br>              number: 7070<br>      <br>      # Add paths for other API services</pre><p>Apply the ingress:</p><pre>kubectl apply -f ingress.yaml</pre><pre># Get external IP<br>kubectl get service ingress-nginx-controller -n ingress-nginx</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZrfuQwT2d_aJ7Z6Kr0yS7w.png" /><figcaption>Online Boutique deployed successfully. See the Load Balancer URL.</figcaption></figure><h3>Step 6: Package with Helm Charts (Production-Ready Deployment)</h3><h3>Why Convert to Helm Charts?</h3><p>Managing 11 individual Kubernetes manifests becomes a nightmare when you need to:</p><ul><li>Deploy across multiple environments (dev, staging, prod)</li><li>Update configurations without editing each file</li><li>Version control your deployments</li><li>Share your setup with team members</li></ul><p>Helm charts solve this by using templates and centralized configuration.</p><h3>Create the Helm Chart Structure</h3><pre># Create the base Helm chart<br>helm create my-online-boutique</pre><pre># This creates the directory structure:<br># my-online-boutique/<br># ├── Chart.yaml<br># ├── values.yaml<br># └── templates/<br>#     ├── deployment.yaml<br>#     ├── service.yaml<br>#     ├── serviceaccount.yaml<br>#     ├── ingress.yaml<br>#     └── hpa.yaml</pre><h3>Configure Chart.yaml</h3><p>Edit Chart.yaml with your project details:</p><pre>apiVersion: v2<br>name: my-online-boutique<br>description: A Helm chart for Online Boutique microservices on AWS EKS<br>type: application<br>version: 0.1.0<br>appVersion: &quot;v1.0&quot;<br>home: https://github.com/your-username/microservices-k8s<br>sources:<br>  - https://github.com/your-username/microservices-k8s<br>maintainers:<br>  - name: Your Name<br>    email: your-email@domain.com</pre><h3>Convert Kubernetes Manifests to Helm Templates</h3><h4>1. Frontend Deployment Template</h4><p>Create templates/frontend-deployment.yaml:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: {{ include &quot;my-online-boutique.frontend.fullname&quot; . }}<br>  namespace: {{ .Values.namespace }}<br>  labels:<br>    {{- include &quot;my-online-boutique.frontend.labels&quot; . | nindent 4 }}<br>spec:<br>  replicas: {{ .Values.frontend.replicaCount }}<br>  selector:<br>    matchLabels:<br>      {{- include &quot;my-online-boutique.frontend.selectorLabels&quot; . | nindent 6 }}<br>  template:<br>    metadata:<br>      labels:<br>        {{- include &quot;my-online-boutique.frontend.selectorLabels&quot; . | nindent 8 }}<br>    spec:<br>      containers:<br>      - name: frontend<br>        image: &quot;{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}&quot;<br>        ports:<br>        - containerPort: {{ .Values.frontend.service.port }}<br>        <br>        # Health checks from values<br>        livenessProbe:<br>          httpGet:<br>            path: {{ .Values.frontend.healthCheck.path }}<br>            port: {{ .Values.frontend.service.port }}<br>          initialDelaySeconds: {{ .Values.frontend.healthCheck.initialDelaySeconds }}<br>          periodSeconds: {{ .Values.frontend.healthCheck.periodSeconds }}<br>        <br>        readinessProbe:<br>          httpGet:<br>            path: {{ .Values.frontend.healthCheck.path }}<br>            port: {{ .Values.frontend.service.port }}<br>          initialDelaySeconds: {{ .Values.frontend.healthCheck.readinessDelay }}<br>          periodSeconds: {{ .Values.frontend.healthCheck.periodSeconds }}<br>        <br>        # Resource limits from values<br>        resources:<br>          requests:<br>            memory: {{ .Values.frontend.resources.requests.memory }}<br>            cpu: {{ .Values.frontend.resources.requests.cpu }}<br>          limits:<br>            memory: {{ .Values.frontend.resources.limits.memory }}<br>            cpu: {{ .Values.frontend.resources.limits.cpu }}<br>        <br>        # Environment variables from external secrets<br>        envFrom:<br>        - secretRef:<br>            name: {{ .Values.externalSecrets.secretName }}<br>        <br>        securityContext:<br>          runAsNonRoot: true<br>          runAsUser: 1000</pre><pre>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: {{ include &quot;my-online-boutique.frontend.fullname&quot; . }}<br>  namespace: {{ .Values.namespace }}<br>spec:<br>  selector:<br>    {{- include &quot;my-online-boutique.frontend.selectorLabels&quot; . | nindent 4 }}<br>  ports:<br>  - port: {{ .Values.frontend.service.port }}<br>    targetPort: {{ .Values.frontend.service.port }}<br>  type: {{ .Values.frontend.service.type }}</pre><h4>2. Configure Values.yaml</h4><p>This is where the magic happens. Instead of showing the entire file (it’s quite long), here’s the structure:</p><pre># Global settings<br>namespace: online-boutique<br>replicaCount: 2</pre><pre># External Secrets configuration<br>externalSecrets:<br>  enabled: true<br>  secretName: frontend-config-secret<br>  secretStore: aws-secrets-store</pre><pre># Frontend service configuration<br>frontend:<br>  enabled: true<br>  replicaCount: 2<br>  image:<br>    repository: your-username/frontend<br>    tag: v1.0<br>    pullPolicy: IfNotPresent<br>  <br>  service:<br>    type: ClusterIP<br>    port: 8080<br>  <br>  resources:<br>    requests:<br>      memory: &quot;64Mi&quot;<br>      cpu: &quot;100m&quot;<br>    limits:<br>      memory: &quot;256Mi&quot;<br>      cpu: &quot;500m&quot;<br>  <br>  healthCheck:<br>    path: /<br>    initialDelaySeconds: 30<br>    readinessDelay: 10<br>    periodSeconds: 5<br>  <br>  autoscaling:<br>    enabled: true<br>    minReplicas: 2<br>    maxReplicas: 10<br>    targetCPUUtilizationPercentage: 70</pre><pre># Cart service configuration<br>cartservice:<br>  enabled: true<br>  replicaCount: 1<br>  image:<br>    repository: your-username/cartservice<br>    tag: v1.0<br>  # ... similar structure for each service</pre><pre># Ingress configuration<br>ingress:<br>  enabled: true<br>  className: nginx<br>  annotations:<br>    nginx.ingress.kubernetes.io/rate-limit: &quot;100&quot;<br>    nginx.ingress.kubernetes.io/rate-limit-rps: &quot;10&quot;<br>  hosts:<br>    - host: your-domain.com<br>      paths:<br>        - path: /<br>          pathType: Prefix<br>          service: frontend</pre><blockquote><strong><em>💡 Note:</em></strong><em> The complete </em><em>values.yaml file is quite extensive (200+ lines) since it configures all 11 microservices. You can find the full file in the </em><a href="https://github.com/DivineTheAnalyst/microservices-k8s"><em>GitHub repository</em></a><em>.</em></blockquote><h4>3. Create Helper Templates</h4><p>Create templates/_helpers.tpl for reusable template functions:</p><pre>{{/*<br>Expand the name of the chart.<br>*/}}<br>{{- define &quot;my-online-boutique.name&quot; -}}<br>{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix &quot;-&quot; }}<br>{{- end }}</pre><pre>{{/*<br>Frontend labels<br>*/}}<br>{{- define &quot;my-online-boutique.frontend.labels&quot; -}}<br>helm.sh/chart: {{ include &quot;my-online-boutique.chart&quot; . }}<br>{{ include &quot;my-online-boutique.frontend.selectorLabels&quot; . }}<br>app.kubernetes.io/version: {{ .Values.frontend.image.tag | quote }}<br>app.kubernetes.io/managed-by: {{ .Release.Service }}<br>{{- end }}</pre><pre>{{/*<br>Frontend selector labels<br>*/}}<br>{{- define &quot;my-online-boutique.frontend.selectorLabels&quot; -}}<br>app.kubernetes.io/name: frontend<br>app.kubernetes.io/instance: {{ .Release.Name }}<br>{{- end }}</pre><h3>Test Your Helm Templates</h3><p>Before deploying, always test your templates:</p><pre># Generate and review the manifests<br>helm template my-online-boutique ./my-online-boutique/ --namespace online-boutique</pre><pre># Check for syntax errors<br>helm lint ./my-online-boutique/</pre><pre># Dry run to validate against cluster<br>helm install my-online-boutique ./my-online-boutique/ --namespace online-boutique --dry-run --debug</pre><h3>Deploy with Helm</h3><pre># Create namespace<br>kubectl create namespace online-boutique</pre><pre># Install the application<br>helm install my-online-boutique ./my-online-boutique/ --namespace online-boutique</pre><pre># Verify deployment<br>helm list -n online-boutique<br>kubectl get pods -n online-boutique</pre><h3>Update and Manage Deployments</h3><p>One of Helm’s biggest advantages is easy updates:</p><pre># Update values and upgrade<br>helm upgrade my-online-boutique ./my-online-boutique/ --namespace online-boutique</pre><pre># Rollback if needed<br>helm rollback my-online-boutique 1 --namespace online-boutique</pre><pre># View deployment history<br>helm history my-online-boutique --namespace online-boutique</pre><h3>Environment-Specific Deployments</h3><p>Create different values files for each environment:</p><pre># Development deployment<br>helm install dev-boutique ./my-online-boutique/ \<br>  --namespace online-boutique-dev \<br>  --values ./my-online-boutique/values-dev.yaml</pre><p><strong>Example </strong><strong>values-dev.yaml:</strong></p><pre>frontend:<br>  replicaCount: 1  # Less replicas for dev<br>  resources:<br>    requests:<br>      memory: &quot;32Mi&quot;  # Lower resource limits<br>      cpu: &quot;50m&quot;</pre><pre>ingress:<br>  hosts:<br>    - host: dev.your-domain.com  # Different hostname</pre><h3>Step 7: Add Monitoring with Prometheus and Grafana</h3><h3>Why Monitoring Matters</h3><p>With 11 microservices running in production, you need visibility into:</p><ul><li><strong>CPU and memory usage</strong> across all services<strong></strong></li><li><strong>Error rates</strong> and failed requests</li><li><strong>Pod scaling behavior</strong> when HPA kicks in</li><li><strong>Resource utilization</strong> for cost optimization</li></ul><h3>Install Prometheus and Grafana Stack</h3><p>The easiest way to get comprehensive monitoring is using the kube-prometheus-stack Helm chart:</p><pre># Add the Prometheus community Helm repository<br>helm repo add prometheus-community https://prometheus-community.github.io/helm-charts<br>helm repo update</pre><pre># Install the complete monitoring stack<br>helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack \<br>  --set grafana.service.type=LoadBalancer \<br>  --namespace monitoring \<br>  --create-namespace</pre><pre># Verify the installation<br>kubectl get pods -n monitoring<br>kubectl get svc -n monitoring</pre><pre># Get the Grafana admin password<br>kubectl --namespace monitoring get secrets kube-prometheus-stack-grafana \<br>  -o jsonpath=&quot;{.data.admin-password}&quot; | base64 -d ; echo</pre><pre># Get the external IP for Grafana<br>kubectl get svc kube-prometheus-stack-grafana -n monitoring</pre><p><strong>Login credentials:</strong></p><ul><li>Username: admin</li><li>Password: (from the command above)</li></ul><h3>Monitor Your Microservices</h3><p>Once logged in, you’ll have access to pre-built dashboards that show:</p><h4>1. Kubernetes Cluster Overview</h4><ul><li><strong>Node resource utilization</strong></li><li><strong>Pod status across namespaces</strong></li><li><strong>Network traffic patterns</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jGjB96hMGHmQKah77l-fVA.png" /></figure><h4>2. Individual Service Metrics</h4><ul><li><strong>CPU and memory usage per service</strong></li><li><strong>Request rates for frontend, cart, and product catalog</strong></li><li><strong>Response times and error rates</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Rs5T8z7HCfVeJk-BGYgLA.png" /></figure><h4>3. Horizontal Pod Autoscaler Behavior</h4><ul><li><strong>Scaling events triggered by load</strong></li><li><strong>Current vs desired replica counts</strong></li><li><strong>Resource utilization trends</strong></li></ul><h3>Key Dashboards to Check</h3><p>Navigate to these pre-configured dashboards:</p><p><strong>Kubernetes / Compute Resources / Namespace (Pods)</strong></p><ul><li>Filter by online-boutique namespace</li><li>Shows resource usage for all your microservices</li></ul><p><strong>Kubernetes / Compute Resources / Pod</strong></p><ul><li>Drill down into individual service performance</li><li>Monitor scaling behaviour</li></ul><p><strong>Node Exporter / Nodes</strong></p><ul><li>Overall cluster health and resource availability</li></ul><h3>Monitor During Load Testing</h3><p>To see monitoring in action, run a load test while watching the dashboards:</p><pre># Generate load on your application<br>ab -n 5000 -c 50 http://&lt;INGRESS-IP&gt;/</pre><pre># Watch HPA scale in real-time<br>kubectl get hpa -w -n online-boutique</pre><h3>Set Up Alerts (Optional)</h3><p>Configure alerts for critical thresholds:</p><ol><li><strong>High CPU usage</strong> (&gt;80% for 5 minutes)</li><li><strong>Memory pressure</strong> (&gt;90% usage)</li><li><strong>Pod crash loops</strong> or restart events</li><li><strong>High error rates</strong> (&gt;5% 4xx/5xx responses)</li></ol><h3>Troubleshooting Monitoring</h3><p><strong>If Grafana doesn’t load:</strong></p><pre># Check if pods are running<br>kubectl get pods -n monitoring</pre><pre># Check service status<br>kubectl describe svc kube-prometheus-stack-grafana -n monitoring</pre><pre># View logs if needed<br>kubectl logs -n monitoring -l app.kubernetes.io/name=grafana</pre><p><strong>If metrics aren’t showing:</strong></p><pre># Verify Prometheus is scraping targets<br>kubectl port-forward -n monitoring svc/kube-prometheus-stack-prometheus 9090:9090<br># Visit http://localhost:9090/targets to see scraping status</pre><h3>Step 8: Test Your Complete Deployment</h3><h3>Check All Components</h3><pre># Verify pods are running<br>kubectl get pods -n online-boutique</pre><pre># Check HPA status<br>kubectl get hpa -n online-boutique</pre><pre># Verify external secrets sync<br>kubectl get externalsecrets -n online-boutique</pre><pre># Test ingress<br>curl -I http://&lt;EXTERNAL-IP&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5SHtTO0sviKwP29TxIi8fg.png" /></figure><h3>Test the Application Features</h3><p>Visit your application URL and test the key functionality:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J76VILX5dLmx9usSEa5TPA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bvdbRJyDIJA0yS8DZfZVjg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*exSZs4abSr56p0f8LRZ85g.png" /></figure><h3>Load Test</h3><pre># Install Apache Bench<br>sudo apt-get install apache2-utils</pre><pre># Test scaling<br>ab -n 1000 -c 50 http://&lt;EXTERNAL-IP&gt;/</pre><pre># Watch HPA scale<br>kubectl get hpa -w</pre><h3>Troubleshooting Common Issues</h3><p><strong>Pods stuck in CreateContainerConfigError:</strong></p><pre>kubectl describe pod &lt;pod-name&gt; -n online-boutique<br># Usually means secrets aren&#39;t syncing<br>kubectl get externalsecrets -n online-boutique</pre><p><strong>External secrets not syncing:</strong></p><pre>kubectl describe externalsecret frontend-config -n online-boutique<br>kubectl logs -n external-secrets-system -l app.kubernetes.io/name=external-secrets</pre><p><strong>Ingress not working:</strong></p><pre>kubectl describe ingress app-ingress -n online-boutique<br>kubectl get endpoints -n online-boutique</pre><h3>Project Structure</h3><pre>microservices-k8s/<br>├── src/                    # Source code for each service<br>│   ├── frontend/<br>│   ├── cartservice/<br>│   └── ...<br>├── k8s-manifests/          # Kubernetes configurations<br>│   ├── deployments/<br>│   ├── hpa/<br>│   ├── external-secrets/<br>│   └── ingress.yaml<br>├── scripts/<br>│   └── build_and_push.sh<br>└── README.md</pre><h3>What You’ve Built</h3><p>✅ <strong>11 microservices</strong> running on AWS EKS</p><p>✅ <strong>Secure secret management</strong> with AWS Secrets Manager</p><p>✅ <strong>Automatic secret sync</strong> with External Secrets Operator</p><p>✅ <strong>Production-ready containers</strong> with security hardening</p><p>✅ <strong>Load balancing and scaling</strong> with HPA</p><p>✅ <strong>Secure ingress</strong> with rate limiting and security headers</p><p>✅ <strong>Health checks</strong> and resource management</p><h3>Next Steps</h3><ul><li>Add monitoring with Prometheus and Grafana</li><li>Implement GitOps with ArgoCD</li><li>Set up centralized logging</li><li>Add service mesh for advanced traffic management</li></ul><p><strong>Complete code:</strong> <a href="https://github.com/DivineTheAnalyst/microservices-k8s">GitHub Repository</a></p><p><strong>Questions?</strong> Email: dnkwocha14@gmail.com</p><p><em>Found this helpful? Give it a clap 👏 and follow for more hands-on DevOps tutorials!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6296b9edde2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You can Build your own Container on a Linux Server.. without Docker]]></title>
            <link>https://medium.com/@dnkwocha14/you-can-build-your-own-container-on-a-linux-server-without-docker-e2d6f54bc45e?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2d6f54bc45e</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[containerization]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Fri, 28 Mar 2025 22:54:42 GMT</pubDate>
            <atom:updated>2025-03-28T22:54:42.186Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OwJg3bGfQKejeD-oAHI6SQ.png" /></figure><h3>Leave Docker Build your own Container on a Linux Server</h3><p>Did you know you can create a container-like environment on a Linux server without using Docker? Many of us are quick to run ‘docker build -t’ whenever we are containerizing an app but do you really know how it works on a lower level? In this guide, we’ll build an isolated application environment using Linux namespaces, cgroups, and chroot. This hands-on approach will help you understand how containerization works <em>under the hood!</em></p><h3>What You’ll Learn</h3><ul><li>How to isolate processes, networks, and filesystems using namespaces.</li><li>How to control resource usage with cgroups.</li><li>How to set up a minimal root filesystem using chroot.</li><li>How to deploy and test an application inside the isolated environment.</li><li>Security hardening using<em> seccomp.</em></li></ul><h3>Prerequisites</h3><p>Before we begin, ensure you have:</p><ul><li>A Linux server (Ubuntu 24.04 recommended).</li><li>Basic command-line knowledge-<em> cd, ls, mkdir</em>.. etc</li><li>Root access (we’ll use sudo for most commands).</li><li>Some understanding of networking (IP addresses, interfaces, etc.).</li></ul><h3>Step 1: Setting Up the Environment</h3><h3>1.1 Update Your Package List</h3><pre>sudo apt update</pre><p>This ensures you have the latest package information before installing new software.</p><h3>1.2 Install Required Packages</h3><pre>sudo apt install -y debootstrap iproute2 cgroup-tools</pre><p>About each package:</p><ul><li>debootstrap: Creates a minimal Linux filesystem.</li><li>iproute2: Manages networking settings.</li><li>cgroup-tools: Controls resource allocation.</li></ul><h3>1.3 Create a Minimal Root Filesystem</h3><pre>sudo debootstrap --arch=amd64 noble /isolated_env http://archive.ubuntu.com/ubuntu/</pre><p>This command sets up a minimal Ubuntu 24.04 (noble) filesystem inside /isolate_env.</p><pre>sudo mkdir -p /isolated_env<br>ls</pre><p>create the /isolated_env directory where the minimal file system has been created</p><h3>1.4 Mount System Files</h3><pre>sudo mount --bind /proc /isolate/proc<br>sudo mount --bind /sys /isolated_env/sys<br>sudo mount --bind /dev /isolated_env/dev</pre><p>The new file system we have created in /isolated_env do not have access to essential file systems on our host machine, they’re just regular folders. To fix this we mount using the bind flag to ensure there is a connection.</p><p>The <strong>/proc</strong> filesystem provides process information. Binding it ensures processes inside the isolated environment can access system details. <strong>/sys </strong>contains kernel and hardware information (needed for system-level operations), and<strong>/dev </strong>Provides access to system devices (needed for /dev/nulletc.).</p><h3>Step 2: Creating an Isolated Namespace</h3><h3>2.1 Start a New Namespace</h3><p>We need the processes in the environment we have created to be totally independent of the host processes- unable to see or interfere.</p><pre>sudo unshare --mount --fork --pid --net chroot /isolated_env /bin/bash</pre><h3>What Does this Whole Command Do?</h3><ol><li><strong>Creates an isolated environment</strong> with its own:</li></ol><ul><li>Filesystem (via chroot).</li><li>Mount points (--mount).</li><li>Process list (--pid).</li><li>Network (--net).</li></ul><ol><li><strong>Runs a Bash shell</strong> inside this environment.</li><li><strong>Protects the host system</strong> — anything done inside won’t affect the outside.</li></ol><p>This isolates the filesystem and starts a new environment, similar to how Docker containers work.</p><p>To test the isolation, run:</p><pre>ps aux</pre><h4>We will be using chroot a lot, so what is it?</h4><p>chroot stands for <strong>&quot;change root.&quot;</strong> It lets you create a <strong>fake root directory</strong> for a program so that it <strong>only sees</strong> that directory as its entire system.</p><p>Think of it like this:<br> 🔹 <strong>Normal System:</strong> A program can access everything on your computer.<br> 🔹 <strong>With </strong><strong>chroot:</strong> The program is <strong>trapped</strong> inside a specific folder and can’t see anything outside it</p><h3>Step 3: Configuring Networking Isolation</h3><p>Is our environment truly isolated if it still shares the same network as the host? Absolutely not! Creating a new network namespace, and setting up a virtual Ethernet (veth) pair to connect it to the host solves this problem.</p><h3>3.1 Create a Virtual Network Interface</h3><pre>sudo ip link add veth-host type veth peer name veth-isolate</pre><p>This creates a virtual Ethernet (veth) pair. One end (veth-host) remains on the host, while the other (veth-isolate) connects to the isolated environment.</p><h3>3.2 Assign a Network Namespace</h3><p>A <strong>network namespace</strong> in Linux is like a separate, isolated network “world” inside your computer.</p><pre>sudo ip netns add isolated_env<br>sudo ip link set veth-isolate netns isolated_env</pre><p>This moves the veth-isolate interface into the new namespace, giving it an isolated network stack.</p><h3>3.3 Assign an IP Address</h3><pre>sudo ip addr add 192.168.1.1/24 dev veth-host<br>sudo ip netns exec isolated_env ip addr add 192.168.1.2/24 dev veth-isolate</pre><p>Assigns the veth-host and veth-isolate interfaces IP addresses for communication.</p><h3>3.4 Activate the Network Interfaces</h3><pre>sudo ip link set veth-host up<br>sudo ip netns exec isolated_env ip link set veth-isolate up</pre><p>Enables both interfaces for communication.</p><h3>3.5 Test Network Connectivity</h3><pre>sudo ip netns exec isolate ping -c 3 192.168.1.1</pre><p>Ensures that the isolated environment can communicate over the virtual network.</p><h3>Step 4: Managing Resource Usage with cgroups</h3><p>cgroup (short for <strong>Control Group</strong>) is a Linux feature that lets you <strong>limit and control</strong> how much CPU, memory, disk, and network a group of processes can use.</p><p>Think of it like this:<br> 🔹 <strong>Without </strong><strong>cgroup:</strong> Any program can use as much CPU and memory as it wants.<br> 🔹 <strong>With </strong><strong>cgroup:</strong> You can <strong>set rules</strong> so that one program doesn’t slow down everything else.</p><h3>4.1 Create a New Cgroup</h3><pre>sudo mkdir /sys/fs/cgroup/isolated_env</pre><p>Creates a dedicated cgroup to limit resource usage for the isolated environment.</p><h3>4.2 Enable Resource Controllers</h3><pre>echo &quot;+cpu +memory +io&quot; | sudo tee /sys/fs/cgroup/isolated_env/cgroup.subtree_control</pre><p>Allows CPU, memory, and disk I/O controls for our cgroup.</p><h3>4.3 Limit CPU Usage</h3><pre>echo &quot;500000 1000000&quot; | sudo tee /sys/fs/cgroup/isolated_env/cpu.max</pre><p>This means the process inside isolated_env can <strong>only use 50% of the CPU every second</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vXzn0u0A6DV2wbbizDFQcg.png" /><figcaption>stress ran on cpu to test limit</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jArOXjqKOh0bzMMiMK6ECg.png" /><figcaption>got terminated before reaching 50%</figcaption></figure><h3>4.4 Set Memory Limits</h3><pre>echo &quot;256M&quot; | sudo tee /sys/fs/cgroup/isolated_env/memory.max</pre><p>Restricts the environment to using a maximum of 256MB RAM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UygxpnWW1aYj6Gv2S5kiIg.png" /><figcaption>stress ran on memory to test limit</figcaption></figure><h3>4.5 Control Disk I/O Operations</h3><pre>echo &quot;254:0 wbps=1048576&quot; | sudo tee /sys/fs/cgroup/isolated_env/io.max</pre><p>Restricts the write speed on device 254:0 to 1MB per second.</p><h3>4.6 Verify Cgroup Controllers</h3><pre>cat /sys/fs/cgroup/isolated_env/cgroup.controllers</pre><p>Displays available controllers to confirm that CPU, memory, and I/O restrictions are in place.</p><h3>Step 5: Adding a Non-Root User (On the Isolated Environment)</h3><p>Just like on our regular host server, there are various users apart from the root user e.g ubuntu. We would like to have that in our isolated environment (/isolated_env), it is best practice to have a user with non-root access, this ensures that processes inside the environment can run securely without unnecessary privileges.</p><pre>sudo chroot /isolated_env<br>apt update<br>apt install -y passwd adduser<br>adduser --disabled-password testuser<br>exit</pre><p>Thereafter, we test this newly created user immediately to see if what we did was right. (On the Host Terminal):</p><pre>sudo unshare --mount --fork chroot /isolated_env su - testuser</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*AQmLVxS1r9th-lzEBMiCMw.png" /></figure><h3>Step 6: Installing Python &amp; Running a Simple Application in our Isolated Environment</h3><p>This is the most exciting part for me! I can install dependencies and run applications in my very own container!</p><p>We now have a secure, isolated environment with a non-root user (congrats). To set up a simple HTTP server, we will install python first. This server will act as a lightweight application running inside the container (/isolated_env). This is to verify that we have done the right thing.</p><pre>sudo chroot /isolated_env<br>apt update<br>apt install -y python3<br>exit</pre><pre>vi script.py</pre><p>The above command installs python on the/isolated_env , then creates a python script.</p><p>Paste the following Python code in it.</p><pre>from http.server import HTTPServer, BaseHTTPRequestHandler<br>class Handler(BaseHTTPRequestHandler):<br>    def do_GET(self):<br>        print(f&quot;Received request: {self.path}&quot;)  <br>        self.send_response(200)<br>        self.send_header(&quot;Content-Type&quot;, &quot;text/plain; charset=utf-8&quot;)<br>        self.end_headers()<br>        self.wfile.write(b&quot;Welcome to Awele Container or Application Isolation Environment&quot;)<br>print(&quot;Starting server on port 8000...&quot;)<br>HTTPServer((&quot;&quot;, 8000), Handler).serve_forever()</pre><p>To round up this step, we copy the script to our isolated environment and set appropriate permissions for successful script execution. (On the Host Terminal):</p><pre>sudo cp script.py /isolated_env/home/testuser/<br>sudo chown 1000:1000 /isolated_env/home/testuser/script.py</pre><p>We cannot edit files in the container, we have to manually copy files into it from the host system.</p><p>Thereafter, the ownership commands ensures that testuser owns the script, rather than the root, thereby allowing the testuser execute it without permission issues.</p><h3>Step 7: Test the Server</h3><p>Now let us see if our script will actually run. This is strong evidence that our container is truly isolated. This command is to be ran on the host terminal:</p><pre>sudo ip netns exec isolated_env unshare --mount --pid --fork chroot /isolated_env su - testuser -c &quot;python3 /home/testuser/script.py&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*defSB8HB7BcUdsMYeVI64w.png" /></figure><p>To test this, open another terminal and run the following command on the host terminal:</p><pre>curl 192.168.1.2:8000</pre><p>Voila! 👇🏽</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*nwSqodERT9gsXZ74sj2m5w.png" /></figure><h3>Errors I faced</h3><p>Here’s a refined version of your “Errors I Faced” section, incorporating the errors you encountered and their resolutions:</p><h3>Errors I Faced</h3><h3>1️. Unable to Edit Files Inside the Isolated Environment</h3><p>I learned the hard way that I couldn’t edit files directly inside my isolated environment. Since the environment is built with a minimal filesystem, essential tools like text editors may not be available. To work around this, I had to <strong>create or modify files on my host machine and copy them into the container</strong> using:</p><pre>sudo cp myfile.txt /isolated_env/test-user/home</pre><h3>2. chroot: failed to run command ‘/bin/bash’: No such file or directory</h3><p>This happened because /bin/bash wasn’t available inside the chroot environment. The issue was that the debootstrap process might not have fully installed the required binaries. I fixed this by ensuring the necessary binaries were included:</p><pre>sudo debootstrap --variant=minbase noble /isolated_env</pre><p>Additionally, I could check for missing dependencies with:</p><pre>ls /isolated_env/bin/</pre><h3>3. Cgroup Configuration Errors</h3><h4>🛑 tee: /sys/fs/cgroup/isolated_env/io.max: No such device</h4><p>This error occurred because the specified device ID (259:0) did not exist or was incorrect. To fix this, I checked the available devices using:</p><pre>lsblk --nodeps -no NAME,MAJ:MIN</pre><p>Then, I used the correct ID in my io.max configuration.</p><h4>🛑 tee: /sys/fs/cgroup/isolate/io.max: No such file or directory</h4><p>This happened because the isolated_env cgroup was not properly created. I fixed it by ensuring the cgroup directory existed:</p><pre>sudo mkdir -p /sys/fs/cgroup/isolated_env</pre><p>Then, I verified that the cgroup.subtree_control was set before applying limits:</p><pre>echo &quot;+cpu +memory +io&quot; | sudo tee /sys/fs/cgroup/cgroup.subtree_control</pre><p>By troubleshooting these errors, I gained a much deeper understanding of how namespaces, cgroups, and chroot environments work under the hood.</p><h3>Conclusion</h3><p>By following these steps, we’ve built an isolated application environment similar to a container. This project gives you hands-on experience with core Linux isolation features like namespaces, cgroups, and chroot — helping you understand how tools like Docker work under the hood.</p><h3>Next Steps</h3><ul><li>Try running more complex applications inside the environment.</li><li>Experiment with additional security restrictions using seccomp or AppArmor.</li><li>Optimize networking to allow external access while maintaining isolation.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2d6f54bc45e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Experience with HNG 12 DevOps Track: Setting Up Nginx on Ubuntu]]></title>
            <link>https://medium.com/@dnkwocha14/my-experience-with-hng-12-devops-track-setting-up-nginx-on-ubuntu-83cb2ff4f671?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/83cb2ff4f671</guid>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 23:35:17 GMT</pubDate>
            <atom:updated>2025-01-29T23:35:17.488Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is HNG and Why Did I Join?</h3><p>HNG (Hotel Ng Internship) is a remote internship program designed to help tech enthusiasts accelerate their learning through hands-on projects, mentorship, and collaboration. It is a highly competitive program that spans multiple tracks, including Software Engineering, UI/UX Design, and DevOps.</p><p>I joined the <a href="https://hng.tech/hire/aws-solutions-architects">DevOps Track</a>.</p><p>All tracks are structured into 11 stages (Stage 0 to Stage 10). Successfully completing all stages earns the title of HNG Finalist — a recognition that demonstrates proficiency in real-world DevOps practices.</p><p>Currently, I am at <strong>Stage 0</strong>, where the first task is to install and configure <strong>NGINX</strong> on an Ubuntu server.</p><h3>Task Overview: DevOps Stage 0 — Nginx Configuration</h3><h3>Task Description</h3><p>The objective of this task is to set up and configure <strong>NGINX</strong> on a fresh Ubuntu server.</p><h3>Task Requirements</h3><h4>1. Nginx Installation and Configuration</h4><h4>2. Documentation</h4><h3>My Approach to Completing the Task</h3><h3>Step 1: Setting Up the Ubuntu Server</h3><p>To begin, I launched an <strong>EC2 instance</strong> on AWS running Ubuntu. Once the instance was up and running, I connected to it using SSH via the public IPv4 address.</p><h3>Step 2: Installing Nginx</h3><p>Since Ubuntu uses the <strong>APT package manager</strong>, I followed these steps:</p><p><strong>Update package lists</strong> to ensure I have the latest package information:</p><p>sudo apt update</p><ol><li><strong>Install Nginx</strong>:</li></ol><p>sudo apt install nginx -y</p><p>This command installs Nginx and ensures it runs automatically.</p><p><strong>2. Confirm the installation</strong> by visiting the public IPv4 address of the instance in a browser. If successful, the default Nginx welcome page should be displayed.</p><h3>Step 3: Configuring Nginx to Serve a Custom HTML Page</h3><p>To customize the default HTML page served by Nginx:</p><p>Navigate to the default web directory:</p><p>cd /var/www/html</p><ol><li>I opened the index.html file using the Vim text editor:</li></ol><p><em>sudo vi index.html</em></p><p>2. Replaced its content with my custom message:</p><p><em>Welcome to DevOps Stage 0 — Ijeawele Divine Nkwocha/awele.yml</em></p><p>3. Save and exit the file.</p><p>4. Restart Nginx to apply changes:</p><p>Refresh the browser using the public IPv4 address to see the custom page.</p><h3>Challenges Faced</h3><p>Surprisingly, I didn’t encounter any significant challenges while setting up Nginx. The installation and configuration were straightforward. However, I made sure to follow best practices, such as updating my package lists before installation and restarting the Nginx service after making changes.</p><h3>How This Task Contributes to My Learning and Professional Goals</h3><p>As someone aiming to become a <strong>Senior DevOps Engineer</strong>, this task helped me strengthen my understanding of:</p><ul><li>How <strong>Nginx functions as a web server</strong> and its role in serving static content.</li><li>How to <strong>install and configure Nginx on an Ubuntu server</strong>.</li><li>How to <strong>customize the default page</strong> served by Nginx.</li><li>The importance of <strong>cloud computing platforms like AWS</strong> in hosting infrastructure.</li></ul><p>This foundational knowledge is essential for working with <strong>reverse proxies, load balancers, and containerized deployments</strong>, which are common in DevOps workflows.</p><h3>Conclusion</h3><p>Completing <strong>Stage 0</strong> of the HNG 12 DevOps track was a great learning experience. The hands-on approach reinforced my understanding of setting up and configuring Nginx on a Linux server. I’m excited for the next stage and look forward to tackling more complex challenges!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83cb2ff4f671" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Journey with Google Cloud, Terraform, and Kubernetes: A Data Engineer’s Tale]]></title>
            <link>https://medium.com/@dnkwocha14/my-journey-with-google-cloud-terraform-and-kubernetes-a-data-engineers-tale-9d9927cf8511?source=rss-91dad3dbfe5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d9927cf8511</guid>
            <category><![CDATA[airflow]]></category>
            <category><![CDATA[data-engineer]]></category>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Ijeawele Nkwocha]]></dc:creator>
            <pubDate>Sat, 13 Jul 2024 12:28:01 GMT</pubDate>
            <atom:updated>2024-07-13T21:30:53.484Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey everyone! I’m currently participating in a 21-day challenge organized by www.amdari.io, where data professionals work on projects to enhance their portfolios. This challenge involves completing projects and discussing it consistently on any social media platform of our choice. I decided to take on a project with a fictional company called Variety Inc., and it’s been quite a journey!</p><p><strong>Brief overview of this Project</strong></p><p><em>Variety Inc</em>. faces challenges with managing the growing volume of movie reviews, user sentiments, and content recommendations, leading to difficulties in scalable infrastructure deployment and analytical latency. To stay competitive in movie review analytics on the Google Cloud Platform (GCP), they need to address data scalability, reduce latency, and optimize their data infrastructure for advanced analytics and real-time insights.</p><p><strong>Diving into Google Cloud and Terraform</strong></p><p>My project started with setting up infrastructure on Google Cloud using Terraform. Terraform is an open-source infrastructure as code tool that allows you to define and provision data center infrastructure using a high-level configuration language. It made managing my cloud resources efficient and automated.</p><p>I began by setting up Google Cloud Storage (GCS) buckets to manage my project’s data. I created a main bucket and a staging bucket, configuring them with appropriate locations and environments.</p><p><strong>Creating a Kubernetes Cluster</strong></p><p>Next, I spun up a Google Kubernetes Engine (GKE) cluster to handle containerized applications. I defined cluster details like node count and machine types using Terraform. I encountered a problem with disk sizes; I initially allocated 100 GB per node, which was too much for my free subscription on GCP. After some research, I reduced it to 50 GB, and everything worked smoothly!</p><p>Here’s the Terraform command that helped me configure `kubectl` to interact with my GKE cluster:<br>```sh<br>gcloud container clusters get-credentials $(terraform output -raw k8s_cluster_name) — region us-central1-a<br>```<br>This command retrieves the credentials and configures `kubectl` to use them.</p><p>`kubectl` is the command-line tool used to interact with Kubernetes clusters. It allows you to run commands against Kubernetes clusters to perform a variety of tasks, such as deploying applications, inspecting and managing cluster resources, and viewing logs.</p><p><strong>Setting Up Apache Airflow on Kubernetes</strong></p><p>With my Kubernetes cluster up and running, I focused on deploying Apache Airflow. Airflow is an open-source tool used to programmatically author, schedule, and monitor workflows. It’s great for managing complex data pipelines.</p><p>To deploy Airflow, I created a namespace called `nfs` and wrote YAML files to define the Kubernetes components. Here’s a glimpse of what I did:</p><ul><li>Created a pod for the NFS server.</li><li>Set up a service for the NFS server to make it accessible within the cluster.</li><li>Deployed the NFS server using a deployment and replicaset.</li></ul><p><strong>Getting Airflow Up and Running</strong></p><p>I continued working on Airflow deployment. I created a GitHub repository named `movie-review-project`, initialized my project folder with `git init`, and added a DAG folder with a `sample.py` file. I committed everything and pushed it to GitHub.</p><p>Using Helm, a package manager for Kubernetes, I deployed Airflow.<br>```sh<br>helm upgrade — install airflow -f airflow-values.yml apache-airflow/airflow — namespace airflow<br>```<br>Helm made it easy to manage the Airflow deployment, specifying custom configurations and ensuring everything was set up correctly.</p><p><strong>What I’ve Learned</strong></p><p>Throughout this journey, I’ve gained a deeper understanding of cloud infrastructure, Kubernetes, and orchestration tools like Terraform and Helm. Here are some key points:</p><p>1. <strong>Infrastructure as Code:</strong> Using Terraform to manage cloud resources is a game-changer. It allows for consistent, repeatable, and automated infrastructure deployments.<br>2. <strong>Kubernetes</strong>: Setting up a GKE cluster and deploying applications on Kubernetes has shown me the power of container orchestration and how it simplifies managing microservices.<br>3. <strong>Apache</strong> <strong>Airflow</strong>: Deploying Airflow on Kubernetes has made me to understand the importance of workflow automation and how tools like Airflow can streamline complex data pipelines.</p><p><strong>Final Thoughts</strong></p><p>This project has been an incredible learning experience. It was challenging at times, especially troubleshooting disk size issues and learning to deploy Airflow. But each hurdle was a valuable learning opportunity.</p><p>I’m very grateful to Amdari for giving me an avenue to push myself and consistently learn no matter how busy I was during day or how tired I am at night.</p><p>Thanks for following along on my journey! Feel free to connect with me on LinkedIn and Twitter to stay updated on my progress and future projects.</p><p>#DataEngineering #CloudComputing #GCP #Terraform #Kubernetes #Airflow #Amdari21DaysDataChallenge #DivineNkAmdariTODDC</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d9927cf8511" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>